• Advertising Value Equivalents
• PR Value
• Positive/negative/neutral ratings
• Media monitoring v media analysis
• CARMA® Methodology – in simple terms
• International Best Practice standards
• Subjectivity / objectivity
• Customising media analysis
• Multi-country reports
• Regional and international analysis
• Foreign language media
• Cost
• Frequency of media analysis reports
• ROI – how to measure it?
• Sales – can PR be measured in terms of money?

Are Advertising Value Equivalents (AVEs) a good measure of media publicity?

So called ‘ad values’ are used by some PR firms and executives as a measure of publicity (ie. volume of publicity multiplied by the advertising rate of the media in which it appears). However, there are fundamental flaws in this method. Advertising is never unfavourable; never promotes competitors; always communicates key messages; and is usually favourably positioned. Editorial publicity may be very favourable and influential. However, editorial also can be negative; it can report on competitors and even favourably compare competitors; it may not contain key messages; and it may not be in key media that reach your target audiences and markets. Also, a client’s mentions may only be part of an editorial article. Few users of Advertising Value Equivalents deduct unrelated content and unfavourable coverage or coverage that misses your target audiences. This method is described as “questionable” and “invalid” by the world’s leading public relations academics, institutes and researchers. More information at the US Institute for Public Relations.

back to top

What about the ‘PR Value’ of media coverage?

The value of public relations is the extent to which it achieves objectives set – whether these are creating awareness, changing attitudes and perceptions, generating inquiries or simply gaining favourable media publicity.

Measurement methods which multiply the advertising cost of space gained as publicity by three or some other multiple have no research basis. These are extremely flawed as editorial publicity may be negative or neutral; it may contain coverage of competitors; it may be poorly positioned; and it may not be in key media that reach your target audience or market. Use of multipliers is described as “dishonest” and “unethical” by the world’s leading public relations research institute, the US Institute for Public Relations.

back to top

Is Positive, Negative, Neutral rating a good method of analysis?

It is better than counting all publicity as good publicity. But positive/negative/neutral ratings are usually subjectively decided. Furthermore, they are simplistic. Articles may be slightly positive or negative, or very negative or positive. A three-point scale is very broad – for instance, a three-point scale is not used in other research such as surveys where five, seven of 10-point scales are most commonly used to provide more precise measurement. Thirdly, they are a univariate method of analysis (ie. they assess a single variable). An article may be positive, but in a medium that does not reach your target audience. Or an article may be small and poorly positioned. Best Practice media content analysis assesses multiple variables about each article to decide its likely impact, including positioning, audience reach, messages communicated, your share of voice in the article, etc.

A good example of the limitation of positive/negative/neutral ratings is during a crisis or issue management when media coverage may be highly negative. Hard work by corporate communication executives may succeed in substantially improving the tone of coverage – but it is unlikely to swing quickly to positive coverage. A simplistic positive/negative/neutral categorisation will not show incremental improvement which can be shown in a more sophisticated scale such as the CARMA® Favourability Rating which rates articles in one-point increments from 0-100 where 50 is neutral.

back to top

I have media monitoring, why do I need media analysis?

Media monitoring is the collection of data – not research or analysis. Media coverage may be favourable – or it may be negative. Even when generally positive, it may not contain your key messages, or it may not reach your key target audiences. Analysis is necessary to evaluate these factors that determine the effectiveness of media coverage. And media analysis should be done quantitatively and qualitatively.

Unlike media advertising where content, positioning and favourability is controlled, editorial media coverage is highly variable. Therefore, analysis of the content of editorial media such as news, current affairs, feature articles, columns, etc is essential.

back to top

What is CARMA’s methodology and why is it better?

The core elements of the CARMA® methodology that are unmatched by other simpler methods are:

• It analyses multiple variables in each editorial media article that determine its impact – such as audience reach (does it reach your target audience?); positioning (page number or position in a broadcast); placement (where you appear in an article); share of voice compared with any other sources quoted in the article and messages communicated (positive and negative)
• It uses a systematic objective coding and scoring method based on pre-determined values for each variable – not someone’s personal rating or interpretation
• It takes account of qualitative factors (tone, messages, etc) – not only quantitative factors
• It uses rigorous quality control measures such as more than one analyst reading articles to reduce subjectivity, intercoder reliability checking and other QA measures.

back to top

What is considered International Best Practice in media analysis?

There are many criteria to achieve International Best Practice in media analysis. Some of the key ones are:

• Multiple variables in each editorial media article that determine its impact should be analysed – such as audience reach (does it reach your target audience?); positioning (page number or position in a broadcast); placement (where you appear in an article); share of voice compared with any other sources quoted in the article and messages communicated (positive and negative)
• It should use a systematic objective coding and scoring method based on pre-determined values for each variable – not someone’s personal rating or interpretation
• It should include qualitative analysis such as tone, messages, etc drawing on semiotics – not only quantitative factors
• It should use rigorous quality control measures such as more than one analyst reading articles to reduce subjectivity, strict Coding Guidelines and employ intercoder reliability assessment to check variance between coders.

Download a free White Paper on Media Content Analysis – International Best Practice.

back to top

How can media analysis be objective – isn’t it very subjective?

Minimisation of subjectivity and reliability of analysis is maximised through five key quality steps in CARMA Asia Pacific:

• Multiple analysts are used on all projects to minimise individual subjectivity in interpretation
• Analysis is based on systematic coding and scoring of variables from a set scale of values – not on personal ratings by analysts
• Analysis is based to a large extent on objective criteria such as media name, circulation or audience, page number, positioning, article length, sources’ names quoted, etc
• Qualitative analysis of Issues and Messages – somewhat more subjective – is undertaken by strict word and phrase matching or presence of acceptable synonyms, with a list of acceptable synonyms provided in written Coding Guidelines to maintain consistency and rigour
• A percentage of articles is coded by more than one analyst and intercoder reliability assessment is conducted for ANOVA and ANCOVA (Analysis of Variance and Analysis of Co-variance). Where variance is higher than our strict standards, data is rejected, analysts are re-briefed and articles are recoded. CARMA Asia Pacific is the only commercial media analysis firm that carries out intercoder reliability assessment.

back to top

Can media analysis be customised to our specific needs?

Yes, media analysis should analyse media coverage to identify how well it reaches your key target audiences and markets; how well it communicates your key messages; how often it reports on the issues that you want to see reported; etc. CARMA Asia Pacific media analysis is fully customised to clients’ needs.

back to top

Can CARMA do regional and international media analysis?

CARMA Asia Pacific is part of CARMA International which is the only truly global media analysis firm. CARMA International has offices in the US, Canada, UK, Europe, Japan, India, SE Asia, Australia and South America. Uniform methodology and standards are used globally. So CARMA Asia Pacific can provide media analysis using internationally recognised methodology locally, regionally or globally.

back to top

Can CARMA do media analysis across multiple countries?

CARMA Asia Pacific is part of CARMA International which is the only truly global media analysis firm. CARMA International has offices in the US, Canada, UK, Europe, Japan, India, SE Asia, Australia and South America. So CARMA Asia Pacific can provide media analysis using internationally recognised methodology locally, regionally or globally.

CARMA Asia Pacific can provide media analysis in a single Asian country or in several countries to suit the needs of the client. Some of our clients want analysis across six, eight, 10 or more countries in the region and we can provide this with our multi-lingual research analysts.

back to top

What about media coverage that is in Mandarin, Japanese, Korean, Thai, etc?
Can you analyse that?

Good question. Some media analysis is based on English language media only or feeds from services such as Factiva. But this captures only a small proportion of the media coverage in Asia Pacific. Alternatively, some media analysis relies on translations. But we all know that translations can be inaccurate. Furthermore, they take time and cost money.

CARMA Asia Pacific employs research analysts who are native speakers of English, Mandarin, Korean, Japanese, Bahasa Malaysia, Bahasa Indonesia, Tamil, Thai and Tagalog and analyses all media coverage ‘in language’ and with an understanding of the media and culture. Therefore, CARMA Asia Pacific media analysis is reliable and comprehensive.

CARMA Asia Pacific also has offices in Asia – unlike some research firms which attempt to analyse media coverage from the US or Europe.

back to top

How much does media analysis cost?

CARMA Asia Pacific costs are fully transparent. Time taken for media analysis depends on the volume of media articles that have to be read and analysed. We charge a rate per article for reading and coding and disclose this rate in our published rate card.

Reports can be presented to clients at any frequency required – eg. monthly, bi-monthly, quarterly or ad hoc project. We charge a rate per report, in addition to the cost of reading and coding articles, and report costs are also listed in our rate card.

Our rates are based on standard research industry hourly charges and our methodology that involves us reading and analysing 12-15 articles per hour. Faster methods skim read and produce less in-depth and less reliable analysis. Contact us for a quotation based on your brief.

back to top

What is the frequency of CARMA reports?

Media analysis reports can be presented to clients at any frequency required – eg. monthly, bi-monthly, quarterly, half-yearly, annually, or ad hoc project. Also, frequency can be changed – for instance, a client may order quarterly reports but during a major launch or an issue management strategy may required more frequently reports. CARMA Asia Pacific can respond to client needs quickly.

back to top

How do you measure the ROI (Return on Investment) of PR?

ROI stands for ‘Return’ on Investment. All communication should have some return – some impact or effect. Return from PR and corporate communication may not be and is unlikely to be sales or money directly. The return required and reasonably expected from PR and corporate communication is usually awareness increase, attitude change, improvement in understanding, generation of inquiries, etc. These are the types of returns that should be measured.

The ‘Return’ that can be reasonably expected of media publicity is favourable positioning of messages, audience reach and, ideally, dominant share of voice over competitors. These can be measured using media content analysis.

Other types of communication such as events and publications and can be measured using surveys.

back to top

Can PR be measured in terms of sales or money?

This is occasionally requested by some management. But PR is not a sales function. It may support sales, but it does not directly generate sales. Nor does HR, the legal department, or even marketing for that matter. Sales and profits are the result of a team effort by many in the organisation and are achieved at the end of a process involving many inputs and contributions.

The ‘Return’ from PR and corporate communication that should be measured is what is stated in your objectives. What is the communication intended to achieve? Is it media profile; awareness; attitude change; inquiries; shareholder support; employee loyalty?

back to top