By Mark Weiner
A study sponsored in part by the Institute for Public Relations and The Arthur Page Society revealed that, while executives who fund PR programs desire measurement to gauge
performance and ROI, they don't know enough about PR to provide direction. At the same time, it revealed that PR execs want to measure their performance but don't know enough about
PR research to initiate the process. As a result, no progress is made.
While media relations is only one form of PR, the following 10 questions, organized by phases, stimulates the media measurement process and provides guidance for getting the best
ROI in media relations research.
Fundamentals
*Do you provide content for the media analysis or do I? An "all-in-one" solution is convenient and may be cost-effective, but if the content is not included, be sure to factor
that incremental cost into your budget. There should be no surcharges or hidden fees when using an all-in-one solution.
*Is your firm's offering based on computerized coding or human coding? Computerized coding relies on artificial intelligence software to divine the meaning of a news story. Humans
may be slower but we tend to be more accurate in recognizing the underlying meaning of complex issues, nuance and sarcasm. Computerized coding systems may be consistent but less
precise, and they usually require significantly more time in establishing and adjusting the criteria by which the media are analyzed.
*How do you check for quality and consistency? Accuracy is a critical element in any research program. With so much data to be uncovered, entered and produced, each media
analysis program may have hundreds of thousands of quality elements (accurate circulation, accurate message analysis, accurate reporting--even simple tasks like spelling need to be
checked).
It is reasonable to expect the service provider's software to eliminate simple errors, that the coders have strict guidelines (known commonly as "the code book"), that the coders
are trained and encouraged to openly discuss judgment calls, that the coders are supervised and evaluated on the quality and accuracy of their work, that the people involved in report
preparation look for abnormalities when they transpose data to charts and graphs, and so on. With this in mind, expect at least six levels of quality control.
*If our program is international, do you do the work "in-country"? Rather than romance-language majors at the local university, "in-country" coders and analysts understand local
priorities and nuance. CSR themes resonate much more strongly, for example, in France than they do in the U.S. If your program is international, you have a right to expect native-
language speakers to perform your coding and analysis. Even if the French coder works in an office in London, the coder should be French and familiar with French culture to
accurately reflect local perceptions.
Getting Started
*How much time is required to set up the program? Initial program setup requires a thorough examination of objectives, strategies and measures of success. A simple program can be
set up same day while more sophisticated programs may take longer. This is an important factor to consider when choosing a media measurement partner generally and an automated or
human-based coding solution in particular.
*Do I have input on designing my own report or do you use templates? One consideration in determining cost and turnaround is the degree of standardization which can be leveraged
when designing your media measurement. Some programs are simple enough to be standardized but others may require custom elements. Keep in mind that your initial needs may be
standard but your future needs may require customization. Flexibility is an important attribute. Also, be certain to weigh the impact of customization as it may greatly increase
your cost and delay turnaround in exchange for elements which may not be as important as the related costs.
*May I meet the people who will be working on my account? Establishing a relationship with your research team is a good idea in the long term, but when it comes time to choose a
measurement firm, it is important to profile the team in terms of whether they are employees or part-timers, on-site versus work-at-home personnel, and whether they have industry
expertise in your category. These elements translate to better research, greater flexibility and enhanced efficiency.
Reporting & Analysis
*How long does it take to produce a report after the end of the reporting period? Do-it-yourself options provide for instantaneous reporting, but if you intend to use an outside
firm, the reporting should be delivered while there's still time to act upon the findings: Content sources can affect delivery, as some digital content feeds are faster than manual
press clippings, but monthly reports should be delivered within 10 days of the month's end; quarterly reports should be delivered within the following month. Faster turnaround times
are usually available for an extra cost.
*Do you provide interpretive analysis and guidance? Some measurement solutions resemble a report card: They tell you how you've done. Other solutions resemble a tutor: They go
beyond "how you did" to answer "why" and "what can be done about it." It is important to know in advance what you are looking for: Every media measurement firm provides data but only
a handful can provide meaningful interpretive analysis and salient strategic guidance.
*How much time is required to adjust my media analysis program when message or media changes are required? Your media relations programs change at "the speed of business," which
is to say moment by moment. Your media measurement program must be just as flexible and responsive.
As in any serious pre-purchase process, ask about the firm's cost-basis, experience in media measurement and expertise in your industry; then, request--and check--references. PRN
CONTACT:
Mark Weiner is the CEO of PRIME Research North America. He can be reached at [email protected].