Tip Sheet: Why Most Measurement Systems Are Bogus (And How To Make Yours Better)

When Henry Ford first started to produce his Model T cars, he used to say that you can have any color you want - so long as it's black. In public relations, we seem to be paraphrasing that when clients ask about measurement - you can have any measurement you want, so long as it's based on circulation or advertising.

But the bottom line on that is that any measurement system based primarily on circulation, impressions, or advertising equivalency is just plain bogus. If you accept the premise that the job of public relations is to drive the right messages to the right audiences in order to change their behavior and achieve corporate or organizational objectives, then the only measurement system that means something is one that measures how effective you were in doing that.

Now, some of you may be saying to yourselves, "Well, I don't accept that premise because our job is to facilitate two-way conversations with our publics." I don't really buy that, because I think almost all organizations hire us to achieve some goal based on that organization's particular point of interest. It could be sales, or a stock price, or community perceptions, etc. But it comes down to effectively driving messages.

Why don't impressions, circulation, or advertising equivalency count? Because they're not measuring real effectiveness. Let's say that you place a large story about your fishing tackle manufacturer client in Good Housekeeping. That would count in most evaluation systems as an impressive placement. Large circulation, great advertising equivalency, lots of impressions. But Good Housekeeping would certainly not be a well targeted publication for fishing tackle, since most studies show that women are not heavy purchasers of fishing tackle. And what if the story simply carried a brief mention of the client's product, or did not include any of the messages necessary to drive sales?

There is any number of systems out there that purport to use complicated metrics to provide quasi-scientific approaches to measuring media results. The Canadian Public Relations Society has developed a system that most agencies in Canada now use, and media distribution and clipping services offer systems in the United States. But all of these are ultimately based on circulation and advertising equivalency, with little more than a nod to message impact.

My firm, like most others, spent years using the same old systems. But as clients increasingly were looking for measurement systems based on effectiveness that they could have confidence in, we developed our own. It's called the Impact Ratingr system, and here's how it works:

Every story that is generated is measured on about six rating scales. While the rating areas may change from client to client, a typical one would contain such questions as:

1.) Did the story contain agreed upon messages that were important?

2.) Did the story appear in media that effectively reached the target audience?

3.) Was the story on the front page, high in a broadcast, or was it in the back of the publication or broadcast?

4.) Was it a big or a small story?

5.) Was there artwork, photography or breakout quotes important to the client?

6.) Was the client the focus of the story or just mentioned in the story?

These six measurements, each on a 1-to-10 scale, produce an overall score for the particular media placement - let's say 8.73. When you add all the stories from a particular month together, you might have a score of 7.79, for example. But it gives you a true measure of how effectively you are driving your client's messages to their audiences.

Only at this point do you note total circulation, to give a general scope for how far your reach has gone. That is, if you have a 7.79 score, but your circulation/viewership is only 40,000, that's not as impressive or effective as a score of 7.79 with a circulation/viewership of 5 million.

It's important to be honest with ourselves, whether we are agencies or corporate public relations departments. Measuring a media relations program based on circulation or advertising equivalency is just silly. It says nothing at all about the quality of the stories or whether they got to the right audience.

As public relations programs get more expensive and managers are asked to justify those expenditures, impact ratings can go a long way toward giving everyone a true look at just how effective your media relations program has been.

CONTACT:

Tom Amberg is the President and CEO of Cushman/Amberg Communications, based in Chicago. He can be reached at [email protected].