Measurement Myths that Hamper Public Relations

As business consultants, one of the most important things we can do is help executives overcome outdated thinking disguised as time-honored traditions. In many cases, the process involves declaring that the way we’ve always done it may not necessarily be the way to do it now. Or ever again.

To keep things simple, let’s refer to these business conventions as myths.  Often based on tradition, they are stories that may be founded on factual origins. In some cases, they're completely fictional.

In PR we contend with many myths that continue to endure despite advances in the profession. Some of the most enduring relate to the science of PR. In this column, we will expose some PR conventions as myths, while explaining why they’re myths and sharing supportive details about the true state of the profession.

It's impossible

Earlier in my career, one prevailing myth was that PR is impossible to measure. PR is an art and one cannot quantify art. Like many myths, this one disguised a prevailing fear. I assumed that since PR measurement appeared in text books, everyone measured (they simply chose not to talk to me about it).

The truth was revealed during a cocktail hour at an international PR conference. The CEO of a well-known agency exclaimed, “Measure? We will never measure! I would gladly forego being a proven success for never being a proven failure.”  Without another word, he turned and headed for the bar.

There I was, standing alone, slack jawed, in the center of the room. I'd learned the truth: it wasn’t that everyone measured and just hid it from me. Very few measured because they feared the results.

Now, 25 years after that experience, the profession has evolved into one in which low-cost/no-cost technology puts measurement ability in the hands of almost every practitioner. Hundreds of technology programs make media analysis and social media listening relatively easy and inexpensive. This has democratized PR measurement.

In 2022, a majority of communicators have technology that monitors and measures media outputs using automated systems. These tools count digital content, gauge its tone/sentiment and identify themes within content.

Ironically, the evolution of technological solutions for media measurement has uncovered a new myth, which may be the most dangerous. This new myth purports that purely automated content analysis is just as good as human analysis, only faster. If it were only true.

Press play, get insights?

It may depend on one’s objectives, but the notion of accurate, real-time content analysis for insights generation and business decision-making is a 21st century myth.  Touted as artificial intelligence, these technologies are fast, consistent and accessible.

Unfortunately, they are not turnkey insights engines. Rather than thinking machines, these tools, as they are popularly called, require an awful lot of human involvement. Humans train the technology, manage it and interpret results, uncovering useable insights.

To provide accurate and reliable insights, one requires:  Technology, sector expertise and statistical acumen.

  • Technology enables the communicator to execute.
  • Sector expertise, in the forms of industry expertise, communication experience and an understanding of the media, ensures that one’s findings are relevant.
  • Statistical acumen guarantees that the findings are accurate.

Sector experts must train the computer to recognize word patterns and assign meaning to them.  For example, an untrained computer could interpret my name as “German currency” and a “sausage.” (Weiner as a last name in the third grade was difficult enough).

Machine learning from humans

Now, consider the complexity of training a computer to uncover meaning in broad reputation topics like innovation, corporate social responsibility and leadership. Journalists and influencers can express these term in thousands of ways.

In 2022, it may be less expensive and convenient to believe the myths that: a) artificial intelligence in its current state is accurate and reliable; and b) that manual coding and analysis is 20th century thinking.

But, like so many man-made solutions derived to disguise our deepest fears, the notion that inexpensive, fully automated content analysis and social listening are reliable and accurate simply isn’t true.  In fact, if the automated content analysis leads to the wrong decision at a critical time, it can be an expensive mistake.

Human-machine collaboration

Until true AI in the form of thinking machines arrives–as it will someday–the most reliable path forward relies on a balance of technology and talent. For now, human intuition is required to understand content's many nuances. Similarly, intuition is needed to comprehend an author's intentions to derive meaning from content and deliver meaningful insights.

Statistical acumen and sector expertise without technology results in very slow execution. Technology and statistical acumen without sector expertise reveals irrelevant results.  Sector expertise with technology but lacking statistical acumen and critical thinking produces inaccurate results. Recognizing these limitations,  we see more and more agencies and corporate communication teams employing one-percenters, who understand communication and love data analysis.

If you’re considering a media analysis platform for content analysis and social listening, beware of claims suggesting that technology alone is sufficient. When you lack the talent to manage a system and uncover insights, you will be wasting resources. Only the combination of talent with technology provides useful insights and uncovers PR’s unique, quantifiable contribution to the enterprise.

Mark Weiner was chief insights officer at PublicRelay and author of “PR Technology, Data and Insights."