AI Augments Humans with PR Measurement, but it’s no Silver Bullet

Pauline Draper-Watts, EVP, Lead, Global Measurement Edelman Intelligence

Setting the Scene

Artificial Intelligence (AI) is influencing our world in dramatic ways. It will continue to do so owing to increased automation, machine learning, and a greater focus on technology.

Our relationship with AI is complex, however, with periods of optimism and hype (and of course, great funding). Following these are AI winters, where disappointing outcomes result in greatly reduced funding.

Realistic Expectations

On returning to a period of hype, as communications professionals, we need to be prudent about what we can and cannot accomplish with AI.

The sub-field of natural language processing focuses on handling of natural language by machines with a human-level understanding of that language. This is a common application of AI within research and media measurement. It has enjoyed varying degrees of success.

This natural language data now gushes from many sources and seems more promising than before. Determining what is relevant and what can be ignored, where trends exist, how to leverage smart tools for analysis, where to devote time to draw relevant insights with the ability to stress test them, and ultimately tell a story is becoming increasingly more complex, though. Can AI help?

School Days: A look at a segment of Talkwalker’s platform undergoing training using the company’s AI engine. Source: The author

It’s All About the Data

As someone who has been working with media analysis and research for more than 25 years, I remember when early social listening providers said that you no longer needed focus groups, that you could just look at social media. For those looking to introduce their new tools into an organization, this was a common sales ploy.

What they failed to understand/account for through this tactic was the full purpose of a focus group in terms of audience: questions you might wish to address, the color and fabric those conversations have, and so much more. Yes, listening tools were a benefit, but they were not the entire answer.

The Tools of our Modern Life

Similarly, today monitoring tools are part of our everyday lives as researchers/analytics professionals. They trawl the internet, pulling in vast swaths of data from websites, news sources, social media channels, and forums, to name a few.

The volume of content is ever-increasing, far more readily available and in a much shorter time period. With structured search to collect content in multiple languages, the trick is to maximize relevant content while minimizing irrelevant material.

AI allows us to take this further, though it requires training to pick up further nuance. Sometimes it is hard to build complex taxonomies (search formulae) to capture what you want to see while excluding material you are trying to avoid. Applying trained AI technology will provide a stronger result.

Let’s take an example. One way to cut through the noise is to train a media-monitoring and analytics tool to recognize relevant content without the need for a strong taxonomy.

We can take this further to improve sentiment detection based on the actual content in context rather than using key words, filter nuanced and more complex concepts and identify key messaging, which may be more implied than explicit. Clearly, it is more appropriate in some situations than others and requires sufficient data to learn the repetition needed to improve accuracy.

The intention behind this engine is to train it to code content as it’s coming into the system with minimal updates and additional education. The level of complexity and nuance determines the length of time needed. Sometimes it can be a few hours; other times it can take much longer periods.

This technique requires reviewing a volume of data to teach the AI platform what is and is not relevant. It also requires ongoing attention to further develop the machine learning.

AI at Work: Quid’s AI software ingested blogs and stories to create this graphic representation of fake news mentions. Source: The author

Good Data

So the starting point must be good data. When computers were in their infancy, an IBM programmer and trainer coined the acronym GIGO – garbage in, garbage out. We need to start with good data and understand the universe of that data so that we are aware of potential limitations.

For instance, if we have not set out our parameters correctly, we may have too much erroneous data or be missing key content. If we are doing a global study and only looking at English-language material, our data will not reflect all key markets or market cultural biases, but will skew to the U.S.

If you are a food brand, you may detect a dietary trend in the U.S., but that does not mean it would be relevant in Brazil.

Key Benefits

  • AI enables us to look at large volumes of data in an intelligence way for meaningful business intelligence
  • In addition it:
  • Provides a starting point, encouraging curiosity and creativity in mining data–overlaying a level of human curation for context as well as address the “so what” and “therefore.”
  • Recognizes trends – AI is good for identifying the patterns in data while humans can dig deeper for the relevance and application. It provides a rigor and consistency, with people providing the validation, implications and interpretation
  • Identifies opportunities for content development based on past performance (of the brand and competitors)
  • Offers data points to confirm or contradict hypotheses

The Watch-Outs

  • AI is here to help but data can be dangerous without interpretation
  • Time is needed to quality check the data and refine processes, and recognize any data bias that may have been introduced in set up or from ongoing improvements
  • AI is constantly developing; it is not static and needs ongoing refinement and development

A Word on Sentiment Detection

One of the beauties of automated sentiment detection is that it is consistent. It is either consistently right or consistently wrong, however.

AI improves on key word determination of sentiment as it provides a level of contextualization, but there is still a long way to go. Within many systems the sentiment is based on the entire article or post rather than an individual brand.

When considering this metric through automation, it is best positioned as a relative measure of change rather than an absolute value. For a more precise measure, it is better to apply a qualitative approach with human coding across a robust sample of the content to determine an absolute value.

A Note on the Importance of Curation

Data curation is as important as ever, but technology certainly helps. It is an interesting observation that Tim Cooke, Apple CEO, chose to have humans select the top stories about the U.S. midterm elections in Apple News.

Similarly, Facebook, in addition to applying machine learning, has hired news credibility specialists to help determine fake news.

With time at a premium for most executives, curating a morning media brief allows them to obtain the news they need, dig deeper where relevant and avoid drowning in additional content. This allows them to focus on what is important to them for that day. Technology provides the starting point, identifying the potential content, and people with knowledge of the company, determine the selection for inclusion.

In Conclusion

AI benefits our world, and it will only get better. But it is not a silver bullet. Simply put: it is not a replacement for the human touch.

As data volumes have increased and aspects of research have become more complex, AI allows us to manage the added complexity, helps us to be smarter and more thorough analysts, and, as consultants, it enables us to focus on what is important for companies, brands, organizations and clients.

And as it changes how we spend our time as researchers, it does not remove us from the equation. I like to think of AI as Augmented Intelligence; it augments what we do, and allows us to delve deeper into quantities of data to make greater sense of it.

CONTACT: [email protected]