[Editor's Note: PRNEWS recently announced the 2023 honorees for PRNEWS People of the Year. We held a virtual roundtable with the our Data & Measurement Game-Changers: Mariorl Calderon, director of PR, CCOMGROUP INC.; Angela Dwyer, head of insights, Fullintel; and Mary Beth Levin, manager, social media strategy and analytics, USPS.
The discussion includes:
- the value of predictive metrics in media analysis and crisis mitigation
- the importance of incorporating a multitude of tools to ascertain sentiment
- resources that communicators can leverage to brush up on their measurement know-how
The conversation was edited for length and clarity.]
PRNEWS: Talk about your approach to measurement, as well as how you've seen it evolve throughout your career.
Mary Beth Levin: This is a second career for me. My background is actually in public health...A lot of what I do is informed by data and how to use data to make people's lives better.
You may be familiar with the heat map I developed [at USPS]; that was actually inspired by a program encouraging adult immunization rates. It's basically taking that worldview and saying, “how can we apply it to the world of social media to make better decisions?...”
Angela Dwyer:…Measurement is really an art and science. And the purpose of measurement is to help us show value and improve performance….Sometimes I find that brands can be a little bit too fixated on the value piece, the scorecard…and sometimes they miss out on opportunities for learning and growth. I lean a little more on the [approach of] so what; why does this matter? How can I learn something from this?
Levin: The nice thing about data is that a lot of PR is based on instinct or people's subjective experience. And you can bring in the data and say, “This is what the data says.” And so there's less tension...because the data is the data; it's not based on one person's individual experience…
Mariorl Calderon: I find that people might not understand what research means—you could read something, but how does it really affect what you're doing? What does it really mean to people?
…When I started [in PR], I was internal...I realized we were looking at things like pieces of content, impressions...We love that as PR practitioners. But when I transitioned to the agency side, I found that my numbers might not mean a lot to the marketing team. So now my approach is to look at qualitative and quantitative KPIs that show what matters to each client to demystify the value of PR…[and] how does that impact, ultimately, the consumer?...
PRNEWS: There's all this talk about PR practitioners proving their value to their clients or to their leadership. What metrics are you using to explain business outcomes?
Calderon: …When we see emerging trends, we try to break it down to get to the root of what the client really wants to understand...At the beginning of every campaign and mid-campaign, we go back to evaluate the goals we had for the program. Then we break down how we might measure that, and set up a standard…[otherwise,] even if we are measuring it, we're not going to show value to our clients, because everybody's saying something different.
Dwyer: When we talk about vanity metrics, they're just numbers; they don't really measure or inform…We've definitely seen failed attempts, things like ad value...So the idea is, how do we actually show value? The metrics that I recommend are what I would call predictive metrics; they incorporate several tested factors for potential impact.
We start with primary research, we start with a survey or action to test these different factors, and then we apply them to the work that we do…I look at factors that would lead someone to remember an article better. Or what would influence the perception of their brand.
I don't care if an article happened; I want to know the impact of that article, or the potential impact. We can't look at the impact every single time [they see] an article…so what we have to do is predict how likely this content would influence this reader...
Calderon: Does that take a lot of human workload, especially if you are trying to measure sentiment?
Dwyer: We use humans to read every article from the readers’ perspective. We can use automation; we can use AI, but there are limitations. It's not going to be as accurate as a human coding sentiment, because we understand sarcasm; we understand humor…
Levin: I approach it from a business intelligence/social listening approach. Our approach is crisis management, brand protection, a better customer experience. We're not interested in metrics for the sake of metrics. We're always asking, how can the data make the world a better place?
For us, predictive analytics are helpful when we want to see an issue that's trending that we need to get ahead of. Looking at engagement rather than impressions is helpful, because that's when people are actually taking action on it, rather than speaking to themselves and their friends…
For example, we often deal with natural disasters…we can get ahead of that and not have a surge in numbers of people asking us for help. Or we can provide feedback and change operations and reduce the number of concerns coming to us and social media by 54%. That's a significant number.
Plus, we can attach dollars; someone is answering a customer concern at $4 per case. If they're answering it by phone, it's $58 per case….our results are statistically significant. That's difficult to argue with.
We also look at online reviews…So, can we measure the level of impact, whether it is for earned media value, or a better online review? People tend to think very traditionally in terms of what's available on social, but there's tons of data available…
Dwyer: It's something that people don't always think about. They say, “these are the metrics my tool gives me, so this is what I use.” You can go much deeper.
One example is we were looking at a skincare product, and we were looking at influencers...If you looked just at impressions and engagement numbers, you still wouldn't really have all the insights. So we analyzed all the comments, and we looked for purchase intent with comments like, “I'm gonna go buy that now” or “I'm getting one for me and my mom”…
Levin: Absolutely. That's where business intelligence comes in. Traditional PR looks at what we are putting out there, what people are putting out there about us, but they're not looking at the reaction….
Dwyer: With crisis, it makes so much sense, especially when you can decide how to respond based on the data…We had a client who was about to respond in a way that would have actually hurt their reputation. But because he did an analysis…we recommended he not say something...In this particular case, we found that the way people were talking, they were already on his side, and that saying something would actually muddy the waters...
Calderon: PR is so much case-by-case; even when we try to set up some type of standard for a particular client, it has to be looked at regularly. For instance, on my side, we have a working standard that's been revised every year, but … one year is too long. Every time that we're submitting a report, we're going back to make sure that these numbers actually make sense with that report…
PRNEWS: Mariorl, Angela mentioned purchase intent from influencer campaigns, with influencers who may or may not have the most followers, for example. So how do you use that data to determine whether to use influencers, and what kind, with a particular campaign?
Calderon: It's a balance between what we would recommend and also what would meet our clients' need for that particular program…
With my client Kenvue, formerly Johnson & Johnson, [we have] what we call the Neutrogena Network; we contract [the influencer] for the year, and they post content that is not necessarily serving product. It's serving overall brand and skin health education…after each piece of content, [we] review not just the numbers that we're getting from platforms … that gives us a very turnkey, quick view of the engagement, the impressions, but we're actually going into each piece of content to measure the sentiment to this type of content.
We are measuring how people responded…compared to previous content, compared to organic content, compared to sponsored content. And that's mostly via comments. We're also pulling organic data from the influencers [including DMs]…
Dwyer: …I did a study in this industry and found that the article type that led to the most engagement… was how to or advice articles.
Think about what you might type into Google, such as how do I get rid of my wrinkles? Or how do I get glowy skin like Meghan Markle? This is the stuff that we're typing in to Google. Those are the type of articles we want to read…
Levin: I'm so glad you brought up search…[It] can tell us what the customer is looking for before they actually ask us for it...
PRNEWS: What are your clients or senior leaders expecting? How do you talk them away from impressions, for example?
Dwyer:…I think it's improving in terms of measurement culture, and what people are looking for. And it definitely comes top-down, so the organizations that I've worked with, that have the best culture are the ones who are really looking for information to guide action, not just a scorecard that might influence their bonuses. The more educated the senior leaders, the more likely to have this action-focused approach.
My basic advice is, at least for media analysis, to start with two things: volume and quality, and look at these together. Because sometimes we're looking at just volume, share of voice; then they're missing [whether it is] good quality, good coverage or bad coverage.
PRNEWS: Are there any tools or courses that people who aren't quite confident in their measurement abilities should look into?
Levin: It’s really helpful to meet with other clients who are using the same software, because they may be using it in a way that you haven't considered…
Also, professional organizations are super helpful, too, because you can compare tools. You can compare how one person is using a different tool for something that actually would benefit your business.
Dwyer: …I've had a lot of success at conferences. One of the resources I’d recommend is IPR (Institute for Public Relations)…they have really great resources that bring together measurement, research and practitioners.
My favorite one is IPRRC, which is the International Public Relations Research Conference…it's a lot of researchers...academics…and it's more sharing, versus presenting. I find those conversations are where I get the insights at that mid-career level.
Calderon: Look at different points of view… On my side, I have to report back to a marketing team. But when I think about the activation on the consumer side, I have to show the value of the brand to a consumer. And it's understanding that the way that you understand data is not the way that everybody's going to consume it, and understand that as well.
PRNEWS: Is there anything else that you think PR pros looking to expand their knowledge and expertise in this area should know?
Levin: People don't know what they don't know. There's a constant, iterative process where you're educating, explaining how your work can help other people's work be easier, be better and more productive…
Calderon: We've talked about measurement, which is something that you do after something is done. But there's a part of data that comes at the beginning. On my side, I work with minority communities, so it goes back to looking at the research for that particular market, and going back to a client or to a brand and see how they can apply it to merge with what they're doing, or what they have going on to impact that community.
Dwyer: Measurement is not something you do at the end. How can you measure if you don't know where you're starting? So...start at the beginning, and [use] research to understand and form what you're doing and know what you're measuring. Because if I don't have that research, I don't know what that key objective or goal is…