We are on the precipice of a watershed moment in measurement, and the tipping point is integration - and not in the traditional sense of breaking down silos between marketing
and PR. That is part of it, true, but the future of measurement hinges in integrating - or merging - multiple measurement factors to develop the most comprehensive metric. Yes,
you read it right - that's 'metric,' not 'metrics,' and one major corporation is defending this with the argument that one is the loneliest number no more.
The Power Of One
"Counting is easy. Evaluating is much harder," says Chris Frank, senior director of market research and insights for Microsoft, identifying the age-old conundrum of how to
make measurement work for the greater organizational good. "How do we take the core of what matters and bring that together with our business needs, and then figure out a metric
that really has impact?"
Surely it's a question that most communications professionals have asked, but few have gone so far as to devise a tangible solution. Frank and his market research team at
Microsoft have, but only after being approached by the corporation's PR department with a challenge: take the company's current measurement standards and develop a "consistent,
global measurement system to assess the effectiveness and impact of PR."
"It's all about having a common language and nomenclature internally," Frank says. "There are so many factors that go into having successful PR. He points to an "influence
model" to identify the basis of the model: trust, from highly influentials, influentials and target buyers/users. "But," he says, "to get people's heads around that, we wanted to
come up with a single metric for comparing scores. The whole goal of a single number is having impact - to compel someone to act."
A Different Kind Of Merger
Mergers may be a hot topic in the context of blending two organizations into one, but Frank's Microsoft example shifts the conversation to merging buzz and advocacy for getting
the most relevant data.
"Buzz is the quantity/volume of coverage," he says. "Advocacy is the opinion embedded in the buzz. What do my consumers advocate for me to do? Building that into the model
is critical."
So was integrating outputs and outcomes. "An effective buzz and advocacy measurement system measures the outputs and outcomes," Frank says. But a balanced combination of
advocacy and buzz, and outputs and outcomes, means nothing without a framework and a methodology. That's where Cymfony enters Microsoft's story, and where yet another merger
takes place - that of automated and manual measurement solutions.
Facing Challenges
The case of Microsoft and Cymfony's quest for a single measurement metric serves as a valuable roadmap for organizations looking for measurement solutions of their own. Frank
acknowledges the inherent challenges - cost, namely. One solution to minimizing costs that many departments have turned to is outsourcing the clips and impressions overseas to be
read and analyzed at a lower cost. However, warns Cymfony President and CEO Andrew Bernstein, "Outsourcing alleviates some of the cost issues, but that introduces quality control
issues. When outsourcing, you must have native speakers. It's also important to combine automated and human analysis, because machines can't always pick up [on nuances].
Another challenge is developing the methodology in the first place, especially when you are looking to home in on one metric. Microsoft's measurement model tackles some of
these roadblocks.
One Step At A Time
Bernstein took an integrated approach when taking on Microsoft's challenge of developing a measurement methodology that would result in a single metric. "We believe in a hybrid
approach," he says. "If you want to have focus on outcomes, you have to believe they are going to be dynamic and change." To anticipate this dynamism, Cymfony took the following
approach:
-
End-to-end quality process: "Extensive and iterative research allowed us to automate the identification of certain themes/topics."
-
Methodology creation: Extensive research on the client side was necessary to understand which were key concepts and how to define them. Messages were then evaluated on a
4-point scale.
-
Leverage key in-house resources in PR research: "We needed to bring in someone who could oversee the project but also be dynamic in understanding the
methodology."
Then, the following process was used to whittle data down to a single metric:
- Step 0: Predefine the Message: Take the message and put it through a "funnel" - product, topic, theme, etc.
- Step 1: Assess Buzz Levels:Measure and count across top sources to track buzz and advocacy.
- Step 2: Evaluate Advocacy:
Categorize articles on tone, message pickup, action taken.
- Step 3: Score PR's Effectiveness: Apply weighting algorithm: PR score = # of impressions * influence of publication (between 0.0 and 1.0) * score on advocacy dimension
(between -1 and 1)
All this adds up to a single number that can then be attached to a given effort. But many are skeptical that a single metric dilutes the complexity of measurement, but the
Microsoft team emphasizes that the depth of the data still exists under the surface.
"If something doesn't look right on the surface, there is still all of the rich data underneath that we can drill down into to see what went wrong," says Jim Nail, chief
marketing and strategy officer of Cymfony. "That's also key in shaping future strategies."
One Byte At A Time
The singular metric is an intriguing concept, to say the least, but many might argue that our current communications environment is too convoluted for such a "simple" summary
of what's going on - for example, how does social media fit into the equation?
"Traditional and social media are merging," Bernstein says. "Are New York Times reporter blogs traditional or social media?" Probably traditional, he argues, because these
blogs are controlled. But the challenge of integrating digital factors into measurement efforts is still a futuristic nightmare for many execs.
"The real measure of social media is engagement," says Katie Paine, CEO of KDPaine & Partners. Frank and Bernstein admit that their measurement effort is still in the early
ages (it was just rolled out last week) and hasn't overcome all barriers, including those imposed by social media, but initial results are promising.
"We wanted to make sure our PR scoring system contributed to bottomline success, influenced corporate reputation and played an increased role in digital marketing," Frank says.
"Right now, our connection to corporate reputation is a long-term goal. We're going to get there. We're just not there today."
(To register for the October 30 measurement Webinar, visit http://www.prnewsonline.com.)
CONTACTS:
Andrew Bernstein, [email protected]; Katie Paine, [email protected]; Jim Nail, [email protected]
Tapping Market Research
For many communications teams, forging partnerships with other internal departments is strategic and cost-effective, especially when taking on initiatives that require massive
amounts of data.
"We have two clients," says Chris Frank, senior director of market research and insights for Microsoft. "One is our PR team, and one is the partnerships with our agencies."
When the PR team wanted a new measurement solution, they tapped the market research department, who then acted as an intermediary between them and Cymfony. Frank identifies
the following reasons why any organization looking at measurement solution - is wise to turn to market researchers:
-
It develops new standards from an outside prespective;
-
Market researchers have a rigorous quantitative background, helping them understand the complexities of data analysis; and,
-
It's a neutral third-party.