We are on the precipice of a watershed moment in measurement, and the tipping point is integration - and not in the traditional sense of breaking down silos between marketing and PR. That is part of it, true, but the future of measurement hinges in integrating - or merging - multiple measurement factors to develop the most comprehensive metric. Yes, you read it right - that's 'metric,' not 'metrics,' and one major corporation is defending this with the argument that one is the loneliest number no more. The Power Of One "Counting is easy. Evaluating is much harder," says Chris Frank, senior director of market research and insights for Microsoft, identifying the age-old conundrum of how to make measurement work for the greater organizational good. "How do we take the core of what matters and bring that together with our business needs, and then figure out a metric that really has impact?" Surely it's a question that most communications professionals have asked, but few have gone so far as to devise a tangible solution. Frank and his market research team at Microsoft have, but only after being approached by the corporation's PR department with a challenge: take the company's current measurement standards and develop a "consistent, global measurement system to assess the effectiveness and impact of PR." "It's all about having a common language and nomenclature internally," Frank says. "There are so many factors that go into having successful PR. He points to an "influence model" to identify the basis of the model: trust, from highly influentials, influentials and target buyers/users. "But," he says, "to get people's heads around that, we wanted to come up with a single metric for comparing scores. The whole goal of a single number is having impact - to compel someone to act." A Different Kind Of Merger Mergers may be a hot topic in the context of blending two organizations into one, but Frank's Microsoft example shifts the conversation to merging buzz and advocacy for getting the most relevant data. "Buzz is the quantity/volume of coverage," he says. "Advocacy is the opinion embedded in the buzz. What do my consumers advocate for me to do? Building that into the model is critical." So was integrating outputs and outcomes. "An effective buzz and advocacy measurement system measures the outputs and outcomes," Frank says. But a balanced combination of advocacy and buzz, and outputs and outcomes, means nothing without a framework and a methodology. That's where Cymfony enters Microsoft's story, and where yet another merger takes place - that of automated and manual measurement solutions. Facing Challenges The case of Microsoft and Cymfony's quest for a single measurement metric serves as a valuable roadmap for organizations looking for measurement solutions of their own. Frank acknowledges the inherent challenges - cost, namely. One solution to minimizing costs that many departments have turned to is outsourcing the clips and impressions overseas to be read and analyzed at a lower cost. However, warns Cymfony President and CEO Andrew Bernstein, "Outsourcing alleviates some of the cost issues, but that introduces quality control issues. When outsourcing, you must have native speakers. It's also important to combine automated and human analysis, because machines can't always pick up [on nuances]. Another challenge is developing the methodology in the first place, especially when you are looking to home in on one metric. Microsoft's measurement model tackles some of these roadblocks. One Step At A Time Bernstein took an integrated approach when taking on Microsoft's challenge of developing a measurement methodology that would result in a single metric. "We believe in a hybrid approach," he says. "If you want to have focus on outcomes, you have to believe they are going to be dynamic and change." To anticipate this dynamism, Cymfony took the following approach: End-to-end quality process: "Extensive and iterative research allowed us to automate the identification of certain themes/topics." Methodology creation: Extensive research on the client side was necessary to understand which were key concepts and how to define them. Messages were then evaluated on a 4-point scale. Leverage key in-house resources in PR research: "We needed to bring in someone who could oversee the project but also be dynamic in understanding the methodology." Then, the following process was used to whittle data down to a single metric: Step 0: Predefine the Message: Take the message and put it through a "funnel" - product, topic, theme, etc. Step 1: Assess Buzz Levels:Measure and count across top sources to track buzz and advocacy. Step 2: Evaluate Advocacy: Categorize articles on tone, message pickup, action taken. Step 3: Score PR's Effectiveness: Apply weighting algorithm: PR score = # of impressions * influence of publication (between 0.0 and 1.0) * score on advocacy dimension (between -1 and 1) All this adds up to a single number that can then be attached to a given effort. But many are skeptical that a single metric dilutes the complexity of measurement, but the Microsoft team emphasizes that the depth of the data still exists under the surface. "If something doesn't look right on the surface, there is still all of the rich data underneath that we can drill down into to see what went wrong," says Jim Nail, chief marketing and strategy officer of Cymfony. "That's also key in shaping future strategies." One Byte At A Time The singular metric is an intriguing concept, to say the least, but many might argue that our current communications environment is too convoluted for such a "simple" summary of what's going on - for example, how does social media fit into the equation? "Traditional and social media are merging," Bernstein says. "Are New York Times reporter blogs traditional or social media?" Probably traditional, he argues, because these blogs are controlled. But the challenge of integrating digital factors into measurement efforts is still a futuristic nightmare for many execs. "The real measure of social media is engagement," says Katie Paine, CEO of KDPaine & Partners. Frank and Bernstein admit that their measurement effort is still in the early ages (it was just rolled out last week) and hasn't overcome all barriers, including those imposed by social media, but initial results are promising. "We wanted to make sure our PR scoring system contributed to bottomline success, influenced corporate reputation and played an increased role in digital marketing," Frank says. "Right now, our connection to corporate reputation is a long-term goal. We're going to get there. We're just not there today." (To register for the October 30 measurement Webinar, visit http://www.prnewsonline.com.) CONTACTS: Andrew Bernstein, firstname.lastname@example.org; Katie Paine, email@example.com; Jim Nail, firstname.lastname@example.org Tapping Market Research For many communications teams, forging partnerships with other internal departments is strategic and cost-effective, especially when taking on initiatives that require massive amounts of data. "We have two clients," says Chris Frank, senior director of market research and insights for Microsoft. "One is our PR team, and one is the partnerships with our agencies." When the PR team wanted a new measurement solution, they tapped the market research department, who then acted as an intermediary between them and Cymfony. Frank identifies the following reasons why any organization looking at measurement solution - is wise to turn to market researchers: It develops new standards from an outside prespective; Market researchers have a rigorous quantitative background, helping them understand the complexities of data analysis; and, It's a neutral third-party.
The Power of 1: Does Measurement’s Future Hinge on a Single Metric?
You might also be interested in: