Measurement Week: How to Evaluate VR and AR

Virtual reality (VR) and augmented reality (AR) are considered innovations mainly because their applications, although known, haven’t yet made it into the mainstream in a significant way.

AR and VR are intended to enhance user experience. AR adds layers of information. VR extends the experience through cognitive, visual, auditive and sometimes even olfactory immersion.

Examples of AR include a Disney application from 2015 that provides 3D renditions of coloring book characters. The U.S. Army’s TAR (Tactical Augmented Reality) offers soldiers additional data points to improve their situational awareness. CMO assembled a list of other impressive AR applications.

The Digital Marketing Institute has a good list of VR examples, including the New York Times’ distribution of Google cardboard glasses to view films, The North Face’s offer to take a virtual hike around Yosemite National park and Volvo’s virtual test drive.

Measurement and Cost

Fun as they might be, these applications require extra resources. So perhaps the most pressing question is how can communicators who recommend these technologies evaluate their effectiveness?

The answer includes two questions and a link to reputation.

A risk with any new technology that generates user data is becoming obsessed with measuring things that are of little value to your business.  The central measurement questions to ask when getting ready to evaluate a technology are:

  • What does success look like?
  • How does this technology help achieve success?

In general terms, strategic communication is concerned with an organization’s long-term reputation. Reputation has seven dimensions, according to the Rep Track model. They are: products & services, innovation, workplace, governance, citizenship, leadership and performance. These will be affected by 7 behaviors: purchase, advocate for, accept, defend, work for and invest.

These behaviors, in AMEC-speak are outcomes. To inspire and influence behaviors over the long term, communicators must think of their activity incrementally. This is why communications objectives usually feature a mix of knowledge, attitude and behavior.

So, when one embarks on an AR/VR adoption journey, the question of what success looks like needs to be a little more nuanced. You must consider the dimensions of reputation you want to influence.

Perceptions of Innovation to Influence Behavior

Let’s look at the New York Times’ VR initiative, which had the traditional newspaper distributing Google cardboard goggles so subscribers could enjoy an enhanced experience from Times' stories. There are two reputation dimensions at play: innovation and products and services.

The important KPIs for the Times are not how many goggles were distributed or the number of times a particular VR video was played. Instead the Times should look at how many subscribers it's converted into VR enthusiasts. The focus should be on net promoter/net advocate scores; surveying Times' subscribers and asking them if they'd recommend the Times VR experience to friends, for example. Those results should be compared over time.

By playing the innovation card, the Times also aims in the longer term to influence perceptions of it as a progressive and innovative company and place to work. Again, evaluate this through surveys and compare it over time.

So, what do you measure when using AR and VR? The outcome (the behavior) and the points leading to it, with more emphasis on attitude (likelihood to consider, to recommend) than on knowledge and awareness.

Ana Adi is professor of PR/Corporate Communications at Quadriga University of Applied Sciences in Berlin. She also is a member of the IPR Commission on Measurement and Evaluation

Follow her: @ana_adi