A Perspective for Retail MarketersBy Scott Eagle, CMO – Conversant ®

MEASURING WHAT MATTERS

INTRODUCTION

A Growing Concern

New Capabilities and New Urgency

HOW THE INDUSTRY ASSESSES DIGITALMARKETING PERFORMANCE TODAY

The Metrics Chosen to Measure

The Motivations for Measurement

The Channels Measured (and Not Measured)

Allocating Credit Versus Measuring Impacts

THE MORE ACCURATE AND SCIENTIFIC ALTERNATIVE

Test and Control Incrementality Measurement

Our Test and Control Process

CONCLUSION

3

4

10

12

Table of Contents

2

As we all know, digital spending has grown incredibly rapidly. eMarketer reports that in 2014 global digital spending will hit $137.5B – just over a quarter of total marketing spend. With its increased importance in the marketing mix, brand leaders are naturally concerned with measuring and proving its efficacy. With Digital, there is no shortage of data to evaluate as part of this process. But all that data brings with it a new challenge – identifying which metrics we should care about, and how to best measure them.

Digital ad spending worldwide, 2012-2018

INTRODUCTION

Source: eMarketer, March 2014

A GROWING CONCERNThis topic is clearly a growing concern for marketing leaders, and one that is contributing to the growth in marketing analytics spending at many brands. According to the February, 2014 CMO Survey, marketing analytics spendingis expected to rise 72% in the next three years.

Increased marketer attention is leading many brands to reconsider why, what and how they measure. Years ago, when digital marketing measurement began in earnest, measurement was seriously limited by the available technology. Now, there is growing recognition that some of our approaches to assessing marketing performance remain rooted to the limited measurement capabilities of the past.

Marketing analytics spending: share of total marketing budgets

Marketing analytics spending is expected to rise 72% in the next three years.

Source: The CMO Survey, February 2014

7.1%

12.2%

Current Levels Next 3 YearsDigital Ad Spending % of Total Media Ad Spending

$104B

20.823.2 25.3 27.0 28.4 29.8 31.1$119B

$154B

$171B$187B

$204B

$137B

2012 2013 2014 2015 2016 2017 2018

3

NEW CAPABILITIESAND NEW URGENCYBut times have changed. Our industry can now focus on scientifically validated measurement methodologies. Methodologies that reveal the causal effects of digital marketing on what most brands are really interested in – total brand sales. More than that – not only can we measure more scientifically – we must if we are to deliver

the sort of results that are expected of us now and in the future. Nearly every marketing leader is under more pressure than ever to drive additional impact with the resources we are allocated.

This paper assesses the state of digital marketing measurement today and then advocates for a scientifically derived measurement approach that addresses its current shortcomings. We’ll review:

• What’s wrong with how the industry measures performance today?

• The more accurate and scientific alternative – test and control incrementality measurement.

Not only can we measure more scientifically – we must if we are to deliver the sort of results that are expected of us.

HOW THE INDUSTRY ASSESSES DIGITAL MARKETING PERFORMANCE TODAYThe most common approaches to marketing measurement today have the following characteristics in common:

THE METRICS CHOSEN TO MEASUREMany brands substitute surrogate metrics like clicks,visits and purchase intent rather than focusing on the overarching KPI – incremental sales effects.

THE MOTIVATIONS FOR MEASUREMENTMany focus primarily on justifying past actions rather than determining the best courses of action for future success.

THE CHANNELS MEASURED(AND NOT MEASURED)When they DO focus on actual sales metrics, brandsusually measure online sales only rather than calculating sales impacts across all channels.

ALLOCATING CREDIT VERSUSMEASURING IMPACTSMost focus on crediting sales to tactics according to predetermined percentages rather than determining which sales were actually caused by each program.

Let’s examine each of these in turn.

4

Instead of focusing on true measures of performance against KPIs, many brands have traditionally chosen to focus on surrogate metrics. Clicks. Impressions. Page views. Interaction times. Why?

• Because they are easy to measure. Years ago, when more sophisticated measurement tools and approaches were unavailable, most brands decided that these easy-to-measure surrogates were the best we could do – and that they provided at least some indication of marketing effectiveness.

• Because brands have used these measures in the past. Precedent likely drives some of the decision to measure certain surrogate metrics like clicks.

• Because they lack the analytics personnel necessary to improve their measurement approaches. Again, according to the same CMO Survey, many marketers feel that they don’t have the right people in place to maximize the effectiveness of their analytics investments.

METRICS

Many brands substitute surrogate metrics like clicks, visits and purchase intent rather than focusing on the overarching KPI – incremental sales effects.

9.0%

24.5%

18.4%18.0%

15.9%

8.2%

6.1%

Does your company have the right talent to fully leverage marketing analytics?

Source: The CMO Survey, February 2014

Does not have the right talent

Has the right talent

1 2 3 4 5 6 7

5

Many brands focus their measurement and research efforts on providing rationale for past programs. In fact, a 2012 eConsultancy study done in partnership with Google revealed that the most important reason why marketers say they measure is to justify digital spending.

Goals for attribution

“We did this and the results were that.”

While such measurement has a valuable role for marketing accountability, I would argue that it plays an even more important role in helping us to optimize program performance in the future. This is a major distinction. When the core objective of a performance measurement system is to demonstrate the value of past performance, accuracy can become subordinate to the desire for big numbers.

When our primary focus is instead on program optimization, the need for accurate measurement becomes paramount.

When the core objective of a performance measurement system is to demonstrate the value of past performance, accuracy can become subordinate to the desire forbig numbers.

JUSTIFYING DIGITAL SPENDING

UNDERSTANDING FUNNELAND SALES CYCLE LENGTH

TO PLAN CAMPAIGNS

CREATE MOST EFFECTIVE MEDIA MIX BASED ON TRUE VALUE

TOWARD CONVERSION

CORRECTLY DETERMINING AFFILIATE PAYMENTS

High Priority Medium Priority Low Priority

57% 36% 7%

47% 42% 11%

62% 28% 10%

36% 39% 25%

Source: Google/eConsultancy, 2012

0% 20% 40% 60% 80% 100%

MOTIVATIONS

Many focus primarily on justifying past actions rather than determining the best courses of action for future success.

6

The industry’s emphasis on justifying past actions versus improving future results is evident when we examine the attribution approaches currently in use. According to a 2013 survey of senior retail marketing leaders underwritten by Conversant, the most popular approach to measurement is last click.

Does your brand use the following approach to digital marketing attribution?

These survey results are disconcerting in light of the fact that marketers view last click as the least accurate method. See this data from the eConsultancy/Google study:

Marketers’ assessment of the effectiveness of attribution models

There are positive signs of change, however. Another Conversant industry survey of both agency- and brand-side professionals revealed that just under a third planned to adopt more accurate methodologies.

As more brands change their approaches, I sincerely hope that they choose a methodology that is truly better – one that doesn’t rely on a rules-based or arbitrary allocation of credit to particular marketing events. Many of the attribution platforms currently getting attention in the marketplace simply use a more complex formula for arbitrarily allocating credit. And greater complexity is no guarantee of greater accuracy.

Percent of marketers/agency leaders who plan to change measurement approach in 2014

MOTIVATIONS (CONT.)

68% 680= LAST CLICK

17% 170= FIRST CLICK

43% 430= LAST VIEW

47% 470= MULTI-TOUCH

38% 380= INCREMENTALITY

Source: Conversant Survey, 2013

(Numbers add up to greater than 100% because many companies use multiple attribution methods)

Source: Conversant Survey, 2014

Yes No Don’t Know

Very Effective

Somewhat Effective

Somewhat Ineffective

Very Ineffective

Source: Google/eConsultancy, 2012

CUSTOMIZED BY CHANNEL

UNIQUE METHODOLOGY

LINEAR

FIRST CLICK

LAST CLICK

0% 20% 40% 60% 80% 100%

43%26%

32%

29% 49% 517%

27% 64% 9%

23% 69% 8%

14% 61% 25%

14% 13%18%55%

7

Most retail brands are only using online sales impact to evaluate and measure digital marketing effectiveness. Again, the primary reason why is that it is far easier to calculate the online sales that correlate with a marketing program than the offline sales.

Measuring online sales impact only is fine for marketing programs designed to drive only online sales, like most affiliate marketing. But for omni-channel marketing strategies and programs, more comprehensive performance measurement is vital.

In most categories, offline sales dwarf online sales. So, by limiting their measurement pool, brands artificially devalue the total impact of their digital marketing programs. Because most methodologies currently in use focus only on online results, total economic impact is often significantly understated.

Most retail brands are using only online sales impact to evaluate and measure digital marketing effectiveness

34% 680= ELECTRONICS & APPLIANCES

22% 440= ENTERTAINMENT & LEISURE

14% 280= APPAREL & FOOTWEAR

9% 180= CONSUMER HEALTHCARE

8% 160= BEAUTY (MASS & LUXURY)

4% 80= PERSONAL CARE

3% 60= HOME & DIY

1% 20= GROCERY

US eCommerce sales, by category:(2012, % of total)

Source: Booz & Company, 2012

CHANNELS

When they do focus on actual sales metrics, brands usually measure online sales only rather than calculating sales impacts across all channels.

8

Many marketing programs allocate credit for all sales to persons reached by marketing programs running at the time of the sale, even when the marketing itself was superfluous to their decisions to buy. The problem is that most online sales that are credited to a marketing program would have occurred anyway, without the marketing activity.

Most? Really? Yes, really! Conversant conducted a massive study across 25 leading retailers in multiple classes of trade and found that 67 to 90 percent of sales that were credited to marketing exposure based upon cookie-centric attribution methodologies were actually NOT caused by that exposure. In other words, the shoppers that converted received the marketing program message, but would’ve made the conversion even if they hadn’t received the marketing message.

Many people use attribution and measurement as interchangeable terms, but they aren’t the same. Measurement and analytics should be about taking scientific approaches to determining the sales impacts of a program. Attribution is focused on crediting a sale (or a portion of a sale) based upon a model.

Cumulative Effects

So one might ask, if standard measurement under-credits in some ways but over-credits in others, does it “all come out in the wash?” In other words, does our flawed methodology actually deliver a reasonable approximation of digital’s total economic impact?

Absolutely not. Again, extensive Conversant research demonstrates that the flawed measurement approaches in use by many retailers do not provide a reasonable approximation of the true sales effects of marketing programs. In addition, these flawed approaches often don’t even reveal a correct ranking of the value marketing tactics bring to a given business.

Most online sales that are credited to a marketing program would have occurred anyway, without the marketing activity.

IMPACTS

Many retailers focus on crediting sales to tactics according to predetermined percentages rather than determining which sales were actually caused by each.

9

Earlier in this paper we identified some major flaws in the most commonly used measurement methodologies. I’d like to use the balance of this paper to outline an alternative approach that mitigates all of these problems.

Test and Control Incrementality Measurement

At Conversant, we believe in “incrementality” measurement and use a scientifically validated test and control methodology to provide precision performance measurement.

The hallmark of this approach is an emphasis on “incrementality.” In other words, what incremental revenue did a program actually cause? This means we don’t take credit for sales that would have happened if a shopper wasn’t exposed to a particular campaign, or take credit for sales effects caused by tactics other than Conversant’s programs.

To suss out incrementality, we conduct constant, scientifically based A/B testing, using statistically matched samples. One group sees the marketing messages in the Conversant program, one sees public service announcements (PSAs) instead. By calculating the sales made to each group, and then subtracting the PSA “control” cell sales from the “test” cell sales, we get a precise measure of the sales that were actually caused by the program. Furthermore, we track revenue across both online and offline sales channels to ensure we get an accurate view of the program’s total impact. By assessing incremental sales impacts across all channels including online, retail, catalog, call center, etc., we can provide a true indicator of a program’s performance.

Track revenue across both online and offline sales channels to ensure you get an accurate view of total impact.

THE MORE ACCURATE AND SCIENTIFIC ALTERNATIVE

Conversant uses a scientifically validated test and control methodology to provide precision performance measurement.

10

To do all this, we follow a four-step process:

1 Step 1: Create test and control groups. In order to identify the incremental sales driven by a campaign, we need to devise test and

control groups. These groups are carefully developed using a scientifically validated method that matches both groups across a host of variables, including interests, shopping behaviors, demographics, marketing exposures and more.

2 Step 2: Show the test group the campaign. Show the control group PSAs. Conversant shows the test group creative from your

campaign, while the control group sees only public service announcements. Each group is exposed to similar amounts of other marketing communications. The only difference in their brand experience is whether or not they are exposed to the Conversant campaign.

3 Step 3: Examine the online and offline sales to each group. The Conversant data platform doesn’t track only your website

sales. It integrates with your POS systems, as well as your catalog and call center sales systems, to calculate total revenue for both online and offline.

4 Step 4: Calculate the true incremental sales. Once we have the sales figures for each group, we calculate incremental sales by

subtracting the control group sales from the test group sales. Because revenue from the control group reflects sales that would have occurred naturally, this calculation reveals the incremental sales that were truly driven by your campaign. We also have the ability to use the test/control methodology to measure key brand attributes and the positive or negative impacts of a given campaign. It’s critical for marketers to understand these effects as well.

Scientifically based incrementality measurement not only measures the past performance of marketing tactics. It also provides the information necessary to optimize current and future programs – thereby driving incremental sales growthover time.

OUR PROCESS

Scientifically based incrementality measurement provides the information necessary to optimize current and future programs.

Our approach to incrementality measurement

1: CREATE

2: EXPOSE

3: MEASURE

4: CALCULATE

Test

Test

Test

Control

Control

Control

PSA

Incremental Sales

11

CONCLUSION

Chances are that your brand can and should make significant improvements in the way that it measures marketing effectiveness. Most brands know that this is something they need to do. But for many, fixing the problem tends to fall low on the priorities list – because the task can seem daunting – and the current situation doesn’t seem “that bad.”

Make no mistake. It may well be “that bad.” Your role, and that of the marketing organization, depend on providing the best results possible. Flawed measurement leads to bad decision-making. Some may believe that instinct combined with existing flawed measurement data can drive great results. The reality is that flawed measurement is of little or no value in decision making. The sooner that you correct the problem, the faster you’ll be able to deliver great results for your brands.

I welcome your comments and ideas on this topic. Please write to me at: ScottEagle@conversantmedia.com

ABOUT THE AUTHOR

Scott Eagle leads Conversant’s global marketing function, including strategy and the integration of marketing programs across our solutions groups. An accomplished senior executive with a strong background in client-side digital marketing and consumer brand management, Mr. Eagle has over 25 years of experience as a marketing leader managing major Fortune 500 brands and building successful new companies. Mr. Eagle has served as CMO for Empowered Careers, eHarmony and Claria Corporation, and he has held management positions at Concentric Network Corporation, MFS Communications and P&G. Mr. Eagle holds a B.S. in economics from The Wharton School, University of Pennsylvania, and serves on the board of Akademos, Inc.

ABOUT CONVERSANT, INC.

Conversant, Inc. (Nasdaq: CNVR) is the leader in personalized digital marketing. Conversant helps the world’s biggest companies grow by creating personalized experiences that deliver higher returns for brands and greater satisfaction for people. We offer a fully integrated personalization platform, personalized media programs and the world’s largest affiliate marketing network - all fueled by a deep understanding of what motivates people to engage, connect and buy.

For more information, please visit: www.conversantmedia.com

Copyright 2014 Conversant, Inc. All Rights Reserved.12