digital analytics lecture4

Information Technology Program

Aalto University, 2015

Dr. Joni Salminen

joolsa@utu.fi, tel. +358 44 06 36 468

DIGITAL ANALYTICS

1

About average levels

• CTR ~5% in SEM, 0.05% in display

• CVR ~1-2%

• Bounce ~40%, if over 60% is usually a bad thing

• CPM $2.80 (Johnston, 2014)

• The numbers obviously vary by industry / firm, but

these levels are typical according to my experience.

2

Standardizing online and offline metrics:

The CPM approach

A. Events

– Participation costs x €, y people participate

– CPM = x / (y/1000)

B. Magazine catalogue

– Distribution costs x €, circulation is y

– CPM = x / (y/1000)

• Compare these to other marketing channels (e.g.

AdWords, Facebook)

3

Standardization: example

4

FESTARI CPM AJANKOHTA KÄVIJÄMÄÄRÄ (2014) VUOKRA

Iskelmä 35€ 25-27.6.2015 34 000 1 200€

Jysäri 77€ 3-4.7.2015 13 000 1 000€

SuomiPop 67€ 9-11.7.2015 15 000 1 000€

Tammerfest 12€ 17-18.7.2015 80 000 1 000€

eCMP: another way to standardize

• impressions: 1,000,000

• cost: 50€

• clicks: 20

• CTR: ??

• CPC: ??

• eCPM: ??

5

eCMP: another way to standardize

• impressions: 1,000,000

• cost: 50€

• clicks: 20

• CTR: 0.002%

• CPC: 0.25€

• eCPM: 0.05€

6

You can use metrics to calculate…more

metrics!

• The problem: our AdWords client wants to know the

estimated reach and number of visitors

• The only number we have is the client’s budget

• How the hell we gonna get those estimates?

7

You can use metrics to calculate…more

metrics! (2/2)

• To calculate estimates for an (AdWords) campaign

plan, you only need to know three figures:

– budget

– goal CTR

– goal CPC

• Out of the previous figures, you can calculate other

metrics:

– clicks = budget / cpc

– impressions = clicks / ctr

8

(An example of the previous)

budget ctr cpc clicks impressions

250 0,05 0,2 1250 25000

9

In addition to being your way out of

complexity, metrics guide the optimization

process. In fact, they become the measures

for continuous improvement by optimization

activities.

10

Choosing the metrics a priori is crucial

• If you don’t rule metrics, they rule you! (remember

analysis paralysis)

• [JONI SHOWS: Facebook Ads reporting metric-o-

mania]

11

How to choose metrics?

There are several ways:

1. platform/channel specific

2. goal specific (derived from business objectives)

3. company type (age, business logic, cf. google

classification)

4. funnel stage (cf. elämyslahjat)

5. direct vs. indirect measures (cf. proxies)

12

Selection criteria for metrics

• is it actionable?

• is it useful?

• That’s it!

13

Engagement: is this good or bad?

14

(Metric: session duration)

Ask yourself:

• is this actionable?

• is this useful?

Engagement: is this good or bad?

15

(Metric: page depth)

Ask yourself:

• is this actionable?

• is this useful?

Neither is good! (I mean, the metrics ARE

good, but I don’t like what I’m seeing in the

data (data hurts :( )

• Solution ideas:

1. automatic lead magnet, when closing the page?

2. changes to product page template, encouraging to

discover gifts

– random product

– gift search engine in the left column

• You know you’ve chosen good metrics, when they

make you think!

16

The idea of ”proxy”

• Proxy is an indirect measure of a phenomenon

• We use proxies when the real data is not available

• For example, an impression is a proxy for capturing

attention (i.e. building ”awareness”)

• (This is similar to construct validity in statistics – how

do you measure e.g. trust?)

• Be careful with proxies, some of them approach

vanity metrics!

17

The proxy problem: How would you

measure ”engagement”?

Amunwa (2014):

– Time on Site

– Avg. Pages per Session

– Pageviews

– Return visits

– Site search keywords

– Submitting a contact form

– Downloading whitepaper / e-book

– Subscribing blog

– Reading one or more key blog posts related to the

offering

18

“How would you measure user engagement

at service like Dropbox?”

”Here are some metrics that I would want to know about to measure

user-engagement for a file storage and sharing platform similar to

Dropbox:

– Unique user accounts (UU): Technically, just the number of sign-

ups is not a user-engagement metric, but this is definitely an

important success metric, and a foundation stone for future user-

engagement.

– Active users (MAU and DAU): A health UU metric is good, but what

matters most is how many of these come back frequently. Monthly

Active Users (MAU) and Daily Active Users (DAU) are two most

popular measures, but depending on your needs, you may have a

different level of granularity.

– Upload frequency: For a service like Dropbox, I would like to know

how frequently users are actually uploading files to their accounts.

An even stronger indicator can be how many users have set-up

automatic sync between their devices and the service.”

19

“How would you measure user engagement

at service like Dropbox?”

”Here are some metrics that I would want to know about to measure

user-engagement for a file storage and sharing platform similar to

Dropbox: – Content access frequency: Another good user-engagement metric is how

frequently the uploaded content is being accessed. This will include

downloads and users accessing their content directly on site and others

accessing users' shared contentthe ettteteee.

– Storage used: Another user-engagement metric specific to this case would be

how much storage is being used. If active users are using up a good share of

what is available to them, that is a good sign.

– Upgrades: With so many free storage options available, if your users decide

to pay you for more storage, they are definitely engaged!

– Referrals: If users consider your service good enough to be referred to their

friends, they definitely like your service. Even if they do this for additional free

storage, this is still a strong positive, as this indicates theirs interest in using

your service more.

– Device-mix: The mix of devices from which your service is being accessed

can tell you a lot about how they intend to use it, and how strong you can

expect the future engagement to be. I would definitely want the device mix to

have a good representation of mobile (smartphones/tablets), as this indicates

users' interest in accessing your service on the go.”

20

Engagement metrics differ by platform

• FB: likes, comments, engagement ratio

• GA: pages/visit, bounce rate, time on site, etc.

• own app: [custom]

21

Platform-specific metrics

• for example,

• Google: PageRank, Quality Score, search

impression share

• Facebook: Relevance Score, engagement ratio

– impSh = ad shown / all possibilities of showing the ad

– engRatio = users who shared, clicked, liked or

commented / all users who saw the post

22

Example of Quality Score (Google, 2010)

BEFORE QUALITY SCORE

Max. CPC CPC Position

Advertiser 1 0,4 0,3 1

Advertiser 2 0,3 0,2 2

Advertiser 3 0,2 0,1 3

Advertiser 4 0,1 - -

+QUALITY SCORE

Max. CPC QS Score Position

Advertiser 1 0,4 1 0,4 -

Advertiser 2 0,3 3 0,9 2

Advertiser 3 0,2 6 1,2 1

Advertiser 4 0,1 8 0,8 3

CLICK PRICE

Max. CPC QS Score CPC

Advertiser 1 0,4 1 0,4 -

Advertiser 2 0,3 3 0,9 (0,80/3) = 0,24

Advertiser 3 0,2 6 1,2 (0,90/6) = 0,15

Advertiser 4 0,1 8 0,8 Minimum price

23

Vickrey-style

second price

sealed auction

QS

changes

positions!

…and

prices

Relevance Score (Facebook, 2015)

• measures the potential of

the ad to succeed in a

chosen target group (1–10)

• good relevance score =

cheaper clicks and

impressions (and vice versa)

24

= if you know the systems, you gain a

competitive advantage

• however, there is a catch…

25

Optimization for platform metrics can be in

conflict with optimizing for business goals

26

Which ad is more successful?

Ad A Ad B

Quality score 10 3

CTR 10 % 3 %

Impressions 1000 1000

Clicks 100 30

Conversions 15 15

Revenue 1500 € 1500 €

Cost 500 € 150 €

Optimization for platform metrics can be in

conflict with optimizing for business goals

27

Ad A Ad B

Quality score 10 3

CTR 10 % 3 %

Impressions 1000 1000

Clicks 100 30

Conversions 15 15

Revenue 1500 € 1500 €

Cost 500 € 150 €

ROI 200 900%

The metric conflict can be seen as an issue

of local vs. global maximum

• This is a common computer science problem

– Platform-specific metrics: local maximum

– Business goals: global maximum

• It can be very very hard to achieve a global

maximum, but metrics should be chosen to support

the path towards it…

28

eCommerce metrics (Fellman, 2015)

• Visitors

• Conversion rate

• Average basket

• Margin

• Example (monthly sales):

• 100,000 x 0.02 x 100 € x 0.40 = 80,000 €

29

Visitors Conversion rate Average

basket

Margin

eCommerce metrics in GA (Promodo, 2013)

Metrics for newsletters

31

Subscribers

x delivery rate

x open rate

x click rate

x conversion rate

x avg. basket

x margin

= profit

(cf. Drake’s equation)

Metrics for newsletters: example

32

Subscribers (20,000)

x delivery rate (0.90 → 18000)

x open rate (0.25 → 4500)

x click rate (0.40 → 1800)

x conversion rate (0.03 → 54)

x avg. basket (100 € → 5400 €)

x margin (0.40)

= profit (2160 €)

(cf. Drake’s equation) …I guess you already

see the importance of

volume & frequency.

damn them

numbers

change!

Company lifecycle (Wojcik, 2013)

1. “Infant: traffic, followers, subscribers, reviews,

social media shares

2. Adolescent: number of sales, revenue,

conversion rate, time on site, customer

satifaction

3. Mature: profit, retention length, churn rate,

revenue per customer, costs of goods sold,

societal/business impact”

First to build awareness, then to make sales, and

finally to optimize. (Follows the logic of company

building.)

33

Startup metrics (startups are cool, yej!)

“There are two types of startups out there:

– ones with very low CAC (usually because they offer

the product for free, and usually having users with low

willingness to pay) and

– ones with very high CAC (usually selling enterprise

software). The best position is to have an offering with

low CAC and strong willingness to pay (translating to

high CLV).”

• Startups usually focus on metrics measuring growth

and viability, as these are their goals. Corporations

tend to be more defensive and focus on efficiency

metrics and market share.

34

AARRR: Startup metrics for pirates

(McClure, 2007)

35

Business logic also matters. Let’s see

how…

36

Sanoma & Facebook both sell ads… Should

they optimize for the same metrics?

Why/why not?

• In my opinion, they should not

• The key difference is not the revenue model, but the

wider business logic, meaning that…

– Sanoma runs on editorial content and media sales

people

– Facebook runs user-generated content and on and

real-time bidding

• For both, engagement, impressions and revenue are

important. But their strategy to achieve them is

different, and so the metrics should be too.

37

The story of Kiosked

• CPC → CPM

• I call it ”funnel transferral”: moving up in the funnel

• from performance-based to awareness-based

• this is the major reason why e.g. Google cancelled

their affiliate program: impressions still sell!

38

Funnel stage

• Assign the following metrics to their proper funnel

stage: CPC, CPA, CPM, CPL

• Awareness

• Interest

• Desire

• Action

39

Funnel stage

• Assign the following metrics to their proper funnel

stage: CPC, CPA, CPM, CPL

• Awareness – CPM

• Interest – CPC

• Desire – CPL

• Action – CPA

40

Actionable metrics vs. vanity metrics

• “An actionable metric is one that ties specific and

repeatable actions to observed results.”

– Examples: conversion rate in a direct response

campaign, number of leads generated from a lead

magnet campaign, bounce rate of campaign + landing

page in comparison to site average

• “The opposite of actionable metrics are vanity

metrics (like web hits or number of downloads)

which only serve to document the current state of

the product but offer no insight into how we got

here or what to do next.”

– Examples: Facebook fans, ad impressions, even

visitors in some cases

41

Fine, you have chosen metrics! Now what??

• …well, you make a dashboard showing them.

42

Let’s build a dashboard!

The case is ElämysLahjat.fi, an ecommerce company

selling activity gifts.

1. First, choose metrics (how do we do this? how many

we take?)

2. Then, let’s build it in Google Analytics…

43

”Ihmisten ymmärryskyky on aika limited.”

44