recsys 2014 tectonic shifts in television advertising including targeting and sell-side applications

Confidential & Proprietary

Tectonic Shifts in Television AdvertisingHOW SET TOP BOX DATA AND DATA MINING TECHNIQUES ARE REVOLUTIONIZING TV

Brendan Kitts – 2014

@adaptv @bkitts1 [email protected]

http://www.google.com/url?sa=i&rct=j&q=&esrc=s&source=images&cd=&cad=rja&uact=8&docid=FqeqPsYRewsAgM&tbnid=7Ry8s3BYLS0N2M:&ved=0CAUQjRw&url=http://dublincore.org/&ei=wjDPU5LRF8j5oASY-oH4CA&bvm=bv.71667212,d.cGU&psig=AFQjCNEhJKjDramX5LXgRcTbTRFqYYzikg&ust=1406173662925873


http://www.google.com/url?sa=i&rct=j&q=&esrc=s&source=images&cd=&cad=rja&uact=8&docid=v5jTA14YtM2HTM&tbnid=tJ8A3owBADQ5dM:&ved=0CAUQjRw&url=http://www.thedrum.com/news/2013/04/05/new-service-adaptv-aims-provide-tv-comparable-online-video-advertising-measurement&ei=GS3PU676Me_DiwL--IHQBA&bvm=bv.71667212,d.cGE&psig=AFQjCNHAgC1eBixJWr77QTqLzQEPC7GpaA&ust=1406172822008319


http://www.google.com/url?sa=i&rct=j&q=&esrc=s&source=images&cd=&cad=rja&uact=8&docid=3BRzm0LJS_-nuM&tbnid=0tjWwiiDkem8cM:&ved=0CAUQjRw&url=http://icons.mysitemyway.com/legacy-icon-tags/email/page/4/&ei=XzHPU7auE8z-oQSf34GoBA&bvm=bv.71667212,d.cGU&psig=AFQjCNGcyOMb1LP7LtxmnXqtIZG4KFoD0g&ust=1406173821395960




Confidential & Proprietary 2

You need to know the numbers…









Who knows?








Confidential & Proprietary 4@adaptv @bkitts1 [email protected]







Confidential & Proprietary@adaptv @bkitts1 [email protected]







Confidential & Proprietary 6@adaptv @bkitts1 [email protected]








• Drunk Nate Silver riding the subway, telling strangers the day they will die

• Drunk Nate Silver predicting the shape of every snowflake

• Drunk Nate Silver looking at strangers from across his doctor's waiting room, then worriedly scribbling in his notebook

• Drunk Nate Silver's McDonald's order comes out to exactly $5.00, after tax. The cashier looks at him in awe

• Drunk Nate Silver placing a wreath next to Schrodinger's cat box









http://www.isnatesilverawitch.com/









TV is dead

14

The Telegraph, 2007, "TV Is Dying Says Google Expert"One of the founding fathers of the internet has predicted the end of traditional television....Vint Cerf,

who helped to build the internet... said...that viewers would soon be downloading most of their favourite programmes onto their computers.









Actual Data

15

NOTICE SOMETHING?









4,000

40,000

1952 1962 1972 1982 1992 2002 2012

Radio Direct Mail Yellow Pages

Magazines Newspapers TOTAL TV

Newspaper

TV

Direct Mail

Radio

Magazines

Yellow Pages

1950 – 2000









4,000

40,000

1952 1962 1972 1982 1992 2002 2012

Radio Direct Mail Yellow Pages

Magazines Newspapers TOTAL TV

Newspapers

Magazines

Yellow pages

1950 – 2000

WHAT HAPPENED?









0%

5%

10%

15%

20%

25%

30%

1990 1995 2000 2005 2010 2015

internet users % of world population


0%

1%

10%

100%

1990 1995 2000 2005 2010 2015











0

10

20

30

40

50

60

70

80

90

100

1940 1950 1960 1970 1980 1990 2000 2010 2020

VCRs % of TVHHs

VCRs % of TVHHs

0

10

20

30

40

50

60

70

80

90

100

1940 1950 1960 1970 1980 1990 2000 2010 2020

Multiset % of TVHHs

Multiset % of TVHHs

0

10

20

30

40

50

60

70

80

90

100

Oct-2006 Feb-2008 Jul-2009 Nov-2010 Apr-2012

HD Display Capable

HD Display Capable

Source: The Nielsen Company-NTI, Jan. each yearSource: The Nielsen Company, Media-Related Universe Estimates.









0

5

10

15

20

25

30

35

40

45

Oct-2006 Feb-2008 Jul-2009 Nov-2010 Apr-2012

DVR % of TVHHs

DVR % of TVHHs

Source: The Nielsen Company, Media-Related Universe Estimates









TV viewing hours

21

0

1

2

3

4

5

6

7

8

9

1940 1950 1960 1970 1980 1990 2000 2010 2020

Hours per day per TVHH

Hours per day per TVHH

Source: The Nielsen Company, NTI Annual Averages, 1994-present estimates based on start of broadcast season September to September. Beginning in 2007, estimates include Live+7 HUT viewing. Prior to 9/87: Audimeter Sample; 9/87 to present: People Meter Sample.









TV viewing hours

22

Source: The Nielsen Company, NTI Annual Averages, 1994-present estimates based on start of broadcast season September to September. Beginning in 2007, estimates include Live+7 HUT viewing plus DVR playback.

0.00

1.00

2.00

3.00

4.00

5.00

6.00

1985 1990 1995 2000 2005 2010 2015

Men Women Teens Children









Recent trends

23

McDonough, P. (2012), The Evolution of The Video Consumer, Audience Measurement 7 presentation, Nielsen Corporation

2.9% switched “on”









Video – Mobile Device

0.00

0.50

1.00

1.50

2.00

2.50

3.00

K2-11 T12-17 A18-24 A25-34 A35-44 A45-54 A55-64 A65+

Video on Mobile

Video on Mobile

12-17 year olds

Pr(VideoOnMobile|AgeGroup=X) / Pr(VideoOnMobile)

Source: Nielsen Three Screen Report 2009 http://blog.nielsen.com/nielsenwire/wp-content/uploads/2009/09/ThreeScreenReport_US_2Q09REV.pdf


http://blog.nielsen.com/nielsenwire/wp-content/uploads/2009/09/ThreeScreenReport_US_2Q09REV.pdf








Video - Internet

0.00

0.50

1.00

1.50

2.00

2.50

K2-11 T12-17 A18-24 A25-34 A35-44 A45-54 A55-64 A65+

Video on internet

Video on internet

18-24 year olds

Pr(VideoOnInternet|AgeGroup=X) / Pr(VideoOnInternet)











Video - Time-shifted

0.00

0.20

0.40

0.60

0.80

1.00

1.20

1.40

1.60

K2-11 T12-17 A18-24 A25-34 A35-44 A45-54 A55-64 A65+

Time-shifted TV

Time-shifted TV

25-34 year olds

Pr(VideoTimeShifted|AgeGroup=X) / Pr(VideoTimeShifted)











Video - Traditional TV

0.00

0.20

0.40

0.60

0.80

1.00

1.20

1.40

1.60

K2-11 T12-17 A18-24 A25-34 A35-44 A45-54 A55-64 A65+

Traditional TV

Traditional TV

65+ year olds

Pr(VideoTraditionalTV|AgeGroup=X) / Pr(VideoTraditionalTV)

Source: Nielsen Three Screen Report 2009 http://blog.nielsen.com/nielsenwire/wp-content/uploads/2009/09/ThreeScreenReport_US_2Q09REV.pdf@adaptv @bkitts1 [email protected]









Traditional TV still dominates every age group

0.0

20.0

40.0

60.0

80.0

100.0

120.0

140.0

160.0

180.0

K2-11 T12-17 A18-24 A25-34 A35-44 A45-54 A55-64 A65+

Lift

vs

Vid

eo

on

In

tern

et

On Traditional TV*

Watching Timeshifted TV*

Using the Internet**

Watching Video on Internet**

Mobile Subscribers Watching Video on a Mobile Phone

20x – 160x video watching on traditional TV versus Internet!

Source: Nielsen Three Screen Report

2009 http://blog.nielsen.com/nielsenwire/wp-content/uploads/2009/09/ThreeScreenReport_US_2Q09REV.

pdf

100x










TV data through 2014

http://www.nielsen.com/content/dam/corporate/us/en/reports-downloads/2014%20Reports/nielsen-cross-platform-report-june-2014.pdf

http://www.marketingcharts.com/wp/wp-content/uploads/2014/07/Nielsen-Traditional-TV-Viewing-by-Age-Q12011-Q12014-Jul2014.png

http://www.marketingcharts.com/wp/wp-content/uploads/2014/07/Nielsen-Traditional-TV-Viewing-by-Age-Q12011-Q12014-Jul2014.png


Original programming

30

0

500

1000

1500

2000

2500

3000

2005 2006 2007 2008 2009 2010 2011 2012

original programs on cable

original programs on cable

Source: NPower. All original programming. Analyzed on programs, not telecasts. Nov 11 v. Nov. 06 1500 original TV Programs now on the air in 2010. This has increased from 750 in 2005.

NCC Presentation 10/20/2011 3:25PM ET

Our Perspective on Local Cable Measurement. Nick Garramone – SVP, eBusiness Operations & Research, NCC Media









Multi-screen & Multi-channel TV programming

31

WHERE YOU WANT IT WHEN YOU WANT IT









The TV landscape

32

FAR FROM BEING COWED BY NEW MEDIA, TV IS COLONIZING IT









Although the popularity of some

programs remains a scientific mystery....









Only two problems with

TV - we can’t target and

we can’t track!

Dancing with the Stars Nielsen@adaptv @bkitts1 [email protected]








TV Ad Targeting using STB DataTechnology Deep-Dive

Brendan Kitts – 2014











Set Top Boxes

70

75

80

85

90

95

1996 2001 2006 2011

% HHs w Set Top Box

Set Top Boxes - 91.5% of US HHs









STBs with return path - 30% of US HHs

Source: platform status report IAB, Platform Status Report: Interactive Television Advertising, AN INTERACTIVE TELEVISIONADVERTISING OVERVIEW, http://www.iab.net/media/file/ITV_Platform_Status_Report.pdf Source: SNL Kagan VOD & ITV Investor Report 1/2009

6%

16%

23%

30%

0%

5%

10%

15%

20%

25%

30%

35%

2009 2010 2011 2012

% HHs w True2Way

% w True2way









• Viewers

• 1000 times more viewers than

traditional panel

• Demographics

• 200 times more variables

• STB volumes:

• 1 billion events per day

Revolution now taking place

Michael Lewis (2003), Moneyball: The Art of Winning an Unfair Game









STB Data

40

TUNE EVENTS, STATE CHANGES, ON/OFF

STB Event = (DeviceID, EventID, DateTime, TimeZone)









CHANNEL SURFING

SESSIONING









Example viewing record

Net Person DateTime M Program Day P

FNEW 10274739 12/5/11 6:30 AM 2 FOX and Friends Mon 1

FNEW 10274739 12/5/11 7:00 AM 30 FOX and Friends Mon 1

OXYG 10274739 12/5/11 7:30 PM 28 The Bad Girls Club Mon 2



OXYG 10274739 12/5/11 9:30 PM 2 Bad Girls Club: Season 8 Preview Mon 2

FNEW 10274739 12/6/11 7:00 AM 29 FOX and Friends Tues 1

LIFE 10274739 12/6/11 8:30 PM 27 America's Supernanny Tues 2

LIFE 10274739 12/6/11 9:00 PM 30 America's Supernanny Tues 2

LIFE 10274739 12/6/11 11:30 PM 30 One Born Every Minute Tues 2

LIFE 10274739 12/7/11 12:00 AM 1 One Born Every Minute Wed 2

FNEW 10274739 12/7/11 7:00 AM 28 FOX and Friends Wed 1

FNEW 10274739 12/7/11 9:30 PM 25 Hannity Wed 1

FNEW 10274739 12/8/11 7:00 AM 21 FOX and Friends Thurs 1

FNEW 10274739 12/9/11 7:00 AM 29 FOX and Friends Fri 1

AFAM 10274739 12/9/11 7:30 PM 3 Santa Claus Is Comin' to Town Fri 3

DSNY 10274739 12/9/11 7:30 PM 20 Beethoven's Christmas Adventure Fri 3

DSNY 10274739 12/9/11 8:00 PM 22 Beethoven's Christmas Adventure Fri 3

AFAM 10274739 12/9/11 8:00 PM 4 The Santa Clause Fri 3

Multiple viewers for the same set top box. The demographics of this viewer are “Female”, Second gender =“Male”, “Married”, “2 children”, “Education=Grad School”. We have hand-labeled column “P” to show what could be the viewing from three different individuals – Person 1 who likes to watch Fox news in the morning, Person 2 who watches young female programming, and Person 3 who watches kids programming.









Targeting Overview: Pr(Buyer|Media)

Media heatmap

P(Buyer|Program=P ^ Station=S ^ Day=D ^ Hour=H ^ Geo=G ^ Demo1=D1 ^ Demo2=D2 ^ ...

Every buyable asset: Rotations, Station, Programs scored

Score buyer concentration in entire spectrum of TV media

Able to predict Pr(Buyer) because STB persons and Buyer persons are both identified with anonymousIDs. We can then find STB persons who bought, and measure concentration of buyer by program, rotation, station, etc









Targeting Overview: Multiple heterogeneous sources

Direct buyer vars (STB)

RPI (phone vars)HighDim vars (STB)Age-Gender-like vars

Buyer scoring more difficult than one might think: TV = Heterogeneous data sources: STB second, STB head-end, Demo panels, Phone

response, web response, etc. All need to be plugged in and used. Standardization necessary Variable selection Missing value handling (real-world requirements) Includes both high coverage variables (demographics) as well as more sparse and more

accurate variables such as BPM “Why” reporting built-in









Targeting Algorithm Classes and Capabilities

101

102

103

104

105

106

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

impressions

Corr

betw

een p

redic

tor

and R

PI

abilitec

rpi

demo

BPM historical buyers/viewers in media

High-dim Demo vector match to media

1MM imp airing (nat broadcast, prime-time)

~50,000 imp airing (natcable)

~10,000 imps and lower: Local market cable, local market broadcast, cable zones, MGM, etc

Prediction accuracy

Airing size log(imps)

RPI direct response measured in media









Targeting: Auto-Target Creation

Includes behavioral, ethnographic, demographic, psychographic elements and more

3,500+ demographic values per person, per HHDemographics Name Demographics Description Customer Pct Index Vs Avg

Luxury SUV - Most Likely to Own 02 15.0% 5.860262

Young Men's Apparel True 3.1% 3.644231

Investing True 28.1% 3.404787

Young Women's Apparel True 11.9% 1.998901

Petite Women's Apparel True 8.5% 1.983947

Infants and Toddler Apparel True 5.8% 1.922626

Occupation - Professional Cosmetologist 5.3% 1.825666

Discretionary Income Lower Discretionary Income Index (15-29) 1.6% 1.789521

Luxury Crossover Cars - Most Likely to Own 01 Most likely to own 21.0% 1.487053

Health - Cholesterol Focus True 7.3% 1.477273

Income $300-500K 1.7% 1.444033

Income $200-300K 2.4% 1.438104

Children's Apparel True 10.0% 1.406801

Luxury SUV - Most Likely to Own 01 Most likely to own 19.9% 1.31996

Home Purchase Year Home Purchased Between 2010-2014 6.6% 1.296542

Income Range Premium $175-200K 2.5% 1.255489


Financial Newsletter Subscriber True 22.3% 1.009264

Income $125-200K 17.8% 0.982331

Income Greater than $500K 1.6% 0.956361

Occupation - Professional Nurse (Registered) 13.9% 0.929024

Luxury Crossover Cars - Most Likely to Own 02 16.7% 0.918896

Hispanicity Assimilation Lowest Assimilation (86-100) 11.3% 0.888074

Discretionary Income Score Highest 1.8% 0.877732

Discretionary Income High Discretionary Income Index (150-164) 3.8% 0.864817

Discretionary IncomeAbove Highest Discretionary Income Index (195-5000 9.5% 0.861194


Jewelry True 24.2% 0.815518


Gender Female 86.3% 0.76957

Highest Vehicle Value 01 Highest 19.0% 0.760943

2nd Vehicle Year 2008 model 8.3% 0.760198

Diabetic Interest True 6.4% 0.755143

Home Market Value $750,000 - $999,999 3.3% 0.739975

Personicx Classic Urban Scramble 1.6% 0.722842

3,500 variable-

values

Auto-target creation creates set of persons, clusters, and demographic vectors. Features are pre-computed.









Inside the Black Box: Auto-Target Creation

Automated Target Creation

1

Acxiom

Advertiser Enrich Profile Clustering

Data file format:Customer, PII, Segment1..N, Value1..NJoe Bloggs, Silver, Basic, 20.0Mary Jane, Gold, FullService, 100.0

Target = Sourcekeylinked to (i) Set of AbilitecIDs (ii) and aggDemographic profile

Targets

Profiles are the % of population who have each demographic variable-value, converted into lifts over US pop. All demo variables are treated the same. Architecture is pluggable for any new demos. Eg. changes to demo set are handled elegantly since architecture just processes them

a

b c d

Auto-profiling of pass-in segments for targeting (eg. “Gold”, “Silver”, “FullService”, etc)

Auto-generation of cluster centroids and then auto-profiling (eg. “NC-1”, “NC-2”, etc)

STB AbilitecIDEnriched Demos

US Pop

e









Inside the Black Box: Scoring Service

Automated Media Scoring

12/12/2013,CNN,Tues,7pm,Piers,Seattle

CNN, Tues, 8pm = 0.5

AC360 = 0.6

Seattle, CNN, Tues, 8pm = 0.55

MAPs (Features)

Rotation to be scored

12/12/2013,CNN,Tues,8pm,AC360,Seattle

Seattle, KIRO, MtoF, 6p-10p

Airings

2

Schedule lookup retrieve program information

CurrQuart,CNN,Tues, 8pm = 0.5

12/12/2013,CNN,Tues,9pm,Erin,Seattle

Model weights (trained & hand-set based on real-world buying conditions and desired missing value failover behavior)

Heterogeneous Scoring Provider handling: MAPType-Standardization transform

Cast to target units (re-standardize)

Score0.5

Explode the rotation into constituent airings (equal rotation)

Missing Values handled elegantly - system always gives the best prediction it can

Same system used for prediction of Imps, Cost, CPM, RPI, BPI

a

b

c

d

e

f g

Pre-computed features (means or demo match scores)









Live System

49@adaptv @bkitts1 [email protected]








Sell-Side Optimizer!

50

• Station x Day x Hour x Program

• Station x Day x Hour x Program x AdvertiserMajorClass


Ph

on

e re

spo

nse

s p

er im

pre

ssio

n

Pr(Buyer)

approx 2MM in spend over 1 yearX-axis = targeting scoreY-axis = RPISize of bubble = Impressions at targeting score


TV Buyer targeting: ROI Landscape








TV Buyer targeting: Revenue per impression

0%

20%

40%

60%

80%

100%

0% 20% 40% 60% 80% 100%

% o

f re

spo

nse

s

% of Impressions

0%

20%

40%

60%

80%

100%

0% 20% 40% 60% 80% 100%

% o

f re

spo

nse

s

% of Impressions

Buyer Targeting Age-Gender TRPs

1,663 airings145,363 spend

0%

20%

40%

60%

80%

100%

0% 20% 40% 60% 80% 100%%

of

resp

on

ses

% of Impressions

0%

20%

40%

60%

80%

100%

0% 20% 40% 60% 80% 100%

% o

f re

spo

nse

s

% of Impressions

11,761 airings2,456,615 spend

Kitts, B., Au, D. and Burdick, B. (2013), A High-Dimensional Set Top Box Ad Targeting Algorithm including Experimental Comparisons to Traditional TV Algorithms, Proceedings of the Thirteenth IEEE International Conference on Data Mining, Dec 7-10, Dallas, TX.

38% lift

37% lift

Inability to discriminate








recsys 2014 tectonic shifts in television advertising including targeting and sell-side applications

Data & Analytics

proprietary drunk nate

proprietary http

tv dead

conventional tv ads

snowflake drunk nate

awe drunk nate silver

television ads

tvtraditional tv wont