dma mac presentation: kajal mukhopadhyay, ph.d

19
Big Data, Small Models: The Age of Fast Analytics March 9, 2015 Carol A. Wolowic Senior Manager, Media Panera Bread Adam Benaroya Director, Digital Insights Mindshare Kajal Mukhopadhyay, Ph.D. VP, Performance & Measurement Xaxis

Upload: kajal-mukhopadhyay-phd

Post on 11-Apr-2017

280 views

Category:

Data & Analytics


4 download

TRANSCRIPT

Page 1: DMA MAC Presentation: Kajal Mukhopadhyay, Ph.D

Big Data, Small Models:The Age of Fast Analytics

March 9, 2015

Carol A. WolowicSenior Manager, MediaPanera Bread

Adam BenaroyaDirector, Digital InsightsMindshare

Kajal Mukhopadhyay, Ph.D.VP, Performance & MeasurementXaxis

Page 2: DMA MAC Presentation: Kajal Mukhopadhyay, Ph.D

Defining Big Data

BIG

DATA

A broad term for data sets so large or complex that they are difficult to process using traditional data processing applications. Challenges include analysis, capture, curation, search, sharing, storage, transfer, visualization, and information privacy

The term big data has largely come to refer simply to the use of predictive analytics or other certain advanced methods to extract value from data, without any required magnitude thereon

-- WIKIPEDIA

VOLUME VARIETY VELOCITY VERACITY

3 Vs or 4 Vs

Page 3: DMA MAC Presentation: Kajal Mukhopadhyay, Ph.D
Page 4: DMA MAC Presentation: Kajal Mukhopadhyay, Ph.D

Small Models: A Few of Everything?

Fewer Goals:Well defined singular KPI (Goals) – key performance indicator/measurement

Fewer Variables:Small set of variables and parameters

Fast Computation:Distributed, Additive, Modular

Forward Looking:Short history, short-term forecast, next action

No fancy acronyms – 3 or 4 F’s

Page 5: DMA MAC Presentation: Kajal Mukhopadhyay, Ph.D

Small model estimation

and fast prediction

Page 6: DMA MAC Presentation: Kajal Mukhopadhyay, Ph.D

Distributional simplification

#(𝑥𝑖∈𝑋𝑖)

𝑁𝑖→ 𝑝𝑖 : convergence in proportion1

𝑥𝑖∈𝑋𝑖𝑥𝑖

𝑁𝑖→ 𝜇𝑖 : convergences in mean2

𝐿𝐿 ≤ 𝑥𝑖 ≤ 𝑈𝐿 : confidence and tests3

𝑥1(𝜇1)

𝑥2(𝜇2)

𝑥3(𝜇3)

𝑥4(𝜇4)

𝑥5(𝜇5)

𝑥6(𝜇6)

Page 7: DMA MAC Presentation: Kajal Mukhopadhyay, Ph.D

Correlation and Confounding Bias Correction

UNIVARIATE RANDOM DESIGNOVER A SINGLE VARIABLE

MANY SMALL EXPERIMENTSOVER A FEW LARGE BETS1

PRE-POST CORRECTIONOVER ANY TEMPORAL SPACE

FAST ADJUSTMENTOVER HISTORICAL BASELINE3

UNIVERSAL CONTROLOVER ENTRIE DATA

COLLECTION PROCESS

RAPID ITERATIONSOVER BIG CAMPAIGNS2

(𝜇𝑡𝑒𝑠𝑡+𝛿𝑏𝑖𝑎𝑠) − (𝜇𝑏𝑎𝑠𝑒+𝛿𝑏𝑖𝑎𝑠) = 𝜇𝑡𝑒𝑠𝑡-𝜇𝑏𝑎𝑠𝑒 lift

Page 8: DMA MAC Presentation: Kajal Mukhopadhyay, Ph.D

Large number of events, audience characteristics and segment populations

𝑃 Action Audience ∈ 𝑋𝑖 =𝑃(𝐴𝑢𝑑𝑖𝑒𝑛𝑐𝑒 ∈ 𝑋𝑖𝑇𝑎𝑘𝑖𝑛𝑔 𝐴𝑐𝑡𝑖𝑜𝑛)

𝑃(𝐴𝑢𝑑𝑖𝑒𝑛𝑐𝑒 ∈ 𝑋𝑖)1

Most Likely

Audience

Index Score =𝑃(Action 1|Audience ∈ 𝑋𝑖)

𝑃(Action 2|Audience ∈ 𝑋𝑖)2Most Likely Action

Odd ratio =𝑃(Action|Audience ∈ 𝑋𝑖)

1 − 𝑃(Action|Audience ∈ 𝑋𝑖)3Event

Chance

Odd Score =𝑃(Action 1 |Audience ∈𝑋𝑖)

1−𝑃(Action 1 |Audience ∈𝑋𝑖)/

𝑃(Action 2 |Audience ∈𝑋𝑖)1−𝑃(Action 2 |Audience ∈𝑋𝑖)4

Event Relevance

Ratio

Page 9: DMA MAC Presentation: Kajal Mukhopadhyay, Ph.D

Correlated vs. Casual Relation

Recommendation table based on conditional probability

Y= 𝜶 + 𝜷𝟏𝑿𝟏 + 𝜷𝟐𝑿2 + 𝜺Regression Framework:

Y

X1 X2

• Causal relationship models• Strong and weak linkages

between variables• Bayesian Network

Page 10: DMA MAC Presentation: Kajal Mukhopadhyay, Ph.D

Fast Prediction

Audience Classifications based on Bayes principle

Audience behaviors based on most likely behavior

Audience actions based most likely behaviorA

ud

ien

ce T

arge

tin

g

Predictive behavior based on actions

Predictive behavior on audience segments

Predictive behaviors based on media trigger P

red

icti

ve B

ehav

ior

Page 11: DMA MAC Presentation: Kajal Mukhopadhyay, Ph.D

ACTION

How do you take action on what, when and where?

Adam BenaroyaDirector, Digital InsightsMindshare

Page 12: DMA MAC Presentation: Kajal Mukhopadhyay, Ph.D

Client Challenge: Local Planning

Client has a significant investment in local digital advertising, but could not predict either efficiency or scale of local markets.

Business Questions - Efficiency• What factors appear to influence local

advertising efficiency?• Can we identify attributes which may

help project the performance of expansion markets for the future?

• Can we determine which new markets best fit this profile and thus are more likely to perform efficiently?

• Can we identify markets which are more likely to be high risk options?

Business Questions - Scale• What is the right amount for each

market to invest in local?• What can we learn from markets

which under-delivered on their budgets?

• Can we find a better way to estimate budgets for each market?

Page 13: DMA MAC Presentation: Kajal Mukhopadhyay, Ph.D

Solution: Cluster Analysis – answering ‘Efficiency’3 explanatory variables were the primary contributors to ‘cost-per-action’. Markets were clustered into 4 groups to project potential campaign efficiency.

Page 14: DMA MAC Presentation: Kajal Mukhopadhyay, Ph.D

Solution: CHAID Analysis – answering ‘Scale’2 explanatory variables were the primary contributors to budget scale. Recommendations for individual local market spend could be determined by the two variables.

Page 15: DMA MAC Presentation: Kajal Mukhopadhyay, Ph.D

Client Challenge: Determining causation between digital touchpoints and conversion

Business Questions • How do paid digital media

channels interact together to drive a final conversion?

• Which digital actions are strongest drivers of final conversions?

• How should I choose a proxy KPI for a sales campaign?

Site Action

A

Conversion

Site Action

B

Video Display

HIGH

Sample Size of Data

LOW

Page 16: DMA MAC Presentation: Kajal Mukhopadhyay, Ph.D

Solution: Bayesian Network to prioritize KPIs

Site Action

A

Conversion

Site Action

B

Video

HIGH

Sample Size of Data

LOW

Contribution: 35%

Contribution: 5%

Display

• Bayesian network built to map the relationship between all trackable digital touchpoints.

• When sample size limitations prevent optimization on a sales conversion KPI, the Bayesian network can also indicate which ‘proxy’ KPIs to use earlier in the campaign flight

Page 17: DMA MAC Presentation: Kajal Mukhopadhyay, Ph.D

ACTION

How do you take action on what, when, and where?

Carol A. WolowicSenior Manager, MediaPanera Bread

Page 18: DMA MAC Presentation: Kajal Mukhopadhyay, Ph.D

Realities of Big Data

Volume

Variety

Velocity

Veracity

• How is Panera using the data from the Data Management Platforms?

• How does Panera use “fast data” to react to market findings in a timely manner?

Leveraging Audience

Intelligence

Using Big Data to address critical, quantifiable goalsB

ig D

ata

Inte

llige

nce

&

Cam

pai

gn P

erfo

rman

ce

Page 19: DMA MAC Presentation: Kajal Mukhopadhyay, Ph.D

THANK YOU

Question and Answers