data days 2014 - volkmar uhlig

CLICK PREDICTIONS FOR 500M MOBILE DEVICES

© 2008-2014

ADVERTISING IS A DATA BUSINESS

Refinement of a commodity: pixel real-estate

Value creation: delivering advertiser message to right client in context

KPI: quality and price

Telekom Shop Bonn-CityFriedensplatz 453111 Bonn

0.5km

© 2008-2014

ADVERTISING IS A BIG DATA BUSINESS

>1B unique devices

~40B impressions per day traded through RTB WW

40TB of new data per day

~500k impressions per second

Latency: 100ms

© 2008-2014

GEOGRAPHIC DISTRIBUTION

50msRTT

50msRTT50ms

RTT50msRTT

© 2008-2014

TECH CHALLENGE: SCALE AND COST

Prediction latency: 3ms

Volume: 500k impressions per second

= 300 predictions per core

= 1,500 cores = 150-200 servers for bidding

Cloud Cost:$200,000 per month

© 2008-2014

PREDICTION

Multiple concurrent predictive models

Likelihood to act (click, land, browse, buy)

Demographic groups

Sociographic groups

Interest groups

Fraud detection

Site classification

100’s of TB input data sets for training

Runs for many hours a day on even more machines

Some refreshed hourly

© 2008-2014

DATA ENRICHMENT

Multiple enrichments per bid request

User

Location

Device

Weather

10’s of Billion records with <1ms access latency

Data store with >1M IOPs/second

© 2008-2014

PREDICTIVE MODEL

Predictive model has substantially cost impactEvery impression individually priced based on KPI estimation (e.g., click)

Basic filtering is insufficient

Continuous model update in multiple time scales

1. Model refinement in real time

2. Full retraining at longer time scales (hour, day, week)

© 2008-2014

REAL-TIME BIG-DATA BUSINESS

Technology and process

Blue print invented by financial world

Layer-cutting integration

Human touch points with distributed ownership

Scales when done right

Real-Time Decisioning

Data Acquisition

Storage

Batch Analytics

Model Testing

Time Series and Cubes

Dashboard and

Reports

10© 2008-2014

THANK YOU

Dr. Volkmar Uhlig

CTO Adello

volkmar@adello.com