feedback control of real-time bidding advertisingsharif.edu/~nobakhti/mimolab...

Feedback Control of Real-Time

Bidding Advertising

Fatemeh Gheshlaghpour

Advisors: Maryam Babazadeh Amin Nobakhti

Sharif University of Technology, EE Department Apr. 2018

Overview• Introduction • Business Model of the RTB Markets

– Key Roles in RTB Market – The Business Process of RTB Ad Delivery

• Second-Price Sealed-Bid Auction • Key Research Issues in the RTB • IPinYou Dataset

– Basic Statistics – User Feedback

• RTB Feedback Control System – Bidding Strategy – Logistic Estimator – Actuator

• Control Issues of the Problem – Reference and Feedback Signals – Model

• Results

2/31

Introduction

• In the previous generations of the display advertising the advertiser paid in two different platforms: – CPM (cost per mille) -> do not getting satisfactory numbers of

clicks. – CPC (cost per click) -> being subjected to click frauds.

• Internet users, on the other hand, faced with lots of irrelevant advertisements on their screens -> Ad-Blockers !

3/31

• The Solution was the RTB in which the advertisers should bid for each impression based on some behavioral and contextual data.

• The business process, including audience identification, auction and ad display, will be finished in exactly 10 to 100 milliseconds, and hence it is named "real-time bidding".

• A basic problem for RTB bidding agents is to figure out how much to bid for an incoming bid request.

Introduction

4/31

Business Model of the RTB Markets

• The key roles in RTB markets ⁻ Advertiser ⁻ Demand side platform (DSP) is a platform that helps

advertisers optimize their strategies. ⁻ Ad exchange (AdX) is an ad exchange market that matches

the buyers and sellers. ⁻ Supply side platform (SSP) is a platform that helps

publishers optimize the strategies. ⁻ Data management platform (DMP) is a platform that

analyzes the cookie data of Internet users. ⁻ Publisher

5/31

Business Model of the RTB Markets

The business process of RTB ad delivery.*

*Yuan, Y., Wang, F., Li, J., & Qin, R. (2014). A survey on real time bidding advertising. Proceedings of the IEEE International Conference on Service Operations and Logistics, and Informatics (SOLI), Qingdao, China (pp. 418-423).

• The Business Process of RTB Ad Delivery Myerson has proved that its optimal mechanism is second-price sealed-bid auction.

6/31

Key Research Issues in the RTB

• Inventory Pricing and Channel Allocation • In RTB markets, publishers and SSPs constitute the supply side

of ad resources. Their key decisions, such as inventory pricing and multi-channel allocation of ad impressions, are major research topics in literatures.

• Business Model and Mechanism Design • Similarly working like the stock markets, AdX can bridge the

gap in RTB markets by matching advertisers to publishers via real-time auctions. The existing works are focused on the design of the business model and auction mechanisms of AdXs and DSPs.

7/31

Key Research Issues in the RTB

• Market Segmentation and Ad Performance Analysis • Via designing the audience classification category and attribute

labels, DSPs can divide the Internet users into large amounts of niche markets with different kinds of demographic characteristics or shopping interests, and display best-matched ads accordingly.

• Bidding Behavior Analysis and Strategy Optimization

• In RTB markets, advertisers and DSPs constitute the demand side of ad resources, seeking to buy best-matched ad impressions via real-time auctioning and bidding. In literatures, bidding behavior analysis and strategy optimization for advertisers and DSPs attract intensive interests.

8/31

Second-Price Sealed-Bid Auction

9/31

• Nash Equilibrium:

• The only NE which does not need to know other players’ private values is to bid truthfully, i.e. to bid same as the private value.

Second-Price Sealed-Bid Auction

10/31

IPinYou Dataset • Fortunately, a leading Chinese advertising technology company

iPinYou decided to release the dataset used in its global RTB algorithm competition in 2013.

• The dataset includes logs of ad auctions, bids, impressions, clicks, and final conversions. These logs reflect the market environment as well as form a complete path of users’ responses from advertisers’ perspective.

11/31

*Zhang, W., Yuan, S., Wang, J., & Shen X. (2014). Real-time bidding benchmarking with ipinyou dataset. arXiv:1407.7073.

The iPinYou data format.*

12/31

Basic Statistics

13/31

User Feedback• Some statistics of user feedback on campaigns 1458 and 3358 are

shown bellow.

14/31

User Feedback

15/31

User Feedback

16/31

The traditional bidding strategy is represented as the bid calculator module in the DSP bidding agent. The controller plays as a role which adjusts the bid price from the bid calculator.

Feedback controller integrated in the RTB system.*

RTB Feedback Control System

*Zhang, W., Yuan, S., & Wang, J. (2014). Optimal real-time bidding for display advertising. Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining (KDD), New York City, NY, USA (pp. 1077-1086).

17/31

•  

Bidding Strategy

00

( ) tb t b θ

θ=

18/31

Logistic Estimator•Logistic estimator predicts the CTR (a real value between 0 and 1) of an ad given a set of features.

•Where fi(ad) is the value of the ith feature for the ad, and wi is the learned weight for that feature.

19/31

• In our experiment, all the features for LR are binary.

• The weekday and hour feature are extracted from timestamps.

• The floor price is processed by buckets of 0, [1,10], [11,50], [51,100] and [101,+∞).

• We do not include the features of Bid ID, Log Type, iPinYou ID, URL, Anonymous URL ID, Bidding Price, Paying Price, Key Page URL.

• In sum, we have 937,748 binary features for LR training and prediction.

Logistic Estimator

20/31

Logistic Estimator

0

150

300

450

600

1458 2259 2261 2821 2997 3358 3386 3427 3479

#ectr > 0.5 #ectr < 0.5

Validation of LR for impressions with click=1

0

175000

350000

525000

700000

1458 2259 2261 2821 2997 3358 3386 3427 3479

#ectr > 0.5 #ectr < 0.5

Validation of LR for impressions with click=0

21/31

•  

Actuator

For example it can be chose to use:

For instance in a PID controller we have:

22/31

Control Issues of the Problem

• Same as any other control problems we should take care of these:

• Reference Signal -> What is our controlling goal?

• Feedback Signal -> Do we have access to this signal?

• Model -> Can we have a static/dynamic model of the process?

23/31

Reference and Feedback Signals • The advertisers are provided with the data of the impressions. So

we can define a feedback signal.

• Reference signal is the output of an optimization problem.

• In order to have smooth budget delivery we should set another constraint, too.

24/31

Reference and Feedback Signals

Illustration of different budget pacing schemes with respect to the portion of the budget spent every time interval.*

*Lee, Kuang-Chih, Ali Jalali, and Ali Dasdan. "Real time bid optimization with smooth budget delivery in online advertising." Proceedings of the Seventh International Workshop on Data Mining for Online Advertising. ACM, 2013. 25/31

• Finally, we should optimize the following goal:

Reference and Feedback Signals

26/31

Model• As the time-constant of the dynamic model is not short enough we

can not have it using the data.

27/31

Results• Here is the AWR signal for the campaign 2998 using impression-based PI controllers.

28/31

• Bellow you can see the overall control performance on AWR for the campaign 2998 using some different methods.

Results

Method ref #Ad #clk total_cost AWR CPC CPM CTR rise-time settling-time rmse-ss

awr_rand_pi 0.6 6055 9 200341 0.5896 20034.1 33086.8 0.0015 4 6 0.00196

control_awr_pi 0.6 6167 13 352707 0.5875 25193.3 57192.6 0.0021 252 2016 0.02587

awr_pi_constant 0.6 6053 7 209847 0.5943 26230.8 34668.2 0.0011 4 6 0.00203

awr_pi 0.6 6052 10 226075 0.5999 20552.2 37355.4 0.0017 4 6 0.00199

static_bid_awr_K 0.6 6191 5 142775 0.6136 23795.8 23061.7 0.0008 15 930 0.03102

Performance of different AWR controllers

29/31

• Bellow you can see the overall control performance on CPC for the campaign 2998 using some different methods.

Results

Performance of different CPC controllers

Method ref #Ad #clk total_cost AWR CPC CPM CTR rise-time settling-time rmse-ss

control_ecpc_pi 10000 1940 4 54510 0.1939 10902 28097.9 0.00206 5250 5250 0.06144

cpc_constant_pi 10000 153 0 10057 0.0152 10057 65732.1 0 134 134 0.00556

cpc_rand_pi 10000 153 0 10057 0.0153 10057 65732.1 0 134 134 0.00556

30/31

Thank you for your time…

31/31