expense constrained bidder optimization in repeated auctions ramki gummadi stanford university...
TRANSCRIPT
Expense constrained bidder optimization in repeated auctions
Ramki Gummadi Stanford University
(Based on joint work with P. Key and A. Proutiere)
Overview
• Introduction/Motivation
• Budgeted Second Price Auctions
• A General Online Budgeting Framework
• Optimal Bids for Micro-Value Auctions
• Conclusion
Three Aspects of Sponsored Search
1. Sequential setting.
2. Micro-transactions per auction.
3. The long tail of advertisers is expense constrained.
Modeling Expense ConstraintsFixed budget over finite horizon => any balance at time is worthless.
Balance
timeT0
B
Modeling Expense ConstraintsStochastic fluctuations could cause spend rate different from target.
Balance
timeT0
B
Modeling Expense Constraints
“…the nature of what this budget limit means for the bidders themselves is somewhat of a mystery. There seems to be some risk control element to it, some purely administrative element to it, some bounded-rationality element to it, and more…”
-- “Theory research at google”, SIGACT News, 2008.
Modeling Expense ConstraintsAdd a fixed income, per unit time to the balance and relax time horizon.
Balance
time0
B
Responsibility for expense constraints Auctioneer Bidder
Bids fixed -- Auction entry throttled.
Bids adjusted dynamically.
Online bipartite matching between queries and bidders.
Online knapsack type problems.
Expense constraints = fixed budget.
Possible to model more general expense constraints.
Bid optimization
Preview
Sequential X-auction with true value v
Static X-auction with virtual value: shade* v
X can be SP, GSP, FP, etc. (any quasi linear utility)
Shade(remaining balance B) =
will be characterized explicitly.
1
1 '( )V B
Preview: Optimal Shading factors
Overview
• Introduction
• Budgeted Second Price auctions
• A General Online Budgeting Framework
• Optimal Bids for Micro-Value Auctions
• Conclusion
Model: Budgeted Second Price
• Discrete time, indexed • Balance: • Constant income per time slot - • I.I.D. environment sampled from– Private valuation (observable) – Competing bid (not observable)
• Decision variable is bid at time – Can depend on and , but not
Model: Budgeted Second Price
Constraint: a.s.
• Utility:
• Objective function:
The Value Function
• : max utility starting with balance
• Can use dynamic programming (“one step look ahead”) to write out a functional fixed point relation.
The Value Function
1 2,
( ) max E 1{ } 1{ }v pu b
v b u p T u p T
( )v p e v b a p
( )e v b a
But boundary conditions can not be inferred from the DP argument.
Currentauction
Loss
Win1T
2T
Future opportunity cost
Characterization of value function
“Effective price” for nominal at balance :
Theorem: Optimal bid is *:i.e: Buy all auctions with “effective price” is a functional fixed point to:
( , )*u b v
,
( ) ( ) ( , )v p
v b e v b a v p b
( , ) ( ( ) ( ))p b p e v b a v b a p
1
,( ) ( ) ( , )i i i
v pv b e v b a v p b
Value Iteration:
𝛽=0.1
Each auction has miniscule utility compared to overall utility:
Value Iteration:
𝛽=0.01
1
,( ) ( ) ( , )i i i
v pv b e v b a v p b
Numerical estimation when is small:• State space quantization errors propagate due
to lack of boundary value.• Need longer iterations over larger state space.
will be studied under scaling:
( ) ( ) ( / )V B v b v B
Limiting case: micro-value auctions
Overview
• Introduction
• Budgeted Second Price Auctions
• A General Online Budgeting Framework
• Optimal Bids for Micro-Value Auctions
• Conclusion
General Online Budgeting ModelDecision Maker Environment
, i.i.d
Unobservable
Observable
Balance:
Utility:
Action
Payment:Income
𝑀𝑎𝑥𝑖𝑚𝑖𝑧𝑒∑𝑡=0
∞
𝑒−𝛽 𝑡𝔼 [𝑔(𝑢 (𝑡 ) , 𝜉 (𝑡 )) ]
Ex1: Second Price Auction
(Random environment) (Observable part) is the bid (Action) (Utility function)
(Payment function)
Ex2: GSP Auction
Random environment: Observable part: is the bidUtility function:Payment function: 1
1
( , ) 1{ } ( )L
l l l ll
g u p u p v p
11
( , ) 1{ }L
l l l ll
c u p u p p
Click events for L slots
Overview
• Introduction
• Budgeted Second Price Auctions
• A General Online Budgeting Framework
• Optimal Bids for Micro-Value Auctions
• Conclusion
Limiting Regime:
( ) ( ) ( / )V B v b v B
Notation:
(( ]) [ , )E g ug u
(( ]) [ , )E c uc u
is an inverseand
is the minimum of:
Theorem
*( ), (0) ,dV
f V VdB
( )x
is the solution to:
*
*( ') , (0) ,V V V
0
( ) sup( ( ) ( ) )u
x ax g u c u x
F
* 0min ( )x x
𝑥
𝑉
𝐵
𝑉 (𝐵)
Theorem
( )x
*
) =
Application to Second Price Auctions𝐸 [𝟏𝑢>𝑝 (𝑣−𝑝 )]
p]
Second Price Auction ExampleOpponents bid p
Private Valuation
𝜙(𝑥 )
Value functions
Optimal bid
0 ( )
( )
sup( ( ) ( ) '( )) sup 1{ }( (1 '( ))
sup 1{ }1 '( )
u u v
u v
g u c u V B uE p v p V B
vu p p
VE
B
F
i.e., Static SP with shaded valuation: 1 '( )
v
V B
* at balance B solves:
Optimal Scaling factor
Optimal Bid: GSP
0
1( ) 1
sup( ( ) ( ) '( ))
sup 1{ }1 '( )
u
L
l l l lu v l
g u c u V B
vp u p p
VE
B
F
Static GSP with “virtual valuation”: 1 '( )
v
V B
Proof Overview
• Variant: Retire with payoff when .
• Value function of variant converges to ODE with initial value .
• But what is the right boundary condition ?To prove:
Because exit payoff optional Next 2 slides
Goal: Exhibit a sequence of policies parametrized by which can achieve a scaled payoff as
Lemma: For any ε > 0, there is a policy * such that ε AND
If could be played continuously, we can get arbitrarily close to ! But every now and then balance is exhausted, so we need a variant of u* that still manages to achieve nearly as much payoff
𝜂∗≤ lim inf 𝑉 𝛽(0)
time
B(t)
B
Play U*
𝜂∗≤ lim inf 𝑉 𝛽(0)
Show that fraction of time spent in green phase by the random walk gets arbitrarily close to 1 as ->0
Overview
• Introduction
• MDP for budgeted SP auctions
• A General Online Budgeting Framework
• Optimal Bids for Micro-Value Auctions
• Conclusion
Conclusion
• A two parameter model for expense constraints in online budgeting problems.
• Optimal bid can be mapped to static auction with a shaded virtual valuation.
• Paper has more contents: MFE analysis and a finite horizon model.