Vijay V. Vazirani

Georgia Tech

A Postmortem of the Last Decade

and

Some Directions for the Future

Although this may seem a paradox,

all exact science is dominated by

the idea of approximation.

Bertrand Russell (1872-1970)

Exact algorithms have been studied

intensively for over four decades,

and yet basic insights are still being obtained.

Since polynomial time solvability is the exception

rather than the rule, it is only reasonable

to expect the theory of approximation algorithms

to grow considerably over the years.

Beyond the list …

Unique Games Conjecture

Simpler proof of PCP Theorem

Online algorithms for AdWords problem

Beyond the list …

Unique Games Conjecture

Simpler proof of PCP Theorem

Online algorithms for AdWords problem

Integrality gaps vs approximability

Raghevendra, 2008: Assuming UGC,

for every constrained satisfaction problem:

Can achieve approximation factor

= integrality gap of “standard SDP”

NP-hard to approximate better.

Future Directions

Status of UGC

Raghavendra-type results for LP-relaxations

Randomized dual growth in

primal-dual algorithms

Approximability: sharp thresholds

For a natural problem:

Can achieve approximation factor in P.

If we can achieve in P

=> complexity-theoretic disaster

α(n)

α(n) − ∈(n)

Conjecture

There is a natural problem

having sharp thresholds

w.r.t. time classes

α1(n) > α 2 (n) > ... > α k (n)

P=T1(n) ⊂ T2 (n) ⊂ ... Tk(n)

Group Steiner Tree Problem

Chekuri & Pal, 2005:

Halperin & Krauthgamer, 2003:

log2−∈n factor algorithm in time

2^ (2^ ( log nO(∈)))

time = 2^(2^( log no(∈)))⇒ subexponential algorithm for 3SAT

What lies at the core of

approximation algorithms?

What lies at the core of

approximation algorithms?

Combinatorial optimization!

Combinatorial optimization

Central problems have LP-relaxations

that always have integer optimal solutions!

ILP: Integral LP

Combinatorial optimization

Central problems have LP-relaxations

that always have integer optimal solutions!

ILP: Integral LP

i.e., it “behaves” like an IP!

Massive accident!

Cornerstone problems in P

Matching (general graph) Network flow Shortest paths Minimum spanning tree Minimum branching

Is combinatorial optimizationrelevant today?

Why design combinatorial algorithms,

especially today that LP-solvers are so fast?

Combinatorial algorithms

Very rich theory

Gave field of algorithms some of its formative

and fundamental notions, e.g. P

Preferable in applications, since efficient

and malleable.

Helped spawn off algorithmic areas,

e.g., approximation algorithms and

parallel algorithms.

Problemsadmitting ILPs

Combinatorial optimization studied:

Problems admittingLP-relaxations with

bounded integrality gaps

Approximation algorithms studied:

Problems admittingLP-relaxations with

bounded integrality gaps

Problemsadmitting ILPs

Rational convex program

A nonlinear convex program that

always has a rational solution (if feasible),

using polynomially many bits,

if all parameters are rational.

Rational convex program

Always has a rational solution (if feasible)

using polynomially many bits,

if all parameters are rational.

i.e., it “behaves” like an LP!

Rational convex program

Always has a rational solution (if feasible)

using polynomially many bits,

if all parameters are rational.

i.e., it “behaves” like an LP!

Do they exist??

KKT optimality conditions

−∇ f0 (x) = yi fii

∑ '(x) + z jj

∑ a j

yi ≥ 0 for 1≤ i ≤ m

yi > 0 ⇒ fi (x) = 0 for 1≤ i ≤ m

fi (x) ≤ 0 for 1≤ i ≤ m

a jT x ≤ b j for 1≤ j ≤ p

Possible RCPs

Pick fi 's linear, and

f0 quadratic or logarithmic.

Quadratic RCPs

fo (x) = xT P x+ qT x

convexity requires:

∇2 f0 f =0, i.e., P f = 0

Two opportunities for RCPs:

Program A: Combinatorial, polynomial time

(strongly poly.) algorithm

Program B: Polynomial time (strongly poly.)

algorithm, given LP-oracle.

Helgason, Kennington & Lall, 1980Single constraint

Minoux, 1984Minimum quadratic cost flow

Frank & Karzanov, 1992Closest point from origin to bipartite perfect

matching polytope.

Karzanov & McCormick, 1997Any totally unimodular matrix.

Combinatorial Algorithms

Ben-Tal & Nemirovski, 1999

Polyhedral approximation of second-order cone

Main technique: Solves any quadratic RCP

in polynomial time, given an LP-oracle.

Ben-Tal & Nemirovski, 1999

Polyhedral approximation of second-order cone

Main technique: Solves any quadratic RCP

in polynomial time, given an LP-oracle.

Strongly polynomial algorithm?

Logarithmic RCPs

f0 (x) = − mii∑ log(ui (x))

where mi > 0 and ui (x) is linear in x.

Logarithmic RCPs

Rationality is the exception to the rule,

and needs to be established piece-meal.

f0 (x) = − mii∑ log(ui (x))

where mi > 0 and ui (x) is linear in x.

Logarithmic RCPs

Optimal solutions to such RCPs capture

equilibria for various market models!

f0 (x) = − mii∑ log(ui (x))

where mi > 0 and ui (x) is linear in x.

Arrow-Debreu Theorem, 1954

Celebrated theorem in Mathematical Economics

Established existence of market equilibrium under very general conditions using a deep theorem from topology - Kakutani fixed point theorem.

Arrow-Debreu Theorem, 1954

Celebrated theorem in Mathematical Economics

Established existence of market equilibrium under very general conditions using a theorem from topology - Kakutani fixed point theorem.

Highly non-constructive!

Arrow-Debreu Theorem, 1954

Celebrated theorem in Mathematical Economics

Established existence of market equilibrium under very general conditions using a theorem from topology - Kakutani fixed point theorem.

Continuous, quasiconcave,

satisfying non-satiation.

Complexity-theoretic question

For “reasonable” utility fns.,

can market equilibrium be computed in P?

If not, what is its complexity?

Short summary

So far, all markets

whose equilibria can be computed efficiently

admit convex or quasiconvex programs,

many of which are RCPs!

Combinatorial Algorithm for Linear Case of Fisher’s Model

Devanur, Papadimitriou, Saberi & V., 2002

By extending primal-dual paradigm to setting of convex programs & KKT conditions

Eisenberg-Gale Program, 1959

max log

. .

:

: 1

: 0

i ii

i ij ijj

iji

ij

m v

s t

i v

j

ij

u xx

x

Eisenberg-Gale Program, 1959

max log

. .

:

: 1

: 0

i ii

i ij ijj

iji

ij

m v

s t

i v

j

ij

u xx

x

prices pj

KKT conditions

1). ∀j : pj ≥0

2). ∀j : pj > 0 ⇒ xij =1i∑

3). ∀i, j :uij

pj

≤vi

m(i)

4). ∀i, j : xij > 0 ⇒uij

pj

=vi

m(i)=

uijxijj∑m(i)

Proof of rationality

Guess positive allocation variables (say k).

Substitute 1/pj by a new variable.

LP with (k + g) equations and

non-negativity constraint for each variable.

Auction for Google’s TV ads

N. Nisan et. al, 2009:

Used market equilibrium based approach.

Combinatorial algorithms for linear case

provided “inspiration”.

Long-standing open problem

Complexity of finding an equilibrium for

Fisher and Arrow-Debreu models under

separable, piecewise-linear, concave utilities?

utility

Piecewise linear, concave

amount of j

Additively separable over goods

Long-standing open problem

Complexity of finding an equilibrium for

Fisher and Arrow-Debreu models under

separable, piecewise-linear, concave utilities?

Equilibrium is rational!

Markets with separable, plc utilitiesare PPAD-complete

Chen, Dai, Du, Teng, 2009

Chen & Teng, 2009

V. & Yannakakis, 2009

Markets with separable, plc utilitiesare PPAD-complete

Chen, Dai, Du, Teng, 2009

Chen & Teng, 2009

V. & Yannakakis, 2009

(Building on combinatorial insights from DPSV)

Theorem (V., 2002): Generalized linear Fisher market to Spending constraint utilities. Polynomial time algorithm for computing equilibrium.

Is there a convex program for this model?

“We believe the answer to this question should be ‘yes’. In our experience, non-trivial polynomial time algorithms for problems are rare and happen for a good reason – a deep mathematical structure intimately connected to the problem.”

EG convex program = Devanur’s program

Price disc. Market

Goel & V.

Spending constraint marketV., 2005

Nash BargainingV., 2008

Eisenberg-Gale MarketsJain & V., 2007

EG[2] MarketsChakrabarty, Devanur & V.

2008

V., 2010: Assuming perfect price

discrimination, can handle:

Continuously differentiable, quasiconcave

(non-separable) utilities, satisfying non-satiation.

V., 2010:

Continuously differentiable, quasiconcave

(non-separable) utilities, satisfying non-satiation.

Compare with Arrow-Debreu utilities!!

continuous, quasiconcave, satisfying non-satiation.

A new development

Orlin, 2009: Strongly polynomial algorithm

for Fisher’s linear case, using scaling.

Open: For rest

Are there other classes of RCPs?

Sturmfels & Uhler, 2009:

S f =0 n×n, sample covariance matrix

G=([n],E) chordal graph

Then the following is an RCP:min log det Σ

s.t. Σij =Sij ∀(i, j)∈E or i = j

EG convex program = Devanur’s program

Price disc. Market

Goel & V.

Spending constraint marketV., 2005

Nash BargainingV., 2008

Eisenberg-Gale MarketsJain & V., 2007

EG[2] MarketsChakrabarty, Devanur & V.

2008

Building on

Karzanov & McCormick, 1997:

Combinatorial algorithm for min cost flow

under concave cost functions on edges.

EG convex program = Devanur’s program

Price disc. Market

Goel & V.

Spending constraint marketV., 2005

Nash BargainingV., 2008

Eisenberg-Gale MarketsJain & V., 2007

EG[2] MarketsChakrabarty, Devanur & V.

2008

Rational (combinatorial)approximations

to convex programs

Problemsadmitting RCPs

EG convex program = Devanur’s program

Price disc. Market

Goel & V.

Spending constraint marketV., 2005

Nash BargainingV., 2008

Eisenberg-Gale MarketsJain & V., 2007

EG[2] MarketsChakrabarty, Devanur & V.

2008

Rational (combinatorial) approximations

to convex programs

Problemsadmitting RCPs