subexponential algorithms for unique games and related problems barriers ii workshop, princeton,...
TRANSCRIPT
Subexponential Algorithms for Unique Games and Related Problems
Barriers II Workshop, Princeton, August 2010
David SteurerMSR New England
Sanjeev Arora Princeton University & CCI
Boaz BarakPrinceton & MSR New England
U
Introduction
Small-Set Expansion
Unique Games
UNIQUE GAMESInput: list of constraints of form xi – xj = cij mod k
Goal: satisfy as many constraints as possible
Input: UNIQUE GAMES instance with k << log n (say)
Goal: Distinguish two cases
YES: more than 1 - ² of constraints satisfiableNO: less than ² of constraints satisfiable
Khot’s Unique Games Conjecture (UGC)For every ² > 0, the following is NP-hard:
UG(²)
Implications of UGCFor many basic optimization problems, it is NP-hard to beat current algorithms (based on simple LP or SDP relaxations)
Examples:
VERTEX COVER [Khot Regev’03], MAX CUT [KhotKindlerMosselO’Donnell’04,
MosselO’DonnellOleszkiewicz’05],any MAX CSP [Raghavendra’08], …
Implications of UGCFor many basic optimization problems, it is NP-hard to beat current algorithms (based on simple LP or SDP relaxations)
Examples:
VERTEX COVER [Khot Regev’03], MAX CUT [KhotKindlerMosselO’Donnell’04,
MosselO’DonnellOleszkiewicz’05],any MAX CSP [Raghavendra’08], …
Unique Games Barrier
Example: (®GW + ²)-approximation for MAX CUTat least as hard as UG(²’)
UNIQUE GAMES is common barrier for improving current algorithms of
many basic problems
Reductions show that beating current algorithms for these problems is harder than UNIQUE GAMES
®GW = 0.878…Goemans–Williamson
bound for Max Cut
Implications of UGCFor many basic optimization problems, it is NP-hard to beat current algorithms (based on simple LP or SDP relaxations)Examples:
VERTEX COVER [KR’03], MAX CUT [KKMO,’04 MOO’05],any MAX CSP [Raghavendra’08], …
Consequences for UGC (*)
Analog of UGC with subconstant ² (say ² = 1/log log n) is false(contrast: subconstant hardness for LABEL COVER [Moshkovitz-Raz’08])
Subexponential Algorithm for Unique Games
In particular: UG(²3) has exp(n²)-time algorithm
Given a UNIQUE GAMES instance with alphabet size ksuch that 1 - ² of constraints are satisfiable,can satisfy 1 - √²/¯
3 of constraints in time exp(k n¯)
NP-hardness reduction for UG(²) requires blow-up npoly(1/²)
rules out certain classes of reductions for proving UGC
(*) assuming 3 SAT does not have subexponential algorithms
poly(n) exp(n)
Concrete Complexity Landscape
2-SAT
MAX 3-SAT(7/8)MAX CUT (®GW)
* assuming Exponential Time Hypothesis [Impagliazzo-Paturi-Zane’01]( 3-SAT has no exp(o(n)) algorithm )
3-SAT (*)
FactoringGraph Isomorphism
exp(n1/2)exp(n1/3)exp(n²)
UG(²3)MAX 3-SAT(7/8+²)LABEL COVER(²)
[Moshkovitz-Raz’08+ Håstad’97]
If UGC true, UNIQUE GAMES is first CSP with intermediate complexity
MAX CUT(®GW + ²)?
UGC-based hardness does not rule out subexponential algorithms, Possibility: exp(n²)-time algorithm for MAX CUT(®GW + ²)
UNIQUE GAMES much easier than LABEL COVER
Implications of UNIQUE GAMES algorithm (*)
Introduction
Small-Set Expansion
Unique Games
d-regular graph Gd
vertex set S
Graph Expansion
expansion(S) = # edges leaving S
d |S|
volume(S) = |S||V|
d-regular graph Gd
vertex set S
Graph Expansion
expansion(S) = # edges leaving S
d |S|
volume(S) = |S||V|
S
expansion(S) = # edges leaving S
d |S|
Graph Expansion
volume(S) = |S||V|
SMALL-SET EXPANSION
Goal: find S with volume(S) < ± so as tominimize expansion(S)
Input: d-regular graph G, parameter ± > 0
Important concept in many contexts:derandomization, network routing, coding theory,Markov chains, differential geometry, group theory
close connection to UNIQUE GAMES [Raghavendra-S.’10]
S
expansion(S) = # edges leaving S
d |S|
Graph Expansion
volume(S) = |S||V|
Important concept in many contexts:derandomization, network routing, coding theory,Markov chains, differential geometry, group theory
Subexponential Algorithm for SMALL-SET EXPANSIONIf there exists S with volume(S) < ± and expansion(S) < ²,we can find S’ with volume(S’) < 2± and expansion(S’) < √²/¯
in time exp(n¯/±)
Subexponential Algorithm for SMALL-SET EXPANSIONIf there exists S with volume(S) < ± and expansion(S) < ²,we can find S’ with volume(S’) < 2± and expansion(S’) < √²/¯
in time exp(n¯/±)
1.few large eigenvalues 2.many large eigenvalues
Distinguish two cases:
large eigenvalues¸i > 1 - ´
Eigenvalues of the random walk matrix G:1 = ¸1 ¸ … ¸ ¸m ¸ ¸m+1 ¸ … ¸ ¸n ¸ -1
(pseudorandom graph) (structured graph?)
Subexponential Algorithm for SMALL-SET EXPANSIONIf there exists S with volume(S) < ± and expansion(S) < ²,we can find S’ with volume(S’) < 2± and expansion(S’) < √²/¯
in time exp(n¯/±)
´ À ²(best: ´ = 100²,
simpler: ´ = ²0.75 )
1 - ´
Eigenvalues of the random walk matrix G:1 = ¸1 ¸ … ¸ ¸m ¸
Subexponential Algorithm for SMALL-SET EXPANSIONIf there exists S with volume(S) < ± and expansion(S) < ²,we can find S’ with volume(S’) < 2± and expansion(S’) < √²/¯
in time exp(n¯/±)
¸ ¸m+1 ¸ … ¸ ¸n ¸ -1
large eigenvalues¸i > 1 - ´
Case 1: Few large eigenvalues: (inspired by [Kolla–Tulsiani ’07] and [Kolla’10])
1 = ¸1 ¸ … ¸ ¸m > 1 - ´ ¸ ¸m+1 ¸ … ¸ ¸n ¸ -1
Expander Mixing Lemma: indicator vector of S livesalmost completely in span of top m eigenvectors
Eigenvalues of the random walk matrix G:
can find set S’ close to S in time exp(m)
Enumerate this space in time exp(m)
Case 2: Many large eigenvalues (m > n¯/±)
can find small non-expanding set S’ around that vertexWill show: 9 vertex whose neighborhoods grow very slowly
Subexponential Algorithm for SMALL-SET EXPANSIONIf there exists S with volume(S) < ± and expansion(S) < ²,we can find S’ with volume(S’) < 2± and expansion(S’) < √²/¯
in time exp(n¯/±)
Case 1: Few large eigenvalues:
1 = ¸1 ¸ … ¸ ¸m > 1 - ´ ¸ ¸m+1 ¸ … ¸ ¸n ¸ -1Eigenvalues of the random walk matrix G:
Case 1: Few large eigenvalues
Subspace enumeration (inspired by [Kolla–Tulsiani ’07] and [Kolla’10])
Eigenvalues of the random walk matrix G:
Suffices to show: indicator vector of S is ²/´-close to U
|S ¢ S’| < ²/´ |S [ S’|
For every set S with expansion(S) < ², can find S’ that is ²/´-close to S in time exp(m)
Algorithm: Let U be span of top-m eigenvectorsFor every vector u in ²-net of unit ball of U,
output all level sets of u
U
1 = ¸1 ¸ … ¸ ¸m > 1 - ´ ¸ ¸m+1 ¸ … ¸ ¸n ¸ -1
Case 1: Few large eigenvalues
Subspace enumeration (inspired by [Kolla–Tulsiani ’07] and [Kolla’10])
Eigenvalues of the random walk matrix G:
Suffices to show: indicator vector of S is ²/´-close to U
(generalization of “easy direction” of Cheeger’s inequality)
hx, G xi > 1 - ² because expansion(S) < ²
x = normalized indicator vector of S
1 = ¸1 ¸ … ¸ ¸m > 1 - ´ ¸ ¸m+1 ¸ … ¸ ¸n ¸ -1
For every set S with expansion(S) < ², can find S’ that is ²/´-close to S in time exp(m)
Case 1: Few large eigenvalues
Subspace enumeration (inspired by [Kolla–Tulsiani ’07] and [Kolla’10])
Eigenvalues of the random walk matrix G:
(generalization of “easy direction” of Cheeger’s inequality)
hx, G xi = hu, G ui + hw, G wi < hu,ui + (1 - ´)hw,wi = 1 - ´hw,wi
Suppose x = u + w for u in U and w orthogonal to U
hx, G xi > 1 - ² because expansion(S) < ²
x = normalized indicator vector of S
1 = ¸1 ¸ … ¸ ¸m > 1 - ´ ¸ ¸m+1 ¸ … ¸ ¸n ¸ -1
Suffices to show: indicator vector of S is ²/´-close to U
For every set S with expansion(S) < ², can find S’ that is ²/´-close to S in time exp(m)
Eigenvalues of the random walk matrix G:1 = ¸1 ¸ … ¸ ¸m > 1 - ´ ¸ ¸m+1 ¸ … ¸ ¸n ¸ -1
Case 2: Many large eigenvalues (m > n¯/±)
Eigenvalues of the random walk matrix G:1 = ¸1 ¸ … ¸ ¸m > 1 - ´ ¸ ¸m+1 ¸ … ¸ ¸n ¸ -1
If m > 1, then 9 S with volume(S) < ½ and expansion(S) < √´and we can find S in poly(n)-time.
Compare: “hard direction” of Cheeger’s inequality
Case 2: Many large eigenvalues (m > n¯/±)
1 = ¸1 ¸ …………………… ¸ ¸m > 1 - ´ ¸ ¸m+1 ¸ … ¸ ¸n ¸ -1Eigenvalues of the random walk matrix G:
If m > n¯/±, then 9 S with volume(S) < ± and expansion(S) < √´/¯
and we can find S in poly(n)-time
Number of large eigenvalues vs. small-set expansion
“higher eigenvalue Cheeger bound”
Heuristic: balls tend to be the least expanding sets in graphs
How can we find small non-expanding sets?
Case 2: Many large eigenvalues (m > n¯/±)
1 = ¸1 ¸ …………………… ¸ ¸m > 1 - ´ ¸ ¸m+1 ¸ … ¸ ¸n ¸ -1Eigenvalues of the random walk matrix G:
If m > n¯/±, then 9 S with volume(S) < ± and expansion(S) < √´/¯
and we can find S in poly(n)-time
Number of large eigenvalues vs. small-set expansion
Suffices to show:9 vertex i such that volume( Ball(i, t) ) < ± for t = (¯/´) log(n)
Volume growth vs. small-set expansion
volume growth < 1+(´/¯) in intermediate step expansion( Ball(i,s) ) < (´/¯) for some s < t
Suppose volume( Ball(i, t) ) < ± for t = (¯/´) log(n)
Case 2: Many large eigenvalues (m > n¯/±)
1 = ¸1 ¸ …………………… ¸ ¸m > 1 - ´ ¸ ¸m+1 ¸ … ¸ ¸n ¸ -1Eigenvalues of the random walk matrix G:
If m > n¯/±, then 9 S with volume(S) < ± and expansion(S) < √´/¯
and we can find S in poly(n)-time
Number of large eigenvalues vs. small-set expansion
Volume growth vs. small-set expansion
How can we relate eigenvalues and volume growth?
Case 2: Many large eigenvalues (m > n¯/±)
1 = ¸1 ¸ …………………… ¸ ¸m > 1 - ´ ¸ ¸m+1 ¸ … ¸ ¸n ¸ -1Eigenvalues of the random walk matrix G:
If m > n¯/±, then 9 S with volume(S) < ± and expansion(S) < √´/¯
and we can find S in poly(n)-time
Number of large eigenvalues vs. small-set expansion
Suffices to show:9 vertex i such that volume( Ball(i, t) ) < ± for t = (¯/´) log(n)
Suffices to show:9 vertex i such that volume( Ball(i, t) ) < ± for t = (¯/´) log(n)
Heuristic:collision probability ¼ 1/|support|
Collision probability decay
|| Gt ei ||2 > 1/(±n)
collision probability of t-step random walk from i
Proof follows from local variant of Cheeger’s inequality(e.g. Dimitriou–Impagliazzo’98)
Case 2: Many large eigenvalues (m > n¯/±)
1 = ¸1 ¸ …………………… ¸ ¸m > 1 - ´ ¸ ¸m+1 ¸ … ¸ ¸n ¸ -1Eigenvalues of the random walk matrix G:
If m > n¯/±, then 9 S with volume(S) < ± and expansion(S) < √´/¯
and we can find S in poly(n)-time
Number of large eigenvalues vs. small-set expansion
Volume growth vs. small-set expansion
Number of large eigenvalues vs. collision probability decay
Collision probability decay vs. small-set expansionSuffices to show:
9 vertex i such that || Gt ei ||2 > 1/(±n) for t = (¯/´) log(n)
= ihei, G2t eii = Trace(G2t) > m¢(1 - ´)2t> 1/±i||Gt ei||2
Case 2: Many large eigenvalues (m > n¯/±)
1 = ¸1 ¸ …………………… ¸ ¸m > 1 - ´ ¸ ¸m+1 ¸ … ¸ ¸n ¸ -1Eigenvalues of the random walk matrix G:
If m > n¯/±, then 9 S with volume(S) < ± and expansion(S) < √´/¯
and we can find S in poly(n)-time
Number of large eigenvalues vs. small-set expansion
Case 1: Few large eigenvalues:
1 = ¸1 ¸ … ¸ ¸m > 1 - ´ ¸ ¸m+1 ¸ … ¸ ¸n ¸ -1Eigenvalues of the random walk matrix G:
can find set S’ ²/´-close to S in time exp(m)
Subspace enumeration: discretize span of top m eigenvectors
Case 2: Many large eigenvalues (m > n¯/±)
can find small non-expanding set S’ around that vertex
Trace bound: 9 vertex where local random walk mixes slowlyU
Subexponential Algorithm for SMALL-SET EXPANSIONIf there exists S with volume(S) < ± and expansion(S) < ²,we can find S’ with volume(S’) < 2± and expansion(S’) < √²/¯
in time exp(n¯/±)
Introduction
Small-Set Expansion
Unique Games
UNIQUE GAMESInput: list of constraints of form xi – xj = cij mod k
Goal: satisfy as many constraints as possible
Constraint Graph Gvariable vertexconstraint edge i
j
xi – xj = cij
mod k
Subexponential Algorithm for Unique Games
In particular: UG(²3) has exp(n²)-time algorithm
Given a UNIQUE GAMES instance with alphabet size ksuch that 1 - ² of constraints are satisfiable,can satisfy 1 - √²/¯
3 of constraints in time exp(k n¯)
can solve UG(²3) in time exp(m)
Few large eigenvalues + strong L1 bounds on top-m eigenvectors
large eigenvalues
Eigenvalues of the random walk matrix of constraint graph G:1 = ¸1 ¸ … ¸ ¸m > 1 - ² ¸ ¸m+1 ¸ … ¸ ¸n ¸ -1
[Arora–Khot–Kolla– S.–Tulsiani–Vishnoi’08]
Expanding constraint graph (m=1)
can solve UG(²) in time poly(n)
[Kolla’10]
UNIQUE GAMES on pseudorandom instances is easy
Algorithms for special instances of Unique Games
Strategypartition general instance into pseudorandom instances by changing only a small fraction of edges [Trevisan’05]
Algorithms for general instances of Unique Games
general instance
pseudorandom instances
decomposition
few constraints between parts
[Arora–Impagliazzo– –Matthews–S.’10]
here: Leighton–Rao decomposition tree, constant depth, using higher
eigenvalue Cheeger boundhere: at most n¯ eigenvalues > 1-²
here: at most√²/¯
3 fraction
Subexponential Algorithm for Unique Games
few large eigenvalues: at most n¯ eigenvalues >1-²
Few-large-eigenvalues decomposition
Every regular graph G can be partitioned into components with few large eigenvalues by removing √²/¯
3 fraction of edges
Unique Games with few large eigenvalues
If every component of G has few large eigenvalues, can solve UG(²2) in time exp(n¯)
few large eigenvalues: at most n¯ eigenvalues >1-²
Unique Games with few large eigenvalues
If every component of G has few large eigenvalues, can solve UG(²2) in time exp(n¯)
few large eigenvalues: at most n¯ eigenvalues >1-²
Unique Games with few large eigenvalues
If every component of G has few large eigenvalues, can solve UG(²2) in time exp(n¯)
label-extended graph G*
ij
xi – xj = cij
mod k
constraint graph G
cloud jcloud i
(i,a) » (j,b) if a-b = cij mod k
assignment satisfying 1- ²2 of constraints
set with volume = 1/kand expansion < ²2
Subspace enumeration: can enumerate all nonexpanding sets if G* has few large eigenvalues
When does G* have herefew large eigenvalues?
Unique Games with few large eigenvalues
label-extended graph G*
ij
xi – xj = c ij
mod k
constraint graph G
cloud jcloud i
(i,a) » (j,b) if a-b = cij mod k
When does G* have herefew large eigenvalues?
•if G is an expander [Kolla–Tulsiani’07]•if G has few large eigenvalues and eigenvectors are well-spread [Kolla’10]•if G has few’ large’ eigenvalues (this work, by comparing collision probabilities)
If every component of G has few large eigenvalues, can solve UG(²2) in time exp(n¯)
few large eigenvalues: at most n¯ eigenvalues >1-²
Recall: (with slightly different parameters)
Analysis: For every subdivision S, charge its expansion to vertices in S
KIf component K has many large eigenvalues, then 9 S in K with |S| < |K|1-¯ and expansion(S) < √²/¯
and we can find S in poly(n)-time
Number of large eigenvalues vs. small-set expansion
Algorithm: Subdivide components until all components have few large eigenvalues
no vertex is charged more than log(1/¯)/¯ timesIf vertex is charged t times, its component has size < n(1-¯)t
few large eigenvalues: at most n¯ eigenvalues larger 1-²
Few-large-eigenvalues decomposition
Every regular graph G can be partitioned into components with few large eigenvalues by removing √²/¯
3 fraction of edges
Open Questions
Example: C-approximation for SPARSEST CUT in time exp(n1/C )
How many large eigenvalues can a small-set expander have?
Is Boolean noise graph the worst case? (polylog(n) large eigenvalues)
Thank you! Questions?
More Subexponential Algorithms
Similar approximation for MULTI CUT and d-TO-1 2-PROVER GAMES
Better approximations for MAX CUT and VERTEX COVER on small-set expanders
What else can be done in subexponential time?
Towards better-than-subexponential algorithms for UNIQUE GAMES
Better approximations for MAX CUT or VERTEX COVER on general instances?