On the Unique Games Conjecture

Subhash Khot NYU Courant

CCC, June 10, 2010

Approximation Algorithms

A C-approximation algorithm for an NP-complete problem computes (C > 1),

for problem instance I , solution A(I) s.t.

Minimization problems :

A(I) C OPT(I)

Maximization problems :

A(I) OPT(I) / C

PCP Theorem

[B’85, GMR’89, BFL’91, LFKN’92, S’92,……] [PY’91] [FGLSS’91, AS’92 ALMSS’92]

Theorem : It is NP-hard to tell whether a MAX-3SAT instance is * Satisfiable (i.e. OPT = 1) or * No assignment satisfies more than 99% clauses (i.e. OPT 0.99).

i.e. MAX-3SAT is 1.01 hard to approximate.

(In)approximability : Towards Tight Hardness

Results • [Hastad’96] Clique n1-

• [Hastad’97] MAX-3SAT 8/7 -

• [Feige’98] Set Cover (1- ) ln n

[Dinur’05] Combinatorial Proof of PCP Theorem !

Open Problems in (In) Approximability

– Vertex Cover (1.36 vs. 2) [DinurSafra’02]

– Coloring 3-colorable graphs (5 vs. n3/14) [KhannaLinialSafra’93, BlumKarger’97]– Sparsest Cut (1+ε vs. (logn)1/2) [AMS’07,

AroraRaoVazirani’04]– Max Cut (17/16 vs 1/0.878… )

[Håstad’97, GoemansWilliamson’94] ………………………..

Unique Games Conjecture [K’02]

Connections to:

• Inapproximability (UGC several problems

inapproximable) • Discrete Fourier Analysis (e.g. KKL, Majority is Stablest)

• Geometry (Isoperimetry, Metric geometry, Integrality gaps)

• Algorithms (Attempts to disprove UGC) • Parallel Repetition (Gap amplification, Foam construction)

Supporting Evidence

Example of Unique Game (2CSP)

OPT = max fraction of equations that can be satisfied by any assignment. x1 - x3 = 2 (mod k)

x5 - x2 = -1 (mod k)

x2 - x1 = k-7 (mod k) …………. ………….

Unique Game 2CSP w/ Permutation

Constraints variable

k labelsHere k=4

constraints

Unique Game2CSP w/ Permutation Constraints

variable

k labelsHere k=4

Permutations or matchings : [k] [k]

OPT(G) = 6/7

Find a labeling that satisfies max # constraints

Unique Game

Unique Games Considered before ……

[Feige Lovasz’92] Parallel Repetition of UG reduces OPT(G).

How hard is approximating OPT(G) ? Observation : Easy to decide whether OPT(G) = 1.

Unique Games Conjecture

For any , , there is integer k(, ), s.t. it is NP-hard to tell whether a UniqueGame with k = k(, ) labels has

OPT 1- or OPT

i.e. Gap-Unique Game (1- , ) is NP-hard.

Gap Projection Game (1, ) is NP-hard. [ PCP Theorem + Raz’s Parallel Repetition Theorem ].

Supporting Evidence

[UGC] Gap-Unique Game (1-ε, ) is NP-hard.

[Feige Reichman’04] Gap-Unique Game (C, ) is NP-hard.

However C --> 0 as C --> ∞. [KV’05] SDP relaxation for UG has “integrality gap” (1-,

).

[KV’05] UGC based predictions were proven correct. Specifically, metric embedding lower bounds.

[Wishful thinking] “There is structure in CS/math”.

Small Set Expansion Conjecture

[Raghavendra Steurer’ 10] Φ (S ) = Edge expansion of set S.

For every ε > 0, there exists δ > 0, such that, it is NP-hard to tell whether in a graph G(V,E),

- There is a set S, |S| = δ |V|, Φ (S) ≤ ε. - For every set S, |S| ≈ δ |V|, Φ (S) ≥ 1- ε.

[Raghavendra Steurer’ 10]

SSE Conjecture Unique Games Conjecture.

S = Optimal labeling.

|S| = 1/k |G ’|.

Φ(S) = 1- OPT(G).

Unique Game and Small Set Expansion

|G’| = n k.

Unique Game G with n variables, k labels

Linear Equations Over Reals [K Moshkovitz’10]

Homogeneous 3LIN(R): x1 – x3 + 2 x5 = 0. ∙∙∙∙∙∙∙∙ eq: xi + .5 x j - x k = 0. Theorem: It is NP-hard to tell if :

• There is a “non-trivial” solution that satisfies 1-ε fraction of equations.

• Any “non-trivial” solution fails on a constant fraction of equations with error Ω(√ε).

3LIN(R) to 2LIN(R) reduction ? 2LIN(R) ≈ Sparsest Cut

Unique Games Conjecture [K’02]

Connections to:

• Inapproximability (UGC several problems

inapproximable) • Discrete Fourier Analysis (e.g. KKL, Majority is Stablest)

• Geometry (Isoperimetry, Metric geometry, Integrality gaps)

• Algorithms (Attempts to disprove UGC) • Parallel Repetition (Gap amplification, Foam construction)

Generic Reduction from Unique Game

[BGS’95 (Long Code), Hastad’97 (Fourier), UGC ,

……]

Generic Reduction from Unique Game

[BGS’95 (Long Code), Hastad’97 (Fourier), UGC ,

……]

Gadget: -1,1 k

PCP Reduction

k labels

Unique Game Instance MAX-CUT Instance

OPT(UG) > 1-ε sized cut. OPT(UG) < δ No cut with size arccos (1-2) /

Match Goemans-Williamson’sSDP rounding Algorithm

1/0.878… Hardness

Gadget : Dictatorships (Long Codes)

Weighted graph, total edge weight = 1.

Picking random edge : x R -1,1 k

y <-- flip every co-ordinate of x with probability ( 0.8)

Noise-sensitivity graph.

x

-1,1 k

y

- Consider f: -1, 1k -1,1, i.e. Cuts.

- Encode label i Є 1,2,…., k by dictatorship function f(x) = xi.

Gadget: Cut that “commits” to co-ordintae i

Fraction of edges cut = Pr(x,y) [xi yi ]

=

Observation : These are the maximum cuts.

xi = 1 xi = -1

Gadget : Cuts not committing to a co-ordinate

How large can be cuts with no influential co-ordinate ? Random Cut : ½ Majority Cut : > arccos (1-2) / > ½ [KKMO’04, MOO’05] Majority Is Stablest (Under Noise) Any cut with no influential co-ordinate has

size at most arccos (1-2) / .

Influence (i, f) = Prx [ f(x) f(x+ei) ]

Integrality Gap

Given : Maximization Problem + SDP relaxation.

• For every problem instance G,

SDP(G) OPT(G)

• Integrality Gap = Sup G SDP(G) / OPT(G)

[Raghavendra’ 08]

• Duality between Algorithms and Hardness.

• For every CSP, write a natural SDP relaxation.

• Integrality gap = β. Implies β-approximation.

• Theorem: Every instance with gap β’ < β can be used to construct a gadget and prove UGC-based β’- hardness result !

• SDPs are optimal algorithm for CSPs.

Unique Games Conjecture [K’02]

Connections to:

• Inapproximability (UGC several problems

inapproximable) • Discrete Fourier Analysis (e.g. KKL, Majority is Stablest)

• Geometry (Isoperimetry, Metric geometry, Integrality gaps)

• Algorithms (Attempts to disprove UGC) • Parallel Repetition (Gap amplification, Foam construction)

Inapproximability and Fourier Analysis

• f : -1,+1 k -1,+1, balanced.

• Sparsest Cut [KV’05, CKKRS’05]

[KKL’88] f has a co-ordinate with influence Ω(log k /k).

[Bourgain’02] If NSε (f) << √ε, then f depends essentially on exp(1/ε2) co-ordinates.

• MAX-CUT [KKMO’04] Majority Is Stablest

[MOO’05] If f has no influential co-ordinate, then NS ε (f) ≥ NS ε (Majority) - o(1).

Inapproximability and Fourier

• f : -1,+1k -1,+1, balanced.

• Vertex Cover [DinurSafra’02, K Regev’03, K Bansal’09]

[Friedgut’98] If total influence is k, then f depends essentially on exp(k) co-ordintaes.

[MOO’05] It Ain’t Over Till It’s Over If f has no influential co-ordinate, then on almost every subcube of -1, +1k of dimension

k/100, f = 1 and f = -1 with constant probability.

Inapproximability and Fourier • f : -1,+1k -1,+1, balanced.

• MAX-k-CSP [Samorodintsky Trevisan ’06]

If f has no influential co-ordinate, then f has low Gowers’ Uniformity norm.

Open: f: [q] k [q], q ≥ 3, no influential co-ordinate.

• f balanced. Is Plurality Stablest ?

• What is the maximum Fourier mass at the first level ?

Unique Games Conjecture [K’02]

Connections to:

• Inapproximability (UGC several problems

inapproximable) • Discrete Fourier Analysis (e.g. KKL, Majority is Stablest)

• Geometry (Isoperimetry, Metric geometry, Integrality gaps)

• Algorithms (Attempts to disprove UGC) • Parallel Repetition (Gap amplification, Foam construction)

Disproving UGC means ..

For small enough (constant) , given a UG with optimum 1- , algorithm that finds a labeling

satisfying (say) 50% constraints, irrespective of k = #labels.

Algorithmic Results Algorithm that finds a labeling satisfying f(, k, n) fraction of constraints.

[K’02] 1- 1/5 k2 [Trevisan’05] 1- 1/3 log1/3 n [Gupta Talwar’05] 1- log n [CMM’05] 1/k , 1- 1/2 log1/2 k [CMM’06] 1- log1/2 k log1/2 n

[AKKSTV’08 , Kolla’10] UG on “mild” expander graphs. [ABS’10] Exp ( n ) time algorithm. None of these disproves UGC. However …

If the UGC is true, then :

• k >> 21/ε .

• Graph of constraints cannot even be a “mild” expander. UG is easy on random graphs.

• Reduction from 3SAT must blow up the size by n1/ε .

• Conjecture does not hold for sub-constant ε, i.e. below 1/log n.

SDP Relaxation of Unique Games [FL’92]

• OPT(G) = 1- ε SDP(G) ≥ 1- ε .

• For i = 1, …, k, ui , vi ≥ 1- ε , up to permutation of indices.

OrthonormalBases for Rk

u1 , u2 ,

… , uk

v1 , v2 ,

… , vkvariables

k labels

Matchings [k] [k]u

v

[K’02, CMM’05] Rounding Algorithm

u1

uk

u2

vk

v2

v1

r r

Pick the label closest to r. Label(u) = Label(v) = 2.

Pr [ Label(u) = Label(v) ] > 1 - 1/5 k2 [K’02]. Pr [ Label(u) = Label(v) ] > 1- 1/2 log1/2 k [CMM’05].

Labeling satisfies 1- 1/2 log1/2 k fraction of constraints in expected sense.

Random ru v

[Trevisan’05] Algorithm

• [Leighton Rao’88] Delete 1% of edges so that all connected components have diameter O(log n).

• Algorithm to solve UG on low diameter graph.

Graph of variables and constraints

[AKKSTV’08 Algorithm]

Algorithm that works on a UG instance s.t. • 1-ε satisfiable and,• Every balanced cut in the graph cuts at least Ω ( √ε ) fraction of edges.

• SDP-based. • “Mild” expansion Almost all SDP vector tuples are nearly

identical Yields a good labeling.

S = Optimal labeling.

|S| = 1/k |G ’|.

Φ(S) = 1- OPT(G).

Unique Game and Small Set Expansion

Label extended Graph

|G’| = n k.

UG G with n variables, k labels

[Arora Barak Steurer’10 Algorithm][Kolla’10, Naor’10]

• Algo. runs in time exp(nε ) on UG that is 1-ε satisfiable.

• Good solution to UG Small non-expanding set S in G’.

• Small non-expanding set in label-extended graph G’

Either corresponds to a good UG solution (useful) Or is a non-expanding set in G (fake).

• Iteratively remove all fake sets from G, sacrificing at most 1% edges.

[Arora Barak Steurer’10 Algorithm][Kolla’10, Naor’10]

Main Lemma (Algorithmic) : If every set of size n1-ε expands by Ω(ε2), then the number of eigenvalues exceeding 1-ε is nO(ε). • The UG solution is found in the linear span of eigenvectors with eigenvalues ≥ 1-ε. [Kolla’10]

• Run-time exp ( nO(ε) ).

Unique Games Conjecture [K’02]

Connections to:

• Inapproximability (UGC several problems

inapproximable) • Discrete Fourier Analysis (e.g. KKL, Majority is Stablest)

• Geometry (Isoperimetry, Metric geometry, Integrality gaps)

• Algorithms (Attempts to disprove UGC) • Parallel Repetition (Gap amplification, Foam construction)

(Gaussian) Isoperimetry

. [MOO’05] Majority Is Stablest reduces via Invariance Principle, to a geometric question:

P: Rn -1,+1 be a partition of Gaussian space into two sets of equal measure.

NSε (P) = Pr [ P(x) ≠ P(y) ], Cor (x,y) = 1-2ε.

Which P minimizes the noise-sensitivity?

[Borell’85] NSε (P) ≥ NSε ( HALF-SPACE THRU ORIGIN ).

(Gaussian) Isoperimetry

Open: q ≥ 3. More Invariance.

[IM’10] MAX-q-CUT Problem. Plurality is Stablest Conjecture. Partition Rn into q equal parts. (Geometric): Standard Simplex Conjecture.

[K Naor’08] Kernel Clustering Problem.

Maximizing Fourier Mass at First Level.

(Geometric): Propeller Conjecture.

Integrality Gap • [Feige Schechtman’01] [Goemans Williamson’92]

1/0.878.. Integrality gap for MAX-CUT.

• SDP with “triangle inequality constraints” ?

• ω(1) Integrality gap for Sparsest Cut?

• UGC NP-hardness These integrality gaps exist.

[KV’05] Integrality Gap for Unique Games

SDP

Unique Game G with

OPT(G) = o(1) SDP(G) = 1-o(1)

OrthonormalBases for Rk

u1 , u2 ,

… , uk

v1 , v2 ,

… , vkvariables

k labels

Matchings [k] [k]u

v

Integrality Gap for MAX-CUT with

Triangle Inequality

-1,1k

u1 , u2 ,

… , uk

u1 u2 u3 ……… uk-1

uk

PCP Reduction

OPT(G) = o(1)

No large cut

Good MAX-CUT SDP solution

MAX-CUT and Sparsest Cut I.G.• [KV’05] MAX-CUT gap matching Goemans-Williamson even with triangle inequality constraints.

• [KV’05, KrauthgamerRabani’05, DKSV’06]

(loglog n) integrality gap for Sparsest Cut SDP.

An n-point “negative type” metric that needs distortion (loglog n) to embed into L1. Refutation of [Goemans Linial’97, ARV’04] conjectures.

• [KS’09, RS’09] Similar gaps for SDP + Sherali-Adams LP. Negative type metric that is L1

embeddable locally but not globally.

Open Problems

• Integrality gaps for the Lasserre SDP Relaxation?

Lasserre Relaxation could potentially disprove

UGC.

• Sparsest Cut (NEG versus L1 Metrics) :

[ARV’04, AroraLeeNaor’05] O(√log n). [LeeNaor’06, CheegerKleiner’06, CKNaor’09]. Ω(logc n), c = ½?

Unique Games Conjecture [K’02]

Connections to:

• Inapproximability (UGC several problems

inapproximable) • Discrete Fourier Analysis (e.g. KKL, Majority is Stablest)

• Geometry (Isoperimetry, Metric geometry, Integrality gaps)

• Algorithms (Attempts to disprove UGC) • Parallel Repetition (Gap amplification, Foam construction)

Gap Amplification Prove UGC in two steps (?):

• Prove “mild” hardness, i.e. GapUG (1-ε’ , 1-ε’’ ) is hard.

• Amplify gap via parallel repetition to GapUG (1-ε , δ).

Note however that even proving “mild” hardness

is a huge challenge.

Strong Parallel Repetition ?

OPT(G) = 1-ε.

[Raz’98] OPT(Gm) ≤ (1-ε32 )m/log k . 2P1R Games

[Holenstein’07] OPT(Gm) ≤ (1-ε3 )m/log k . 2P1R Games

[Rao’08] OPT(Gm) ≤ (1-ε2)m . Projection Games (UG).

GapUG (1-ε , δ ) is NP-hard iff GapUG (1-ε , 1 - √ε C(ε) ) is NP-hard where C(ε) –> ∞ as ε –> 0. [Raz’08] The rate (1-ε2)m cannot be improved further.

Raz’s Example Optimal Foam

Problem: Tiling Rd using a “shape” of unit volume and minimum surface area.

Great for tiling:

Surface area = 2d

Not good for tiling:

Surface area ≈ √d

[Kindler O’Donnell Rao Wigderson ’08, Alon Klartag’09]

There exists a tiling shape with unit volume and surface area O(√d ) !

Conclusion

• (Dis)Prove Unique Games Conjecture.

• Intermediate between P and NP-complete?

• Prove hardness results bypassing UGC.

• TSP, Steiner Tree, Scheduling Problems ?

• More techniques, connections, results …