How to Round Any CSP

Prasad RaghavendraUniversity of Washington, Seattle

David Steurer,Princeton University

(In Principle)

Constraint Satisfaction ProblemA Classic Example : Max-3-SAT

Given a 3-SAT formula,Find an assignment to the variables that satisfies the maximum number of clauses.

))()()(( 145532532321 xxxxxxxxxxxx Equivalently the

largest fraction of clauses

Variables : {x1 , x2 , x3 ,x4 , x5} Constraints : 4 clauses

Constraint Satisfaction Problem

Instance :• Set of variables.• Predicates Pi applied on variables

Find an assignment that satisfies the largest fraction of constraints.

Problem :

Domain : {0,1,.. q-1}Predicates : {P1, P2 , P3 … Pr}

Pi : [q]k -> {0,1}

Max-3-SAT

Domain : {0,1}Predicates :

P1(x,y,z) = x ѵ y ѵ z

))()()(( 145532532321 xxxxxxxxxxxx

Theorem: [Raghavendra 08]Assuming Unique Games Conjecture, For every CSP, “a simple semidefinite program (SDP1) gives the best approximation computable efficiently.”

[Raghavendra08]A generic rounding scheme for (SDP1) that is optimal for every CSP under UGC.

Independent of UGC, for 2CSPs, the generic rounding scheme for (SDP1) achieves an

Approximation Ratio ≥ (1-²) Integrality Gap of SDP.

Rounding Algorithm

minimumover all instances

=

value of rounded solution

value of SDP solution

rounding – ratioA ( ¦ )(approximation ratio)

≥ (1-²) integrality gap ( ¦ )

=

value of optimal solution

value of SDP solution

minimumover all instances

For any CSP ¦ and any ²>0, there exists an efficient algorithm A,

Unconditionally, the algorithm A as good as all known algorithms for CSPs

Very Simple : No Invariance Principle, Dictatorship Tests, Unique Games.

Drawbacks•Running Time(A) On CSP over alphabet size q, arity k

•No explicit approximation ratio)(2

)/1,,(2 npolyqkpoly

Computing Integrality Gaps

Theorem:

For any CSP ¦ and any ²>0, there exists an algorithm A to compute integrality gap (¦) within an accuracy ²

Running Time(A) On CSP over alphabet size q, arity k

)/1,,(22qkpoly

Previous Work SDP ALGORITHMS[Charikar-Makarychev-Makarychev 06]

MaxCut [Goemans-Williamson] [Charikar-Wirth]

[Lewin-Livnat-Zwick][Charikar-Makarychev-Makarychev 07]

[Hast] [Charikar-Makarychev-Makarychev 07]

[Frieze-Jerrum][Karloff-Zwick]

[Zwick SODA 98][Zwick STOC 98]

[Zwick 99][Halperin-Zwick 01]

[Goemans-Williamson 01][Goemans 01]

[Feige-Goemans][Matuura-Matsui]

[Trevisan-Sudan-Sorkin-Williamson]

[O’Donnell-Wu] Optimal rounding schemes for MaxCut

ALGORITHM OUTLINERounding Any Constraint Satisfaction Problem

Max Cut

10

15

3

7

11

Max CUTInput : A weighted graph G

Find :A cut with maximum fraction of crossing edges

Eji

jiij vvw),(

2||4

1

Semidefinite Program

Variables : v1 , v2 … vn

| vi |2 = 1

Maximize

Max Cut SDP

10

15

3

7

11

1

1

1

-1

-1

-1

-1-1

-1

v1

v2

v3

v4

v5

MaxCut Rounding Problem

Given a graph on the n - dimensional unit ball,Find the maximum cut of the graph.

Approximation using Finite Models

¦-CSP Instance =

¦-CSP Instance =finite

variablefolding

(identifyingvariables)

optimal solution for

=finite

approximate solution

for =

unfolding ofthe assignment

constant time

Challenge: ensure = finite has a good solution

10

15

3

7

11

1

1

-1

-1

-1

-1-1

-1

-1

1

1

11

Approximation using Finite Models

[Frieze-Kannan]For a dense instance =, it is possible to construct finite model

=finite

OPT(=finite) ≥ (1-ε) OPT(=)

General Method for CSPs

What we will do :

SDP value (=finite) > (1-ε)SDP value (=)

PTAS for dense instances

Analysis of Rounding Scheme

¦-CSP Instance =

¦-CSP Instance =finite

SDP value ®

SDP value > ® - ²

OPT value¯

rounded value¯

010001001010001001

Hence: rounding-ratio for = < (1+²) integrality-ratio for = finite

unfolding

CONSTRUCTING FINITE MODELS (MAXCUT)

Rounding Any Constraint Satisfaction Problem

v1

v2

v3

v4

v5

STEP 1 : Dimension Reduction

• Pick d = 1/ Є4 random Gaussian vectors {G1 , G2 , .. Gd} • Project the SDP solution along these directions.Map vector V

V → V’ = (V G∙ 1 , V G∙ 2 , … V G∙ d)v

1

v3v

4 v5

Constant dimensions

STEP 2 : SurgeryScale every vector V’ to unit length

STEP 3 : Discretization•Pick an Є –net for the

d dimensional sphere• Move every vertex to the nearest point in the Є –net

v2v

2

FINITE MODEL Graph on Є –net points

To Show:

SDP value (=finite) > (1-ε)SDP value (=)

Lemma : “Inner Products are almost preserved under random

projections”

If V’,U’ are random projections of U, V on 1/ ε4 directions,

Pr [ |V U – V’ U’| > ∙ ∙ ε] < ε2

STEP 1 : Dimension Reduction•Project the SDP solution along 1/ Є4 random directions.

STEP 2 : SurgeryScale every vector V’ to unit length

STEP 3 : Discretization•Pick an Є –net for the

d dimensional sphere• Move every vertex to the nearest point in the Є –net

For SDP value (=)Contribution of an edge e = (U,V)

|U-V|2 = 2-2 V U ∙

To Show:

SDP value (=finite) > (1-ε)SDP value (=)SDP Vectors for =finite = Corresponding vectors in Є –net

STEP 1With probability > 1- Є2 ,

| |U-V|2 - |U’-V’|2 | < 2Є

STEP 2With probability > 1- 2Є2 ,

1+ Є < |V’|2 ,|U’|2 < 1- Є, Normalization changes distance by at most 2Є

STEP 3Changes edge length by at most 2Є

For SDP value (=)Contribution of an edge e = (U,V)

|U-V|2 = 2-2 V U ∙

To Show:

SDP value (=finite) > (1-ε)SDP value (=)SDP Vectors for =finite = Corresponding vectors in Є –net

STEP 1With probability > 1- Є2 ,

| |U-V|2 - |U’-V’|2 | < 2Є

STEP 2With probability > 1- 2Є2 ,

1+ Є < |V’|2 ,|U’|2 < 1- Є, Normalization changes distance by at most 2Є

STEP 3Changes edge length by at most 2Є

ANALYSISWith probability 1-3Є2,The contribution of edge e changes by < 6Є

In expectation,For (1-3Є2) edges, the contribution of edge e changes by < 6Є

SDP value (=finite) > SDP value (=) - 6Є – 3Є2

FINITE MODELS FOR GENERAL CSPRounding Any Constraint Satisfaction Problem

Semidefinite Program for CSPs

Variables :For each variable Xa

Vectors {V(a,0) , V(a,1)}

For each clause P = (xa ν xb ν xc),Scalar variables

μ(P,000) , μ(P,001) , μ(P,010) , μ(P,100) , μ(P,011) , μ(P,110) , μ(P,101) , μ(P,111)

))()()(( 145532532321 xxxxxxxxxxxx

Xa = 1 V(a,0) = 0 V(a,1) = 1

Xa = 0 V(a,0) = 1 V(a,1) = 0

If Xa = 0, Xb = 1, Xc = 1

μ(P,000) = 0 μ(P,011) = 1μ(P,001) = 0 μ(P,110) = 0μ(P,010) = 0 μ(P,101) = 0μ(P,100) = 0 μ(P,111) = 0

Objective Function :

PClauses sassignment

PP

3}1,0{

),()(

Constraints : For each clause P,

0 ≤μ(P,α) ≤ 1

For each clause P (xa ν xb ν xc), For each pair Xa , Xb in P,

consitency between vector and LP variables.

V(a,0) V∙ (b,0) = μ(P,000) + μ(P,001) V(a,0) V∙ (b,1) = μ(P,010) + μ(P,011) V(a,1) V∙ (b,0) = μ(P,100) + μ(P,101) V(a,1) V∙ (b,1) = μ(P,100) + μ(P,101)

1),(

P

Semidefinite Relaxation for CSPSDP solution for =:

SDP objective:

for every constraint Á in =- local distributions ¹Á over

assignments to the variables of Á

Example of local distr.: Á = 3XOR(x3, x4, x7)

x3 x4 x7 ¹Á0 0 0 0.10 0 1 0.010 1 0 0 …1 1 1 0.6for every variable xi in =

- vectors vi,1 , … , vi,q

constraints

(also for first moments)

Explanation of constraints:first and second moments of distributions are consistent and form PSD matrix

maximize

Strong and WeakSTRENGTHFor every clause Á in =- local distributions ¹Á over assignments to the variables of Á

Vector variables vi,a within a clause Á satisfy all valid constraints (like triangle inequality)

– the inner products are in the integral hull.WEAKNESS

The above hard constraint is only for variables that participate together in a clause

Throwing away constraints

{vi,a } { μ …}

-Infeasible SDP solution for a instance = , it does not satisfy the consistency for a clause P.

Consider instance =‘ = = - {P}

Now {vi,a } { μ … } is a good SDP solution for =‘

Throw away clauses from CSP

Throw away constraints from SDP relaxation

v1

v2

v3

v4

v5v

1

v3v

4 v5

Constant dimensions

v2v

2

FINITE MODEL CSP on Є –net points

STEP 1 : Dimension Reduction•Project the SDP solution along d =1/ Є4 random directions.

STEP 3 : Discretization•Pick an Є –net for the

d dimensional sphere• Move every variable to the nearest point in the Є –net =finite = discretized instance

STEP 2 : Throw away Discard clauses for which the corresponding inner products are not preserved within Є.

=‘ = New instance

To Show:

SDP value (=finite) > (1-ε)SDP value (=)

SDP Vectors for =finite = Corresponding vectors in Є –net

LP variables { μ …}?

Problem :

The inner products of vectors corresponding to a clause P might not be in the integral hull.( For example : 3 arbitrary vectors in a Є –net are not guaranteed to satisfy triangle inequality )

The initial SDP solution satisfied all the constraints

STEP 1 : Dimension Reduction•Project the SDP solution along d =1/ Є4 random directions.

STEP 3 : Discretization•Pick an Є –net for the

d dimensional sphere• Move every variable to the nearest point in the Є –net =finite = discretized instance

STEP 2 : Throw away Discard clauses for which the corresponding inner products are not preserved within Є.

=‘ = New instance

From STEP 2,

We have discarded clauses for which inner products are not preserved within Є

Discarding a clause P

Forget about constraints corresponding to P

Discretization changes inner product by Є

For every remaining clause, all inner products are within 2Є of what it was.

Smoothing Operation

Canonical SDP SolutionUniform Distribution over all Integral solutions.

Example:Va,0 V∙ a,0 = Va,1 V∙ a,1 = ½Va,0 V∙ b,0 = Va,0 V∙ b,1 = Va,1 V∙ b,0 = Va,1 V∙ b,1 = 1/4

Є –net SolutionSDP Vectors for =finite =

Corresponding vectors in Є –net (1-Є) X

+

=Final SDP solution

IntegralHull

Є X

Є

Consider the inner products corresponding to a single clause P

SDP Objective value remains roughly the same.

Conclusions

• Rounding stronger SDPs.

• More efficient rounding? Can this SDP be solved in constant dimensional space directly?

• Integrality gaps for stronger SDP relaxation of Unique Games

Thank You

Good finite Models from SDP solutions – Dimension Reduction & Discretization

¦-CSP Instance =

¦-CSP Instance =finite

SDP solution for =

compute

Dimension Reduction

Project on randomlow dimensional

subspace

almostSDP solution

for =

Discretize

Move vectors to closest point

on ²-net

almostSDP solution

for =

Rn Rd

identify variableswith same vectors

Theorem: SDP value (=finite) > SDP value (=)

Idea: use almostSDP solution and

do surgery

finite number ofdifferent vectrs

Constraint Satisfaction Problems (CSP)CSP ¦

finite set of allowed types of constraints Á : [q]k {0,1} (alphabet [q], arity k)e.g. ¦ = { 3XOR, 3SAT, 3NAE}

¦-CSP Instance =- variables x1,…,xn

- list of constraints Á of type ¦ on subsets of variables

Goal: Find assignment x 2 [q]n so as to maximize fraction of satisfied constraints opt(=)

Examples: Max-Cut, Max-3SAT,…

PCP Theorem: NP-hard to distinguish opt(=)=1 and opt(=)<0.9 (even for constant k and q)

Approximation Algorithms: Goemans-Williamson, Zwick, CMM, …