satyen kale (yahoo! research) joint work with sanjeev arora (princeton)

21
A Combinatorial, Primal- Dual Approach to Semidefinite Programming Satyen Kale (Yahoo! Research) Joint work with Sanjeev Arora (Princeton)

Upload: daniel-bradford

Post on 14-Dec-2015

217 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Satyen Kale (Yahoo! Research) Joint work with Sanjeev Arora (Princeton)

A Combinatorial, Primal-Dual Approach to Semidefinite Programming

Satyen Kale (Yahoo! Research)

Joint work withSanjeev Arora (Princeton)

Page 2: Satyen Kale (Yahoo! Research) Joint work with Sanjeev Arora (Princeton)

Semidefinite Programming

Semidefinite Program (SDP): find X s.t.A1 ² X · 0A2 ² X · 0

Am ² X · 0

X º 0 Tr(X) = 1

All eigenvalues are non-negative´ 9 v1, v2, …, vn s.t. Xij = vi ¢ vj

A ² B = Tr(AB) = ij AijBij

Density matrix

Page 3: Satyen Kale (Yahoo! Research) Joint work with Sanjeev Arora (Princeton)

Semidefinite ProgrammingMax Cut [GW’95]

Graph Coloring [KMS’98, ACC’06]

(a Ç :b) Æ (:a Ç c) Æ (a Ç b) Æ (:c Ç b)

Constraint Satisfaction [ACMM’05, R ’08, RS ‘09,..]1 0 0 0 1 1 0 1

SDP

Balanced Partitioning [ARV’04]

Control Theory

Page 4: Satyen Kale (Yahoo! Research) Joint work with Sanjeev Arora (Princeton)

Algorithms for SDP

Ellipsoid method [GLS’81]:• O(n8) iterations

Interior point methods [NN’90, A’95]:• O(√n ¢ m3) time

Lagrangian Relaxation [KL’95], [AHK’05]:• Reduction to eigenvectors• poly(1/) dependence on , limits applicability

A1 ² X · 0 An ² X · 0

i wiAi ² X · 0

Combinatorial, Primal-Dual algorithms

(analogous to flow algorithms via LP)

Page 5: Satyen Kale (Yahoo! Research) Joint work with Sanjeev Arora (Princeton)

Challenges in designing efficient, combinatorial algorithms for SDP

Positive semidefiniteness hard to maintain

Rounding algorithms exploit geometric structure (e.g. negative-type metrics)

Matrix operations inefficient to implement

Page 6: Satyen Kale (Yahoo! Research) Joint work with Sanjeev Arora (Princeton)

Our Work

A general scheme that yields fast,

combinatorial, primal-dual approximation algorithms for various

combinatorial optimization problems using SDP relaxations

Page 7: Satyen Kale (Yahoo! Research) Joint work with Sanjeev Arora (Princeton)

Our Results: Primal-Dual algorithms

Problem Previous best:

O(√log n) apx

Our algorithm:O(√log n)

apx

Our algorithm:

O(log n) apx

UndirectedSparsest Cut

Õ(n2)[AHK’04]

Õ(n2) Õ(m + n1.33)

UndirectedBalanced Sep.

Õ(n2)[AHK’04]

Õ(n2) Õ(m + n1.33)

DirectedSparsest Cut

Õ(n9.5) Õ(m1.33 + n2) Õ(m1.33)

DirectedBalanced Sep.

Õ(n9.5) Õ(m1.33 + n2) Õ(m1.33)

Min UnCut Õ(n9.5) Õ(n3) ---

Min 2CNF Deletion

Õ(n9.5) Õ(nm1.33 + n3)

---

Õ(m + n1.33) [S’09]

Õ(m + n1.33) [S’09]Unknown how to achieve using LP!

Page 8: Satyen Kale (Yahoo! Research) Joint work with Sanjeev Arora (Princeton)

Near-linear time alg for MAXCUT

Also yields Õ(m) time algorithm for approximating MAXCUT SDP Gives first near-linear implementation of

Goemans-Williamson approximation algorithm

Previous best: Õ(nm) [KL’95]

Page 9: Satyen Kale (Yahoo! Research) Joint work with Sanjeev Arora (Princeton)

LP via Multiplicative Weights [L’88]

a1 ¢ x · 0a2 ¢ x · 0

am ¢ x · 0

x ¸ 0i xi = 1

Multiplicative WeightsInitialize: x1 = (1/n, …, 1/n)Update:xt+1 = xt £ exp(- ait

)/t

t = normalization factor

Violation CheckerFind it s.t. ait

¢ xt >

xt

ait

Thm: Finds -feasible x in O(2log(n)/2) iterations.

= maxij |aij| (“width”)

Convex combination of constraints allowed:i yi ai ¢ xt > , where yi ¸ 0, i yi = 1.

Hence Primal-Dual.

Analysis uses t as potential function

Coordinate-wise

a1 ¢ x · a2 ¢ x ·

am ¢ x ·

x ¸ 0i xi = 1

Page 10: Satyen Kale (Yahoo! Research) Joint work with Sanjeev Arora (Princeton)

SDP via Matrix Multiplicative Weights

A1 ² X · 0A2 ² X · 0

Am ² X · 0

X º 0Tr(X) = 1

Matrix MWInitialize: X1 = (1/n)IUpdate:Xt+1 = exp(- t

s=1 Ais)/t

t = normalization factor

Violation CheckerFind it s.t. Ait

² Xt >

Xt

Ait

Thm: Finds -feasible X in O(2log(n)/2) iterations.

= maxi kAik (“width”)

Convex combination of constraints allowed:i yi Ai²Xt > , where yi ¸ 0, i yi = 1.

Hence Primal-Dual.

Analysis uses t as potential function

Page 11: Satyen Kale (Yahoo! Research) Joint work with Sanjeev Arora (Princeton)

The Matrix Exponential Matrix exponential:exp(A) = I + A + A2/2! +

A3/3! + …

Always PSD: exp(A) º 0

Tricky beast:exp(A+B) = exp(A) exp(B)

Computation: O(n3) time Computing exp(A)v:

Õ(m) time (arises as soln of ODE)

Golden-Thompson inequality:Tr(exp(A+B)) · Tr(exp(A)exp(B)) Matrix MW

Initialize: X1 = (1/n)IUpdate:Xt+1 = exp(- t

s=1 Ais)/t

t = normalization factor

Page 12: Satyen Kale (Yahoo! Research) Joint work with Sanjeev Arora (Princeton)

Approximation via SDP Relaxations

Input

QuadraticProgram

SDP:Vector

variables

Reduction

SDP Solver

RoundingAlgorithm

SDP Opt: XXij = vi ¢ vj

v1

vn

v2

RelaxationPrimal-Dual SDP

Page 13: Satyen Kale (Yahoo! Research) Joint work with Sanjeev Arora (Princeton)

Primal-Dual SDP Framework

Reduction

RelaxationRoundingAlgorithm

v1

vn

v2

MatrixMWX

y1,…, ym

Primal-Dual SDP

SDP:Vector

variables

Input

Xij = vi ¢ vj

QuadraticProgram

Page 14: Satyen Kale (Yahoo! Research) Joint work with Sanjeev Arora (Princeton)

Approximating Balanced Separator

Cut (S, S’) is c-balanced if |S|, |S’| ¸ cn

Min c-Balanced Separator: c-balanced cut of min

capacity

Numerous applications: Divide-and-conquer algs Markov chains Geometric embeddings Clustering Layout problems …

G = (V, E)

S

S’

Page 15: Satyen Kale (Yahoo! Research) Joint work with Sanjeev Arora (Princeton)

SDP for Balanced Separator

¼kvi – vjk2 = 0 if i, j on same side,

= 1 otherwise

SDPmin i,j 2 E ¼kvi – vjk2

8i: kvik2 = 1

i, j ¼kvi – vjk2 ¸ c(1-c)n2

8ijk: kvi–vjk2 + kvj–vkk2 ¸ kvi–vkk2

vi = -1 vi = 1

G = (V, E)

S

S’

Page 16: Satyen Kale (Yahoo! Research) Joint work with Sanjeev Arora (Princeton)

Finding Violated Constraints

SDP(C1) i,j 2 E ¼kvi – vjk2 <

(C2) 8i: kvik2 = 1

(C3) i, j ¼kvi – vjk2 ¸ c(1-c)n2

(C4) 8ijk: kvi–vjk2 + kvj–vkk2 ¸ kvi–vkk2

To find violated constraints, given X (thus, vi): Check (C2) and (C3) To check (C1) and (C4): find multicommodity

flow s.t.1. Total flow from any node is at most d = Õ(/n)2. Total flow on any edge is at most 13. i,j fijkvi – vjk2 ¸ 4

fij = total flow between i, j

Bounds width

Binary search parameter

Either (C1) or (C4) violated

Page 17: Satyen Kale (Yahoo! Research) Joint work with Sanjeev Arora (Princeton)

Finding flow: O(log n) approx

Xij = vi ¢ vj

v1

vn

v2

d

d

d

d

d

d

d

To find flow s.t. ij fijkvi – vjk2 ¸ 4

Thm: This yields either:1. a flow s.t. ijfijkvi – vjk2 ¸ 42. a cut of value O(log(n))

kvik2 = 1ij ¼kvi – vjk2 ¸ c(1-c)n2

Max flow

Page 18: Satyen Kale (Yahoo! Research) Joint work with Sanjeev Arora (Princeton)

Xij = vi ¢ vj

v1

vn

v2

Thm: This yields either:1. a flow s.t. ijfijkvi – vjk2 ¸ 42. a cut of value O(√log(n))

d

d

dd

dd

d

d

Multicommodity flow

To find flow s.t. ij fijkvi – vjk2 ¸ 4kvik2 = 1ij ¼kvi – vjk2 ¸ c(1-c)n2

Finding flow: O(plog n) approx

Page 19: Satyen Kale (Yahoo! Research) Joint work with Sanjeev Arora (Princeton)

Other Applications of Matrix MWDerandomization• Deterministic Expander Graphs Constructions [K’07, WX

’07]• Deterministic Approximation via Covering SDPs [K ’07, WX

’07]Quantum Computing• QIP = PSPACE [JJUW ’10] and related results [JUW ’09, JW

’09]• Learnability of Quantum States [A ’06, K ’07]

Machine Learning• Online Variance Minimization [WK ’06]• Online PCA [WK ’06]

Other SDP based Approximation Algorithmss• Near Linear-Time Approximation Algorithms for 2-CSPS [S

’10]• Near-Linear Time Balanced Graph Partitioning [OV ’11]

Page 20: Satyen Kale (Yahoo! Research) Joint work with Sanjeev Arora (Princeton)

Future Work

Make SDP solver efficient in practice Sampling, rescaling techniques might be

useful ``Combinatorialize’’ interior-point

methods?

Near linear-time algs for approx max flow?

Page 21: Satyen Kale (Yahoo! Research) Joint work with Sanjeev Arora (Princeton)

Thank You!