Approximating the Cut-Norm

Hubert Chan

• “Approximating the Cut-Norm via Grothendieck’s Inequality”

Noga Alon, Assaf Naor

appearing in STOC ‘04

Problem Definition

.

sum themaximizes that }1,1{, find

),(matrix real an Given

ijjiij

ji

ij

yxa

yx

aAnm

_ __ ++++

++

++

__

_

_

_ +

+

.||||by denoted , of norm the

sum maximized thecall We

AA

Main Result

• The problem is MAX SNP hard.

• Randomized polynomial algorithm that gives expected 0.56-approximation.

For maximization problem, approximation ratio always less than 1.

The authors showed a deterministic algorithm that gives 0.03-approximation.

De-randomization: paper by Mahajan and Ramesh

Road Map

• Motivation• Hardness Result

• General Approach

• Outline of Algorithm

• Conclusion

Motivation

• Inspired by the MAX-CUT problem

Frieze and Kannan proposed decomposition scheme for solving problems on dense graphs

• Estimating the norm of a matrix is a key step in the decomposition scheme

Comparing with Previous result

• Previously, computes norm with additive errormn

• This is good only for a matrix whose norm is large.

• The new algorithm approximates norm for all real matrices within constant factor 0.56 in expectation.

Road Map

• Motivation

• Hardness Result• General Approach

• Outline of Algorithm

• Conclusion

MAX-SNP

A maximization problem is MAX-SNP hard if

.factor ithin solution w optimal theeapproximat

can algorithm timepolynomial no 0

For example, there is a well-known polynomial algorithm for MAX-CUT that returns a cut with size at least 0.5 of the maximum cut.

However, there is no polynomial algorithm that gives 16/17-approximation.

MAX-CUTGraph G=(V,E)

W V\W

The problem is MAX SNP hard

• Reduction from MAX-CUT• Given graph G = (V,E),

construct 2|E| x |V| matrix A:

for each edge e = (u,v),

4

1 ,

4

14

1 ,

4

1

,2,,2,

,1,,1,

veue

veue

AA

AA

eu

v

u v

e,1e,2

1/41/4-1/4

-1/4

MAX-CUT · ||A||§

otherwise. 1- , if 1Set

cut.max a forms )\,( Suppose

Wjy

WVW

j u v

1/4 -1/4

_u v

1/4 -1/41/4-1/4

+

+

_

For e = (u,v) not in max cut, there is no contribution no matter what xe,1 and xe,2 are.

For e = (u,v) in max cut, we can set xe,1 and xe,2 to give contribution 1.

MAX-CUT ¸ ||A||§.|||| attains s' and s' of choice some Suppose Ayx ji

}.1:{Set jyjWu v

1/4 -1/4

_u v

1/4 -1/41/4-1/4

+

+

_

For e = (u,v) not in cut (W,V\W), there is no contribution no matter what xe,1 and xe,2 are.

For e = (u,v) in cut (W, V\W), the contribution from rows e,1 and e,2 is at most 1.

Road Map

• Motivation

• Hardness Result

• General Approach• Outline of Algorithm

• Conclusion

Relaxation Schemes

}1,1{,

max||||,

ji

jji

iij

yx

yxaA

• Recall the problem:

Objective function not linear Could introduce extra variables, but rounding might

be tricky How about Semidefinite Program Relaxation?

Linear Programming Relaxation?

Semidefinite Program Relaxation

.product dot with tion multiplica Replace

.or with vect variableReplace

.or with vect variableReplace

jiji

jj

ii

vuyx

vy

ux

||A||SDP = max ij aij ui vj

ui ² ui = 1

vj ² vj = 1

where ui and vj are vectors in

m+n dimensional Euclidean space

Remarks about SDP.|||| |||| that Note AA SDP

² Are (m+n)-dimensions sufficient?

Yes, since any m+n vectors in a higher dimensional Euclidean space lie on an (m+n)-dimensional subspace.

² Fact:

There exists an algorithm that given > 0, returns solution vectors ui’s and vj’s that attains value at least ||A||SDP - in time polynomial in the length of input and the logarithm of 1/.

Are we done?We need to convert the vectors back to integers in {-1,1}!

General strategy:

1. Obtain optimal vectors ui and vj for the SDP.

2. Use some randomized procedure to reconstruct integer solutions xi, yj 2 {-1,1} from the vectors.

3. Give good expected bound:Find some constant > 0 such that

E[ij aij xi yj] ¸ ||A||SDP ¸ ||A||§

Road Map

• Motivation

• Hardness Result

• General Approach

• Outline of Rounding Algorithm• Conclusion

Random Hyperplane

)(

)(Set

.r unit vecto random Generate

zv sign y

zu sign x

z

jj

ii

+_

z

Recall we need to show:E[ij aij xi yj] ¸ ij aij ui ² vj

Analyzing E[xy]

z u

v

Unit vectors u and v such that cos = u ² v

A random unit vector z determines a hyperplane.

Pr[u and v are separated] = /

Set x = sign(u ² z), y = sign(v ² z).

E[xy] = (1 - / ) - /

= 1 - 2 /

= 2/ ( / 2 - )

= 2/ arcsin(u ² v)

How do sine and arcsine relate?

.2

|arcsin

| 1 ],11[For

t

t,-t

Is this good news?

Performance Guarantee?

)arcsin( ][ ][ jiij

ijij

jiijij

jiij vuayxEayxaE

• We have term by term constant factor approximation.

• Bad news: cancellation because terms have different signs

• Hence, we need global approximation.

An Equivalent Way to Round Vectors

+_

R

Generate standard, independent Gaussian random variables

r1, r2, …, rm+n

R = (r1, r2, …, rm+n)

Set xi = sign(ui ² R), yj = sign(vj ² R)

What we would like to see….

.0constant somefor

, )arcsin(][ ?

c

vucvuyxE jijiji

This is impossible because arcsin is not a linear function.

What we can prove……

)]},()([{2

][ Rj

gRi

fEj

vi

uj

yi

xE

where fi is a function depending on ui

and gj is a function depending on vj.

Important property of fi and gj:

E[fi2] = E[gj

2] = /2 – 1 < 1.

Inner Product and E[f g]

gf,fE

(R)f(R)g(R) fE

vuvuk

kk

g][

can write We

g][

Compare

Recall the SDP

Are (m+n)-dimensions sufficient?

Yes, since any m+n vectors in a higher dimensional Euclidean space lie on an (m+n)-dimensional subspace.

||A||SDP = max ij aij ui vj

ui ² ui = 1

vj ² vj = 1

where ui and vj are vectors in

m+n dimensional Euclidean space

Wait a minute…We need unit vectors!

SDPjiij

ij

jjii

Agfa

ggff

||||)12

( |,|

12

, ,

Constant factor approximation

},{2

][ jg

ifa

jv

iua

jy

ixaE

ijij

ijij

ijij

},||{||2

jg

ifaA

ijijSDP

}||||)12

( ||{||2

SDPSDP AA

.||||273.0 ||||273.0

||||)14

(

AA

A

SDP

SDP

What are functions f and g?

).(2

)( and

)(2

)( where

)]},()([{2

][

RvsignRvRg

RusignRuRf

Rj

gRi

fEj

vi

uj

yi

xE

iij

iii

Properties of Gaussian Measure

vuvu

rrEvu

rvruERvRuE

ppp

pqqpqp

p qqqpp

][

][)])([(

(a) Mean 0, Variance 1

(b) Multi-dimensional Gaussian spherical symmetric

vuRvsignRuE 2

)]()[(

Recap1. Solve for optimal vectors ui and vj for the SDP.

2. Generate multi-dimensional Gaussian random vector R.

Set xi = sign(ui ² R), yj = sign(vj ² R).

3. Relate E[xi yj] to ui ² vj.

)]}()([{2

][ Rj

gRi

fEj

vi

uj

yi

xE

4. Use (1) ui and vj are optimal vectors and

(2) E[fi gj] can be considered as an inner product.

E[ij aij xi yj] ¸ 0.273 ||A||§

What we would like to see….

.0constant somefor

, )arcsin(][ ?

c

vucvuyxE jijiji

This is impossible because arcsin is not a linear function.

What if…

jij

ji

vcuvu

v u

c

i

)arcsin(

such that and rsunit vecto

and 0constant aexist thereSuppose

''

''

If this is possible….

jijiji

jjii

vuc

vuyxE

zvsignyzusignx

2

)arcsin(2

][

)( ),(

''''

''''

Recall z is the random unit vector.

This is indeed possible!

jij

ji

vcuvu

v u

c

i

)arcsin(

such that and rsunit vecto

exist there),21ln(With

''

''

).sin( that Note ''jij vcuvu

i

Another Semidefinite Program

jvv

iuu

jivcuvu

jj

jij

ii

i

,1

,1

, ),sin(

''

''

''

Better Constant Approximation

.

''''

||||56.0

||||2

2

)arcsin(2

][

A

Ac

vuac

vuayxaE

SDP

jiij

ij

jiij

ijjiij

ij

Road Map

• Motivation

• Hardness Result

• General Approach

• Outline of Algorithm

• Conclusion

Main Ideas

• Semidefinite Program Relaxation

- a powerful tool for optimization problems

• Randomized Rounding Scheme

- random hyperplane

- multi-dimensional Gaussian

• Apply similar techniques directly to approximate MAX-CUT