![Page 1: Approximability & Sums of Squares Ryan O’Donnell Carnegie Mellon](https://reader034.vdocuments.net/reader034/viewer/2022051316/56649c745503460f9492672f/html5/thumbnails/1.jpg)
Approximability& Sums of Squares
Ryan O’DonnellCarnegie Mellon
![Page 2: Approximability & Sums of Squares Ryan O’Donnell Carnegie Mellon](https://reader034.vdocuments.net/reader034/viewer/2022051316/56649c745503460f9492672f/html5/thumbnails/2.jpg)
Basic Optimization Problems
Minimum-Balanced-Separator:
Given G=(V,E), partition V into 2 parts,each of size at least n/3,
minimize # of edges crossing partition.
![Page 3: Approximability & Sums of Squares Ryan O’Donnell Carnegie Mellon](https://reader034.vdocuments.net/reader034/viewer/2022051316/56649c745503460f9492672f/html5/thumbnails/3.jpg)
Basic Optimization Problems
Minimum-Balanced-Separator:
Minimum-Vertex-Cover:
Given G=(V,E), partition V into 2 parts,each of size at least n/3,
minimize # of edges crossing partition.
Given G=(V,E), choose the smallestsubset S ⊆ V such that each edge touches S.
![Page 4: Approximability & Sums of Squares Ryan O’Donnell Carnegie Mellon](https://reader034.vdocuments.net/reader034/viewer/2022051316/56649c745503460f9492672f/html5/thumbnails/4.jpg)
Both are NP-hard
n-vbl 3SATformula F
O(n)-vtxgraph G
poly(n) time
, β
F satisfiable
F unsatisfiable
⇒
⇒
Min-BS(G) = β
Min-BS(G) > β
Distinguishing requires* at least
2Ω(n) time.⇒
Distinguishing requires* at least
2Ω(n) time.
![Page 5: Approximability & Sums of Squares Ryan O’Donnell Carnegie Mellon](https://reader034.vdocuments.net/reader034/viewer/2022051316/56649c745503460f9492672f/html5/thumbnails/5.jpg)
Approximate Optimization
“C-approximation algorithm”
Guaranteed to find a solution with value at most C times the minimum.
“C-certification algorithm”
• Output form: “I certify the minimum is ≥ α”.• Must always be correct.• Guaranteed that α ≥ (true minimum) / C.
Stronger
![Page 6: Approximability & Sums of Squares Ryan O’Donnell Carnegie Mellon](https://reader034.vdocuments.net/reader034/viewer/2022051316/56649c745503460f9492672f/html5/thumbnails/6.jpg)
Minimum Balanced-Separator
Is there a 1.01-approximationalgorithm running in O(n) time?
Is there a 10000-certification
algorithm running in 2n.99 time?
DON’TKNOW
DON’TKNOW
[AMS’11]: Cannot* 1.0000000000000001-certify in poly(n) time.
![Page 7: Approximability & Sums of Squares Ryan O’Donnell Carnegie Mellon](https://reader034.vdocuments.net/reader034/viewer/2022051316/56649c745503460f9492672f/html5/thumbnails/7.jpg)
![Page 8: Approximability & Sums of Squares Ryan O’Donnell Carnegie Mellon](https://reader034.vdocuments.net/reader034/viewer/2022051316/56649c745503460f9492672f/html5/thumbnails/8.jpg)
Minimum Vertex-Cover
Can 2-approximate in linear time.
Cannot* 1.17-certify even in 2n.99999 time.
Cannot* 1.36-certify even in 2n.000001 time.
Can you 1.5-certify in polynomial time?
DON’TKNOW
![Page 9: Approximability & Sums of Squares Ryan O’Donnell Carnegie Mellon](https://reader034.vdocuments.net/reader034/viewer/2022051316/56649c745503460f9492672f/html5/thumbnails/9.jpg)
How could you show that you can’t 1.5-certify Min-VC in poly time?
n-vbl 3SATformula F
O(n)-vtxgraph G
poly(n) time
, β
F satisfiable
F unsatisfiable
⇒⇒
Min-VC(G) ≤ β
Min-VC(G) > 1.5 β
This would show 1.5-certifying Min-VCrequires* superpolynomial time.
DON’TKNOWHOW
![Page 10: Approximability & Sums of Squares Ryan O’Donnell Carnegie Mellon](https://reader034.vdocuments.net/reader034/viewer/2022051316/56649c745503460f9492672f/html5/thumbnails/10.jpg)
How could you show that you can’t 1.5-certify Min-VC in poly time?
give evidence that
Show that known powerful poly-timeoptimization techniques fail to do it.
![Page 11: Approximability & Sums of Squares Ryan O’Donnell Carnegie Mellon](https://reader034.vdocuments.net/reader034/viewer/2022051316/56649c745503460f9492672f/html5/thumbnails/11.jpg)
Prehistory: Linear programming can’t 1.999999-certify Min-VC.
[ABL’02]: Lovász-Schrijverd Super-LP
can’t 1.999999-certify Min-VC.
[GK’95]: Semidefinite programming can’t 1.999999-certify Min-VC.
[GMPT’07]: Lovász-Schrijverd Super-SDP
can’t 1.999999-certify Min-VC.
[BCGM’10]: Sherali-Adamsd Super-Duper-SDP
can’t 1.999999-certify Min-VC.
+++
nO(d) time
![Page 12: Approximability & Sums of Squares Ryan O’Donnell Carnegie Mellon](https://reader034.vdocuments.net/reader034/viewer/2022051316/56649c745503460f9492672f/html5/thumbnails/12.jpg)
[KS’09], [RS’09]: Sherali-Adamsd Super-Duper-SDP
can’t 10000-certify Min-Bal-Sep.
For Min-Balanced-Separator, a similar situation:
![Page 13: Approximability & Sums of Squares Ryan O’Donnell Carnegie Mellon](https://reader034.vdocuments.net/reader034/viewer/2022051316/56649c745503460f9492672f/html5/thumbnails/13.jpg)
Prehistory: Linear programming can’t 1.999999-certify Min-VC.
I.e., there are graphs G on n vertices such that:
• Min-VC(G) ≥ .999999 n• LP(G) = “I certify Min-VC(G) ≥ .500001 n”
α = minimize: ∑v∈V Xv
subject to: Xv ∈ {0,1} for all v∈V
Xu + Xv ≥ 1 for all (u,v)∈E
[0,1]
LP certif. alg. for Min-VC outputs α, where
![Page 14: Approximability & Sums of Squares Ryan O’Donnell Carnegie Mellon](https://reader034.vdocuments.net/reader034/viewer/2022051316/56649c745503460f9492672f/html5/thumbnails/14.jpg)
I.e., there are graphs G on n vertices such that:
• Min-VC(G) ≥ .999999 n
• SAd(G) = “I certify Min-VC(G) ≥ .500001 n”
[BCGM’10]: Sherali-Adamsd Super-Duper-SDP
can’t 1.999999-certify Min-VC.
Specifically, this is true for “Frankl-Rödl graphs” [FR’87]:
V = {0,1}m, E = {(x,y) : ∆(x,y)=.999 m}
![Page 15: Approximability & Sums of Squares Ryan O’Donnell Carnegie Mellon](https://reader034.vdocuments.net/reader034/viewer/2022051316/56649c745503460f9492672f/html5/thumbnails/15.jpg)
I.e., there are graphs G on n vertices such that:
• Min-BS(G) ≥ β
• SAk(G) = “I certify Min-BS(G) ≥ ”.
Specifically, this is true for “Khot-Vishnoi graphs” [KV’05].
[KS’09], [RS’09]: Sherali-Adamsd Super-Duper-SDP
can’t 10000-certify Min-Bal-Sep.
![Page 16: Approximability & Sums of Squares Ryan O’Donnell Carnegie Mellon](https://reader034.vdocuments.net/reader034/viewer/2022051316/56649c745503460f9492672f/html5/thumbnails/16.jpg)
These are tough instances.
We, the mathematicians, can analyze their opt.
But our strongest poly-time algorithms cannot.
![Page 17: Approximability & Sums of Squares Ryan O’Donnell Carnegie Mellon](https://reader034.vdocuments.net/reader034/viewer/2022051316/56649c745503460f9492672f/html5/thumbnails/17.jpg)
Actually…
![Page 18: Approximability & Sums of Squares Ryan O’Donnell Carnegie Mellon](https://reader034.vdocuments.net/reader034/viewer/2022051316/56649c745503460f9492672f/html5/thumbnails/18.jpg)
There is one more algorithm…
![Page 19: Approximability & Sums of Squares Ryan O’Donnell Carnegie Mellon](https://reader034.vdocuments.net/reader034/viewer/2022051316/56649c745503460f9492672f/html5/thumbnails/19.jpg)
It’s even stronger, but hard to analyze…
![Page 20: Approximability & Sums of Squares Ryan O’Donnell Carnegie Mellon](https://reader034.vdocuments.net/reader034/viewer/2022051316/56649c745503460f9492672f/html5/thumbnails/20.jpg)
The “Lasserred Super-Duper-Ultra-SDP”…
![Page 21: Approximability & Sums of Squares Ryan O’Donnell Carnegie Mellon](https://reader034.vdocuments.net/reader034/viewer/2022051316/56649c745503460f9492672f/html5/thumbnails/21.jpg)
Also known as…
SOSd
nO(d) time
![Page 22: Approximability & Sums of Squares Ryan O’Donnell Carnegie Mellon](https://reader034.vdocuments.net/reader034/viewer/2022051316/56649c745503460f9492672f/html5/thumbnails/22.jpg)
Our Results
[OZ’13]: SOS4 is a C-certification algorithm
(for some small C, maybe 5)for Min-BS on Khot-Vishnoi graphs.
SOSd is also pretty good for Max-Cut
on Khot-Vishnoi graphs.
[KOTZ’13]: SOSd is essentially a 1-certif.
alg. for Min-VC on all but the‘hardest’ Frankl-Rödl graphs.
![Page 23: Approximability & Sums of Squares Ryan O’Donnell Carnegie Mellon](https://reader034.vdocuments.net/reader034/viewer/2022051316/56649c745503460f9492672f/html5/thumbnails/23.jpg)
So your whole result is that
one particular algorithm
does well on one particular
instance?
![Page 24: Approximability & Sums of Squares Ryan O’Donnell Carnegie Mellon](https://reader034.vdocuments.net/reader034/viewer/2022051316/56649c745503460f9492672f/html5/thumbnails/24.jpg)
An Old Joke
Q: Why did the complexity theorist work on algorithms?
A: To get lower bounds on his lower bounds.
SOSd is a dozen years old, but hard to analyze.
The Dream: it’s great certification alg. not justfor these known hard graphs, but for all graphs.
![Page 25: Approximability & Sums of Squares Ryan O’Donnell Carnegie Mellon](https://reader034.vdocuments.net/reader034/viewer/2022051316/56649c745503460f9492672f/html5/thumbnails/25.jpg)
Our Inspiration:
STOC’12 paper of Barak, Brandão, Harrow, Kelner, Steurer, and Zhou.
• Showed SOS4 is good certification alg.
on known hard instances of “Unique-Games”.
• Somewhat demystified analysis of SOSd.
![Page 26: Approximability & Sums of Squares Ryan O’Donnell Carnegie Mellon](https://reader034.vdocuments.net/reader034/viewer/2022051316/56649c745503460f9492672f/html5/thumbnails/26.jpg)
So what is SOSd?
![Page 27: Approximability & Sums of Squares Ryan O’Donnell Carnegie Mellon](https://reader034.vdocuments.net/reader034/viewer/2022051316/56649c745503460f9492672f/html5/thumbnails/27.jpg)
“Min-Balanced-Separator(G) > α”
⇔
has no real solutions”
“
![Page 28: Approximability & Sums of Squares Ryan O’Donnell Carnegie Mellon](https://reader034.vdocuments.net/reader034/viewer/2022051316/56649c745503460f9492672f/html5/thumbnails/28.jpg)
infeasibility certificate:
identity −1 = Q0 + Q1P1 + Q2P2+ ••• +QmPm
where each Qi is a “sum of squares”:
Qi = Ri12 + ••• + Rik
2
![Page 29: Approximability & Sums of Squares Ryan O’Donnell Carnegie Mellon](https://reader034.vdocuments.net/reader034/viewer/2022051316/56649c745503460f9492672f/html5/thumbnails/29.jpg)
Positivstellensatz
Subject to some mild technical conditions,every infeasible system has such a certificate.
Caveat: Qi’s might need to have high degree.
SOSd algorithm: [Shor’87,Lasserre’00,Parrilo’00]
If there exists an infeasibility certificate
where all the Qi’s have degree ≤ d,
finds it in time nO(d).
![Page 30: Approximability & Sums of Squares Ryan O’Donnell Carnegie Mellon](https://reader034.vdocuments.net/reader034/viewer/2022051316/56649c745503460f9492672f/html5/thumbnails/30.jpg)
E.g.: SOSd for Min-VC(G)
“Min-VC(G) > α” ⇔
Xv2 = Xv for all v∈V,
Xu+Xv ≥ 1 for all (u,v)∈E,
∑v Xv ≤ α
infeasible
−1 = Q0 + Q1 (α−∑ Xv) + ∑ Quv (Xu+Xv−1) + •••
existence of sum-of-squares Q’s such that
Find largest α such that degree-d Q’s exist.
⇐
![Page 31: Approximability & Sums of Squares Ryan O’Donnell Carnegie Mellon](https://reader034.vdocuments.net/reader034/viewer/2022051316/56649c745503460f9492672f/html5/thumbnails/31.jpg)
Our Results
[OZ’13]: SOS4 is a C-certification algorithm
(for some small C, maybe 5)for Min-BS on Khot-Vishnoi graphs.
I.e., for Khot-Vishnoi graphs G, there are degree-4 SOS Q’s certifying
“Min-Bal-Sep(G) > α” for some α > (true Min-Bal-Sep) / C.
![Page 32: Approximability & Sums of Squares Ryan O’Donnell Carnegie Mellon](https://reader034.vdocuments.net/reader034/viewer/2022051316/56649c745503460f9492672f/html5/thumbnails/32.jpg)
One Slide How-ToThm: Min-VC in this graph is ≥ .999nProof: … vertex isoperimetry…
… inductive argument…
Thm: Min-BS in this graph is ≥ blahProof: … hypercontractivity…
“Check out these polynomials.”
“Check out these polynomials.”
![Page 33: Approximability & Sums of Squares Ryan O’Donnell Carnegie Mellon](https://reader034.vdocuments.net/reader034/viewer/2022051316/56649c745503460f9492672f/html5/thumbnails/33.jpg)
Tiny Taste
A bit of the analysis for Max-Cut:
Lemma: Let a,b,c ∈ {−1,1}. If a ≠ c then either a ≠ b or b ≠ c.
Formalization with polynomials:
SOS Proof:
![Page 34: Approximability & Sums of Squares Ryan O’Donnell Carnegie Mellon](https://reader034.vdocuments.net/reader034/viewer/2022051316/56649c745503460f9492672f/html5/thumbnails/34.jpg)
Open Problems
Can you give an SOS proof of…
• Vertex Isoperimetric Theorem in {0,1}n:
If A, B ⊆ {0,1}n, |A|,|B| ≥ .1·2n,
then ∃x∈A,y∈B with ∆(x,y) ≤
• Central Limit Theorem
![Page 35: Approximability & Sums of Squares Ryan O’Donnell Carnegie Mellon](https://reader034.vdocuments.net/reader034/viewer/2022051316/56649c745503460f9492672f/html5/thumbnails/35.jpg)
Thanks!
+