2 gamarnik
TRANSCRIPT
![Page 1: 2 gamarnik](https://reader030.vdocuments.net/reader030/viewer/2022032420/55a4f1411a28ab38038b45a5/html5/thumbnails/1.jpg)
Power and limits of local algorithms for sparse
random graphs
David Gamarnik
MIT
Joint work with
David Goldberg, Theophane Weber and Madhu Sudan
Workshop on Random Graphs and Their Applications
Яндекс
October, 2013
![Page 2: 2 gamarnik](https://reader030.vdocuments.net/reader030/viewer/2022032420/55a4f1411a28ab38038b45a5/html5/thumbnails/2.jpg)
• Real life natural, technological communication networks are large
(WWW 109, neurons in a human brain 1011)
• This makes computational models traditionally deemed scalable
(polynomial time), too impractical
• Local algorithms – promising framework
• Correlation Decay – property which validates accuracy of local
algorithms
Some high level thoughts…
![Page 3: 2 gamarnik](https://reader030.vdocuments.net/reader030/viewer/2022032420/55a4f1411a28ab38038b45a5/html5/thumbnails/3.jpg)
CD arises in statistical phyisics -- uniqueness of Gibbs measures on infinite
graphs, Dobrushin [60-70s]
Correlation Decay Property
![Page 4: 2 gamarnik](https://reader030.vdocuments.net/reader030/viewer/2022032420/55a4f1411a28ab38038b45a5/html5/thumbnails/4.jpg)
Correlation Decay Property
Decisions based on local neighborhood ¼ decisions based on the entire
network:
![Page 5: 2 gamarnik](https://reader030.vdocuments.net/reader030/viewer/2022032420/55a4f1411a28ab38038b45a5/html5/thumbnails/5.jpg)
Electrical Engineering: Belief Propagation (BP) algorithm
Wainwright & Jordan [2008], Mezard & Montanari [2009]
Theoretical Computer Science:
Nguyen & Onak [2008], Rubinfeld, Tamir, Vardi & Xie [2011], Suomella
[2011]
Mathematics:
Hatami, Lovasz & Szegedy [2012], Elek & Lippner [2010], Lyons &
Nazarov [2011], Czoka & Lippner [2012], Aldous [2012].
Context – graph limits.
Local Algorithms
![Page 6: 2 gamarnik](https://reader030.vdocuments.net/reader030/viewer/2022032420/55a4f1411a28ab38038b45a5/html5/thumbnails/6.jpg)
Local algorithms and CD for combinatorial optimization.
Maximum Independent Set (MIS) problem
![Page 7: 2 gamarnik](https://reader030.vdocuments.net/reader030/viewer/2022032420/55a4f1411a28ab38038b45a5/html5/thumbnails/7.jpg)
Local algorithms and CD for combinatorial optimization.
Maximum Independent Set (MIS) problem
Berman and Karpinski [1999]. NP-hard to approximate for
![Page 8: 2 gamarnik](https://reader030.vdocuments.net/reader030/viewer/2022032420/55a4f1411a28ab38038b45a5/html5/thumbnails/8.jpg)
• Extensions to arbitrary constant degree graphs with weights which are
mixtures of exponentials.
• Proof technique – local algorithm (Cavity Expansion) and correlation decay
Maximum Weight Independent Set (MWIS) problem
![Page 9: 2 gamarnik](https://reader030.vdocuments.net/reader030/viewer/2022032420/55a4f1411a28ab38038b45a5/html5/thumbnails/9.jpg)
Computational results.
G & Yedidia [2013]
Instance I
UB: 569
LB: 568
GREEDY: 420
Instance II
UB: 542
LB: 540
GREEDY: 397
Instance III
UP: 514
LB: 513
GREEDY 392
Instance IV
UB: 499
LB: 498
GREEDY: 384
![Page 10: 2 gamarnik](https://reader030.vdocuments.net/reader030/viewer/2022032420/55a4f1411a28ab38038b45a5/html5/thumbnails/10.jpg)
Cavity Expansion Algorithm for general (non-tree) graphs
![Page 11: 2 gamarnik](https://reader030.vdocuments.net/reader030/viewer/2022032420/55a4f1411a28ab38038b45a5/html5/thumbnails/11.jpg)
Cavity Expansion
![Page 12: 2 gamarnik](https://reader030.vdocuments.net/reader030/viewer/2022032420/55a4f1411a28ab38038b45a5/html5/thumbnails/12.jpg)
Cavity Expansion
Problem: node cavity ! set cavity
![Page 13: 2 gamarnik](https://reader030.vdocuments.net/reader030/viewer/2022032420/55a4f1411a28ab38038b45a5/html5/thumbnails/13.jpg)
Solution inspired by Weitz [2006] – expand set cavity as a sum of node cavities
![Page 14: 2 gamarnik](https://reader030.vdocuments.net/reader030/viewer/2022032420/55a4f1411a28ab38038b45a5/html5/thumbnails/14.jpg)
Cavity Expansion
Theorem. Operator
is contracting – correlation decay takes place.
![Page 15: 2 gamarnik](https://reader030.vdocuments.net/reader030/viewer/2022032420/55a4f1411a28ab38038b45a5/html5/thumbnails/15.jpg)
Cavity Expansion Algorithm
1. Compute using depth-m computation tree
2. Include iff
- Computation tree
![Page 16: 2 gamarnik](https://reader030.vdocuments.net/reader030/viewer/2022032420/55a4f1411a28ab38038b45a5/html5/thumbnails/16.jpg)
Correlation decay property:
Probability of error
Computation time PTAS
![Page 17: 2 gamarnik](https://reader030.vdocuments.net/reader030/viewer/2022032420/55a4f1411a28ab38038b45a5/html5/thumbnails/17.jpg)
Generalization:Optimization on networks with random rewards Economic theory of teams
![Page 18: 2 gamarnik](https://reader030.vdocuments.net/reader030/viewer/2022032420/55a4f1411a28ab38038b45a5/html5/thumbnails/18.jpg)
Theorem. [G, Goldberg and Weber 2009]. Suppose
Then the model exhibits the correlation decay property. As a result the
Cavity Expansion algorithm provides an asymptotically optimal solution.
Generalization: Optimization on networks with random rewards
![Page 19: 2 gamarnik](https://reader030.vdocuments.net/reader030/viewer/2022032420/55a4f1411a28ab38038b45a5/html5/thumbnails/19.jpg)
Random n-node d-regular graph
Local algorithms as i.i.d. factors. Mathematicians
perspective
![Page 20: 2 gamarnik](https://reader030.vdocuments.net/reader030/viewer/2022032420/55a4f1411a28ab38038b45a5/html5/thumbnails/20.jpg)
Local algorithms – i.i.d. factors
1. Fix r>0 and
2. Generate U1,…,Un uniformly at random and apply at every node
i=1,…,n of the graph
3. Output
![Page 21: 2 gamarnik](https://reader030.vdocuments.net/reader030/viewer/2022032420/55a4f1411a28ab38038b45a5/html5/thumbnails/21.jpg)
Local algorithms – i.i.d. factors
Conjecture. Hatami, Lovasz & Szegedy [2012] Max-Independent-Set
on random regular graph problem can be solved by means of local
algorithms. Namely there exists a sequence of functions such that
Conjecture holds for Max-Matching:
Lyons & Nazarov [2011]
Abert, Csoka, Lippner & Terpai [2012]
Best known bound 0.4361 for 3-regular graph is obtained by i.i.d. factors
Csoka, Gerencsér, Harangi, & Virag [2013]
![Page 22: 2 gamarnik](https://reader030.vdocuments.net/reader030/viewer/2022032420/55a4f1411a28ab38038b45a5/html5/thumbnails/22.jpg)
Asymptotics of optimal solutions
Theorem. Bayati,G,Tetali 2010 problem exists w.h.p.
Theorem. Achlioptas & Peres [2003] K-SAT problem. Satisfiability
threshold
Theorem. Frieze & Luczak 1992
![Page 23: 2 gamarnik](https://reader030.vdocuments.net/reader030/viewer/2022032420/55a4f1411a28ab38038b45a5/html5/thumbnails/23.jpg)
Best known algorithms
Greedy algorithms for Max Ind Set problem
Greedy algorithms for K-SAT problem. Finds satisfying assignment when
the clauses to variables ratio is at most
![Page 24: 2 gamarnik](https://reader030.vdocuments.net/reader030/viewer/2022032420/55a4f1411a28ab38038b45a5/html5/thumbnails/24.jpg)
Theorem. [G & Sudan 2012] HLS conjecture is not valid. No local algorithm
can produce an independent set larger than factor of the
optimal for large enough d.
Proof relies on the Spin Glass Theory - clustering (shattering) phenomena
Main Result
![Page 25: 2 gamarnik](https://reader030.vdocuments.net/reader030/viewer/2022032420/55a4f1411a28ab38038b45a5/html5/thumbnails/25.jpg)
Mezard & Parisi [1980s]
Achlioptas & Ricci-Tersenghi [2004]
Mezard, Mora & Zecchina [2005]
Krzakala, Montanari, Ricci-Tersenghi, Semerjian, Zdeborova [2007]
Clustering Phenomena. K-SAT
![Page 26: 2 gamarnik](https://reader030.vdocuments.net/reader030/viewer/2022032420/55a4f1411a28ab38038b45a5/html5/thumbnails/26.jpg)
Clustering Phenomena. Independent Sets
Theorem. [Coja-Oghlan & Efthymiou 2010, G & Sudan 2012]
For every large enough d and every there exist
such that no two independent sets of size
have intersection size between and
![Page 27: 2 gamarnik](https://reader030.vdocuments.net/reader030/viewer/2022032420/55a4f1411a28ab38038b45a5/html5/thumbnails/27.jpg)
Proof of non-existence of local algorithms: if an algorithm exists then
one can construct two independent sets with intersection in the non-
existent region
First direct link between clustering and algorithmic hardness
![Page 28: 2 gamarnik](https://reader030.vdocuments.net/reader030/viewer/2022032420/55a4f1411a28ab38038b45a5/html5/thumbnails/28.jpg)
Proof of non-existence of local algorithms: if an algorithm exists then
one can construct two independent sets with intersection in the non-
existent region
Suppose exists such that
Construct two independent sets I and J using independent sources
U1,…,Un and V1,…,Vn
Then
![Page 29: 2 gamarnik](https://reader030.vdocuments.net/reader030/viewer/2022032420/55a4f1411a28ab38038b45a5/html5/thumbnails/29.jpg)
Proof of non-existence of local algorithms: if an algorithm exists then
one can construct two independent sets with intersection in the non-
existent region
Continuously interpolate between U1,…,Un and V1,…,Vn : let
For each vertex , let with probability p and with
probability 1-p.
Consider independent set K obtained from
K=I when p=1 and K=J when p=0.
Fact: is continuous in p. Then we obtain intersection sizes for
all points in
- contradiction.
![Page 30: 2 gamarnik](https://reader030.vdocuments.net/reader030/viewer/2022032420/55a4f1411a28ab38038b45a5/html5/thumbnails/30.jpg)
Theory is good not when it is correct but when it is interesting…
![Page 31: 2 gamarnik](https://reader030.vdocuments.net/reader030/viewer/2022032420/55a4f1411a28ab38038b45a5/html5/thumbnails/31.jpg)
Ongoing work
Local algorithms (Belief Propagation and Survey Propagation) to solve
random instance of NAE-K-SAT problem
This would refute a conjecture put forward by physicists regarding the
power of these algorithms
![Page 32: 2 gamarnik](https://reader030.vdocuments.net/reader030/viewer/2022032420/55a4f1411a28ab38038b45a5/html5/thumbnails/32.jpg)
Thank you