local computation algorithms ning xie, csail mit based on joint works with noga alon, ronitt...

Local Computation Local Computation AlgorithmsAlgorithms

Ning XieNing Xie, CSAIL MIT, CSAIL MITBased on joint works with Based on joint works with

Noga Alon, Ronitt Rubinfeld, Gil Tamir Noga Alon, Ronitt Rubinfeld, Gil Tamir andand Shai Shai VardiVardi

Large answersLarge answers Combinatorial optimization/searchCombinatorial optimization/search

scheduling, coloring, SAT, LP, etcscheduling, coloring, SAT, LP, etc Coding theoreticCoding theoretic Linear algebraLinear algebra

BUT, only need small portion of answer?BUT, only need small portion of answer?

Large data sets

and large solutions

Example problemExample problem

Maximal independent setsMaximal independent sets

given a sequence vgiven a sequence v11, v, v22, , , v, vkk (assuming k<<n), which are in an (assuming k<<n), which are in an MIS? MIS?

All answers must be consistent!

In general there are

many different

legal solutions…

Previous and related Previous and related workwork

Locally decodable Locally decodable codes (codes (Katz-TrevisanKatz-Trevisan))

Online reconstruction Online reconstruction ((Ailon, Chazelle, Kale, Ailon, Chazelle, Kale, Liu, Peres, Saks, Liu, Peres, Saks, Seshadhri)

Distributed Distributed computing (local computing (local algorithms)algorithms)

Local Computation Algorithms

Rest of the talkRest of the talk

1. Model of local computation 1. Model of local computation algorithms (LCA)algorithms (LCA)

2. An LCA for maximal independent 2. An LCA for maximal independent set set

3. An LCA for hypergraph coloring 3. An LCA for hypergraph coloring

Part 1: Local Computation Part 1: Local Computation Algorithms (LCAs)Algorithms (LCAs)

F: a computation problem F: a computation problem input x (|x|=n) input x (|x|=n) set of solutions is F(x)={yset of solutions is F(x)={y11, y, y22, …, y, …, yss} }

Want: a randomized algorithm Want: a randomized algorithm A A that that implements the oracle access to implements the oracle access to somesome y yii

y1

y2

ys

Local computation Local computation algorithmalgorithm

Algorithm A

input tape (RAM)

random tape (RAM)

work tape

Local computation Local computation algorithmalgorithm

Algorithm A

input tape (RAM)

random tape (RAM)

work tape

i1

Local computation Local computation algorithmalgorithm

Algorithm A

input tape (RAM)

random tape (RAM)

work tape

i1 yi1

Local computation Local computation algorithmalgorithm

Algorithm A

input tape (RAM)

random tape (RAM)

work tape

i1 yi1

i2

Local computation Local computation algorithmalgorithm

Algorithm A

input tape (RAM)

random tape (RAM)

work tape

i1 yi1

i2 yi2

Local computation Local computation algorithmalgorithm

Algorithm A

input tape (RAM)

random tape (RAM)

work tape

i1 yi1

i2 yi2...

...

Local computation Local computation algorithmalgorithm

Algorithm A

input tape (RAM)

random tape (RAM)

work tape

i1 yi1

i2 yi2...

...

iq

Local computation Local computation algorithmalgorithm

Algorithm A

input tape (RAM)

random tape (RAM)

work tape

i1 yi1

i2 yi2...

...

iq yiq

Local computation algorithms Local computation algorithms (cont.)(cont.)

For any sequence of queries iFor any sequence of queries i11, i, i22, ..., , ..., iiqq, , AA can answer y can answer yiijj

with with

at most t(n) time at most t(n) time at most s(n) spaceat most s(n) space

All queries are consistent: there All queries are consistent: there exists some yexists some y22 F(x) that agrees F(x) that agrees with all answerswith all answers

AA is correct for all queries w.p. 1- is correct for all queries w.p. 1-δδ(n)(n)AA is a (t(n),s(n), is a (t(n),s(n),δδ(n)) local computation (n)) local computation algorithmalgorithm

Would like LCAs to have…Would like LCAs to have…

Both t(n) and s(n) are at most poly-Both t(n) and s(n) are at most poly-logarithmiclogarithmic

Support parallelism Support parallelism Query order obliviousQuery order oblivious

A generic LCAA generic LCA

Generate a random string r Generate a random string r On query i, do the following On query i, do the following

deterministicallydeterministically read some bits of input x and random read some bits of input x and random

string r string r compute ycompute yii

Part 2: Maximal Part 2: Maximal Independent SetIndependent Set

Input: An undirected graph G(V, E)Input: An undirected graph G(V, E) Output: A subset of vertices which is Output: A subset of vertices which is

a maximal independent set (MIS) a maximal independent set (MIS) Query: is v in MIS?Query: is v in MIS?

11stst ingredient: Parnas- ingredient: Parnas-RonRon

Local distributed approx. algorithm Local distributed approx. algorithm sublinear approx. algorithmsublinear approx. algorithm

Query and (usually) time complexity: Query and (usually) time complexity: O(dO(dkk) ) d: max. degree of graphd: max. degree of graph k: max. rounds of the distributed k: max. rounds of the distributed

algorithmalgorithm

22ndnd ingredient: Luby’s ingredient: Luby’s distributed algorithm for distributed algorithm for

MISMIS

repeat O(logn) times in parallelrepeat O(logn) times in parallel each vertex selects itself w.p. 1/2deach vertex selects itself w.p. 1/2d if v selects itself and none in N(v) is if v selects itself and none in N(v) is

selectedselected add v to MISadd v to MIS remove v and N(v) from the graphremove v and N(v) from the graph

Combining the first 2 Combining the first 2 ingredientsingredients

Luby+Parnas-Ron Luby+Parnas-Ron n nO(1)O(1) running running timetime But we want poly-logarithmic…But we want poly-logarithmic…

Run Luby’s algorithm a small num. Run Luby’s algorithm a small num. of rounds [of rounds [Marko-RonMarko-Ron] ] Large independent set but not Large independent set but not

maximalmaximal……

33rdrd ingredient: Beck’s ingredient: Beck’s ideaidea

[[Beck’91Beck’91] Algorithmic approach to the ] Algorithmic approach to the Lovasz Local LemmaLovasz Local Lemma First run randomized algorithm to get a First run randomized algorithm to get a

“partially good” solution“partially good” solution Show that remaining problems are small Show that remaining problems are small

enough for brute-force searchenough for brute-force search

Two-phase algorithmTwo-phase algorithm

Phase 1: run Luby’s algorithm for Phase 1: run Luby’s algorithm for O(d log d) roundsO(d log d) rounds Hope: remaining graph has only small Hope: remaining graph has only small

componentscomponents Phase 2: run greedy (LFMIS) on the Phase 2: run greedy (LFMIS) on the

connected components in which vconnected components in which vii lieslies

Original graphOriginal graph

After Phase 1After Phase 1

Running Phase 2Running Phase 2

Running Phase 2Running Phase 2

After Phase 2After Phase 2

Why fast? Phase 1Why fast? Phase 1 Apply Parnas-Ron reduction to this Apply Parnas-Ron reduction to this

locallocal distributed algorithm distributed algorithm Number of rounds is O(d·log d) (instead of Number of rounds is O(d·log d) (instead of

O(log n))O(log n)) Running time of Phase 1 is dRunning time of Phase 1 is dO(d·log d)O(d·log d)

Why fast? Phase 2Why fast? Phase 2

Main LemmaMain Lemma: With prob. at least 1-: With prob. at least 1-1/n, after running Luby’s algorithm 1/n, after running Luby’s algorithm O(d·log d) rounds, all connected O(d·log d) rounds, all connected components of surviving vertices are components of surviving vertices are of size O(poly(d)·log n)of size O(poly(d)·log n) Proof: “Beck-like” analysisProof: “Beck-like” analysis Running time of Phase 2 is O(poly(d)·log Running time of Phase 2 is O(poly(d)·log

n) n)

What about consistency?What about consistency? Remember all previous random bits used in Remember all previous random bits used in

Luby’s algorithm (each vertex, each round)Luby’s algorithm (each vertex, each round) Naïve implementation: linear spaceNaïve implementation: linear space

But our algorithm is local…But our algorithm is local… O(log n)-wise independence suffices for O(log n)-wise independence suffices for

BeckBeck ((Alon-Babai-ItaiAlon-Babai-Itai) k-wise independent ) k-wise independent

random variables with seed length O(k·log random variables with seed length O(k·log n)n)

4th ingredient: k-wise 4th ingredient: k-wise independenceindependence

Making the space Making the space complexity smallcomplexity small

Put everything Put everything together…together…

TheoremTheorem: For any : For any degree-boundeddegree-bounded graph, there is an (O(log n), graph, there is an (O(log n), O(logO(log22n), 1/n) local computation n), 1/n) local computation algorithm for MIS.algorithm for MIS.

Part 3: Hypergraph Part 3: Hypergraph coloringcoloring

Hypergraph H=(V, E)Hypergraph H=(V, E) V: finite set of verticesV: finite set of vertices E: a family of subsets of VE: a family of subsets of V (called edge) (called edge) H is H is k-uniformk-uniform (every edge is of size k) (every edge is of size k)

every edge every edge intersects at most dintersects at most d other other edgesedges

Size of the problem: N=# of edgesSize of the problem: N=# of edges H is H is 2-colorable2-colorable if we can color all the if we can color all the

vertices vertices redred and and blueblue so that no edge is so that no edge is monochromaticmonochromatic

A 3-regular hypergraphA 3-regular hypergraph

11stst ingredient: Alon’s ingredient: Alon’s algorithmalgorithm

((Beck’91, Alon’91Beck’91, Alon’91) Under certain ) Under certain conditions, there is a quasi-linear conditions, there is a quasi-linear algorithm that finds a two-coloring algorithm that finds a two-coloring of a hypergraphof a hypergraph

The algorithm runs in 2 (or 3) The algorithm runs in 2 (or 3) phases, depending on performance phases, depending on performance requirementrequirement

Overview of Alon’s Overview of Alon’s algorithmalgorithm

Phase 1 coloringPhase 1 coloring Uniformly at random color all vertices Uniformly at random color all vertices

sequentiallysequentially An edge becomes An edge becomes dangerousdangerous if too many of its if too many of its

vertices colored monochromaticallyvertices colored monochromatically Save all the rest uncolored vertices in that edgeSave all the rest uncolored vertices in that edge An edge is called An edge is called survivedsurvived if it does not have both if it does not have both

colors at the end of Phase 1colors at the end of Phase 1 Phase 2 coloringPhase 2 coloring brute-force search for a 2-coloring for each brute-force search for a 2-coloring for each

connected component of survived edgesconnected component of survived edges

Phase 1 coloringPhase 1 coloring

Phase 1 coloringPhase 1 coloring

Phase 1 coloringPhase 1 coloring

Phase 1 coloringPhase 1 coloring

Phase 1 coloringPhase 1 coloring

Phase 2 coloringPhase 2 coloring

Phase 2 coloringPhase 2 coloring

Overview of Alon’s Overview of Alon’s algorithmalgorithm

Key Lemma:Key Lemma: At the end of Phase 1, At the end of Phase 1, almost surely all connected almost surely all connected components of survived edges are of components of survived edges are of size O(log N)size O(log N)

A simple observationA simple observation

Alon’s algorithm works for any Alon’s algorithm works for any ordering of the verticesordering of the vertices

So we can simply take the order that So we can simply take the order that queries arrive!queries arrive!

Are we done?Are we done?

Space complexity can be linear as we Space complexity can be linear as we need to “remember” all previous answersneed to “remember” all previous answers

Not query-obliviousNot query-oblivious

New algorithmNew algorithm

Run Alon’s algorithm Run Alon’s algorithm Instead of using query order to color Instead of using query order to color

vertices vertices Use random ranking order [Use random ranking order [Nguyen-Nguyen-

OnakOnak]] Recursively color only vertices that we have Recursively color only vertices that we have

to to

2nd ingredient: Nguyen-Onak

Randomly assign a rank r(v)Randomly assign a rank r(v)22[0,1] [0,1] for each vertexfor each vertex

v’s color depends only on color’s of v’s color depends only on color’s of neighbors with lower rankneighbors with lower rank

In In expectationexpectation, only need O(e, only need O(edd) ) recursive callsrecursive calls But we need worst case boundBut we need worst case bound

Coloring dependencyColoring dependency

Coloring dependencyColoring dependency

Coloring dependencyColoring dependency

Query treeQuery tree

Query treeQuery tree

0.7

0.60.4 0.8

Query treeQuery tree

0.7

0.40.6

Query treeQuery tree

0.7

0.20.5 0.5 0.6 0.10.8

0.40.6

Query treeQuery tree

0.7

0.20.5 0.1

0.40.6

Bounding the size of Bounding the size of query treequery tree

Query tree has bounded degreeQuery tree has bounded degree H is k-uniform and each edge intersects H is k-uniform and each edge intersects

at most d other edges at most d other edges max. degree max. degree D=kd D=kd

LemmaLemma: For any vertex v, with prob. : For any vertex v, with prob. at least 1-1/Nat least 1-1/N22, the query tree rooted , the query tree rooted at v has size at most c(D)(log N)at v has size at most c(D)(log N)D+1 D+1 , , where c(D) is some constantwhere c(D) is some constant

Proof of the lemmaProof of the lemma

Partition the tree into (D+1) levels Partition the tree into (D+1) levels based on the values of their random based on the values of their random ranks ranks

Upper bound the sizes of trees on Upper bound the sizes of trees on each leveleach level Apply results in branching processes Apply results in branching processes

theory (Galton-Waton process)theory (Galton-Waton process)

Obs. 1Obs. 1: the exact values of random ranks : the exact values of random ranks do not matter, only relative orderings dodo not matter, only relative orderings do

Obs. 2Obs. 2: our computations are local, so : our computations are local, so limited independence sufficeslimited independence suffices

DefinitionDefinition: An ordering function [N] : An ordering function [N] NN is is k-wise independentk-wise independent if for any index if for any index subset of size k, all the k! relative subset of size k, all the k! relative orderings of these k integers happen orderings of these k integers happen with equal prob.with equal prob.

3rd ingredient: k-wise independent 3rd ingredient: k-wise independent orderingsorderings

What about space complexity?What about space complexity?

Reducing the space Reducing the space complexity complexity

Construction of k-wise independent Construction of k-wise independent ordering (seed length=polylog(N))ordering (seed length=polylog(N))

Replace true random strings with k-Replace true random strings with k-wise independent random strings for wise independent random strings for the random colors of vertices (k=O(log the random colors of vertices (k=O(log n), seed length=polylog(N))n), seed length=polylog(N))

total randomness is at most poly-

logarithmic

Put everything Put everything together…together…

TheoremTheorem: under certain conditions, : under certain conditions, there is a (polylog n, polylog n, 1/n)-there is a (polylog n, polylog n, 1/n)-local computation algorithm for local computation algorithm for hypergraph coloring.hypergraph coloring.

ConclusionsConclusions

Propose the notion of local Propose the notion of local computation algorithms (LCAs)computation algorithms (LCAs)

Develop some techniques for Develop some techniques for designing LCAsdesigning LCAs

Open question: LCAs for more Open question: LCAs for more problems?problems?

Thank You!