l1-embeddings and algorithmic applications

`1-embeddings and algorithmic applications

Grigory Yaroslavtsev(proofs from “The design of approximation algorihms”

by Williamson and Shmoys)

Pennsylvania State University

March 12, 2012

Grigory Yaroslavtsev (PSU) March 12, 2012 1 / 17

Metric embeddings and tree metrics

A finite metric space is a pair (V , d), where V is a set of n points andd : X × X → R+ is a distance function (three axioms).

A metric embedding of (V , d) is a metric space (V ′, d ′), such thatV ⊆ V ′ and for all u, v ∈ V we have du,v ≤ d ′u,v .

Distortion = maxu,v∈V

d ′u,v/du,v .

A tree metric is a shortest path metric in a tree.

Theorem (Fakcharoenphol, Rao, Talwar)

Given a distance metric (V , d), there is a randomized polynomial-timealgorithm that produces a tree metric (V ′,T ), V ⊆ V ′, such that for allu, v ∈ V , duv ≤ Tu,v and E[Tuv ] ≤ O(log n)duv .


Metric embeddings and tree metrics

Theorem (Fakcharoenphol, Rao, Talwar)

Given a distance metric (V , d), there is a randomized polynomial-timealgorithm that produces a tree metric (V ′,T ), V ⊆ V ′, such that for allu, v ∈ V , duv ≤ Tu,v and E[Tuv ] ≤ O(log n)duv .

With a single tree Ω(n) distortion for a cycle (Steiner vertices don’thelp).

Distribution on trees [Alon, Karp, Peleg, West]: O(2√

log n log logn).

With Steiner points [Bartal]: O(log n log log n).

Lower bound for any tree metric [Bartal]: Ω(log n).

With `1-embeddable metrics (more general), distributions and Steinerpoints are not needed.


Embeddings into Rk and `2-embeddings

Embedding of (V , d) into (Rk , `p): d`p(x , y) =(∑k

i=1 |xi − yi |p)1/p

.

Some facts about `2-embeddings:

If (V , d) is exactly `2-embeddable ⇒ it is exactly `p-embeddable for1 ≤ p ≤ ∞.

Distortion: O(log n) [Bourgain’85] (dimension n is enough).

Minimum distortion embedding can be computed via SDP.

Lower bound Ω(log n) via dual SDP (for expander graphs).

Dimension reduction: n-point `2-metric can be embedded into

RO(

log n

ε2

)with distortion 1 + ε [Johnson, Lindenstrauss ’84].

Dimension above is optimal ([Jayram, Woodruff, SODA’11]).

Multiple applications.


`1-embeddings

Some facts about `1-embeddings:

Embedding with distortion O(log n) and dimension O(log2 n) (later).

JL-like dimension reduction impossible [Brinkman, Charikar; Lee,Naor]: for distortion D dimension nΩ(1/D2) is needed.

Any tree metric is `1-embeddable, converse is false.

Representable as a convex combination of cut metrics (later).


`1-embeddings and cut metrics

Definition (Cut metric)

For S ⊆ V , a cut metric is χS(u, v) = 1 if |u, v ∩ S | = 1, otherwiseχS(u, v) = 0.

Lemma

If (V , d) is an `1-embeddable metric with an embedding f , then thereexist λS ≥ 0 for all S ⊆ V such that for all u, v ∈ V ,

‖f (u)− f (v)‖1 =∑S⊆V

λS · χS(u, v)

If f is an embedding into Rm then ≤ mn of the λS are non-zero.


`1-embeddings and cut metrics

If (V , d) is an `1-embeddable metric with an embedding f , then thereexist λS ≥ 0 for all S ⊆ V such that for all u, v ∈ V ,

‖f (u)− f (v)‖1 =∑S⊆V

λS · χS(u, v)

If f is an embedding into Rm then ≤ mn of the λS are non-zero.

Proof.

If m = 1, then f embeds V into n points on a line.

Let xi = f (i) and assume that x1 ≤ · · · ≤ xn.Consider cuts Si = 1, . . . , i.Let λSi = xi+1 − xi , then |xi − xj | =

∑j−1k=i λSk

.

|xi − xj | =∑n−1

k=1 λSkχSk

(i , j).

If m > 1, do the same for each coordinate separately ⇒ ≤ mnnon-zero λS , which can be computed efficiently.


Computing an `1-embedding

Theorem (Bourgain; Linial, London, Rabinovich)

Any metric (V , d) embeds into `1 with distortion O(log n). The

embedding f : V → RO(log2 n) can be computed w.h.p. in polynomial time.

Theorem (Aumann, Rabani; Linial, London, Rabinovich)

Given a metric (V , d) and k pairs of terminals si , ti ∈ V , we can compute

in polynomial time an embedding f : V → RO(log2 k) such that w.h.p:

1 ‖f (u)− f (v)‖1 ≤ r · O(log k) · duv , for all u, v ∈ V ,

2 ‖f (si )− f (ti )‖1 ≥ r · dsi ti , for all 1 ≤ i ≤ k,

for some r > 0.

Second theorem is more general ⇒ O(log k) approximation for sparsestcut (later today).


Frechet embedding

Definition (Frechet embedding)

For a metric space (V , d) and p subsets A1, . . . ,Ap ⊆ V a Frechetembedding f : V → Rp is defined for all u ∈ V as:

f (u) = (d(u,A1), . . . , d(u,Ap)) ∈ Rp,

where d(u, S) = minv∈S d(u, v) for a subset S ⊆ V .

Lemma

For a Frechet embedding f : V → Rp of (V , d), we have‖f (u)− f (v)‖1 ≤ pdu,v for all u, v ∈ V .

Proof.

For each 1 ≤ i ≤ p, we have |d(u,Ai )− d(v ,Ai )| ≤ duv .


Proof of the main theorem

Idea: pick O(log2 k) sets Aj randomly, such that w.h.p.:

‖f (si )− f (ti )‖1 = Ω(log k)dsi ti , for all (si , ti ),

then by taking r = Θ(log k) we’re done by the previous lemma.

Let size of T = ∪isi , ti be a power of two and τ = log2(2k).

Let L = q log k for some constant q.

Let At,` for 1 ≤ t ≤ τ , 1 ≤ ` ≤ L be sets of size 2k/2t , chosenrandomly with replacement from T .

We have Lτ = O(log2 k) sets.

Will show: ‖f (si )− f (ti )‖1 ≥ Ω(Ldsi ti ) = Ω(log k) · dsi ti w.h.p.



Want to show: ‖f (si )− f (ti )‖1 ≥ Ω(Ldsi ti ) w.h.p.

(Open) ball Bo(u, r) = v ∈ T |du,v<≤ r

Let rt be minimum r , such that |B(si , r)| ≥ 2t and |B(ti , r)| ≥ 2t .

Let t = minimum t, such that rt ≥ 14 dsi ti .

Will show: for any 1 ≤ ` ≤ L, 1 ≤ t ≤ t we have (w.l.o.g.):

Pr[(At` ∩ B(si , rt−1) 6= ∅) ∧ (At` ∩ Bo(ti , rt) = ∅)] ≥ const

By Chernoff:∑L

`=1 |d(si ,At`)− d(ti ,At`)| ≥ Ω(L(rt − rt−1)), w.h.p.

Because ‖f (si )− f (ti )‖1 ≥∑t

t=1

∑L`=1 |d(si ,At`)− d(ti ,At`)|, we have:

‖f (si )− f (ti )‖1 ≥t∑

t=1

Ω(L(rt − rt−1)) = Ω(Lrt) = Ω(Ldsi ti ) .



Want to show: for any 1 ≤ ` ≤ L, 1 ≤ t ≤ t we have (w.l.o.g.):

Pr[(At` ∩ B(si , rt−1) 6= ∅) ∧ (At` ∩ B(ti , rt) = ∅)] ≥ const

Let event Et` = (At` ∩ B(si , rt−1) 6= ∅) ∧ (At` ∩ B(ti , rt) = ∅).

Let G = B(si , rt−1), B = Bo(ti , rt) and A = At`.

Pr[E t`] = Pr[A ∩ B = ∅ ∧ A ∩ G 6= ∅]= Pr[A ∩ G 6= ∅|A ∩ B = ∅] · Pr[A ∩ B = ∅]≥ Pr[A ∩ G 6= ∅] · Pr[A ∩ B = ∅].

Recall, that |A| = 2τ−t , |B| < 2t and |G | ≥ 2t−1.

Pr[A ∩ B = ∅] =(

1− |B||T |)|A|≥ (1− 2τ−t)2τ−t ≥ 1

4 .

Pr[A ∩ G 6= ∅] = 1−(

1− |G ||T |)|A|≥ 1− e−|G ||A|/|T | ≥ 1− e−1/2.


Approximation for sparsest cut

Sparsest cut: given an undirected graph G (V ,E ), costs ce ≥ 0 for e ∈ Eand k pairs (si , ti ) with demands di , find S , which minimizes:

ρ(S) =

∑e∈δ(S) ce∑

i :|S∩si ,ti|=1 di.

LP relaxation (variables yi ≥ 0, 1 ≤ i ≤ k and xe ≥ 0, ∀e ∈ E ):

minimize:∑e∈E

cexe

subject to:k∑

i=1

diyi = 1,∑e∈P

xe ≥ yi ∀P ∈ Pi , 1 ≤ i ≤ k,

where Pi is the set of all si − ti paths.Grigory Yaroslavtsev (PSU) March 12, 2012 13 / 17



minimize:∑e∈E

cexe

subject to:k∑

i=1

diyi = 1,∑e∈P

xe ≥ yi ∀P ∈ Pi , 1 ≤ i ≤ k,

where Pi is the set of all si − ti paths.Intended solution: if we separate pairs D = di1 , . . . , dit with a cut S :

xe =χS(e)∑

t dit

, yi =1D(i)∑

t dit

.


Approximation for sparsest cut: rounding

Given a solution xe, define a shortest path metric dx(u, v).

Find an embedding f : (V , dx)→ RO(log2 k) with distortion O(log k).

Find ≤ O(n log2 k) values λS : ‖f (u)− f (v)‖1 =∑

S⊆V λSχS(u, v).

Return S∗, such that ρ(S∗) = minS : λS>0

ρ(S).

ρ(S∗) = minS : λS>0

∑e∈δ(S) ce∑

i : |S∩si ,ti|=1 di= min

S : λS>0

∑e∈E ceχS(e)∑i diχS(si , ti )

≤∑

S⊆V λS∑

e∈E ceχS(e)∑S⊆V λS

∑i diχS(si , ti )

=

∑e∈E ce

∑S⊆V λSχS(e)∑

i di∑

S⊆V λSχS(si , ti )

=

∑e=(u,v)∈E ce‖f (u)− f (v)‖1∑

i di‖f (si )− f (ti )‖1≤

r · O(log k)∑

e=(u,v)∈E cedx(u, v)

r ·∑

i didx(si , ti ).




minimize:∑e∈E

cexe (1)

subject to:k∑

i=1

diyi = 1, (2)∑e∈P

xe ≥ yi ∀P ∈ Pi , 1 ≤ i ≤ k, (3)

where Pi is the set of all si − ti paths.

ρ(S∗) ≤ O(log k)

∑e=(u,v)∈E cedx(u, v)∑

i didx(si , ti )

(3)

≤ O(log k)

∑e=(u,v)∈E cexe∑

i diyi

(2)= O(log k)

∑e∈E

cexe(1)

≤ O(log k)OPT .


Conclusion

What we saw today:

`1-embedding into RO(log2 n) with distortion O(log n).

O(log k)-approximation for sparsest cut.

Extensions:

Cut-tree packings, approximating cuts by trees [Racke; Harrelson,Hildrum, Rao].

Balanced sparsest cut: O(√

log n)-approximation [Arora, Rao,Vazirani].


l1-embeddings and algorithmic applications

Business

sparsest cutlp relaxation

si ti paths

ce xe ee

sparsest cut

metric space

ce dx

grigory yaroslavtsev

ce xe