optimal network locality in distributed services

Optimal NetworkLocality in DistributedServicesGwendal SimonDepartment of Computer ScienceInstitut Telecom - Telecom Bretagne2010

Telecom Bretagne

Institut Telecom: graduate engineering schoolsTelecom Bretagne: 1200 students (200 PhD)Computer Science: 20 full-time research lecturers

2 / 27 Gwendal Simon Network Locality in Distributed Services

Credits

Funding:Thomson R&D (now Technicolor)French grant with Orange, NDS Tech. and INRIA

Co-authors:Jimmy Leblet (post-doc)Yiping Chen (PhD student)Zhe Li (PhD student)Gilles Straub (senior researcher Thomson)Di Yuan (Ass. Professor: Linkopping Univ. Sweden)


Service Delivery Network

CLOUD

end user

data-center CDN

in-network servers

set-top-boxset-top-box

set-top-box

set-top-box



CLOUD

end user

data-center

CDN

in-network servers


set-top-box

set-top-box



CLOUD

end user

data-center CDN

in-network servers


set-top-box

set-top-box


Toward a Decentralized Architecture

servers’ capacities scale down1

services scale up2

=⇒ multi-servers multi-component architectures3

1J. He, A. Chaintreau and C. Diot. “A performance evaluation of scalablelive video streaming with nano data centers” Computer Networks, 2009

2J. Pujol, V. Erramilli, and P. Rodriguez, “Divide and Conquer:Partitioning Online Social Networks” Arxiv preprint arXiv:0905.4918, 2009.

3R. Baeza-Yates, A. Gionis, F. Junqueira, V. Plachouras, and L. Telloli,“On the feasibility of multi-site web search engines”, in ACM CIKM 2009


Problem Modellingand Analysis


Problem Formulation (Assumptions)

About the n servers and the k components:only one component per serverno capacity boundscomponents are uniformly accessed

About the global service architecture:client requests are routed toward the closest servercharacteristics of links between servers are known

a generic distance (cost) function dĳ


Problem Formulation (Definition)

Rainbow distance for a server i :total cost to fetch all missing components

i j1

j2

j3

j4

dĳ1 = 2dĳ2

= 3dĳ3

= 4dĳ4

=5

dĳ1 = 2

dĳ4=

5d(i) = dĳ1 + dĳ4 = 7


Problem Formulation (Objective)

Global goal: assign components to servers

Optimization: minimize sum of rainbow distances∑

0<i≤nd(i)

Motivations:network operator: reduce cross-domain trafficservice provider: reduce overall latencyacademic: funny unknown problem


Problem Complexity

The problem is NP-complete:closely related with domatic partition

012

3

4

5

6

7

8 9


Problem Complexity


012

3

4

5

6

7

8 9

6

5

4


Problem Complexity


012

3

4

5

6

7

8 983

0


Problem Complexity


012

3

4

5

6

7

8 91

2

7

9


Problem Complexity


012

3

4

5

6

7

8 9

6

5

4

12

7

983

0


Integer Programming

xic =

{1 if component c is allocated at server i0 otherwise

y cĳ =

{1 if i obtains component c from j0 otherwise

Minimizen∑

i=1

k∑c=1

n∑j=1

d(i , j)y cĳ

Subject tok∑

c=1

xic = 1, only one component per server∑j 6=i

y cĳ = 1− xic , a server has c or has exactly one pointer to c

y cĳ ≤ xjc , a server has c from another server if this latter has c


Related Works


Related Works

Facility Location Problem⇒ open a subset of facilities with minimal overall cost

u1c1 = 9

u2c2 = 3

u3c3 = 5

i1 i2

3 6 8 12 11 7


Related Works


u1c1 = 9

u2c2 = 3

u3c3 = 5

i1 i2

3 6 8

12 11 7


Related Works


u1c1 = 9

u2c2 = 3

u3c3 = 5

i1 i2

3 6 8 12 11 7


Related Works

Facility Location Problemmost variants are NP-completeclose variant is k-PUFLP: a

(32k − 1

)-approx. algo4

possible transformation from our prob. to k-PUFLP

4H. C. Huang and R. Li, “A k-product uncapacitated facility locationproblem”, European Journal of Op. Res., vol. 185, no. 2, 2008.


Related Works

Facility Location Problema(32k − 1

)-approx. algo


Related Works


)-approx. algo

Content Delivery Networksk-median problem: no multiple servers


Related Works


)-approx. algo

Content Delivery Networksk-median problem: no multiple servers

Nano data centers powered by set-top-boxesuniform random allocation of components to servers


Our Algorithms


Approximate Algorithm

For a server i :1. compute distance d̄(i) to k − 1 closest servers2. wait until every server j with smaller d̄(j) are OK3. try to optimize locally −→ optimized state4. if impossible −→ saved state5. uncolored saved nodes get furthest components



1

2

3

4

5

67

8

9

10

11

12

13

14

15

16

17

18

2

1

4

5

optimized2

1

4

5

3816

optimized

2

1

4

5

3816

conflict

saved but colored

2

1

4

5

3816

11

12

13

9

15

7

conflict

saved and uncolored

2

1

4

5

3816

11

12

13

9

15

10

67

colored by node 10

2

1

4

5

3816

11

12

13

9

15

10

6

1417

7

only node uncolored

2

1

4

5

3816

11

12

13

9

15

10

6

1417

7

18

choose farthest component

sorted list nearest neighbors2 1,4,53 1,8,161 2,3,168 3,11,125 1,2,411 8,12,134 2,5,716 1,3,512 8,9,1115 1,10,1110 2,6,1514 3,16,1717 5,14,1613 11,12,157 2,4,66 2,7,109 8,12,1418 4,5,17



1

2

3

4

5

67

8

9

10

11

12

13

14

15

16

17

18

2

1

4

5

optimized

2

1

4

5

3816

optimized

2

1

4

5

3816

conflict

saved but colored

2

1

4

5

3816

11

12

13

9

15

7

conflict

saved and uncolored

2

1

4

5

3816

11

12

13

9

15

10

67

colored by node 10

2

1

4

5

3816

11

12

13

9

15

10

6

1417

7

only node uncolored

2

1

4

5

3816

11

12

13

9

15

10

6

1417

7

18





1

2

3

4

5

67

8

9

10

11

12

13

14

15

16

17

18

2

1

4

5

optimized2

1

4

5

3816

optimized

2

1

4

5

3816

conflict

saved but colored

2

1

4

5

3816

11

12

13

9

15

7

conflict

saved and uncolored

2

1

4

5

3816

11

12

13

9

15

10

67

colored by node 10

2

1

4

5

3816

11

12

13

9

15

10

6

1417

7

only node uncolored

2

1

4

5

3816

11

12

13

9

15

10

6

1417

7

18




Proof

2

1

4

5

38 16

11

12

13

9

15

10

6

1417

7

18

2 4

7

i

4

i′18

i

14

i′

17

j1

d̄(i), cost to i ’s k − 1 nearest neighbors.d(i), rainbow cost of i .

optimized node: d(i) = d̄(i)

node conflicting with its nearest neighbor:

optimized node at one hop

d(i) ≤ (k − 2)d̄(i)

node with two conflicting nearest neighbors:

optimized node at two hops

d(i) ≤ ( 32 k − 5

2 )d̄(i)


Proof

2

1

4

5

38 16

11

12

13

9

15

10

6

1417

7

18

2 4

7

i

4

i′

18

i

14

i′

17

j1

d̄(i), cost to i ’s k − 1 nearest neighbors.d(i), rainbow cost of i .

optimized node: d(i) = d̄(i)

node conflicting with its nearest neighbor:

optimized node at one hop

d(i) ≤ (k − 2)d̄(i)

node with two conflicting nearest neighbors:

optimized node at two hops

d(i) ≤ ( 32 k − 5

2 )d̄(i)


A Heuristic Algorithm

Idea: use the similarity with domatic partitiondomatic coloring of a proximity graph

1. build a k-nearest neighbor graph O(n · log n)

2. augment it into an interval graph O(n)

3. build the domatic partition O(n)


Another Heuristic Algorithm

Based on a k-nearest neighbor graph, two rounds1. explore surroundings: do not pick a component

hosted by a direct neighborhosted by a peer that considers you as a direct neighborhosted by one of its direct neighbors

2. try to maximize the benefitspick component satisfying in average the direct neighbors


Simulations


Configurations

Several contexts have been considered:network of latencies

select randomly n peers among 20, 000 entries⇒ minimize the global latency

network of Autonomous Systemsput µ peers into every ASinter-AS routing

⇒ minimize the cross-domain traffic


Comparing to Exact Solutions


Going Further


Cross-Domain Gain

3,778

1,346

1,072 1,074 1,0853

4

5

6

Av

era

ge

Ho

ps

2,695 2,764 2,746

0

1

2

Random k-nearest Topo k-nearest Rela k-PUFLP

Av

era

ge

Ho

ps

Peering Transit


Conclusion


Only Preliminary Works

Many theoretical results can be obtained:relax assumptions (esp. capacity, number ofcomponents)study families of instancesbetter approximation

Many realistic variants can be formulated:take into account network architectureobjective of fairness


Any question?

<no image>


optimal network locality in distributed services

Technology

gwendal simon network

optimal network locality

gwendal simonnetwork

network servers cloud

n servers

server i

capacity bounds components

missing components j45