deterministic distributed resource discovery
Post on 01-Jan-2016
29 Views
Preview:
DESCRIPTION
TRANSCRIPT
Deterministic Distributed Resource Discovery
Shay Kutten TechnionDavid Peleg Weizmann Inst.Uzi Vishkin Univ. of Maryland & Technion
Distributed Resource Discovery [HLL99]
pointer
The corresponding directed graph (on nodes A,B,C):
(assumption: weakly connected)Problem: compute the connected component
router
A
C
B
A B C
Distributed Resource Discovery-[Harcol-Balter, Leighton, Lewin 1999]
“)Names Dropper” protocol(
Licensed to AKAMAI, a leading Web caching provider
The resource discovery task: the set of servers changes Find the current set (towards deciding which Akamai server to assign to each user)
Akamai servers need to learn how to reach each other via the Internet (towards deciding which Akamai server is “close” to the user).
Akamai Server
Internet
user
CNN
Think Distributed
A B
a pointer used by the algorithm (one per node, initially pointing at the node itself)
A Address “known” to Node A (“knowledge graph”)
Mr. and Ms. B132.12.16.64 IP street
Interland
a pointer used by the algorithm
A?
When the Algorithm is started
A
C
D
B
Weakly connected directed graph
a pointer used by the algorithm (one per node, initially pointing at the node itself)
A Address “known” to Node A (“knowledge graph”)
Algorithm Actions
A
C
D
BC
(1) Node A learns C’s addr. From B, thus, directed edge (A C) is added to knowledge graph.
Algorithm Actions
A
C
D
BC
(1) Node A learns C’s addr. From B, thus, directed edge (A C) is added to knowledge graph.
(2) Node D who knows of C, change PTR to point at C, thus, pointer graph changes.
D
Algorithm Actions
A
C
D
BC
(1) Node A learns C’s addr. From B, thus, directed edge (A C) is added to knowledge graph.
(2) Node D who knows of C, change PTR to point at C, thus, pointer graph changes.
D
change
Algorithm’s result
pointers form star, knowledge graph is strongly connected,Plus, possibly, additional links (possibly clique) .
Model [HLL99]- synchronous (not necessary for our alg)
- simultaneously start (for our alg: just “close”)
- complete comm. Graph
Complexity: #Msgs (connections [HLL99]),
#bits (#pointers*log n), time (#rounds)
Motivation (2) GnutellaNapster server (mp3 music files)
I have “Love Song”I want “Love Song”
I have “Madonna”
Motivation (2) Gnutella
Napster server
I have Love SongI want Love Song
A
Ask A
Motivation (2) Gnutella
Napster server
I want Love SongA
Ask A
Motivation (2) Gnutella
Napster server
A
Ask A
Love Song
Motivation (2) Gnutella vs. Napster
Napster serverIf the court closes The service collapsed
Motivation (2) Gnutella vs. Napster
Napster serverIf the court closes The service collapsed
Motivation (2) Gnutella vs. Napster
Napster server If the court closes the service collapses
Motivation (2) Gnutella vs. Napster
Napster serverIf the court closes the service collapses
Gnutella: attempts tosolve by having everyclient=server
Same problem: user knows some others find more increase connectivity to withstand disconnections
Motivation(2) Gnutella
Users get disconnected often
Bob
Carol
Initially Alice knows only Bob & Carol (personally, or from ICQ, or from Google…) and can connect via themTo others they know.
Alice
David
Motivation(2) gnutellaUsers get disconnected often
Bob
Carol
Initially Alice knows only Bob & Carol (personally, or from ICQ, or from Google…) Alice learned from Carol how to reach David, so that when Bob & Carol are not online Alice is still connected.
Alice
David
Motivation: “Peer to Peer” Applications(P2P)
(1) Akamai http://www.akamai.com 1999(2) Gnutella http://gnutella.wego.com (originally
Nullsoft (Winamp)) 2000 (3) Gpulp: general Purpose Location Protocol http://www.gpulp.com (4) Genny http://www.gnutelladev.com (5) Freenet http://freenet.sourceforge.net 2000(6) JXTA http://www.jxta.org (Sun Microsystems) April 2001(7) Retsina: Discovery of Infrastructure in Multi-Agent
Systems (CMU)
This paper [HLL99]
deterministic randomized termination detection no detection O(log n) rounds O(log n)
O(n log n) messages O(n log n)
O(E log n) bits O(n log n)
2
2
2 320
Algorithm strategies
D
E
F
G
H
I
(1) Shrink tall trees to make them stars [SV82]
12
5
(2) Merge stars to get one tree (changed from [SV82])
(3) Carefully connect weakly connected(explained later; one of the differences from]Shiloach, Vishkin 82[)
Example of a technique taken from ]SV82[
(Handling tall trees in “pointer graph”) D
E
F
G
H
I
C
B
AShortcuts ]SV82[
D
E
F
G
H
I
C
B
A
Handing tall trees in “pointer graph” (cont.)
Shortcuts (]SV82[
Each child learnsthe pointer to its grandparent
D
E
F
G
H
I
C
B
A
Shortcuts
Handing tall trees in “pointer graph” (cont.)
D
E
F
G
H
I
C
B
A
Algorithm: finding who to merge with ]SV82[
C
E,E,F,G,H,I,C
node I, on becoming a child of root D, tells D about C
Add edges to knowledge graph:
Algorithm (2) handling active star roots
12
5join
3
join
Star roots ask to connect
24jo
in
Algorithm (2) handling active star roots
12
5join
3
join
-An undirected subgraph is created by joinMessages. -Each root connects to local minima (ID) rootOn undirected subgraph.
24jo
in
Algorithm (2) handling active star roots
12
5
3-An undirected subgraph is created by joinMessages. -Each root connects to local minima (ID) rootOn undirected subgraph.
24
Algorithm (2) handling active star roots
9
7 24
20
12
join
(note differences from [SV82])
Non-stars do not join
Algorithm (2) handling active star roots
9
7 24
20
12
5join
3
joinjoin
Join smallest id-prevents cycles
5 joins 12!!! : smallest
id neighbor
(only stars join, as opposed to [SV82])
Algorithm (2) handling active star roots
12
9
7 24
20
5join
3
join
join
16join
Algorithm (2) handling active star roots
12
9
7 24
20
5join
3
join
join
16join
Algorithm (2) handling active star roots
12
9
7 24
20
5join
3
join
join
16join
Algorithm handling passive star roots
24
5join
9
Node 9 is “passive” since it does not know any Node it cannot initiate any joining
Algorithm handling passive star roots
12
9
7 24
20
5join
3
join
8join
Node 12 is passive:
It does not know Any non-child.
It cannot Initiate any
progress.
Algorithm handling passive star roots
12
9
7 24
20
5join
3
join
8join
Node 12 joins its lowest “suitor” 3.Suitors of 12 join 3 too.
Algorithm: An Example of a Techniquedifferent than [SV82]
(weakly connected directed graph)Without an additional technique, may take (n) time:
A
Worst case (only A is active, its star grows by 1per phase):
Phase 1
(weakly connected diagraph)
Without an additional technique, may take (n) time:
A
Worst case (only A is active, its star grows by 1per phase):
phase 2
(weakly connected diagraph)
Without an additional technique, may take (n) time:
A
Worst case (only A is active, its star grows by 1per phase):
phase n-1
(weakly connected diagraph)
Idea: star root A picks 2 neighbors (not just 1) in phase i.(In phase i there are only n/2 stars)
i
Prevent (n) time, and still do not send too many messages:
Example: i=2 2 =4
A
i
i
(weakly connected diagraph)
Idea: A picks 2 neighbors (not just 1) in phase i.i
Prevent (n) time:
Some delicate points: resolving collisions, mixed topologies prove convergence in O(log n) phases
A
CorrectnessLemma: No cycles are created (hook on smaller + non-star does not connect)
Assured by a new technique:
37Possible because ofWeak connectivity!!
14
3 7 50
30
200
No connection No cycle
Lemma: always some active star
Complexity
Two kinds of progress in each phase:
(1) A tall tree gets shallower by a constant factor.
Obstacles:- # of star trees may grow because of (1)- Trees may get taller because of (2)
But (with the right combination) there is a progress in the combination.
(2) Active star roots merge.
Conclusion
- improved complexity in all measures (verifies [HLL99] conjecture, forecasting a simple algorithm with these complexities).- Deterministic.- Terminates (answers an open problem of Lipton).
Further research
- Adaptive algorithms (vs. “one shot”) - Lower bounds? (in [CGK95] O(n) msgs for undirected case).
top related