www3.informatik.uni-wuerzburg.de institute of computer science department of distributed systems...
TRANSCRIPT
www3.informatik.uni-wuerzburg.de
Institute of Computer ScienceDepartment of Distributed Systems
Prof. Dr.-Ing. P. Tran-Gia
Efficient Simulation of Large-Scale P2P Networks:Compact Data Structures
PDP 2007, Naples, Italy, February 7-9, 2007
Andreas Binzenhöfer, University of Würzburg
2Andreas Binzenhöfer
Authors
University of Würzburg
Andreas Binzenhöfer Tobias Hoßfeld
Technical University of Munich
Gerald Kunzmann
Hamburg University of Technology
Kolja Eger
Technical Report Daniel Schlosser (Würzburg) Robert Nagel (Munich) Jens Oberender (Passau) Ivan Dedinski (Passau)
3Andreas Binzenhöfer
Categories of Simulation Efficiency
0
50
100
0
50
1000
20
40
60
80
100
Event EfficiencyState Efficiency
Qu
eu
e E
ffic
ien
cy Calendar
Queue
UnoptimizedSimulation
OptimizedStates Optimized
Events
OptimizedSimulation
Calendar Queue withOptimized States Calendar Queue with
Optimized Events
4Andreas Binzenhöfer
Outline
Queue Efficiency Calendar Queue in P2P Simulations
Event Efficiency Periodic Events Dynamic Events Kademlia Bucket Refreshes
State Efficiency Process Handlers
Conclusion
5Andreas Binzenhöfer
Queue Efficiency
Reducing the time needed to handle events
6Andreas Binzenhöfer
Problem Definition
0 200 400 600 800 10000
100
200
300
400
500
600
700
800
900
1000
Number of Events in Queue
Ho
ld T
ime
O(log(n))
O(n)
O(1)
Hold Time = Enqueue + Dequeue Operation
list
heap
calendar queue
7Andreas Binzenhöfer
Calendar Queue
Works like a regular desktop calendar Each year has Nd (=365) day
Each day consist of Td (=24) hours
Enqueue operation: Calculate index as
Dequeue operation: Move from day to day until event found
Day 1 Day 2 Day 3 Day NdScheduled for:
current year
next year
year after next year*
*
1 mod dd
timestampindex N
T
8Andreas Binzenhöfer
Calendar Queue: Problems
Too many days:
Dequeue Operation no longer in O(1)
Very long days: Enqueue Operationno longer in O(1)
9Andreas Binzenhöfer
4096 days of length 100ms each
0 1000 2000 3000 4000 50000
50
100
150
200
250
Day of the year (100ms each)
Nu
mb
er
of
ev
en
ts
Spike at currentday of the year
10Andreas Binzenhöfer
32768 days of length 1ms each
0 0.5 1 1.5 2 2.5 3 3.5
x 104
0
2
4
6
8
10
12
14
16
Day of the year (1ms each)
Nu
mb
er
of
ev
en
ts
11Andreas Binzenhöfer
Two Queues for User and Simulation Events
0.5 1 1.5 2 2.5 3 3.5
x 104
0
10
20
30
40
50
Day of the year (1ms each)
Nu
mb
er
of
ev
en
ts
All events
User specific eventsSimulation events
joinsleavessearches
messagestimeoutsmaintenance routines
• event file can be given to different simulators• new user models without having to change the simulator• log files and real traces can be translated into simulator input• simulation time reduced due to the pre-calculated events
event-file
12Andreas Binzenhöfer
Event Efficiency
Reducing the number of events
14Andreas Binzenhöfer
Kademlia Bucket Refreshes
Roughly log2(n) buckets for each peer (where n = overlay size) Refresh bucket as soon as it has not been used for one hour Example:
4380
3960 5820
peer X
bucket 1
bucket 2 bucket 3
Time of next refresh
15Andreas Binzenhöfer
Kademlia Bucket Refreshes
Refresh event moved every time peer uses bucket
bucket i
Simple Solution (no additional memory required):
Obsolete refresh events are being skipped
bucket i
Simple Solution (additional memory required):
Time consuming
Algorithm:• search event• delete event• insert new event
Algorithm:• search event• delete event• insert new event• store time of next refresh at peer
X X XMany redundant events
16Andreas Binzenhöfer
Kademlia Bucket Refreshes
bucket i
Improved Solution (Simulation time):
Store actual time of next refresh for each bucket
Improved Solution (Memory):
bucket 1
bucket 2
bucket 3
currentminimum
log2(n)·n refresh events:
1.7 million for 100000 peers
Use only one single event for all buckets of a peer
Algorithm:• search event• delete event• insert new event• store time of next refresh at peer
17Andreas Binzenhöfer
State Efficiency
Reduce memory needed to represent states
18Andreas Binzenhöfer
Process Handler
Avoid redundant information of parallel processes Example – Parallel search:
Each participating peer stores similar information This information is stored multiple times in RAM
Day 1 Day 2 Day 3 Day Nd
process handler
Ra = 3xyz = 48
19Andreas Binzenhöfer
Process Handler
Ra : Number of remaining pointers to the process handler
Last pointer frees memory Advantage: Avoid redundant information
Day 1 Day 2 Day 3 Day Nd
process handler
Ra = 3xyz = 48
20Andreas Binzenhöfer
Example: Search Handler
Search Handler stores Search initiator Searched ID Timeout values Number of (positive) answers …
process handler
Ra=3sender=48query=667timeout=4567answers=2
21Andreas Binzenhöfer
Conclusion
Typical structure of P2P events allows for queue hold time of O(1) does not require dynamic adaptation of Calendar queue
Careful design of events can dramatically reduce simulation time required memory example: Kademlia bucket refreshes
Process handlers exploit parallel/distributed nature of P2P algorithms reduce redundant information
22Andreas Binzenhöfer
Q&A