end-to-end bandwidth estimation in the wide internet daniele croce phd dissertation, april 16, 2010
DESCRIPTION
Inter-connected networks –Different technologies, many operators –No global view The Internet 3 Net1 Net2 Net3 Net4 Objective: characterize the E2E performanceTRANSCRIPT
![Page 1: End-to-end Bandwidth Estimation in the Wide Internet Daniele Croce PhD dissertation, April 16, 2010](https://reader035.vdocuments.net/reader035/viewer/2022062317/5a4d1b0d7f8b9ab05998ca4e/html5/thumbnails/1.jpg)
End-to-end Bandwidth EstimationEnd-to-end Bandwidth Estimationin the Wide Internetin the Wide Internet
Daniele CrocePhD dissertation, April 16, 2010
![Page 2: End-to-end Bandwidth Estimation in the Wide Internet Daniele Croce PhD dissertation, April 16, 2010](https://reader035.vdocuments.net/reader035/viewer/2022062317/5a4d1b0d7f8b9ab05998ca4e/html5/thumbnails/2.jpg)
• “Breakfast Can Wait. The Day’s First Stop Is Online.” [NYTimes‘09]
but is our connection performing well?
Internet is wonderful
2
![Page 3: End-to-end Bandwidth Estimation in the Wide Internet Daniele Croce PhD dissertation, April 16, 2010](https://reader035.vdocuments.net/reader035/viewer/2022062317/5a4d1b0d7f8b9ab05998ca4e/html5/thumbnails/3.jpg)
• Inter-connected networks– Different technologies, many operators– No global view
The Internet
3
Net1
Net2Net3
Net4
Objective: characterize the E2E performance
![Page 4: End-to-end Bandwidth Estimation in the Wide Internet Daniele Croce PhD dissertation, April 16, 2010](https://reader035.vdocuments.net/reader035/viewer/2022062317/5a4d1b0d7f8b9ab05998ca4e/html5/thumbnails/4.jpg)
• Simple metrics– Packet loss– Delay (One-Way, RTT), jitter– (TCP) throughput
• Advanced metrics– End-to-end capacity
C=min(Ci)– End-to-end available bandwidth (AB)
• i.e., the unused capacityA=min(Ai)
Performance metrics
4
![Page 5: End-to-end Bandwidth Estimation in the Wide Internet Daniele Croce PhD dissertation, April 16, 2010](https://reader035.vdocuments.net/reader035/viewer/2022062317/5a4d1b0d7f8b9ab05998ca4e/html5/thumbnails/5.jpg)
• On a generic link i :Available Bandwidth
5T
TBCTA iii
),0(),0(
![Page 6: End-to-end Bandwidth Estimation in the Wide Internet Daniele Croce PhD dissertation, April 16, 2010](https://reader035.vdocuments.net/reader035/viewer/2022062317/5a4d1b0d7f8b9ab05998ca4e/html5/thumbnails/6.jpg)
• An example:Narrow link and tight link
6
Narrow Link Tight Link
100 Mbps90 Mbps
1000 Mbps400 Mbps
155 Mbps20 Mbps
Available bandwidth
C =AB =
Capacity
![Page 7: End-to-end Bandwidth Estimation in the Wide Internet Daniele Croce PhD dissertation, April 16, 2010](https://reader035.vdocuments.net/reader035/viewer/2022062317/5a4d1b0d7f8b9ab05998ca4e/html5/thumbnails/7.jpg)
• Tools require access to both end hosts– Impossible between different organizations!
Three single-ended tools
Contribution 1
7
Net1
Net2Net3
Net4
![Page 8: End-to-end Bandwidth Estimation in the Wide Internet Daniele Croce PhD dissertation, April 16, 2010](https://reader035.vdocuments.net/reader035/viewer/2022062317/5a4d1b0d7f8b9ab05998ca4e/html5/thumbnails/8.jpg)
• Large-scale deployments of active AB tools– Routing, P2P optimization, improve TCP
Performance evaluation of AB techniquesin large-scale measurement systems
Contribution 2
8
Net1
Net2Net3
Net4
![Page 9: End-to-end Bandwidth Estimation in the Wide Internet Daniele Croce PhD dissertation, April 16, 2010](https://reader035.vdocuments.net/reader035/viewer/2022062317/5a4d1b0d7f8b9ab05998ca4e/html5/thumbnails/9.jpg)
• Three AB measurement paradigms exist:– PRM (Probe Rate Model)
• “Is rate higher than the AB?”– PGM (Probe Gap Model)
• “Has the Inter-Packet Gap increased?”– PDM (Probe Delay Model)
• “Has the packet queued?”• Only analytical or simulative studies• Better than PRM or PGM?
Real implementation and comparison with other classic PRM and PGM tools
Contribution 3
9
NEW!!!
![Page 10: End-to-end Bandwidth Estimation in the Wide Internet Daniele Croce PhD dissertation, April 16, 2010](https://reader035.vdocuments.net/reader035/viewer/2022062317/5a4d1b0d7f8b9ab05998ca4e/html5/thumbnails/10.jpg)
SINGLE-ENDED TECHNIQUES
10
![Page 11: End-to-end Bandwidth Estimation in the Wide Internet Daniele Croce PhD dissertation, April 16, 2010](https://reader035.vdocuments.net/reader035/viewer/2022062317/5a4d1b0d7f8b9ab05998ca4e/html5/thumbnails/11.jpg)
Non-cooperative estimation
• RTT = OWDf
DSLAM
ACK probes
TCP RSTsSender Receiver
Can we separate the effects of the two paths?
Sender Receiver
ACKs
Sender
RSTs
+ OWDr
11
![Page 12: End-to-end Bandwidth Estimation in the Wide Internet Daniele Croce PhD dissertation, April 16, 2010](https://reader035.vdocuments.net/reader035/viewer/2022062317/5a4d1b0d7f8b9ab05998ca4e/html5/thumbnails/12.jpg)
Where is the tight link?
12
Sender
ACKs
Sender
RSTs
• RSTs are always 40 Bytes• No matter the size of the ACK probes
• By varying the ACK size We can load the two paths equally (SACK = SRST) We can load the downlink more than the uplink (SACK > SRST) We can NOT load the uplink more than the downlink (SACK <
SRST)
![Page 13: End-to-end Bandwidth Estimation in the Wide Internet Daniele Croce PhD dissertation, April 16, 2010](https://reader035.vdocuments.net/reader035/viewer/2022062317/5a4d1b0d7f8b9ab05998ca4e/html5/thumbnails/13.jpg)
ABw-Probe (ABP)
13
• Measuring the downlink (no uplink traffic)
• Impact of “cross”-traffic on the uplink
cooperative
non-coop.
![Page 14: End-to-end Bandwidth Estimation in the Wide Internet Daniele Croce PhD dissertation, April 16, 2010](https://reader035.vdocuments.net/reader035/viewer/2022062317/5a4d1b0d7f8b9ab05998ca4e/html5/thumbnails/14.jpg)
Uplink cross-traffic
14
![Page 15: End-to-end Bandwidth Estimation in the Wide Internet Daniele Croce PhD dissertation, April 16, 2010](https://reader035.vdocuments.net/reader035/viewer/2022062317/5a4d1b0d7f8b9ab05998ca4e/html5/thumbnails/15.jpg)
Filtering uplink cross-traffic
• Cross-traffic is not just MTU packets– Use DT to remove
large packets– Then use RR for
refining15
![Page 16: End-to-end Bandwidth Estimation in the Wide Internet Daniele Croce PhD dissertation, April 16, 2010](https://reader035.vdocuments.net/reader035/viewer/2022062317/5a4d1b0d7f8b9ab05998ca4e/html5/thumbnails/16.jpg)
FAB-probe (large-scale)
16
Do we really need a 40 kbps precision?
![Page 17: End-to-end Bandwidth Estimation in the Wide Internet Daniele Croce PhD dissertation, April 16, 2010](https://reader035.vdocuments.net/reader035/viewer/2022062317/5a4d1b0d7f8b9ab05998ca4e/html5/thumbnails/17.jpg)
Real-world experience• Tested on 1244 ADSL hosts, 10 different ISPs
– Participating in Kademlia DHT (eMule)• Used KAD crawler (ACM IMC 2007)• Selected ADSL using Maxmind
1. Capacity of the ADSL link2. A snapshot of the available bandwidth3. Average AB on over 10 days
– 82 hosts online for over one month– Static IP address– Measured every 5 minutes
• On average 6 seconds per measurement17
![Page 18: End-to-end Bandwidth Estimation in the Wide Internet Daniele Croce PhD dissertation, April 16, 2010](https://reader035.vdocuments.net/reader035/viewer/2022062317/5a4d1b0d7f8b9ab05998ca4e/html5/thumbnails/18.jpg)
Capacity estimation• Comparison of 2 large ISPs
The policy used by Free is quite uncommon (see IMC07)
0.7Mbps
2.5Mbps 0.3Mbps
1Mbps
Downlink
Uplink
18
![Page 19: End-to-end Bandwidth Estimation in the Wide Internet Daniele Croce PhD dissertation, April 16, 2010](https://reader035.vdocuments.net/reader035/viewer/2022062317/5a4d1b0d7f8b9ab05998ca4e/html5/thumbnails/19.jpg)
Available bandwidth (I)• Snapshot of 1244 (eMule) hosts
Hosts are divided in congested or idle
19
![Page 20: End-to-end Bandwidth Estimation in the Wide Internet Daniele Croce PhD dissertation, April 16, 2010](https://reader035.vdocuments.net/reader035/viewer/2022062317/5a4d1b0d7f8b9ab05998ca4e/html5/thumbnails/20.jpg)
Available bandwidth (II)• 82 hosts, 10 days average
– Each point is an average of one user over 10 days
• 30% congested, 30-40% frequently idle20
![Page 21: End-to-end Bandwidth Estimation in the Wide Internet Daniele Croce PhD dissertation, April 16, 2010](https://reader035.vdocuments.net/reader035/viewer/2022062317/5a4d1b0d7f8b9ab05998ca4e/html5/thumbnails/21.jpg)
ANALYSIS OF LARGE-SCALEAB MEASUREMENT SYSTEMS
21
![Page 22: End-to-end Bandwidth Estimation in the Wide Internet Daniele Croce PhD dissertation, April 16, 2010](https://reader035.vdocuments.net/reader035/viewer/2022062317/5a4d1b0d7f8b9ab05998ca4e/html5/thumbnails/22.jpg)
Motivation• We have a dream: measure AB everywhere
– Route selection, server selection – Overlay performance optimization– Improve TCP– ...
• Naïve approach:– pick one of the existing techniques!
• BUT what if we all do the same simultaneously?
Interference between measurements22
![Page 23: End-to-end Bandwidth Estimation in the Wide Internet Daniele Croce PhD dissertation, April 16, 2010](https://reader035.vdocuments.net/reader035/viewer/2022062317/5a4d1b0d7f8b9ab05998ca4e/html5/thumbnails/23.jpg)
In brief• Existing techniques
– Pathload, Spruce, pathChirp• Experimental testbed
– All tools suffer from mutual interference• But not in the same way!!!
– High intrusiveness and overhead• Analytical models
– Probability of interference– Measurement bias
• What can we do?23
![Page 24: End-to-end Bandwidth Estimation in the Wide Internet Daniele Croce PhD dissertation, April 16, 2010](https://reader035.vdocuments.net/reader035/viewer/2022062317/5a4d1b0d7f8b9ab05998ca4e/html5/thumbnails/24.jpg)
Pathload – Packet Trains• Probing strategy:
– Iteratively send N trains at different rates– Binary search to converge to the AB
• Inference:– Detect One-Way Delay increase (rate > AB)
24
![Page 25: End-to-end Bandwidth Estimation in the Wide Internet Daniele Croce PhD dissertation, April 16, 2010](https://reader035.vdocuments.net/reader035/viewer/2022062317/5a4d1b0d7f8b9ab05998ca4e/html5/thumbnails/25.jpg)
Spruce – Packet Pairs• Probing strategy:
– Two packets with specific inter-packet gap• Inference:
– Measure dispersion (gap increase) of the pair
– Accuracy is debated, out of our scope
∆in ∆in Bottlene
ck
∆out
∆out
25
![Page 26: End-to-end Bandwidth Estimation in the Wide Internet Daniele Croce PhD dissertation, April 16, 2010](https://reader035.vdocuments.net/reader035/viewer/2022062317/5a4d1b0d7f8b9ab05998ca4e/html5/thumbnails/26.jpg)
Interference in Spruce• One pair interfering…
• What is the probability that this happens?– Hint: similar to ALOHA protocol
0
26
∆in ∆out
100% error!
![Page 27: End-to-end Bandwidth Estimation in the Wide Internet Daniele Croce PhD dissertation, April 16, 2010](https://reader035.vdocuments.net/reader035/viewer/2022062317/5a4d1b0d7f8b9ab05998ca4e/html5/thumbnails/27.jpg)
pathChirp – Packet “chirps”
• Probing strategy:– One train with exponentially increasing rate
• Inference:– Detect One-Way Delay increase
27
ABwLimit
![Page 28: End-to-end Bandwidth Estimation in the Wide Internet Daniele Croce PhD dissertation, April 16, 2010](https://reader035.vdocuments.net/reader035/viewer/2022062317/5a4d1b0d7f8b9ab05998ca4e/html5/thumbnails/28.jpg)
Testbed results• 62 hosts running linux
– Half are senders, half receivers• Single bottleneck (10 Mbps), CBR traffic
– Ideal conditions for ABw tools– Errors are due to mutual interference only
28
![Page 29: End-to-end Bandwidth Estimation in the Wide Internet Daniele Croce PhD dissertation, April 16, 2010](https://reader035.vdocuments.net/reader035/viewer/2022062317/5a4d1b0d7f8b9ab05998ca4e/html5/thumbnails/29.jpg)
Pathload
29
![Page 30: End-to-end Bandwidth Estimation in the Wide Internet Daniele Croce PhD dissertation, April 16, 2010](https://reader035.vdocuments.net/reader035/viewer/2022062317/5a4d1b0d7f8b9ab05998ca4e/html5/thumbnails/30.jpg)
Spruce
True?? How much OVERHEAD?
30
Results are biased
![Page 31: End-to-end Bandwidth Estimation in the Wide Internet Daniele Croce PhD dissertation, April 16, 2010](https://reader035.vdocuments.net/reader035/viewer/2022062317/5a4d1b0d7f8b9ab05998ca4e/html5/thumbnails/31.jpg)
pathChirpResults seem better
31
True?? High OVERHEAD!
![Page 32: End-to-end Bandwidth Estimation in the Wide Internet Daniele Croce PhD dissertation, April 16, 2010](https://reader035.vdocuments.net/reader035/viewer/2022062317/5a4d1b0d7f8b9ab05998ca4e/html5/thumbnails/32.jpg)
Intrusiveness
x10 x100
32
![Page 33: End-to-end Bandwidth Estimation in the Wide Internet Daniele Croce PhD dissertation, April 16, 2010](https://reader035.vdocuments.net/reader035/viewer/2022062317/5a4d1b0d7f8b9ab05998ca4e/html5/thumbnails/33.jpg)
Possible Solutions• Mutual interference
– Direct probing more promising• Simple, Spruce-like algorithms. No binary search
– Identify interference (and correct it)??
• Overhead– “In-band” measurements (piggy-backing)
• Best, no overhead at all• Complex! (SIGCOMM09) + delay constraints
– “Out-of-band” measurements• At least, make the overhead scale with the ABw!
• Lets help each other! Network Tomography33
![Page 34: End-to-end Bandwidth Estimation in the Wide Internet Daniele Croce PhD dissertation, April 16, 2010](https://reader035.vdocuments.net/reader035/viewer/2022062317/5a4d1b0d7f8b9ab05998ca4e/html5/thumbnails/34.jpg)
Conclusions• Non-cooperative estimation
– Three highly optimized tools– No need to install software or buy new equipment
• An Italian ISP already interested!
• Analysis of large-scale AB measurements– Tools can not be used off-the-shelf
• Mutual interference, Intrusiveness, Overhead
– Interference can be predicted and modeled– Discussed possible solutions
• Future work includes– Technologies different from ADSL (cable, FTTH)– New, lightweight techniques (passive?), tomography
34
![Page 35: End-to-end Bandwidth Estimation in the Wide Internet Daniele Croce PhD dissertation, April 16, 2010](https://reader035.vdocuments.net/reader035/viewer/2022062317/5a4d1b0d7f8b9ab05998ca4e/html5/thumbnails/35.jpg)
BACKUP
35
![Page 36: End-to-end Bandwidth Estimation in the Wide Internet Daniele Croce PhD dissertation, April 16, 2010](https://reader035.vdocuments.net/reader035/viewer/2022062317/5a4d1b0d7f8b9ab05998ca4e/html5/thumbnails/36.jpg)
Collision with ON-OFF meas.
36
Few hosts cause > 10%
collisions!
![Page 37: End-to-end Bandwidth Estimation in the Wide Internet Daniele Croce PhD dissertation, April 16, 2010](https://reader035.vdocuments.net/reader035/viewer/2022062317/5a4d1b0d7f8b9ab05998ca4e/html5/thumbnails/37.jpg)
Non-cooperative estimation• Who is answering to what (Monarch, IMC’06)
![Page 38: End-to-end Bandwidth Estimation in the Wide Internet Daniele Croce PhD dissertation, April 16, 2010](https://reader035.vdocuments.net/reader035/viewer/2022062317/5a4d1b0d7f8b9ab05998ca4e/html5/thumbnails/38.jpg)
Measurement bias: Spruce• Measurement error in Spruce
– Depends on the # of interfering pairs n :
• The average number of interf. pairs is
• This explains why Spruce bias is proportional to
38
![Page 39: End-to-end Bandwidth Estimation in the Wide Internet Daniele Croce PhD dissertation, April 16, 2010](https://reader035.vdocuments.net/reader035/viewer/2022062317/5a4d1b0d7f8b9ab05998ca4e/html5/thumbnails/39.jpg)
Pathload interference, two trains
39
![Page 40: End-to-end Bandwidth Estimation in the Wide Internet Daniele Croce PhD dissertation, April 16, 2010](https://reader035.vdocuments.net/reader035/viewer/2022062317/5a4d1b0d7f8b9ab05998ca4e/html5/thumbnails/40.jpg)
PathChirp, two chirps
With only two trains, errors up to
80%!!!
40
![Page 41: End-to-end Bandwidth Estimation in the Wide Internet Daniele Croce PhD dissertation, April 16, 2010](https://reader035.vdocuments.net/reader035/viewer/2022062317/5a4d1b0d7f8b9ab05998ca4e/html5/thumbnails/41.jpg)
Measurement Overhead• Spruce
– Overhead = min(240kbps, 5% of Bneck Capacity)– Few hosts can consume a LOT of Bw!
• Pathload– Overhead ≈ ABw– Cons: measurements consume all the ABw– Pro: overhead “scales” with the ABw
• pathChirp– Overhead = 300kbps (tunable parameter)– What if 10 hosts are measuring? If 100?
41
![Page 42: End-to-end Bandwidth Estimation in the Wide Internet Daniele Croce PhD dissertation, April 16, 2010](https://reader035.vdocuments.net/reader035/viewer/2022062317/5a4d1b0d7f8b9ab05998ca4e/html5/thumbnails/42.jpg)
With traffic load• 20 hosts running, ABw=6 Mbps
42
![Page 43: End-to-end Bandwidth Estimation in the Wide Internet Daniele Croce PhD dissertation, April 16, 2010](https://reader035.vdocuments.net/reader035/viewer/2022062317/5a4d1b0d7f8b9ab05998ca4e/html5/thumbnails/43.jpg)
All tools together• 9 hosts per type (27 senders)
43
![Page 44: End-to-end Bandwidth Estimation in the Wide Internet Daniele Croce PhD dissertation, April 16, 2010](https://reader035.vdocuments.net/reader035/viewer/2022062317/5a4d1b0d7f8b9ab05998ca4e/html5/thumbnails/44.jpg)
Delay-based tools• Consider a single server queue
– The utilization can be computed as
• 0 is the probability of the queue being empty
– Probe-Delay-Model (PDM) tools estimate 0
• PDM tools– Make no assumptions on cross-traffic– Inject very little overhead
• no need for high probing rates44
![Page 45: End-to-end Bandwidth Estimation in the Wide Internet Daniele Croce PhD dissertation, April 16, 2010](https://reader035.vdocuments.net/reader035/viewer/2022062317/5a4d1b0d7f8b9ab05998ca4e/html5/thumbnails/45.jpg)
Forecaster Model
45
The AB is estimated by “projecting” the
utilization
![Page 46: End-to-end Bandwidth Estimation in the Wide Internet Daniele Croce PhD dissertation, April 16, 2010](https://reader035.vdocuments.net/reader035/viewer/2022062317/5a4d1b0d7f8b9ab05998ca4e/html5/thumbnails/46.jpg)
Threshold problem
46
-100 0 100 200 300 400 500 600 700 800 9000
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
One Way Delay(usec)
CD
F
CDF of OWD with 20% cross traffic
delay in this area is considered not suffered from queueing
Time Threshold
In our experiments, must allow ~100us for inaccuracies!