internet performance measurements and measurement techniques jim kurose department of computer...
TRANSCRIPT
![Page 1: Internet Performance Measurements and Measurement Techniques Jim Kurose Department of Computer Science University of Massachusetts/Amherst kurose](https://reader035.vdocuments.net/reader035/viewer/2022070400/56649f135503460f94c277a2/html5/thumbnails/1.jpg)
Internet Performance Measurements and
Measurement Techniques
Jim KuroseDepartment of Computer Science
University of Massachusetts/Amhersthttp://www.cs.umass.edu/~kurose
![Page 2: Internet Performance Measurements and Measurement Techniques Jim Kurose Department of Computer Science University of Massachusetts/Amherst kurose](https://reader035.vdocuments.net/reader035/viewer/2022070400/56649f135503460f94c277a2/html5/thumbnails/2.jpg)
Overview
• Introduction– why and what to measure
• Measuring per-hop performance– tricks, successes, “failures”
• End-to-end measurements– correlation in end-end loss, delay– “confidence” in measurements
• What lies ahead?
![Page 3: Internet Performance Measurements and Measurement Techniques Jim Kurose Department of Computer Science University of Massachusetts/Amherst kurose](https://reader035.vdocuments.net/reader035/viewer/2022070400/56649f135503460f94c277a2/html5/thumbnails/3.jpg)
What “performance” to measure?
• packet delay• packet loss• link or path capacity/availability• where?
• over what time scale?– sub-second, minute, hours?
end-to-end
per-hop
![Page 4: Internet Performance Measurements and Measurement Techniques Jim Kurose Department of Computer Science University of Massachusetts/Amherst kurose](https://reader035.vdocuments.net/reader035/viewer/2022070400/56649f135503460f94c277a2/html5/thumbnails/4.jpg)
Why measure?end-end measurements:
• benchmarking, monitoring (e.g., Imeter)
• fault identification (e.g., routing instabilities)
• understanding end-end perf– misordering, loss (e.g., tcp
studies by Paxson)– correlation-time scale for
end-end loss, delayuse in adaptive applications
per-hop measurements:
• network operations (proprietary?)
• understanding where in end-end path performance impairments occur – use in reliable
multicast protocols, active services, network modeling
![Page 5: Internet Performance Measurements and Measurement Techniques Jim Kurose Department of Computer Science University of Massachusetts/Amherst kurose](https://reader035.vdocuments.net/reader035/viewer/2022070400/56649f135503460f94c277a2/html5/thumbnails/5.jpg)
Measuring Per-hop performance• Question: what is loss, delay, capacity at given hop?• Question: what per-hop delays does a packet see?
• Complication:– routers do not report performance stats to end users
• need to infer performance statistics– cleverly use little “machinery” that we have– develop an inferencing methodology
per-hop
![Page 6: Internet Performance Measurements and Measurement Techniques Jim Kurose Department of Computer Science University of Massachusetts/Amherst kurose](https://reader035.vdocuments.net/reader035/viewer/2022070400/56649f135503460f94c277a2/html5/thumbnails/6.jpg)
Clever use of existing protocols
• traceroute, pathchar: – use ICMP packets and time-to-live (TTL) field– each router decrements TTL on forwarding– TTL = 0 results in ICMP error msg back to sender
• Used to discover all routers on path to destination
ttl=3ttl=2
ttl=1
ICMP errrouter = x
x
![Page 7: Internet Performance Measurements and Measurement Techniques Jim Kurose Department of Computer Science University of Massachusetts/Amherst kurose](https://reader035.vdocuments.net/reader035/viewer/2022070400/56649f135503460f94c277a2/html5/thumbnails/7.jpg)
Clever use of exiting protocols (cont.)
• ICMP/TTL-field trick also gives link bandwidth:– find min roundtrip delay to x-
1 (use many probe pkts)– find min rt delay to hop x– difference gives prop. delay
plus transmission delay– vary pkt size to get link
bandwidth
• gives variable queueing delay, loss of path to x– isolating hop x behavior
difficult
hop x-1 hop x
data packet (d bits)
ICMP reply (r bits)
d/bw
2*prop
r/bw
time
![Page 8: Internet Performance Measurements and Measurement Techniques Jim Kurose Department of Computer Science University of Massachusetts/Amherst kurose](https://reader035.vdocuments.net/reader035/viewer/2022070400/56649f135503460f94c277a2/html5/thumbnails/8.jpg)
Can we measure per-hop delays?
• motivation - a typical modeling paper: “We model the network as a single link ….”
• is this a valid assumption?– does a packet generally experience “most”
of its delay at one link?
![Page 9: Internet Performance Measurements and Measurement Techniques Jim Kurose Department of Computer Science University of Massachusetts/Amherst kurose](https://reader035.vdocuments.net/reader035/viewer/2022070400/56649f135503460f94c277a2/html5/thumbnails/9.jpg)
Measuring per-hop delays:• send unicast probes along path• use “IP options” on probes to gather timestamps
– packet passing through specified router timestamped
• problem: only 4 timestamps in each packet• solution: send multiple probes at one time:
x
y
x y
y
xts(x)
y
xts(x)
ts(y)
probe 1
probe 2
x x x xx x x x
![Page 10: Internet Performance Measurements and Measurement Techniques Jim Kurose Department of Computer Science University of Massachusetts/Amherst kurose](https://reader035.vdocuments.net/reader035/viewer/2022070400/56649f135503460f94c277a2/html5/thumbnails/10.jpg)
Measuring per-hop delays:• problem: IP options packets treated differently
– data packets forwarded on fast path– IP options packet detoured (hopefully briefly)
• solution: send non-option packet with probes– only analyze probes when non-option packet delay
close to probe delay (hope: negligible options processing delays)
probe 1 x x x xprobe 2 x x x x
options
forwarding
non-option pkt
![Page 11: Internet Performance Measurements and Measurement Techniques Jim Kurose Department of Computer Science University of Massachusetts/Amherst kurose](https://reader035.vdocuments.net/reader035/viewer/2022070400/56649f135503460f94c277a2/html5/thumbnails/11.jpg)
Analyzing the per-hop data
• Consider only probes with e-e queueing delays > 100ms
• filter cases where probe and option pkt delays “close” (20 ms)
• Hypothesis: e-e delays of filtered probes from same distribution as all probes– hypothesis rejected with
negligible probability of being wrong :-(
Title:
Creator:gnuplotPreview:This EPS picture was not savedwith a preview included in it.Comment:This EPS picture will print to aPostScript printer, but not toother types of printers.
![Page 12: Internet Performance Measurements and Measurement Techniques Jim Kurose Department of Computer Science University of Massachusetts/Amherst kurose](https://reader035.vdocuments.net/reader035/viewer/2022070400/56649f135503460f94c277a2/html5/thumbnails/12.jpg)
Can we measure per-hop packet delay/loss?
• timestamping approach not statistically valid
• inspiration (!) from another on-going effort: multicast loss
• question: where in a multicast tree does loss occur?– backbone? edges? – implications for design of
reliable multicast protocols
Title: /tmp/xfig-export003787Creator: fig2devCreationDate: Sat May 11 23:53:21 1996
![Page 13: Internet Performance Measurements and Measurement Techniques Jim Kurose Department of Computer Science University of Massachusetts/Amherst kurose](https://reader035.vdocuments.net/reader035/viewer/2022070400/56649f135503460f94c277a2/html5/thumbnails/13.jpg)
Using multicast to infer per-hop performance
• correlation of received mcast pkts provides glimpse inside
• simple loss model– independent loss probabilitiesk on link k
• method– multicast n packets from source– data: list of packets received at each receiver– check consistency of data with independent loss model– analysis: Maximum Likelihood Estimator
• find which maximizes Prob[data | ]
R1 R2
R1 R2
1
32
![Page 14: Internet Performance Measurements and Measurement Techniques Jim Kurose Department of Computer Science University of Massachusetts/Amherst kurose](https://reader035.vdocuments.net/reader035/viewer/2022070400/56649f135503460f94c277a2/html5/thumbnails/14.jpg)
Multicast inference: evaluation
• through ns simulations– 2-8 receivers– different topologies– TCP, on/off background sources
• approach tracks probe loss well• good estimate of back- ground traffic loss
![Page 15: Internet Performance Measurements and Measurement Techniques Jim Kurose Department of Computer Science University of Massachusetts/Amherst kurose](https://reader035.vdocuments.net/reader035/viewer/2022070400/56649f135503460f94c277a2/html5/thumbnails/15.jpg)
Multicast Inference: to-do listObservations:• multicast-based inference promising for loss• applicable to delaysResearch questions:• what if topology partially unknown?• can we identify bottleneck links?Potential Applications:• Internet weather map• use in adaptive applications
UMass collaboration with AT&T, LBNL
![Page 16: Internet Performance Measurements and Measurement Techniques Jim Kurose Department of Computer Science University of Massachusetts/Amherst kurose](https://reader035.vdocuments.net/reader035/viewer/2022070400/56649f135503460f94c277a2/html5/thumbnails/16.jpg)
End-End Loss Delay Characteristics
• Question: time correlation of e-e loss, delay ?
• Application:– adjustment of FEC for audio, video, data– playout delay adjustment for audio– analytic models: how many “states” needed in
Markovian models?
• Approach: collect/analyze point-point, multicast traces of periodically generated UDP probes
![Page 17: Internet Performance Measurements and Measurement Techniques Jim Kurose Department of Computer Science University of Massachusetts/Amherst kurose](https://reader035.vdocuments.net/reader035/viewer/2022070400/56649f135503460f94c277a2/html5/thumbnails/17.jpg)
Analysis Issues:
• stationarity of traces:– look for increasing trends in avg, variance
over trace– non-stationary traces not considered
• removal of clock skew– algorithm for removing constant clock drift
• how “confident” are we in the measured value?• 150 hours of measurement data
– there’s an exception to every “typical” result
![Page 18: Internet Performance Measurements and Measurement Techniques Jim Kurose Department of Computer Science University of Massachusetts/Amherst kurose](https://reader035.vdocuments.net/reader035/viewer/2022070400/56649f135503460f94c277a2/html5/thumbnails/18.jpg)
Analysis Metrics:
• delay autocorrelation: dj: measured delay of pkt j
• loss autocorrelation: xj = 0 if pkt j received
= 0 if pkt j lost
• conditional average delay given loss:
dad d d d
d dii
j i j
j
( )( )
( ), ,...
21
E d ld
ljj j i
k i
k[ | ] , ,...
1 1
lax x x x
x xii
j i j
j
( )( )
( ), ,...
21
![Page 19: Internet Performance Measurements and Measurement Techniques Jim Kurose Department of Computer Science University of Massachusetts/Amherst kurose](https://reader035.vdocuments.net/reader035/viewer/2022070400/56649f135503460f94c277a2/html5/thumbnails/19.jpg)
Delay Autocorrelation
• Note: typically autocorrelation dies down quickly
Title:
Creator:gnuplotPreview:This EPS picture was not savedwith a preview included in it.Comment:This EPS picture will print to aPostScript printer, but not toother types of printers.
![Page 20: Internet Performance Measurements and Measurement Techniques Jim Kurose Department of Computer Science University of Massachusetts/Amherst kurose](https://reader035.vdocuments.net/reader035/viewer/2022070400/56649f135503460f94c277a2/html5/thumbnails/20.jpg)
Conditional Delay Given Loss:
• Interesting behavior!• Loss appears to be predictor of near term
higher-than average delays
Title:
Creator:gnuplotPreview:This EPS picture was not savedwith a preview included in it.Comment:This EPS picture will print to aPostScript printer, but not toother types of printers.
![Page 21: Internet Performance Measurements and Measurement Techniques Jim Kurose Department of Computer Science University of Massachusetts/Amherst kurose](https://reader035.vdocuments.net/reader035/viewer/2022070400/56649f135503460f94c277a2/html5/thumbnails/21.jpg)
Loss Autocorrelation:
• generally: loss correlation timescale < 500 ms
• modeling: length of consecutive losses, successful reception can be modeled accurately by 2 or 3 state Markov process
Title:/usr/mcast/yajnik/lossdata/traces3-loss-autoc.epsCreator:MATLAB, The Mathworks, Inc.Preview:This EPS picture was not savedwith a preview included in it.Comment:This EPS picture will print to aPostScript printer, but not toother types of printers.
![Page 22: Internet Performance Measurements and Measurement Techniques Jim Kurose Department of Computer Science University of Massachusetts/Amherst kurose](https://reader035.vdocuments.net/reader035/viewer/2022070400/56649f135503460f94c277a2/html5/thumbnails/22.jpg)
How many states needed in analytic model?
• For n-state Markov model, determine transition probabilities from observed data
Title:/usr/mcast/yajnik/lossdata/20Dec97.uni/baddist.epsCreator:MATLAB, The Mathworks, Inc.Preview:This EPS picture was not savedwith a preview included in it.Comment:This EPS picture will print to aPostScript printer, but not toother types of printers.
Title:/usr/mcast/yajnik/lossdata/20Dec97.uni/gooddist.epsCreator:MATLAB, The Mathworks, Inc.Preview:This EPS picture was not savedwith a preview included in it.Comment:This EPS picture will print to aPostScript printer, but not toother types of printers.
• needed: rigorous hypothesis testing of agreement between model and observed distributions
![Page 23: Internet Performance Measurements and Measurement Techniques Jim Kurose Department of Computer Science University of Massachusetts/Amherst kurose](https://reader035.vdocuments.net/reader035/viewer/2022070400/56649f135503460f94c277a2/html5/thumbnails/23.jpg)
“Confidence” in loss probability estimates
• suppose: we send 10 packets and see 3 lost– view loss as a random process– is loss rate “really” 30%?– could be true loss rate is 20% or 50% !– if we sample more, we’d have more “confidence” in
the estimate
• goal: interval estimator for loss rate– e.g.: 95% confident that true loss in range [p1,p2]
• use: adaptive applications (e.g., using RTCP)
![Page 24: Internet Performance Measurements and Measurement Techniques Jim Kurose Department of Computer Science University of Massachusetts/Amherst kurose](https://reader035.vdocuments.net/reader035/viewer/2022070400/56649f135503460f94c277a2/html5/thumbnails/24.jpg)
Example:
Bernoulli loss process: each pkt lost independently
with probability p
95% confidence interval around MLE: find [ p1, p2 ] such that Pr{loss[k, n] | p = p1} = Pr{loss[0, k] | p = p2} = 0.025
n = 10 k = 3 MLE = k/n = 0.3
Loss probability confidence: model
p1 0.07 p2 0.65
0 1 2 3 4 5 6 7 8 9 10
0.1
0.2
0.3
0.4
0.5
0 1 2 3 4 5 6 7 8 9 10
0.1
0.2
0.3
0.4
0.5
![Page 25: Internet Performance Measurements and Measurement Techniques Jim Kurose Department of Computer Science University of Massachusetts/Amherst kurose](https://reader035.vdocuments.net/reader035/viewer/2022070400/56649f135503460f94c277a2/html5/thumbnails/25.jpg)
Loss probability estimation: intervals
0 1000 2000 3000 4000 5000
0.2
0.4
0.6
0.8
1
MLE = 0.50
0.10
0.05
0.02
0.01
number of packets sent n
95%confidenceintervalwidthrelativeto MLE
![Page 26: Internet Performance Measurements and Measurement Techniques Jim Kurose Department of Computer Science University of Massachusetts/Amherst kurose](https://reader035.vdocuments.net/reader035/viewer/2022070400/56649f135503460f94c277a2/html5/thumbnails/26.jpg)
What’s ahead?
• need for statistically rigorous, empirically verified, end-user oriented performance measurement tools and technique– research just beginning
• middleware: network-to-user performance feedback?– when and in what form?
• informed use of performance measurements in:– adaptive applications– active services
![Page 27: Internet Performance Measurements and Measurement Techniques Jim Kurose Department of Computer Science University of Massachusetts/Amherst kurose](https://reader035.vdocuments.net/reader035/viewer/2022070400/56649f135503460f94c277a2/html5/thumbnails/27.jpg)
For More Information …..
• This talk: ftp://gaia.cs.umass.edu/pub/kurose/intel98.ps
• Group publications: http://gaia.cs.umass.edu/papers• WWW sites:
– Cooperative Association for Internet Data Analysis www.caida.org– National Laboratory for Applied Network Research www.nlanr.net