trajectory sampling for direct traffic oberservation n.g. duffield and matthias grossglauser...
TRANSCRIPT
![Page 1: Trajectory Sampling for Direct Traffic Oberservation N.G. Duffield and Matthias Grossglauser IEEE/ACM Transactions on Networking, Vol. 9, No. 3 June 2001](https://reader035.vdocuments.net/reader035/viewer/2022070404/56649f395503460f94c55813/html5/thumbnails/1.jpg)
Trajectory Sampling for Direct Traffic Oberservation
N.G. Duffield and Matthias Grossglauser
IEEE/ACM Transactions on Networking, Vol. 9, No. 3 June 2001
![Page 2: Trajectory Sampling for Direct Traffic Oberservation N.G. Duffield and Matthias Grossglauser IEEE/ACM Transactions on Networking, Vol. 9, No. 3 June 2001](https://reader035.vdocuments.net/reader035/viewer/2022070404/56649f395503460f94c55813/html5/thumbnails/2.jpg)
Problem: Which (spatial) path does traffic take?
Circuit switched networks (e.g. telephone):
Per-call state is maintained
=>trivial
IP networks:Don’t maintain per-
flow information
?
![Page 3: Trajectory Sampling for Direct Traffic Oberservation N.G. Duffield and Matthias Grossglauser IEEE/ACM Transactions on Networking, Vol. 9, No. 3 June 2001](https://reader035.vdocuments.net/reader035/viewer/2022070404/56649f395503460f94c55813/html5/thumbnails/3.jpg)
Why is this interesting? Quality of Service depends on traffic
management Traffic control
Timescale: seconds no human intervention
Traffic engineering Timescale: minutes - months Resource allocation Pricing Failover strategies
![Page 4: Trajectory Sampling for Direct Traffic Oberservation N.G. Duffield and Matthias Grossglauser IEEE/ACM Transactions on Networking, Vol. 9, No. 3 June 2001](https://reader035.vdocuments.net/reader035/viewer/2022070404/56649f395503460f94c55813/html5/thumbnails/4.jpg)
Options
Indirect measurement
Uses information on Network model Network state
Direct measurement
Direct observation of traffic at multiple points in the network
![Page 5: Trajectory Sampling for Direct Traffic Oberservation N.G. Duffield and Matthias Grossglauser IEEE/ACM Transactions on Networking, Vol. 9, No. 3 June 2001](https://reader035.vdocuments.net/reader035/viewer/2022070404/56649f395503460f94c55813/html5/thumbnails/5.jpg)
Problems with indirect measurement Behavior of network elements
depends on vendor-specific design choices
Deliberate sources of randomness to avoid collision
Events outside domain (route advertising by neighboring domains)
Interactions may be too complex to predict
![Page 6: Trajectory Sampling for Direct Traffic Oberservation N.G. Duffield and Matthias Grossglauser IEEE/ACM Transactions on Networking, Vol. 9, No. 3 June 2001](https://reader035.vdocuments.net/reader035/viewer/2022070404/56649f395503460f94c55813/html5/thumbnails/6.jpg)
Direct measurement:Sampling of packets Sample packets that traverse each
link Subset of packets used as
representative
Problem: How do we get the actual
path?
![Page 7: Trajectory Sampling for Direct Traffic Oberservation N.G. Duffield and Matthias Grossglauser IEEE/ACM Transactions on Networking, Vol. 9, No. 3 June 2001](https://reader035.vdocuments.net/reader035/viewer/2022070404/56649f395503460f94c55813/html5/thumbnails/7.jpg)
Key idea of the paper Use a deterministic hash function
over the packet’s content to determine subset of packets
Use the same hash function throughout the domain
Use second hash function to label packets
![Page 8: Trajectory Sampling for Direct Traffic Oberservation N.G. Duffield and Matthias Grossglauser IEEE/ACM Transactions on Networking, Vol. 9, No. 3 June 2001](https://reader035.vdocuments.net/reader035/viewer/2022070404/56649f395503460f94c55813/html5/thumbnails/8.jpg)
Theory Measurement domain represented as a
directed graph Packets
enter at ingress node exit at egress node
Invariance function Packet content without changing fields, e.g.time-to-live field which is decremented each
hop
![Page 9: Trajectory Sampling for Direct Traffic Oberservation N.G. Duffield and Matthias Grossglauser IEEE/ACM Transactions on Networking, Vol. 9, No. 3 June 2001](https://reader035.vdocuments.net/reader035/viewer/2022070404/56649f395503460f94c55813/html5/thumbnails/9.jpg)
Sampling Hash Function Decides whether or not a given
packet should be sampled Deterministic function of the
invariant packet content Same function on each link Results in L-bit binary number
![Page 10: Trajectory Sampling for Direct Traffic Oberservation N.G. Duffield and Matthias Grossglauser IEEE/ACM Transactions on Networking, Vol. 9, No. 3 June 2001](https://reader035.vdocuments.net/reader035/viewer/2022070404/56649f395503460f94c55813/html5/thumbnails/10.jpg)
Identification Hash Function Entire packet content could be used Aim: limit traffic to measurement
collection system Results in m-bit binary number Additional information may be
included Length of packet Source, destination
![Page 11: Trajectory Sampling for Direct Traffic Oberservation N.G. Duffield and Matthias Grossglauser IEEE/ACM Transactions on Networking, Vol. 9, No. 3 June 2001](https://reader035.vdocuments.net/reader035/viewer/2022070404/56649f395503460f94c55813/html5/thumbnails/11.jpg)
Invariant content Header: three categories of fields
Variable fields (not included) E.g., TTL, header checksum, etc.
Low entropy fields (not included) Content changes little between packets E.g., version, header length, protocol
High entropy fields (included) Source and destination IP, etc.
Part of remainder of packet
![Page 12: Trajectory Sampling for Direct Traffic Oberservation N.G. Duffield and Matthias Grossglauser IEEE/ACM Transactions on Networking, Vol. 9, No. 3 June 2001](https://reader035.vdocuments.net/reader035/viewer/2022070404/56649f395503460f94c55813/html5/thumbnails/12.jpg)
Ambiguities (f-h)
![Page 13: Trajectory Sampling for Direct Traffic Oberservation N.G. Duffield and Matthias Grossglauser IEEE/ACM Transactions on Networking, Vol. 9, No. 3 June 2001](https://reader035.vdocuments.net/reader035/viewer/2022070404/56649f395503460f94c55813/html5/thumbnails/13.jpg)
Dealing with ambiguities Probability that trajectory can be
disambiguated depends on network topology and traffic => renormalization of results necessary
Safer to discard all duplicate labels (greater loss of samples)
![Page 14: Trajectory Sampling for Direct Traffic Oberservation N.G. Duffield and Matthias Grossglauser IEEE/ACM Transactions on Networking, Vol. 9, No. 3 June 2001](https://reader035.vdocuments.net/reader035/viewer/2022070404/56649f395503460f94c55813/html5/thumbnails/14.jpg)
Specification of Hash Functions Ordered bits of invariant part of
packet content x are considered as binary integers: (x)
Sampling hash functionh((x)) = (x) mod A
Identification hash functiong((x)) = (x) mod B
with A, B positive integers
![Page 15: Trajectory Sampling for Direct Traffic Oberservation N.G. Duffield and Matthias Grossglauser IEEE/ACM Transactions on Networking, Vol. 9, No. 3 June 2001](https://reader035.vdocuments.net/reader035/viewer/2022070404/56649f395503460f94c55813/html5/thumbnails/15.jpg)
Identical Packets Automatically ambiguous
=> lead to biased estimatorsQuestion: How much packet content is
needed to avoid collisions?Answer: 40 bytes lead to collision
probability smaller than 10-3
![Page 16: Trajectory Sampling for Direct Traffic Oberservation N.G. Duffield and Matthias Grossglauser IEEE/ACM Transactions on Networking, Vol. 9, No. 3 June 2001](https://reader035.vdocuments.net/reader035/viewer/2022070404/56649f395503460f94c55813/html5/thumbnails/16.jpg)
Implementation of hashing 40 byte “numbers” are represented by
vector of 16 bit wordsz = (zk ,zk-1,…,z0) = i=0
k zi 216i
Use 32 bit long division Iteratively compute
(zk ,zk-1,…,z0) mod A
= (zk-1+ 216(zk mod A),…,z0) mod A
![Page 17: Trajectory Sampling for Direct Traffic Oberservation N.G. Duffield and Matthias Grossglauser IEEE/ACM Transactions on Networking, Vol. 9, No. 3 June 2001](https://reader035.vdocuments.net/reader035/viewer/2022070404/56649f395503460f94c55813/html5/thumbnails/17.jpg)
Sampling independent of packet content? Note: IP address of source and
destination are included in the invariant content!
Chi-squared test 40 byte packet prefix
=> 95% confidence level 20 byte packet prefix results in strong
dependence
![Page 18: Trajectory Sampling for Direct Traffic Oberservation N.G. Duffield and Matthias Grossglauser IEEE/ACM Transactions on Networking, Vol. 9, No. 3 June 2001](https://reader035.vdocuments.net/reader035/viewer/2022070404/56649f395503460f94c55813/html5/thumbnails/18.jpg)
Optimal Sampling Tradeoff
More unambiguous samples => more accuracy
More samples => more measurement traffic
Optimize for given measurement traffic mn(m bits per sample, n samples) Small m increases collisions Large m means smaller n
![Page 19: Trajectory Sampling for Direct Traffic Oberservation N.G. Duffield and Matthias Grossglauser IEEE/ACM Transactions on Networking, Vol. 9, No. 3 June 2001](https://reader035.vdocuments.net/reader035/viewer/2022070404/56649f395503460f94c55813/html5/thumbnails/19.jpg)
(Question to the authors Doesn’t the measurement traffic
itself get sampled and thereby add another source of error?
… may be part of their future work statement)
![Page 20: Trajectory Sampling for Direct Traffic Oberservation N.G. Duffield and Matthias Grossglauser IEEE/ACM Transactions on Networking, Vol. 9, No. 3 June 2001](https://reader035.vdocuments.net/reader035/viewer/2022070404/56649f395503460f94c55813/html5/thumbnails/20.jpg)
ExampleService provider wants to determine what
fraction of packets on a certain backbone link belongs to a certain customer
Compare customer packets observed both on backbone
and on access link Total number of packets observed on
backbone Real and estimated fractions largely
within error bars
![Page 21: Trajectory Sampling for Direct Traffic Oberservation N.G. Duffield and Matthias Grossglauser IEEE/ACM Transactions on Networking, Vol. 9, No. 3 June 2001](https://reader035.vdocuments.net/reader035/viewer/2022070404/56649f395503460f94c55813/html5/thumbnails/21.jpg)
Implementation issues Can trajectory sampling be part of next
generation of high-speed interfaces? Authors claim “yes”:
Compute both hash functions in parallel Processor cost negligible compared with
cost of interface cards Processor speed doubles every 18 months,
maximum trunk speed every 21 months
![Page 22: Trajectory Sampling for Direct Traffic Oberservation N.G. Duffield and Matthias Grossglauser IEEE/ACM Transactions on Networking, Vol. 9, No. 3 June 2001](https://reader035.vdocuments.net/reader035/viewer/2022070404/56649f395503460f94c55813/html5/thumbnails/22.jpg)
Other Common Approaches Aggregation-
based approaches e.g., sum of
packets traversing a link
Sampling-based approaches sample subset of
observations
![Page 23: Trajectory Sampling for Direct Traffic Oberservation N.G. Duffield and Matthias Grossglauser IEEE/ACM Transactions on Networking, Vol. 9, No. 3 June 2001](https://reader035.vdocuments.net/reader035/viewer/2022070404/56649f395503460f94c55813/html5/thumbnails/23.jpg)
Aggregation-based Approaches Link measurements (direct)
Traffic statistics (# of bytes / # of packets transferred / dropped)
Measurements reported periodically Flow aggregation (indirect)
Flow: sequence of packets with common field in header
Relies on emulation of routing protocol
![Page 24: Trajectory Sampling for Direct Traffic Oberservation N.G. Duffield and Matthias Grossglauser IEEE/ACM Transactions on Networking, Vol. 9, No. 3 June 2001](https://reader035.vdocuments.net/reader035/viewer/2022070404/56649f395503460f94c55813/html5/thumbnails/24.jpg)
Sampling-based Approaches Active end-to-end probes (direct)
Hosts send probe packets to one or more other hosts
Packet loss rate Round-trip delay
End-to-end path characteristics Variation: collect and exchange
measurements of multicast session
![Page 25: Trajectory Sampling for Direct Traffic Oberservation N.G. Duffield and Matthias Grossglauser IEEE/ACM Transactions on Networking, Vol. 9, No. 3 June 2001](https://reader035.vdocuments.net/reader035/viewer/2022070404/56649f395503460f94c55813/html5/thumbnails/25.jpg)
Related Work Measure end-to-end performance
of individual flows ATM cells sampled at ingress and
egress points Determine QoS for a single
connection, e.g., delay and loss rate
![Page 26: Trajectory Sampling for Direct Traffic Oberservation N.G. Duffield and Matthias Grossglauser IEEE/ACM Transactions on Networking, Vol. 9, No. 3 June 2001](https://reader035.vdocuments.net/reader035/viewer/2022070404/56649f395503460f94c55813/html5/thumbnails/26.jpg)
Extensions and Other Applications Distributed denial of service attacks
Attackers use packet spoofing Filtering
A configurable packet filter may allow trajectory sampling for a subset of packets
Probe Packets Packet content may be constructed to
ensure sampling
![Page 27: Trajectory Sampling for Direct Traffic Oberservation N.G. Duffield and Matthias Grossglauser IEEE/ACM Transactions on Networking, Vol. 9, No. 3 June 2001](https://reader035.vdocuments.net/reader035/viewer/2022070404/56649f395503460f94c55813/html5/thumbnails/27.jpg)
Conclusions Simple processing No Router state required Packets directly observed