network emulator
DESCRIPTION
TRANSCRIPT
![Page 1: Network emulator](https://reader033.vdocuments.net/reader033/viewer/2022061210/548fd848b4795927058b4fab/html5/thumbnails/1.jpg)
Network Emulation using tc
Jeromy Fu
![Page 2: Network emulator](https://reader033.vdocuments.net/reader033/viewer/2022061210/548fd848b4795927058b4fab/html5/thumbnails/2.jpg)
Agenda
• Why emulation• What to be emulated• How TC works• Emulation Howto• Compared with Nistnet/WANem• Other references
![Page 3: Network emulator](https://reader033.vdocuments.net/reader033/viewer/2022061210/548fd848b4795927058b4fab/html5/thumbnails/3.jpg)
Mathematic model
• Mathematic model analysis can provide import insight on the behavior of a system
• But, sometimes difficult because of too many factors combined
![Page 4: Network emulator](https://reader033.vdocuments.net/reader033/viewer/2022061210/548fd848b4795927058b4fab/html5/thumbnails/4.jpg)
Network simulator
• Network simulator is a software program that imitates the working of a computer network
• fast and inexpensive• controlled and reproducible environment
![Page 5: Network emulator](https://reader033.vdocuments.net/reader033/viewer/2022061210/548fd848b4795927058b4fab/html5/thumbnails/5.jpg)
Network emulator
• A network emulator emulates the network which connects end-systems, not the end-systems themselves
• transmit actual network traffic• Can use real code
![Page 6: Network emulator](https://reader033.vdocuments.net/reader033/viewer/2022061210/548fd848b4795927058b4fab/html5/thumbnails/6.jpg)
Real world test
• Impractical experiments• Planet Lab helps• not reproducible
![Page 7: Network emulator](https://reader033.vdocuments.net/reader033/viewer/2022061210/548fd848b4795927058b4fab/html5/thumbnails/7.jpg)
Why emulator
• Complementary
Like
lihoo
d of
Ri
sk O
ccur
renc
e
Consequence of Risk Occurrence
More reality
M
ore
unco
ntro
lled
Trade off
![Page 8: Network emulator](https://reader033.vdocuments.net/reader033/viewer/2022061210/548fd848b4795927058b4fab/html5/thumbnails/8.jpg)
Agenda
• Why emulation• What to be emulated• How TC works• Emulation Howto• Compared with Nistnet/WANem• Other references
![Page 9: Network emulator](https://reader033.vdocuments.net/reader033/viewer/2022061210/548fd848b4795927058b4fab/html5/thumbnails/9.jpg)
What to be emulated
• Bandwidth.• Shaped(Policed) bandwidth more specifically.• Capacity can’t be emulated.
![Page 10: Network emulator](https://reader033.vdocuments.net/reader033/viewer/2022061210/548fd848b4795927058b4fab/html5/thumbnails/10.jpg)
What to be emulated
• RTT
• Jitter
• Queuing delay
![Page 11: Network emulator](https://reader033.vdocuments.net/reader033/viewer/2022061210/548fd848b4795927058b4fab/html5/thumbnails/11.jpg)
What to be emulated
• Duplicate/Disorder/Corrupt• Lossrate• Loss burstiness, a longest sequence beginning
and ending with a loss, consecutive received packets is less than some value Gmin
![Page 12: Network emulator](https://reader033.vdocuments.net/reader033/viewer/2022061210/548fd848b4795927058b4fab/html5/thumbnails/12.jpg)
Agenda
• Why emulation• What to be emulated• How TC works• Emulation Howto• Compared with Nistnet/WANem• Other references
![Page 13: Network emulator](https://reader033.vdocuments.net/reader033/viewer/2022061210/548fd848b4795927058b4fab/html5/thumbnails/13.jpg)
What is TC• TC is abbr. of Traffic Control
- Rate control
- Bandwidth management
- Active Queue Management(AQM)
- Network Emulator, pkt loss, pkt disorder, pkt duplication, pkt delay
- QoS ( diffserv + rsvp )
- Many more …
![Page 14: Network emulator](https://reader033.vdocuments.net/reader033/viewer/2022061210/548fd848b4795927058b4fab/html5/thumbnails/14.jpg)
How TC works
![Page 15: Network emulator](https://reader033.vdocuments.net/reader033/viewer/2022061210/548fd848b4795927058b4fab/html5/thumbnails/15.jpg)
How TC works
![Page 16: Network emulator](https://reader033.vdocuments.net/reader033/viewer/2022061210/548fd848b4795927058b4fab/html5/thumbnails/16.jpg)
TC basic concepts
• Classification(Filter)
- Used to distinguish among different classes of packets and process each class in a specific way.
• Qdisc(Queue discipline)
- Decide which ones to send first, which ones to delay, and which ones to drop
- class/classful Qdisc: Qdisc with/without configurable internal subdivision
![Page 17: Network emulator](https://reader033.vdocuments.net/reader033/viewer/2022061210/548fd848b4795927058b4fab/html5/thumbnails/17.jpg)
TC basic concepts• Class
Classes either contain other Classes, or a Qdisc is attached
Qdiscs and Classes are intimately tied together
• Action
Actions get attached to classifiers and are invoked after a successful classification. Common used actions includes instantly drop, modify or redirect packets, etc.
Works on ingress only.
![Page 18: Network emulator](https://reader033.vdocuments.net/reader033/viewer/2022061210/548fd848b4795927058b4fab/html5/thumbnails/18.jpg)
TC basic concepts
![Page 19: Network emulator](https://reader033.vdocuments.net/reader033/viewer/2022061210/548fd848b4795927058b4fab/html5/thumbnails/19.jpg)
TC Commands
• OPTIONS: options are effective for all sub commands
• OBJECTS: the object of the tc command operates on
• COMMAND: the sub command for each object
![Page 20: Network emulator](https://reader033.vdocuments.net/reader033/viewer/2022061210/548fd848b4795927058b4fab/html5/thumbnails/20.jpg)
TC Qdisc
• Operations on qdisc: add | del | replace | change | show
• Handle: qdisc handle used to identify qdisc• root|ingress|parent CLASSID(handle), specify
the parent node
![Page 21: Network emulator](https://reader033.vdocuments.net/reader033/viewer/2022061210/548fd848b4795927058b4fab/html5/thumbnails/21.jpg)
qdisc handle
• Qdisc handle is used to identify Qdisc
- {none|major[:]}
- none, autogen by kernel
- major is 16bits HEX number(Without ‘0x’ prefix)
- : is optional
• Internally, qdisc_handle = major<<16
![Page 22: Network emulator](https://reader033.vdocuments.net/reader033/viewer/2022061210/548fd848b4795927058b4fab/html5/thumbnails/22.jpg)
TC class
• Class’s parent can be class or qdisc, classid should have the same major with parent
• classid, {[major]:minor} - major/minor are both 16bits HEX numbers(Without ‘0x’ prefix), major is optional
• Internally, classid = (major<<16)|minor
![Page 23: Network emulator](https://reader033.vdocuments.net/reader033/viewer/2022061210/548fd848b4795927058b4fab/html5/thumbnails/23.jpg)
TC filter
• Perf(prio): priority of matching.• Protocol: protocol on which the filter must
operate, ip/icmp etc, see /etc/protocols.• root|classid CLASSID|handle FILTERID, specify
the class or qdisc attached.
![Page 24: Network emulator](https://reader033.vdocuments.net/reader033/viewer/2022061210/548fd848b4795927058b4fab/html5/thumbnails/24.jpg)
Classful qdisc example
![Page 25: Network emulator](https://reader033.vdocuments.net/reader033/viewer/2022061210/548fd848b4795927058b4fab/html5/thumbnails/25.jpg)
Classful qdisc example
![Page 26: Network emulator](https://reader033.vdocuments.net/reader033/viewer/2022061210/548fd848b4795927058b4fab/html5/thumbnails/26.jpg)
Agenda
• Why emulation• What to be emulated• How TC works• Emulation Howto• Compared with Nistnet/WANem• Other references
![Page 27: Network emulator](https://reader033.vdocuments.net/reader033/viewer/2022061210/548fd848b4795927058b4fab/html5/thumbnails/27.jpg)
topology
• Client, Emulator and Server are in the same subnet. Add route.
![Page 28: Network emulator](https://reader033.vdocuments.net/reader033/viewer/2022061210/548fd848b4795927058b4fab/html5/thumbnails/28.jpg)
topology
• Client, Emulator in one subnet, server in another subnet. Use NAT.
![Page 29: Network emulator](https://reader033.vdocuments.net/reader033/viewer/2022061210/548fd848b4795927058b4fab/html5/thumbnails/29.jpg)
tc-tbf
• Tokens are added at a fixed rate• Check if the bucket contains sufficient tokens
![Page 30: Network emulator](https://reader033.vdocuments.net/reader033/viewer/2022061210/548fd848b4795927058b4fab/html5/thumbnails/30.jpg)
Bernoulli loss model
• Model uncorrelated loss events, “loss probability” p.
• Two state, one independent parameter.
![Page 31: Network emulator](https://reader033.vdocuments.net/reader033/viewer/2022061210/548fd848b4795927058b4fab/html5/thumbnails/31.jpg)
Simple Gilbert model
• A system with “consecutive loss events”, which can be characterized by a “loss probability”. (p)and a “burst duration” (1-r).
• Two state, two independent parameters.• 1-r = p -> Bernoulli
![Page 32: Network emulator](https://reader033.vdocuments.net/reader033/viewer/2022061210/548fd848b4795927058b4fab/html5/thumbnails/32.jpg)
Gilbert model
• Within the Bad state there is a probability h that a packet is transmitted.
• “loss probability” (p), a “burst duration”(1-r) and a “loss density”(1-h).
• Two state, three independent parameters.• h=0->Simple
![Page 33: Network emulator](https://reader033.vdocuments.net/reader033/viewer/2022061210/548fd848b4795927058b4fab/html5/thumbnails/33.jpg)
Gilbert-Elliot model
• k is the probability that the packet is transmitted while the system is in Good state.
• In good state, loss events appear as “isolated” and independent with each other
• Two state, four independent parameters.• k=1-> Gilbert
![Page 34: Network emulator](https://reader033.vdocuments.net/reader033/viewer/2022061210/548fd848b4795927058b4fab/html5/thumbnails/34.jpg)
4-state Markov chain
![Page 35: Network emulator](https://reader033.vdocuments.net/reader033/viewer/2022061210/548fd848b4795927058b4fab/html5/thumbnails/35.jpg)
difference
![Page 36: Network emulator](https://reader033.vdocuments.net/reader033/viewer/2022061210/548fd848b4795927058b4fab/html5/thumbnails/36.jpg)
tc-netem
• Loss random(independent loss probability, correlation can be added)| Loss state | Loss gemodel | ecn
![Page 37: Network emulator](https://reader033.vdocuments.net/reader033/viewer/2022061210/548fd848b4795927058b4fab/html5/thumbnails/37.jpg)
tc-netem
• crand(n) = corr*crand(n-1) + (1-corr)*rand()• delay(n) = delay + distri(jitter, crand(n))• duplicate, corrupt , loss, reorder aslo use
crand.• Delay should specified if need reorder(packets
should be queued first)• If gap not specified, gap = 1 will be used.
![Page 38: Network emulator](https://reader033.vdocuments.net/reader033/viewer/2022061210/548fd848b4795927058b4fab/html5/thumbnails/38.jpg)
Distribution table
![Page 39: Network emulator](https://reader033.vdocuments.net/reader033/viewer/2022061210/548fd848b4795927058b4fab/html5/thumbnails/39.jpg)
Why not loss correlation
• Correlation changed the distribution
![Page 40: Network emulator](https://reader033.vdocuments.net/reader033/viewer/2022061210/548fd848b4795927058b4fab/html5/thumbnails/40.jpg)
Netem example
• tc qdisc add dev eth0 root netem delay 100ms 20ms 25% distribution normal
• tc qdisc add dev eth0 root netem loss 0.3% 25%
• tc qdisc add dev eth0 root netem duplicate 1% corrupt 0.1%
• tc qdisc add dev eth0 root netem delay 10ms reorder 25% 50% gap 5
![Page 41: Network emulator](https://reader033.vdocuments.net/reader033/viewer/2022061210/548fd848b4795927058b4fab/html5/thumbnails/41.jpg)
Bandwidth emulation - tbf
• Tc-tbf
• bfifo is the default child qdisc of tbf,• can be replace by other qdiscs such as pfifo.
![Page 42: Network emulator](https://reader033.vdocuments.net/reader033/viewer/2022061210/548fd848b4795927058b4fab/html5/thumbnails/42.jpg)
Bandwidth emulation - tbf
• limit - limit is the size (in bytes) of bfifo, bfifo is the queue which stores the packets.
• rate - the bandwidth cap we need to enforce• burst/buffer/maxburst - this is the bucket size
of the first tbf. Its value should be larger than rate/HZ to achieve the specified throughput, the larger value means more burst when traffic starts(tokens are accumulated in large bucket).
![Page 43: Network emulator](https://reader033.vdocuments.net/reader033/viewer/2022061210/548fd848b4795927058b4fab/html5/thumbnails/43.jpg)
Bandwidth emulation - tbf
• peakrate - if we only have one bucket, the burst rate will be larger than rate we set, so we need peakrate to limit the burst. the peakrate should be no less than rate.
• mtu/minburst - most of the time, set this to MTU of the interface, larger values means larger burst.
![Page 44: Network emulator](https://reader033.vdocuments.net/reader033/viewer/2022061210/548fd848b4795927058b4fab/html5/thumbnails/44.jpg)
Policing and shaping
• Policer: Rate limiting without buffering, typically set at ingress, un-conformed packets are dropped directly.
• Shaper: Rate limiting with buffering, typically set at exgress, and can be buffered and then if no extra buffer, then be dropped, will add extra queuing delay.
![Page 45: Network emulator](https://reader033.vdocuments.net/reader033/viewer/2022061210/548fd848b4795927058b4fab/html5/thumbnails/45.jpg)
Policing and shaping
![Page 46: Network emulator](https://reader033.vdocuments.net/reader033/viewer/2022061210/548fd848b4795927058b4fab/html5/thumbnails/46.jpg)
Shape emulation
• No delay
Bandwidth cap 1mbit/s, don't allow burst traffic, then burst = max(MTU, rate/8/HZ) = (3000,1000000/8/100) = (3000,1250) = 3000,
if queuing delay 100ms, set latency 100ms or we set limit = qdelay*rate/8/1000+burst = 100*1000000/8/1000+3000 = 13750
![Page 47: Network emulator](https://reader033.vdocuments.net/reader033/viewer/2022061210/548fd848b4795927058b4fab/html5/thumbnails/47.jpg)
Shape emulation
• With delay
attach netem to the engress first, and then add tbf to the child qdisc of netem.
use limit parameter for tbf here, if using latency, tbf will not include the extra buffer needed for netem
limit = tbf_burst + netem_qsize + tbf_qsize = max(rate/8/hz, MTU) + delay*rate/8000 + qdelay*rate/8000.
![Page 48: Network emulator](https://reader033.vdocuments.net/reader033/viewer/2022061210/548fd848b4795927058b4fab/html5/thumbnails/48.jpg)
Police emulation
• policer drops packets directly for the Non-conformant packets, and it has no buffer
• tc-tbf with very small buffer.• tc-tbf use bfifo as the default child qdisc,
queue length(in bytes) is set automatically by specifying 'limit' or 'latency', which ensures that the queue length is no less than token bucket depth(introduce queuing delay).
![Page 49: Network emulator](https://reader033.vdocuments.net/reader033/viewer/2022061210/548fd848b4795927058b4fab/html5/thumbnails/49.jpg)
Police emulation
• Workaround is replace the bfifo with pfifo
• You can also use police on ingress. tc-police also use token bucket to do bandwidth cap, but it don't own queue, so there's no qdelay introduced.
![Page 50: Network emulator](https://reader033.vdocuments.net/reader033/viewer/2022061210/548fd848b4795927058b4fab/html5/thumbnails/50.jpg)
Burst emulation
• Most adsl will allow some burst traffic, this kind of burst is caused by the large token bucket size, which accumulates many tokens when transmission starts.
• To emulate the burst, we only need to turn the 'burst' parameter.
![Page 51: Network emulator](https://reader033.vdocuments.net/reader033/viewer/2022061210/548fd848b4795927058b4fab/html5/thumbnails/51.jpg)
Burst emulation
• For example, we allow for 2mbit/s in the first second in the following case. 1m*t + burst = 2m*t => burst = 1m*t => burst=1m=125k
![Page 52: Network emulator](https://reader033.vdocuments.net/reader033/viewer/2022061210/548fd848b4795927058b4fab/html5/thumbnails/52.jpg)
Burst emulation
• What if adding delay? Any problem?• Extra burst will use the netem buffer and cause
extra queuing delay.• Separate the buffer using ifb(Intermediate
Functional Block device)• If traffic is redirect to ifb dev, it is returned back
to the original point when dequeueing from ifb.• Can add qdisc for ifb dev
![Page 53: Network emulator](https://reader033.vdocuments.net/reader033/viewer/2022061210/548fd848b4795927058b4fab/html5/thumbnails/53.jpg)
Burst emulation
• Using ifb
![Page 54: Network emulator](https://reader033.vdocuments.net/reader033/viewer/2022061210/548fd848b4795927058b4fab/html5/thumbnails/54.jpg)
Agenda
• Why emulation• What to be emulated• How TC works• Emulation Howto• Compared with Nistnet/WANem• Other references
![Page 55: Network emulator](https://reader033.vdocuments.net/reader033/viewer/2022061210/548fd848b4795927058b4fab/html5/thumbnails/55.jpg)
Compared with Nistnet/WANem
![Page 56: Network emulator](https://reader033.vdocuments.net/reader033/viewer/2022061210/548fd848b4795927058b4fab/html5/thumbnails/56.jpg)
How Nistnet work
![Page 57: Network emulator](https://reader033.vdocuments.net/reader033/viewer/2022061210/548fd848b4795927058b4fab/html5/thumbnails/57.jpg)
How Nistnet work
• Bandwidth limitation is implemented as adding delay, just like a packet go through a bottleneck link.
• Determine the amount of time to delay a
packet. This is the maximum of two quantities: 1. Probabilistic packet delay time 2. Bandwidth-limitation delay time
![Page 58: Network emulator](https://reader033.vdocuments.net/reader033/viewer/2022061210/548fd848b4795927058b4fab/html5/thumbnails/58.jpg)
How Nistnet work• probdelay = correlatedtabledist(&tableme->ltEntry.lteIDelay);
if (hitme->hitreq.bandwidth) { fixed_gettimeofday(&our_time); //last queue delay bandwidthdelay = timeval_diff(&hitme->next_packet, &our_time);
if (bandwidthdelay < 0) { bandwidthdelay = 0; hitme->next_packet = our_time; } //add transmission delay packettime = (long)skb->len*(MILLION/hitme->hitreq.bandwidth) + ((long)skb->len*(MILLION%hitme->hitreq.bandwidth) + hitme->hitreq.bandwidth/2)/hitme->hitreq.bandwidth; timeval_add(&hitme->next_packet, packettime); bandwidthdelay += packettime; }
delay = probdelay > bandwidthdelay ? probdelay : bandwidthdelay;
![Page 59: Network emulator](https://reader033.vdocuments.net/reader033/viewer/2022061210/548fd848b4795927058b4fab/html5/thumbnails/59.jpg)
Nistnet drawbacks
• Bandwidth model not emulate the real one.• Queuing delay and one way delay are
combined.• Buffer size can only be tune by DRD.• Only ip:port filter supported.• Not support 4 state loss burst model• Only DRD (Derivative Random Drop) AQM
supported.
![Page 60: Network emulator](https://reader033.vdocuments.net/reader033/viewer/2022061210/548fd848b4795927058b4fab/html5/thumbnails/60.jpg)
WANem
• WANem is just a WEB UI which use tc underneath.
![Page 61: Network emulator](https://reader033.vdocuments.net/reader033/viewer/2022061210/548fd848b4795927058b4fab/html5/thumbnails/61.jpg)
WANem
• WEB UI, Easy to use• Add connection disconnect• Queue size need patches to work• No burst settings• No settings for GE or 4-state loss model• Queuing delay can controlled directly
![Page 62: Network emulator](https://reader033.vdocuments.net/reader033/viewer/2022061210/548fd848b4795927058b4fab/html5/thumbnails/62.jpg)
Agenda
• Why emulation• What to be emulated• How TC works• Emulation Howto• Compared with Nistnet/WANem• Other references
![Page 63: Network emulator](https://reader033.vdocuments.net/reader033/viewer/2022061210/548fd848b4795927058b4fab/html5/thumbnails/63.jpg)
Reference
• NEWT (Network Emulator for Windows Toolkit) in vs2010
• Introducing True Network Emulation in Visual Studio 2010
• Network Emulator Toolkit• dummynet • Nistnet FAQ