perfsonar mdm and tcp buffers domenico vicinanza dante, cambridge, uk [email protected]...
TRANSCRIPT
![Page 1: PerfSONAR MDM and TCP buffers Domenico Vicinanza DANTE, Cambridge, UK domenico.vicinanza@dante.net EGI Technical Forum 2013, Madrid, Spain](https://reader034.vdocuments.net/reader034/viewer/2022051516/56649ee15503460f94bf20e8/html5/thumbnails/1.jpg)
perfSONAR MDM and TCP buffers
Domenico Vicinanza DANTE, Cambridge, UK
EGI Technical Forum 2013, Madrid, Spain
![Page 2: PerfSONAR MDM and TCP buffers Domenico Vicinanza DANTE, Cambridge, UK domenico.vicinanza@dante.net EGI Technical Forum 2013, Madrid, Spain](https://reader034.vdocuments.net/reader034/viewer/2022051516/56649ee15503460f94bf20e8/html5/thumbnails/2.jpg)
2Connect | Communicate | Collaborate
perfSONAR MDM
Multi-domain monitoring service
Based on monitoring probes installed in the network
Based on a standard (perfSONAR) protocol
Hundreds of deployments around the world
Web interface!
![Page 3: PerfSONAR MDM and TCP buffers Domenico Vicinanza DANTE, Cambridge, UK domenico.vicinanza@dante.net EGI Technical Forum 2013, Madrid, Spain](https://reader034.vdocuments.net/reader034/viewer/2022051516/56649ee15503460f94bf20e8/html5/thumbnails/3.jpg)
3Connect | Communicate | Collaborate
perfSONAR in a nutshell
Performance focused Service Oriented Network monitoring Architecture
International collaboration for network monitoring
Contributors are GÉANT, Internet2, ESnet, and RNP
Two main implementations committed to interoperate:
perfSONAR MDM within GÉANT: http://perfsonar.geant.net
perfSONAR PS within I2/ESnet: http://psps.perfsonar.net/
Open OGF protocol to exchange data
Web-service based
Design goals: flexibility, extensibility, openness, and decentralization.
![Page 4: PerfSONAR MDM and TCP buffers Domenico Vicinanza DANTE, Cambridge, UK domenico.vicinanza@dante.net EGI Technical Forum 2013, Madrid, Spain](https://reader034.vdocuments.net/reader034/viewer/2022051516/56649ee15503460f94bf20e8/html5/thumbnails/4.jpg)
4Connect | Communicate | Collaborate
How does perfSONAR work?
perfSONAR MP
perfSONAR MP
Available BandwidthOne Way DelayJitterIP Route Tracing
Web UI
![Page 5: PerfSONAR MDM and TCP buffers Domenico Vicinanza DANTE, Cambridge, UK domenico.vicinanza@dante.net EGI Technical Forum 2013, Madrid, Spain](https://reader034.vdocuments.net/reader034/viewer/2022051516/56649ee15503460f94bf20e8/html5/thumbnails/5.jpg)
5Connect | Communicate | Collaborate
perfSONAR web user interface
![Page 6: PerfSONAR MDM and TCP buffers Domenico Vicinanza DANTE, Cambridge, UK domenico.vicinanza@dante.net EGI Technical Forum 2013, Madrid, Spain](https://reader034.vdocuments.net/reader034/viewer/2022051516/56649ee15503460f94bf20e8/html5/thumbnails/6.jpg)
8Connect | Communicate | Collaborate
First measurement: Geneva-Frankfurt
570Km
4.5ms
![Page 7: PerfSONAR MDM and TCP buffers Domenico Vicinanza DANTE, Cambridge, UK domenico.vicinanza@dante.net EGI Technical Forum 2013, Madrid, Spain](https://reader034.vdocuments.net/reader034/viewer/2022051516/56649ee15503460f94bf20e8/html5/thumbnails/7.jpg)
9Connect | Communicate | Collaborate
Geneva-Frankfurt1Kbyte buffer
1K=1024Byte
Bandwidth<2Mb/s!!
![Page 8: PerfSONAR MDM and TCP buffers Domenico Vicinanza DANTE, Cambridge, UK domenico.vicinanza@dante.net EGI Technical Forum 2013, Madrid, Spain](https://reader034.vdocuments.net/reader034/viewer/2022051516/56649ee15503460f94bf20e8/html5/thumbnails/8.jpg)
10Connect | Communicate | Collaborate
What happens with TCP buffer=1KB?
Geneva FrankfurtDelay: 4.5ms
1024 Byte
1024Bytes sent every 4.5ms
Max Throughput≤(TCP buffer)/Delay1.8 Mbit/sBandwidth limited by the buffer!
![Page 9: PerfSONAR MDM and TCP buffers Domenico Vicinanza DANTE, Cambridge, UK domenico.vicinanza@dante.net EGI Technical Forum 2013, Madrid, Spain](https://reader034.vdocuments.net/reader034/viewer/2022051516/56649ee15503460f94bf20e8/html5/thumbnails/9.jpg)
11Connect | Communicate | Collaborate
Optimal TCP Window size:Geneva-Frankfurt
RTT: 9ms
1Gb/s network interface
Calculating Bandwidth Delay Product
BDP=
Optimal TCP window = BDP
TCP window = 1MByte for Geneva-Frankfurt
![Page 10: PerfSONAR MDM and TCP buffers Domenico Vicinanza DANTE, Cambridge, UK domenico.vicinanza@dante.net EGI Technical Forum 2013, Madrid, Spain](https://reader034.vdocuments.net/reader034/viewer/2022051516/56649ee15503460f94bf20e8/html5/thumbnails/10.jpg)
12Connect | Communicate | Collaborate
Geneva-Frankfurt1Mbyte TCP Buffer
1M=1048756Byte
Bandwidth ~0.9 Gb/s!!
![Page 11: PerfSONAR MDM and TCP buffers Domenico Vicinanza DANTE, Cambridge, UK domenico.vicinanza@dante.net EGI Technical Forum 2013, Madrid, Spain](https://reader034.vdocuments.net/reader034/viewer/2022051516/56649ee15503460f94bf20e8/html5/thumbnails/11.jpg)
13Connect | Communicate | Collaborate
Second measurement: London – New York
5570Km
50ms
![Page 12: PerfSONAR MDM and TCP buffers Domenico Vicinanza DANTE, Cambridge, UK domenico.vicinanza@dante.net EGI Technical Forum 2013, Madrid, Spain](https://reader034.vdocuments.net/reader034/viewer/2022051516/56649ee15503460f94bf20e8/html5/thumbnails/12.jpg)
14Connect | Communicate | Collaborate
London-New York1Kbyte Buffer
1K=1024Byte
Bandwidth<200Kb/s!!
![Page 13: PerfSONAR MDM and TCP buffers Domenico Vicinanza DANTE, Cambridge, UK domenico.vicinanza@dante.net EGI Technical Forum 2013, Madrid, Spain](https://reader034.vdocuments.net/reader034/viewer/2022051516/56649ee15503460f94bf20e8/html5/thumbnails/13.jpg)
15Connect | Communicate | Collaborate
Optimal TCP Window size:London-New York
RTT: 100ms
1Gb/s network interface
Calculating Bandwidth Delay Product
BDP=
Optimal TCP window = BDP
TCP window = 12MByte for London-New York
![Page 14: PerfSONAR MDM and TCP buffers Domenico Vicinanza DANTE, Cambridge, UK domenico.vicinanza@dante.net EGI Technical Forum 2013, Madrid, Spain](https://reader034.vdocuments.net/reader034/viewer/2022051516/56649ee15503460f94bf20e8/html5/thumbnails/14.jpg)
16Connect | Communicate | Collaborate
London-New York12Mbyte Buffer
12M=12582912Byte
Bandwidth ~0.95Gb/s!!
![Page 15: PerfSONAR MDM and TCP buffers Domenico Vicinanza DANTE, Cambridge, UK domenico.vicinanza@dante.net EGI Technical Forum 2013, Madrid, Spain](https://reader034.vdocuments.net/reader034/viewer/2022051516/56649ee15503460f94bf20e8/html5/thumbnails/15.jpg)
17Connect | Communicate | Collaborate
Live demo: Madrid-Tallin
27ms
![Page 16: PerfSONAR MDM and TCP buffers Domenico Vicinanza DANTE, Cambridge, UK domenico.vicinanza@dante.net EGI Technical Forum 2013, Madrid, Spain](https://reader034.vdocuments.net/reader034/viewer/2022051516/56649ee15503460f94bf20e8/html5/thumbnails/16.jpg)
18Connect | Communicate | Collaborate
Optimal TCP Window size:Madrid-Tallinn
RTT: 54ms
1Gb/s network interface
Calculating Bandwidth Delay Product
BDP=
Optimal TCP window = BDP
TCP window = 6.43MByte for Madrid-Tallin
![Page 17: PerfSONAR MDM and TCP buffers Domenico Vicinanza DANTE, Cambridge, UK domenico.vicinanza@dante.net EGI Technical Forum 2013, Madrid, Spain](https://reader034.vdocuments.net/reader034/viewer/2022051516/56649ee15503460f94bf20e8/html5/thumbnails/17.jpg)
19Connect | Communicate | Collaborate
Let’s try on the perfSONAR UI!
We’ll set:
Source: GEANT Madrid
Destination: GEANT Tallinn
TCP Window size to 6.43MByte=6750000 Byte
![Page 18: PerfSONAR MDM and TCP buffers Domenico Vicinanza DANTE, Cambridge, UK domenico.vicinanza@dante.net EGI Technical Forum 2013, Madrid, Spain](https://reader034.vdocuments.net/reader034/viewer/2022051516/56649ee15503460f94bf20e8/html5/thumbnails/18.jpg)
20Connect | Communicate | Collaborate
Madrid-TallinnTest with optimal TCP window
6.43MByte=6750000 Byte
![Page 19: PerfSONAR MDM and TCP buffers Domenico Vicinanza DANTE, Cambridge, UK domenico.vicinanza@dante.net EGI Technical Forum 2013, Madrid, Spain](https://reader034.vdocuments.net/reader034/viewer/2022051516/56649ee15503460f94bf20e8/html5/thumbnails/19.jpg)
21Connect | Communicate | Collaborate
Madrid-Tallinn
Sometimes we have the possibility to check the buffer set in kernel
Max receiver (Tallinn) buffer: 67108864 bytes = 64MBytes
If we set TCP buffers> 67108864 bytes
there will be packet loss!
Let’s try!
![Page 20: PerfSONAR MDM and TCP buffers Domenico Vicinanza DANTE, Cambridge, UK domenico.vicinanza@dante.net EGI Technical Forum 2013, Madrid, Spain](https://reader034.vdocuments.net/reader034/viewer/2022051516/56649ee15503460f94bf20e8/html5/thumbnails/20.jpg)
22Connect | Communicate | Collaborate
TCP Window too large Packet drops
TCP window size: 160MByteMax buffer (Kernel): 64MByte
Bandwidth drop due to packet loss<160Mb/s!!
![Page 21: PerfSONAR MDM and TCP buffers Domenico Vicinanza DANTE, Cambridge, UK domenico.vicinanza@dante.net EGI Technical Forum 2013, Madrid, Spain](https://reader034.vdocuments.net/reader034/viewer/2022051516/56649ee15503460f94bf20e8/html5/thumbnails/21.jpg)
23Connect | Communicate | Collaborate
Conclusions
Buffers are critical in TCP communication
Not too small
But not too big!
Particular care with long-haul, high-bandwidth network
Long Fat Networks, LFN
Buffers can be the limiting factor
perfSONAR MDM
Add interactive, on-demand testing capability
Particularly useful for TCP debugging
Measurement strategy
Calculate the BDP
Set the TCP Window size=BDP
![Page 22: PerfSONAR MDM and TCP buffers Domenico Vicinanza DANTE, Cambridge, UK domenico.vicinanza@dante.net EGI Technical Forum 2013, Madrid, Spain](https://reader034.vdocuments.net/reader034/viewer/2022051516/56649ee15503460f94bf20e8/html5/thumbnails/22.jpg)
24Connect | Communicate | Collaborate
www.geant.net
www.twitter.com/GEANTnews | www.facebook.com/GEANTnetwork | www.youtube.com/GEANTtv
Connect | Communicate | Collaborate
Thank you!