stacking it up experimental observations on the operation of dual stack services
DESCRIPTION
Stacking it Up Experimental Observations on the operation of Dual Stack Services. Geoff Huston, APNIC Labs. If working with one protocol has its problems …. Then just how much fun can we have by using two protocols at once?. Some Dual Stack Questions. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Stacking it Up Experimental Observations on the operation of Dual Stack Services](https://reader036.vdocuments.net/reader036/viewer/2022070504/56816974550346895de14f9c/html5/thumbnails/1.jpg)
Stacking it Up
Experimental Observations on the operation of Dual Stack Services
Geoff Huston, APNIC Labs 1
![Page 2: Stacking it Up Experimental Observations on the operation of Dual Stack Services](https://reader036.vdocuments.net/reader036/viewer/2022070504/56816974550346895de14f9c/html5/thumbnails/2.jpg)
If working with one protocol has its problems …
2
![Page 3: Stacking it Up Experimental Observations on the operation of Dual Stack Services](https://reader036.vdocuments.net/reader036/viewer/2022070504/56816974550346895de14f9c/html5/thumbnails/3.jpg)
Then just how much fun can we have by using two protocols at once?
3
![Page 4: Stacking it Up Experimental Observations on the operation of Dual Stack Services](https://reader036.vdocuments.net/reader036/viewer/2022070504/56816974550346895de14f9c/html5/thumbnails/4.jpg)
Some Dual Stack Questions
• How many clients are capable of IPv6 access?
• What forms of IPv6 access are they using?
• Is their experience over Dual Stack better or worse than IPv4?
4
![Page 5: Stacking it Up Experimental Observations on the operation of Dual Stack Services](https://reader036.vdocuments.net/reader036/viewer/2022070504/56816974550346895de14f9c/html5/thumbnails/5.jpg)
Setting the scene• Adding IPv6 to your website may have risks
– Will your clients still be able to ‘see’ you?– What % of clients will experience issues?
• Finding out in advance what to expect is useful– A way to measure end-user behavior– Without affecting your own website investment
• Measuring failure is hard!– Website logs only measure successful connections
![Page 6: Stacking it Up Experimental Observations on the operation of Dual Stack Services](https://reader036.vdocuments.net/reader036/viewer/2022070504/56816974550346895de14f9c/html5/thumbnails/6.jpg)
Adding IPv6 may have risks
• Older Windows XP hosts experience problems with dual-stack (IPv4, IPv6) DNS records– May refuse to connect to the IPv4 address
• Some hosts cannot process IPv6 DNS properly– Not supported in all DHCP backed configurations
• ‘Partial IPv6’ problems– Locally IPv6 enabled, no IPv6 route to global Internet
• Loss of eyeballs = Loss of revenue?– When your core business presents via the web, what risks
to loss of web access are you willing to take?
![Page 7: Stacking it Up Experimental Observations on the operation of Dual Stack Services](https://reader036.vdocuments.net/reader036/viewer/2022070504/56816974550346895de14f9c/html5/thumbnails/7.jpg)
Finding out in advance what to expect
• Measure client’s IPv6 behavior without having to add IPv6 to your website– Leverage cross-site URL fetches
• Integrate these measurements into existing tracking methods, and analytics framework– No new tools needed
![Page 8: Stacking it Up Experimental Observations on the operation of Dual Stack Services](https://reader036.vdocuments.net/reader036/viewer/2022070504/56816974550346895de14f9c/html5/thumbnails/8.jpg)
Measuring failure is hard!
• Web logs record completed TCP/IP events– Even 4xx and 5xx responses in logs are completed valid TCP/IP
sessions• What about the people who fail to complete the
connection?– Not in access- or error- logs
• Only partially visible on-the-wire– Characteristic missing ‘SYN/ACK’ sequence in TCP signals failure
to complete a 2-way handshake• But (inside a time limit) client knows what worked or
failed: and can report back.
![Page 9: Stacking it Up Experimental Observations on the operation of Dual Stack Services](https://reader036.vdocuments.net/reader036/viewer/2022070504/56816974550346895de14f9c/html5/thumbnails/9.jpg)
APNIC’s web measurement systemhttp://labs.apnic.net
• Built on google ‘analytics’ method– Javascript, highly portable– Asynchronous, runs in the background
• after page render already complete– Uses DNS wildcards, uncacheable
• Data integrated into google analytics reports– Graphs of ‘events’ to monitor IPv4, IPv6 and dual-stack
• Configurable by website manager– Sample or every connection, extra tests etc
![Page 10: Stacking it Up Experimental Observations on the operation of Dual Stack Services](https://reader036.vdocuments.net/reader036/viewer/2022070504/56816974550346895de14f9c/html5/thumbnails/10.jpg)
Measuring by 1x1 invisible pixels
• Javascript requests sequence of 1x1 pixel images– Images fetched but not included in the DOM so not displayed– Image fetches take place after DOM render, – does not add delay to page view, invisible
(may be seen in browser status bar, error report windows)– Javascript callback records success/time
• Image fetches from unique DNS names– Every client is a fresh name, no cached state
• Client reports timing, connect failures – to your analytics report as a results/summary field – Can account for ‘unable to connect’ TCP/IP failure
![Page 11: Stacking it Up Experimental Observations on the operation of Dual Stack Services](https://reader036.vdocuments.net/reader036/viewer/2022070504/56816974550346895de14f9c/html5/thumbnails/11.jpg)
What is tested?• Basic test set is dual-stack, IPv4, IPv6
– Dual stack enabled DNS behind all fetches• Additional (optional) tests
– IPv6 literal (bypasses many Windows Teredo IPv6 suppression settings)
– IPv6 DNS (can be visible to user, stress-tests DNS)
– Auto-Tunnel detectionURLs only reachable from Teredo and 6to4 source IP addresses
• Results reported over IPv4-only URL
![Page 12: Stacking it Up Experimental Observations on the operation of Dual Stack Services](https://reader036.vdocuments.net/reader036/viewer/2022070504/56816974550346895de14f9c/html5/thumbnails/12.jpg)
Additional Measurements
We extended this technique into Flash, and created an anonymous banner ad
The IPv6 capability test is built into the Flash code
12
![Page 13: Stacking it Up Experimental Observations on the operation of Dual Stack Services](https://reader036.vdocuments.net/reader036/viewer/2022070504/56816974550346895de14f9c/html5/thumbnails/13.jpg)
Banner Ad FunNo clicks needed
(indeed we would prefer that clients did NOT click the ad, as it costs us more for a click!)
Impressions are really cheap$25 per day buys around 25,000 impressionsEvery impression carries the complete IPv6 test set
But many users are ad-intolerantUsers tend to browse away from pages containing the ad in a far shorter time intervalWe see a higher number of aborted test runs with the ad
13
![Page 14: Stacking it Up Experimental Observations on the operation of Dual Stack Services](https://reader036.vdocuments.net/reader036/viewer/2022070504/56816974550346895de14f9c/html5/thumbnails/14.jpg)
Some Results
• How much IPv6 is out there in terms of end host capability?
• What forms of IPv6 access are clients using?
14
![Page 15: Stacking it Up Experimental Observations on the operation of Dual Stack Services](https://reader036.vdocuments.net/reader036/viewer/2022070504/56816974550346895de14f9c/html5/thumbnails/15.jpg)
IPv6 capability, as seen by Google
15http://www.google.com/intl/en/ipv6/statistics/
![Page 16: Stacking it Up Experimental Observations on the operation of Dual Stack Services](https://reader036.vdocuments.net/reader036/viewer/2022070504/56816974550346895de14f9c/html5/thumbnails/16.jpg)
IPv6 capability, as seen by APNIC
16
0.0%
0.2%
0.4%
0.6%
![Page 17: Stacking it Up Experimental Observations on the operation of Dual Stack Services](https://reader036.vdocuments.net/reader036/viewer/2022070504/56816974550346895de14f9c/html5/thumbnails/17.jpg)
Is This All There Is?
• 0.3% – 0.4% of clients is a very low number
• And most of the IPv6 access we see here is using unicast IPv6
• Where are all the 6to4 and Teredo auto-tunnels?
• Lets look harder by testing with an IPv6-only image
17
![Page 18: Stacking it Up Experimental Observations on the operation of Dual Stack Services](https://reader036.vdocuments.net/reader036/viewer/2022070504/56816974550346895de14f9c/html5/thumbnails/18.jpg)
IPv6 ONLY, as seen by APNIC
1%
2%
3%
5%
Nov18
Dec Jan Feb
4%
Mar MayApr Jun
![Page 19: Stacking it Up Experimental Observations on the operation of Dual Stack Services](https://reader036.vdocuments.net/reader036/viewer/2022070504/56816974550346895de14f9c/html5/thumbnails/19.jpg)
IPv6: “could” vs “will”
1%
2%
3%
5%
IPv6 Preferred
IPv6 Capable
Nov19
Dec Jan Feb
4%
Mar MayApr Jun
![Page 20: Stacking it Up Experimental Observations on the operation of Dual Stack Services](https://reader036.vdocuments.net/reader036/viewer/2022070504/56816974550346895de14f9c/html5/thumbnails/20.jpg)
Is This All There Is?
• 3% - 4% of clients is still a very low number
• Most of the access in IPv6-only is via 6to4 auto-tunnelling
• Where is Teredo?• Lets look harder by testing with an image
that does not require a DNS lookup: http://[2401:2000:6660::f003]/1x1.png
20
![Page 21: Stacking it Up Experimental Observations on the operation of Dual Stack Services](https://reader036.vdocuments.net/reader036/viewer/2022070504/56816974550346895de14f9c/html5/thumbnails/21.jpg)
IPv6: “can” vs “could” vs “will”
21
![Page 22: Stacking it Up Experimental Observations on the operation of Dual Stack Services](https://reader036.vdocuments.net/reader036/viewer/2022070504/56816974550346895de14f9c/html5/thumbnails/22.jpg)
How Much IPv6 is Out There?
• Around 0.4% of the Internet’s clients can and will use IPv6 in a Dual Stack scenario– And these clients are generally using a “native” IPv6
service• Around 4% of the Internet’s clients can use IPv6 in an
IPv6-only scenario– And the additional clients are generally using 6to4 auto-
tunnelling• Around 25% of the Internet’s clients are equipped
with IPv6 capability that can be exposed– And the additional clients are using Teredo auto-tunnelling
22
![Page 23: Stacking it Up Experimental Observations on the operation of Dual Stack Services](https://reader036.vdocuments.net/reader036/viewer/2022070504/56816974550346895de14f9c/html5/thumbnails/23.jpg)
Performance Observations
23
![Page 24: Stacking it Up Experimental Observations on the operation of Dual Stack Services](https://reader036.vdocuments.net/reader036/viewer/2022070504/56816974550346895de14f9c/html5/thumbnails/24.jpg)
Performance and Tunnels
V6 Unicast
6to4
Teredo
+4 Secs
+2 Secs
-2 Secs
0 Sec
24
-4 SecsNov Dec Jan Feb Mar MayApr
![Page 25: Stacking it Up Experimental Observations on the operation of Dual Stack Services](https://reader036.vdocuments.net/reader036/viewer/2022070504/56816974550346895de14f9c/html5/thumbnails/25.jpg)
Performance and Tunnels
25
• Unicast IPv6 performance is on average equivalent to IPv4 performance for web object retrieval
• Auto-tunnel performance is on average considerably worse– Teredo is highly variable with 1 – 3 seconds of
additional delay per retrieval– 6to4 is more consistent with an average 1.2
seconds additional delay per retrieval
![Page 26: Stacking it Up Experimental Observations on the operation of Dual Stack Services](https://reader036.vdocuments.net/reader036/viewer/2022070504/56816974550346895de14f9c/html5/thumbnails/26.jpg)
Performance and Tunnels
Two causes of incremental delay:– Tunnel setup time
• Stateful Teredo tunnels require initial packet exchanges to set the tunnel up (min 1 x RTT)
– Tunnelling can extend the RTT delay• addition of tunnel relays between the source
and destination• This is exacerbated when the forward and
reverse paths are asymmteric26
![Page 27: Stacking it Up Experimental Observations on the operation of Dual Stack Services](https://reader036.vdocuments.net/reader036/viewer/2022070504/56816974550346895de14f9c/html5/thumbnails/27.jpg)
V4-OnlyNetwork
Dual-StackNetwork
6to4 Packet Path
27
ClientDual-Stack
Server
192.88.99.1 Relay
2002::/16 Relay
![Page 28: Stacking it Up Experimental Observations on the operation of Dual Stack Services](https://reader036.vdocuments.net/reader036/viewer/2022070504/56816974550346895de14f9c/html5/thumbnails/28.jpg)
V4-OnlyNetwork
Dual-StackNetwork
Partial Mitigation of 6to4 Packet Path
28
ClientDual-Stack
Server 2002::/16
Relay
192.88.99.1 Relay
![Page 29: Stacking it Up Experimental Observations on the operation of Dual Stack Services](https://reader036.vdocuments.net/reader036/viewer/2022070504/56816974550346895de14f9c/html5/thumbnails/29.jpg)
6to4 Performance
Setup Time
29
![Page 30: Stacking it Up Experimental Observations on the operation of Dual Stack Services](https://reader036.vdocuments.net/reader036/viewer/2022070504/56816974550346895de14f9c/html5/thumbnails/30.jpg)
Tunnel RTT Cost
6to4 Performance
30
![Page 31: Stacking it Up Experimental Observations on the operation of Dual Stack Services](https://reader036.vdocuments.net/reader036/viewer/2022070504/56816974550346895de14f9c/html5/thumbnails/31.jpg)
Teredo PerformanceTunnel Setup Time
31
![Page 32: Stacking it Up Experimental Observations on the operation of Dual Stack Services](https://reader036.vdocuments.net/reader036/viewer/2022070504/56816974550346895de14f9c/html5/thumbnails/32.jpg)
Tunnel RTT Cost
Teredo Performance
32
![Page 33: Stacking it Up Experimental Observations on the operation of Dual Stack Services](https://reader036.vdocuments.net/reader036/viewer/2022070504/56816974550346895de14f9c/html5/thumbnails/33.jpg)
IPv6 Performance
• Unicast IPv6 appears to be as fast as IPv4 for object retrieval
• Auto-tunnelling IPv6 attracts major performance overheads– these are strongly context dependent– widespread deployment of 6to4 relays and Teredo relays
and servers would mitigate this, to some extent– Dual Stack servers may want to consider using local 6to4
relays to improve reverse path performance for auto-tunnelling clients
33
![Page 34: Stacking it Up Experimental Observations on the operation of Dual Stack Services](https://reader036.vdocuments.net/reader036/viewer/2022070504/56816974550346895de14f9c/html5/thumbnails/34.jpg)
Failure Observations
34
![Page 35: Stacking it Up Experimental Observations on the operation of Dual Stack Services](https://reader036.vdocuments.net/reader036/viewer/2022070504/56816974550346895de14f9c/html5/thumbnails/35.jpg)
Dual Stack Failure
How many clients retrieve the V4 only object but DON’T retrieve the Dual Stack objects?i.e. how many clients exhibit “Dual Stack Failure”?
35
![Page 36: Stacking it Up Experimental Observations on the operation of Dual Stack Services](https://reader036.vdocuments.net/reader036/viewer/2022070504/56816974550346895de14f9c/html5/thumbnails/36.jpg)
Dual Stack Loss Rate
![Page 37: Stacking it Up Experimental Observations on the operation of Dual Stack Services](https://reader036.vdocuments.net/reader036/viewer/2022070504/56816974550346895de14f9c/html5/thumbnails/37.jpg)
Dual Stack Loss
• 4 in 1000 clients are unable to fetch a web URL if presented with a dual-stack DNS name
• Older (windows XP) hosts, browsers
![Page 38: Stacking it Up Experimental Observations on the operation of Dual Stack Services](https://reader036.vdocuments.net/reader036/viewer/2022070504/56816974550346895de14f9c/html5/thumbnails/38.jpg)
Connection FailureTo attempt to look more precisely for some instances of connection failure, lets looking for connections that fail after the initial TCP SYN
Note that this approach does not detect failure of the initial SYN packet, so the results are a lower bound of total connection failure rates
38
Client
Server
SYN
SYN + ACK
ACK
X Response fails
![Page 39: Stacking it Up Experimental Observations on the operation of Dual Stack Services](https://reader036.vdocuments.net/reader036/viewer/2022070504/56816974550346895de14f9c/html5/thumbnails/39.jpg)
Connection Failure
39
![Page 40: Stacking it Up Experimental Observations on the operation of Dual Stack Services](https://reader036.vdocuments.net/reader036/viewer/2022070504/56816974550346895de14f9c/html5/thumbnails/40.jpg)
IPv6 Connection Failure
40
![Page 41: Stacking it Up Experimental Observations on the operation of Dual Stack Services](https://reader036.vdocuments.net/reader036/viewer/2022070504/56816974550346895de14f9c/html5/thumbnails/41.jpg)
Is Teredo really THAT good?
41
![Page 42: Stacking it Up Experimental Observations on the operation of Dual Stack Services](https://reader036.vdocuments.net/reader036/viewer/2022070504/56816974550346895de14f9c/html5/thumbnails/42.jpg)
Teredo Connection FailureTeredo uses an initial ICMPv6 exchange to assist in the Teredo Server / Relay state setup
Note that this approach does not detect failure of the initial ICMPv6 echo request , so the results are a lower bound of total connection failure rates
42
Client
Server
SYN
SYN + ACK
ACK
XSYN fails
ICMPv6Echo Req
ICMPv6 Echo Resp
X XICMP fails
![Page 43: Stacking it Up Experimental Observations on the operation of Dual Stack Services](https://reader036.vdocuments.net/reader036/viewer/2022070504/56816974550346895de14f9c/html5/thumbnails/43.jpg)
IPv6 Connection Failure
43
![Page 44: Stacking it Up Experimental Observations on the operation of Dual Stack Services](https://reader036.vdocuments.net/reader036/viewer/2022070504/56816974550346895de14f9c/html5/thumbnails/44.jpg)
IPv6 Connection Failure
44
• Some 2%-5% of IPv6 unicast connections fail!– This rate is better than IPv6 auto-tunnels, but is still 20x
the rate of IPv4 connection failure• Some 12% - 20% of 6to4 connections fail!
– This is a very high failure rate!– The failure is most likely a protocol 41 filter close to the
client that prevents incoming 6to4 packets reaching the client
• Some 35% of Teredo connections fail!– This is an amazingly high failure rate!– Is STUN just broken? And/or …?
![Page 45: Stacking it Up Experimental Observations on the operation of Dual Stack Services](https://reader036.vdocuments.net/reader036/viewer/2022070504/56816974550346895de14f9c/html5/thumbnails/45.jpg)
Can we improve Dual Stack Performance?
We need to understand how client systems behave in a dual stack environment in order to understand how we can improve the situation
45
![Page 46: Stacking it Up Experimental Observations on the operation of Dual Stack Services](https://reader036.vdocuments.net/reader036/viewer/2022070504/56816974550346895de14f9c/html5/thumbnails/46.jpg)
Serialization
46
Client
DNS Web Server
AAAA Query A Query
AAAA ResponseA Response
V6 SYN
V6 SYN+ACK
V6 ACK
DNS Phase TCP Connection Phase
![Page 47: Stacking it Up Experimental Observations on the operation of Dual Stack Services](https://reader036.vdocuments.net/reader036/viewer/2022070504/56816974550346895de14f9c/html5/thumbnails/47.jpg)
Serialization and Failure
47
Client
DNS Web Server
AAAA Query A Query
AAAA ResponseA Response
V6 SYNs V4 SYN
V4 SYN+ACK
V4 ACK
V6 TCP SYN Timeout
DNS Phase TCP Connection Phase
![Page 48: Stacking it Up Experimental Observations on the operation of Dual Stack Services](https://reader036.vdocuments.net/reader036/viewer/2022070504/56816974550346895de14f9c/html5/thumbnails/48.jpg)
Serialization and Failure
In response to poor performance associated with auto-tunnelling many OS stacks have responded by altering the local protocol preference table to depref 6to4 BELOW V4, and to try and not use Teredo at all!
48
![Page 49: Stacking it Up Experimental Observations on the operation of Dual Stack Services](https://reader036.vdocuments.net/reader036/viewer/2022070504/56816974550346895de14f9c/html5/thumbnails/49.jpg)
Parallelization
• In response to an open() call from the application, set off two independent streams (V4 and V6) and perform in parallel:– DNS query– TCP SYN exchange
• ACK the first TCP SYN+ACK to be received, and present this back to the application as the “working” TCP connection
• RST the other49
![Page 50: Stacking it Up Experimental Observations on the operation of Dual Stack Services](https://reader036.vdocuments.net/reader036/viewer/2022070504/56816974550346895de14f9c/html5/thumbnails/50.jpg)
Web ServerDNS
Parallelization
50
Client
DNS Web Server
AAAA Query
A Query
AAAA Response
A Response
V6 SYN
V6 SYN+ACK
V4 SYN
V6 SYN+ACK
V6 ACK …
V4 RST
Protocol section pointV4 thread
V6 thread
![Page 51: Stacking it Up Experimental Observations on the operation of Dual Stack Services](https://reader036.vdocuments.net/reader036/viewer/2022070504/56816974550346895de14f9c/html5/thumbnails/51.jpg)
Parallelization
Trade offs:+ Faster client experience- Higher client state overhead- Higher server SYN load for dual stack servers
“Happy Eyeballs: Trending Towards Success with Dual-Stack Hosts” draft-wing-v6ops-happy-eyeballs-ipv6-01
51
![Page 52: Stacking it Up Experimental Observations on the operation of Dual Stack Services](https://reader036.vdocuments.net/reader036/viewer/2022070504/56816974550346895de14f9c/html5/thumbnails/52.jpg)
Conclusions
What can we say about the performance and robustness of a Dual Stack network environment as a result of these observations?
52
![Page 53: Stacking it Up Experimental Observations on the operation of Dual Stack Services](https://reader036.vdocuments.net/reader036/viewer/2022070504/56816974550346895de14f9c/html5/thumbnails/53.jpg)
For an Online Service…
Converting a service to operate as a Dual Stack service is a viable option in today’s environment
But:– a small fraction of existing clients will experience a
much slower service– a very small fraction of existing clients will fail to
connect to the dual stack service at all53
![Page 54: Stacking it Up Experimental Observations on the operation of Dual Stack Services](https://reader036.vdocuments.net/reader036/viewer/2022070504/56816974550346895de14f9c/html5/thumbnails/54.jpg)
What about IPv6-Only Services?
Is an IPv6-only service a viable option today?
Not really.– Only ~4% of the existing client base would successfully
connect to an IPv6-only service– And many would experience poor performance relative to
IPv4 services
54
![Page 55: Stacking it Up Experimental Observations on the operation of Dual Stack Services](https://reader036.vdocuments.net/reader036/viewer/2022070504/56816974550346895de14f9c/html5/thumbnails/55.jpg)
What about Dual Stack Transition?
55
![Page 56: Stacking it Up Experimental Observations on the operation of Dual Stack Services](https://reader036.vdocuments.net/reader036/viewer/2022070504/56816974550346895de14f9c/html5/thumbnails/56.jpg)
What about Dual Stack Transition?
End-host auto-tunnelling is not a solution!
56
![Page 57: Stacking it Up Experimental Observations on the operation of Dual Stack Services](https://reader036.vdocuments.net/reader036/viewer/2022070504/56816974550346895de14f9c/html5/thumbnails/57.jpg)
What about Dual Stack Transition?
End-host auto-tunnelling is not a solution!– Auto-tunnelling appears to encounter many more
performance and reliability problems than it solves in terms of IPv6 connectivity
– Auto-tunnelling is not proving to be a useful mainstream transition tool for IPv6
57
![Page 58: Stacking it Up Experimental Observations on the operation of Dual Stack Services](https://reader036.vdocuments.net/reader036/viewer/2022070504/56816974550346895de14f9c/html5/thumbnails/58.jpg)
What about Dual Stack Transition?
If we want this transition to operate in a manner where IPv6 operates at least as well as IPv4 then end hosts really need to be connected to a IPv6 Unicast service delivered from their service provider
58
![Page 59: Stacking it Up Experimental Observations on the operation of Dual Stack Services](https://reader036.vdocuments.net/reader036/viewer/2022070504/56816974550346895de14f9c/html5/thumbnails/59.jpg)
Thank You
Questions?
59