link virtualization based on xen
DESCRIPTION
We implement link virtualization based on Xen. Link virtualization is a basic building block for network virtualizaiton that allows the co-existence of different Internet protocols. To minimize virtualize overhead, we use SR-IOV with Intel 82576TRANSCRIPT
![Page 1: Link Virtualization based on Xen](https://reader035.vdocuments.net/reader035/viewer/2022062319/55515311b4c905e1708b469d/html5/thumbnails/1.jpg)
Sponsored by:
&
Link Virtualization based on Xen
ShinHyoung Lee, Chuck Yoo
![Page 2: Link Virtualization based on Xen](https://reader035.vdocuments.net/reader035/viewer/2022062319/55515311b4c905e1708b469d/html5/thumbnails/2.jpg)
ContentsIntroduction
Future Internet
Virtual Network
Link Virtualization
Related Works802.1q
VRouter
Trellis
GENI and FIRE
Network IsolationMAC-in-UDP tunneling
vARP
Bandwidth Isolation
Weight Based Control
Bandwidth Based Control with Pri-
ority
Performance Evaluation
Network Isolation
Bandwidth Isolation
Virtual Link
Conclusion
![Page 3: Link Virtualization based on Xen](https://reader035.vdocuments.net/reader035/viewer/2022062319/55515311b4c905e1708b469d/html5/thumbnails/3.jpg)
introductionThe Future Internet
RequirementVarious network protocols can be existed in the Future Internet
ChallengeHow to isolate different networks
Network Virtualization is a good solutionVirtualization layer is an innovative substrate for Future Internet
allowing multiple virtual networks
isolating virtual networks
![Page 4: Link Virtualization based on Xen](https://reader035.vdocuments.net/reader035/viewer/2022062319/55515311b4c905e1708b469d/html5/thumbnails/4.jpg)
Virtual Network
Node VirtualizationImplemented by Router Virtualization
e.g.) Xen
Link VirtualizationImplemented by NIC Virtualization
e.g.) Paravirtualization on Xen, SR-IOV
![Page 5: Link Virtualization based on Xen](https://reader035.vdocuments.net/reader035/viewer/2022062319/55515311b4c905e1708b469d/html5/thumbnails/5.jpg)
Example of Virtual Network
Virtual network for IPv4
Virtual network for CCN
Physical network
![Page 6: Link Virtualization based on Xen](https://reader035.vdocuments.net/reader035/viewer/2022062319/55515311b4c905e1708b469d/html5/thumbnails/6.jpg)
Performance network virtualization
Cannot support over 10Gbps traffic
Some virtualization techniques are try to solve per-formance problemParavirtualization with Xen
SR-IOV with PCIe
We use both of them
![Page 7: Link Virtualization based on Xen](https://reader035.vdocuments.net/reader035/viewer/2022062319/55515311b4c905e1708b469d/html5/thumbnails/7.jpg)
SR-IOVSR-IOV minimize I/O virtual-
ization overhead
SR-IOV device has physical function (PF) and virtual function (VF)
Physical functionUse all function of device
Initialize and configure device
Virtual functionCommunication directly with
domain
HW
SR-IOV NIC
Switch and Filters
PF VF VF
IO MMU / vT-d
Xen
IOVM
PF
Driver Domain
VF
Guest Do-main 1
VF
Guest Do-main 2
DMA and Message Signal Interrupts
Initialization and Configuration
![Page 8: Link Virtualization based on Xen](https://reader035.vdocuments.net/reader035/viewer/2022062319/55515311b4c905e1708b469d/html5/thumbnails/8.jpg)
Related Work
802.1q vlanvlan tag in MAC header
![Page 9: Link Virtualization based on Xen](https://reader035.vdocuments.net/reader035/viewer/2022062319/55515311b4c905e1708b469d/html5/thumbnails/9.jpg)
Related Work (cont.)Vrouter at Lancaster University
Virtualized router based on Xen
Control plane is placed in guest domain
Data plane is placed in driver domain because of performance
Cannot guarantee network isolation
![Page 10: Link Virtualization based on Xen](https://reader035.vdocuments.net/reader035/viewer/2022062319/55515311b4c905e1708b469d/html5/thumbnails/10.jpg)
Related Work (cont.)
TrellisContainer based virtualization
Use Ethernet over GRE tunneling
Hard to guarantee network isolation
Geni and FIRENo definition of link virtualization
Support network and bandwidth isolation indirectly through virtual router resource control
![Page 11: Link Virtualization based on Xen](https://reader035.vdocuments.net/reader035/viewer/2022062319/55515311b4c905e1708b469d/html5/thumbnails/11.jpg)
Link Virtualization
Connect between Virtual Nodes through virtualized NIC
Network IsolationNode that is member of a virtual network cannot see other
virtual networks packets
Bandwidth IsolationVirtual link shared physical link’s bandwidth
A virtual link cannot intrude other virtual links’ bandwidth
![Page 12: Link Virtualization based on Xen](https://reader035.vdocuments.net/reader035/viewer/2022062319/55515311b4c905e1708b469d/html5/thumbnails/12.jpg)
Network Isolation
802.1q vlanvlan tag is placed in MAC header and it cannot deliver across the
node that do not support 802.1q
Every node must support virtual network
TunnelingEncapsulation/decapsulation overhead
Not every node must support virtual network
We choose tunnelingEvolution of processing power
It is impossible that every node support virtual network
![Page 13: Link Virtualization based on Xen](https://reader035.vdocuments.net/reader035/viewer/2022062319/55515311b4c905e1708b469d/html5/thumbnails/13.jpg)
MAC-in-UDP TunnelingWe use SR-IOV for performance
SR-IOV NIC support 5-tuple filter
Source/destination IP address, source/destination TCP/UDP port number, and protocol
We use UDP port number over 50K as virtual network id
SR-IOV NIC can filter via virtual network id through hardware
Minimize filtering overhead
MAC-in-UDP tunneling header
Physical Network Virtual Network
MAC MACIP UDP Payload
![Page 14: Link Virtualization based on Xen](https://reader035.vdocuments.net/reader035/viewer/2022062319/55515311b4c905e1708b469d/html5/thumbnails/14.jpg)
Mac-in-UDP Tunneling (cont.)
Encapsulation/decapsulation is done in guest domainDriver domain do not process the packets
Avoid domain switch
Minimize performance overhead
Guest domain must know all information for tunnelingPhysical network information
MAC, IP and UDP header
Virtual network informationMAC header
![Page 15: Link Virtualization based on Xen](https://reader035.vdocuments.net/reader035/viewer/2022062319/55515311b4c905e1708b469d/html5/thumbnails/15.jpg)
vARP
ARP – matching between MAC address and IP address
vARP – matching between virtual network MAC ad-dress and physical network IP addressUser input only physical IP address of other side node on vir-
tual link
vARP protocol gets physical source/destination MAC ad-dress, virtual MAC address automatically
![Page 16: Link Virtualization based on Xen](https://reader035.vdocuments.net/reader035/viewer/2022062319/55515311b4c905e1708b469d/html5/thumbnails/16.jpg)
vARP flowSource Node Next Hop Destination Node
VN PN PN VN
…vARP req.
ARP req.
ARP rep.(Physical MAC)
vARP req.
vARP rep.(Virtual MAC)
vARP rep.(Physical MAC <next hop, himself>)
![Page 17: Link Virtualization based on Xen](https://reader035.vdocuments.net/reader035/viewer/2022062319/55515311b4c905e1708b469d/html5/thumbnails/17.jpg)
Bandwidth Isolation
It is possible that many virtual networks share one physical networkMany virtual networks share physical bandwidth
Total bandwidth has upper cap by physical link
Driver domain control guest domains sending ratethe received packets have already used the network re-
source
the transmission rate is the same as the receive rate of the next node
![Page 18: Link Virtualization based on Xen](https://reader035.vdocuments.net/reader035/viewer/2022062319/55515311b4c905e1708b469d/html5/thumbnails/18.jpg)
Bandwidth Isolation StructureDriver Domain
SR-IOV NIC
Guest Domain 1
PFDriver
Descriptor FetchArbiter
Pool 1DQ0
DQ1
…
Guest Domain 2
Guest Domain n…
Packet Filter
Select transmit en-able VF
Packet Transmit
Read VF Bandwidth BICModule
Pool 2DQ0
DQ1
Pool nDQ
DQnn-1
Call BIC function using timer
VFTERegister
BandwidthStatisticRegister
![Page 19: Link Virtualization based on Xen](https://reader035.vdocuments.net/reader035/viewer/2022062319/55515311b4c905e1708b469d/html5/thumbnails/19.jpg)
Policy of sharing physical bandwidth
Weight Based Control
Bandwidth Based Control with priority
![Page 20: Link Virtualization based on Xen](https://reader035.vdocuments.net/reader035/viewer/2022062319/55515311b4c905e1708b469d/html5/thumbnails/20.jpg)
Weight Based Control
Each virtual machines assign weight
Virtual machines are send as assigned weight
Example (Total 1Gbps)
VM1 VM2 VM3
Weight 1 2 3
Used BW 166Mbps=1/6Gbps
333Mbps=2/6Gbps
500Mps=3/6Gbps
![Page 21: Link Virtualization based on Xen](https://reader035.vdocuments.net/reader035/viewer/2022062319/55515311b4c905e1708b469d/html5/thumbnails/21.jpg)
Bandwidth Based Control with priority
Each virtual machines assign bandwidth
Virtual machines cannot send more data than assigned bandwidth
Priority is used when summary of virtual networks bandwidth is more than physical network
To avoid disconnect virtual link that have low priority, we guarantee minimum bandwidth
![Page 22: Link Virtualization based on Xen](https://reader035.vdocuments.net/reader035/viewer/2022062319/55515311b4c905e1708b469d/html5/thumbnails/22.jpg)
Performance EvaluationPhysical machine environment
Intel XEON X5650 (2.67GHz, 6-cores) * 2
12GB physical memory
Intel 82576 NIC (1Gbps with SR-IOV support)
Software environmentXen 4.0
Guest OS Ubuntu 10.04 LTS with Paravirtualization (Kernel ver 2.6.37.1)
4 cores VCPU
2GB memory
NICIgbvf (Intel 82576 VF) 1.1.3 for SR-IOVE1000 (Xen PV NIC model)
![Page 23: Link Virtualization based on Xen](https://reader035.vdocuments.net/reader035/viewer/2022062319/55515311b4c905e1708b469d/html5/thumbnails/23.jpg)
Tunneling Overhead
We compare PV model and SR-IOV model
Flow of packets are different
PV Model SR-IOV Model
![Page 24: Link Virtualization based on Xen](https://reader035.vdocuments.net/reader035/viewer/2022062319/55515311b4c905e1708b469d/html5/thumbnails/24.jpg)
Tunneling Overhead
0
100,000
200,000
300,000
400,000
pps
0
100,000
200,000
300,000
400,000
500,000
600,000
700,000
pps
Performance of E1000 (PV)Performance of Intel 82576 (SR-IOV)
![Page 25: Link Virtualization based on Xen](https://reader035.vdocuments.net/reader035/viewer/2022062319/55515311b4c905e1708b469d/html5/thumbnails/25.jpg)
Weight Based Control PerformanceWeight BW (Mbps) Ratio
VM1:VM2:VM3 VM1 VM2 VM3 total VM1:VM2:VM3
NA 314 314 314 942
1:1:1 314 314 314 942 1:1:1
1:1:8 94 94 752 940 1:1:8
1:2:3 157 314 471 942 1:2:3
1:2:4 135 268 538 941 1:1.99:3.99
1:3:3 135 403 403 941 1:2.99:2.99
1:3:6 94 283 565 942 1:3.01:6.01
1:4:5 95 377 471 943 1:3.97:4.96
2:2:3 269 269 403 941 2:2:3
2:3:4 209 314 418 941 2:3:4
3:3:4 283 283 376 942 3:3:3.99
![Page 26: Link Virtualization based on Xen](https://reader035.vdocuments.net/reader035/viewer/2022062319/55515311b4c905e1708b469d/html5/thumbnails/26.jpg)
Bandwidth Based Control Performance
Assigned Bandwidth Priority Measured Bandwidth (Mbps)
VM1:VM2:VM3 VM1:VM2:VM3 VM1 VM2 VM3 total
300:200:100 1:2:3 286 191 95.4 572.4
600:400:200 1:2:3 571 273 95.3 939.3
600:400:200 3:2:1 366 381 191 938
1200:600:300 1:2:3 751 95.4 95.4 941.8
![Page 27: Link Virtualization based on Xen](https://reader035.vdocuments.net/reader035/viewer/2022062319/55515311b4c905e1708b469d/html5/thumbnails/27.jpg)
Link Virtualization Performance
We use 2 physical machines
Each physical machine has 3 guest domains (vir-tual networks)
10.0.0.10
10.0.0.20Vid 80
10.0.0.20Vid 90
10.0.0.20Vid 100
10.0.0.11
10.0.0.21Vid 80
10.0.0.21Vid 90
10.0.0.21Vid 100
![Page 28: Link Virtualization based on Xen](https://reader035.vdocuments.net/reader035/viewer/2022062319/55515311b4c905e1708b469d/html5/thumbnails/28.jpg)
Link Virtualization with Weight Based Control Performance
Weight BW (Mbps) Ratio
VM1:VM2:VM3 VM1 VM2 VM3 total VM1:VM2:VM3
1:1:1 305 305 305 915 1:1:1
1:1:2 228 229 457 914 1:1:2
1:1:8 92 92 730 914 1:1:7.93
1:2:3 152 305 457 914 1:2.01:3.01
1:2:4 130 261 522 913 1:2.01:4.02
1:3:3 131 392 392 915 1:2.99:2.99
1:3:6 92 274 548 914 1:2.98:5.96
1:4:5 92 366 457 915 1:3.97:4.97
2:2:3 261 261 392 914 2:2:3
2:3:4 203 305 406 914 2:3:4
3:3:4 274 274 366 914 3:3:4.01
![Page 29: Link Virtualization based on Xen](https://reader035.vdocuments.net/reader035/viewer/2022062319/55515311b4c905e1708b469d/html5/thumbnails/29.jpg)
Link Virtualization with Bandwidth Based Con-trol Performance
Bandwidth Priority BW (Mbps)
VM1:VM2:VM3 VM1:VM2:VM3 VM1 VM2 VM3 Total
300:200:100 1:2:3 278 185 92.7 555.7
600:400:200 1:2:3 556 264 92.8 912.8
600:400:200 3:2:1 356 371 185 912
1200:600:300 1:2:3 728 92.8 92.8 913.6
![Page 30: Link Virtualization based on Xen](https://reader035.vdocuments.net/reader035/viewer/2022062319/55515311b4c905e1708b469d/html5/thumbnails/30.jpg)
Conclusion
Network virtualization is the core technology of the fu-ture Internet
Link virtualization is necessary for network virtualiza-tion
We propose and implement link virtualization on Xen with SR-IOV
We minimize virtualize overhead through Xen and SR-IOV
![Page 31: Link Virtualization based on Xen](https://reader035.vdocuments.net/reader035/viewer/2022062319/55515311b4c905e1708b469d/html5/thumbnails/31.jpg)
THANK YOU