networking solutions for a server virtualization environment
DESCRIPTION
Networking Solutions for A Server Virtualization Environment. APRICOT 2011 Russell Cooper [email protected]. What you will get from this session. 1. Talk: about challenges Server Virtualization technologies brings for the data center networks. - PowerPoint PPT PresentationTRANSCRIPT
NETWORKING SOLUTIONS FOR A SERVER VIRTUALIZATION ENVIRONMENT
APRICOT 2011
Russell Cooper
2
WHAT YOU WILL GET FROM THIS SESSION
1. Talk: about challenges Server Virtualization technologies brings for the data center networks.
2. Demonstrate: standards based approach, where available, to improve the experience and economics in a virtualized environment.
3
AGENDA
1. Market Drivers
2. Limitations of legacy network
3. Solutions Simplification Infrastructure Enhanced services
4. Summary
4
THE EVOLUTION OF SERVER VIRTUALIZATION
PHASE 1 PAST
Server Consolidation
Guiding Principle: Improve utilization of physical resources
Driver: Power and space Improvements in server utilization Savings
Network had no role
PHASE 2 FUTURE
Business Agility
Guiding Principle: : Improve utilization of a pool of resources
Driver: Adapt quickly to new demands Heightened compliance & security Better disaster management Cloud Based Computing Models
Network has a huge role
5
LEGACY NETWORKS RESTRICT AGILITY
VM2 VM3
SERVER 1
NIC
VM2 VM3VM1
SERVER 2
NIC
VM1
COMPLEX:Too Many Devices
to ManageAdditional virtual
switches
INFRASTRUCTURE: LACK OF ADDITIONAL SERVICES:
POOR PERFORMANCEMultiple layersAcross North-South path
PROPRIETARY:Pre-standard protocols
MOBILITY:North-south pathScale & scope of L2 adjacenciesAcross sites
SECURITY:Silo’ed , unavailable across domains Intra-VM traffic
MANAGEABILITY:Orchestration between the physical and virtual network
6
NETWORK SIMPLIFICATION FOR SUPPORTING SERVER VIRTUALIZATION
VM2 VM3
SERVER 1
NIC
VM2 VM3VM1
SERVER 2
NIC
VM1
INFRASTRUCTURE: LACK OF ADDITIONAL SERVICES:
POOR PERFORMANCEMultiple layersAcross North-South path
PROPRIETARY:Pre-standard protocolsInteroperability Lock-in
MOBILITY:North-south pathScale & scope of L2 adjacenciesAcross sites
SECURITY:Silo’ed , unavailable across domains Intra-VM traffic
MANAGEABILITY:Orchestration between the physical and virtual network
HIGH PERFORMANCE
INFRASTRUCTURE THAT IS:
OPEN, STANDARDS
BASED
MOBILITY
MANAGEABILITY
SECURITY
ENHANCED SERVICES NEEDED
COMPLEX:Too Many Devices
to ManageAdditional virtual
switchesSIMPLIFICATION
7
BEFORE AFTER
Fewer devices to manage: 44 -> 4
SIMPLIFICATION
NETWORK DEVICE CLUSTERING
8
TECHNOLOGY APPROACHES
Facts Simplify operations Behaves as a single node
both at L2 & L3 layers so it inherits all benefits found in L2 Table Synch approach
Control Plane Unification
Facts Distributed link
aggregation (LAG) plus some L2/L3 protocols enhancements to minimize interchassis link load
L2 Table Synch
Multiple Devices – One Control PlaneMultiple Devices – Enhanced
Protocols
9
INFRASTRUCTURE THAT IS:
OPEN STANDARDS BASED
SIMPLIFICATION
HIGH PERFORMANCE MOBILITY
MANAGEABILITY
SECURITY
ENHANCED SERVICES NEEDED
OPEN, STANDARDS
BASED
10
VM2VM1
NIC
VM3VM2VM1
NIC
VM3VM2VM1
NIC
COMMUNICATION BETWEEN THE VIRTUAL MACHINES
1. In the hypervisor vendor’s switch(e.g.VM Ware vSwitch)
2. In the NIC 3. In the existing external physical switch (VEPA)
VM3
11
COMPARING VEPA AND VEB
VM2VM1
NIC
VM3 VM2VM1
NIC
VM3
Virtual Ethernet Port Aggregator (VEPA)
North – South optimizedFull functioned hardware
switch
Virtual Ethernet Bridge (VEB)
East – West optimizedLimited function software
switch
Hypervisor/softwareswitch
Physical switch
Network servicesin hardware
Network servicesin software
12
COMPARISON OF OPTIONS
1 2 3
Switching done in Software Hardware Hardware
Customer’s Time to adopt solution
Low – comes in- built with hypervisor
UnknownLow - simple
software upgrade
Latency for switching Very LowVery Low
Low
vSwitch NIC VEPA
Industry support (standards based)
NA Unknown Yes
Virtual switching managed by
Server admin UnknownNetwork Admin
Customers’ Cost to adopt
Low – comes with hypervisor
UnknownFree - software
upgrade
Compatibility with any existing network
Yes Unknown Yes
Feature Richness Very Low Low High
13
VEPA
Virtual Ethernet Port Aggregator Uses external physical network for intra-
server VM to VM communication It’s an evolving open standard IEEE
802.1Qbg / 802.1Qbh Supported by almost all the major IT
vendors For more information
http://www.ieee802.org/1/files/public/docs2009/new-bg-thaler-par-1109.pdf http://www.ieee802.org/1/pages/802.1bg.html
VEPA brings the evolved Ethernet functionality to virtual networking
VM2VM1
NIC
VM3
14
TOP 3 BENEFITS OF VEPA
Features & Scale
Switching where it belongs – on the switches
Elegant
VEPA is a non-disruptive and cost-effective
Open
Server and hypervisor agnostic, maximum flexibility.
15
INFRASTRUCTURE THAT IS:
HIGH PERFORMANCE
SIMPLIFICATION
OPEN, STANDARDS
BASED
MOBILITY
MANAGEABILITY
SECURITY
ENHANCED SERVICES NEEDED
HIGH PERFORMANCE
16
LATENCY WITH LEGACY NETWORK
Every hop adds additional latency
Increases load on uplinks
Requires VLANs to span multiple access switches to support VM migration
BA
17
VIRTUALIZATION WITH CHASSIS CLUSTERING
Clustered Access
Switches
10x latency improvement by eliminating trip to upper layers
Single-point lookup model
Works with any Hypervisor
BA
18
INFRASTRUCTURE THAT IS:
MOBILITY
SIMPLIFICATION
OPEN, STANDARDS
BASED
MANAGEABILITY
SECURITY
ENHANCED SERVICES NEEDED
HIGH PERFORMANCE MOBILITY
19
NETWORK REQUIREMENTS FOR VM MOBILITY
IP network with 622 Mbps is required.
The maximum latency between the two servers < 5 milliseconds (ms).
Access to the IP subnet & data storage location
Access from vCenter Server and vSphere Client.
Same IP subnet & broadcast domain Layer 2 adjacency VLAN stretch
20
VM MIGRATION SCENARIOS
Within Same Data Center
Rack A
Layer 2 domain across racks
Scenario #1
Clustered Access Switches
Rack A
Data Centers in the same City - two different locations
Layer 2 domain across fiber connected data centers
Scenario #2
Clustered Access Switches
Data Center Data Center
Layer 2 domain across virtual private LAN
Scenario #3
Clustered Access Switches
Data Center Data Center
VPLS
Data Centers in different Cities
Remember the vMotion Requirements!Bandwidth/Latency/IP Subnet/VLAN
21
Top-of-Rack / End-of-Row Clustered
Switches
RACK TO RACK
RACK 1 RACK 2
Managed as a single device
Automatic VLAN update propagation.
Sub 10us latency
VM2 VM5VM3
NIC NIC
VM4VM1
22
VM2VM1 VM5VM4VM3
NIC NIC
VM2VM1 VM5VM4VM3
NIC NIC
POD TO POD
CoreClustered Chassis
Extends L2 domain across multiple Rows/Pods in a DC
Extends L2 adjacency to over 10,000 1GbE servers
Eliminates STP
Core managed as a single device
VM2 VM5
NIC NIC
POD NPOD 1
Clustered Access Switches
VM3 VM4VM1
23
ACROSS DC/CLOUDS
Extends L2 domain across DC /clouds
Allows VM Motion across locations.
VPLS can be provisioned or orchestrated using vendor tools and scripts
VLAN to VPLS mapping
DB/Storage mirroringVM2VM1 VM5VM4VM3
NIC NIC
VM2VM1 VM5VM4VM3
NIC NIC
VM2 VM5VM4
NIC NIC
VM2VM1 VM5VM4VM3
NIC NIC
VM2VM1 VM5VM4VM3
NIC NIC
VM2VM1 VM5VM3
NIC NIC
VM6
VPLS Over MPLS Cloud
Routers with VPLS
Core Switches
AccessSwitches
RoutersWith VPLS
VM3 VM4
CoreSwitches
AccessSwitches
VM1
24
INFRASTRUCTURE THAT IS:
MANAGEABILITY
SIMPLIFICATION
OPEN, STANDARDS
BASEDSECURITY
ENHANCED SERVICES NEEDED
HIGH PERFORMANCE MOBILITY
MANAGEABILITY
25
Network Admin
Server Admin
DC MANAGEABILITY CHALLENGES WITH SERVER VIRTUALIZATION
1. Blurred roles between the server and network admin.
2. No automation/orchestration to sync-up the 2 networks.
3. VM Migration can fail.
4. Proprietary products & protocols
B
AVirtual n/w
Physical n/w
PP
VM1 VM2 VM3 VM1 VM2
A
26
ONE STEP ORCHESTRATION
1. Clear roles and responsibilities
2. Automated orchestration between physical and virtual networks
3. Scalable solution – allows VMs to move freely
4. Open Architecture
Network Admin
Server Admin
VM1 VM2
Orchestration Tools
A
AA
A
Virtual n/w
Physical n/w
PPA A
VM2 VM3VM1
27
INFRASTRUCTURE THAT IS:
SECURITY
SIMPLIFICATION
OPEN, STANDARDS
BASED
ENHANCED SERVICES NEEDED
HIGH PERFORMANCE MOBILITY
MANAGEABILITY
SECURITY
28
VIRTUAL NETWORK
SECURITY IMPLICATIONS OF VIRTUAL SERVERS
PHYSICAL NETWORK
ES
X H
os
t
Physical Security is “Blind” toTraffic Between Virtual Machines
Firewall/IPS InspectsAll Traffic Between Servers
HYPERVISOR
VM1 VM2 VM3
29
APPROACHES TO SECURING VIRTUAL SERVERS:THREE METHODS
2. Agent-based
Each VM has a software firewall
Drawback: Significant performance implications; Huge management overhead of maintaining software and signature on 1000s of VMs
ES
X H
ost
VM1 VM2 VM3
FW Agents
HYPERVISOR
3. Kernel-based Firewall
VMs can securely share VLANs
Inter-VM traffic always protected
High-performance from implementing firewall in the kernel
Micro-segmenting capabilities
ES
X H
ost
FW as Kernel Module
VM1 VM2 VM3
HYPERVISOR
1. VLAN Segmentation
ES
X H
ost
Each VM in separate VLAN
Inter-VM communications must route through the firewall
Drawback: Possibly complex VLAN networking
HYPERVISOR
VM1 VM2 VM3
30
Hypervisor Kernel Stateful Firewall
Purpose-built virtual firewall Secure Live-Migration (VMotion) Security for each VM by VM ID Fully stateful firewall
Tight Integration with Virtual Platform Management, e.g. VMware vCenter
Fault-Tolerant Architecture
ES
X H
ostKERNEL VF
INTRODUCING THE IDEA OF A STATEFUL KERNEL FIREWALL
SecurityPolicy
Management
Data CenterFirewall
AccessSwitch
NetworkSecurity
InformationAnd Event
Management
VM1 VM2 VM3
31
ES
X H
ost
FOLLOW-ME POLICIES
Data Centre Firewall
Access Switch
ES
X H
ost
Access Switch
When a VM migrates, the network policies of the VM are migrated to the new server port.
Traffic between VMs still gets re-directed to the same appliance in the Services cluster
No migration of services state is required
Policy
VM2 VM3 VM3VM2
KERNEL VF KERNEL VF
Policy
VM1
32
SIMPLIFCATION: Few DevicesFewer Devices to
Manage
SUMMARY OF SOLUTIONS FOR SERVER VIRTUALIZATION
INFRASTRUCTURE: ADDITIONAL SERVICES
HIGH PERFORMANCEFew layersClustered Switches
OPEN:VEPAStandards Based
MOBILITY:VPLSClustered Switch domains
SECURITY:Kernel Stateful FirewallsIntegration with DC FWs for follow me policies
MANAGEABILITY:VEPAOrchestration Tools
Routers
Core Switch
Clusters
Data Center Firewalls
Access Switch Clusters
VM2VM3
SERVER 1
NIC
VM2 VM3VM1
SERVER 2
NIC
VM1