hawaii tech day- dc switching design
TRANSCRIPT
![Page 1: Hawaii Tech Day- DC Switching Design](https://reader034.vdocuments.net/reader034/viewer/2022052418/58f0efe21a28abb2198b45cf/html5/thumbnails/1.jpg)
Data Center Switching Design
Scott HarrisConsulting Systems Engineer, [email protected]
based on Cisco Live presentation BRK-2208
![Page 2: Hawaii Tech Day- DC Switching Design](https://reader034.vdocuments.net/reader034/viewer/2022052418/58f0efe21a28abb2198b45cf/html5/thumbnails/2.jpg)
Agenda
• Data Center Ethernet Design• evolution of traditional designs
• migrating to an ACI design
• Fibre Channel & Ethernet Convergence
• Cisco Nexus & MDS Switch Overview: very brief
![Page 3: Hawaii Tech Day- DC Switching Design](https://reader034.vdocuments.net/reader034/viewer/2022052418/58f0efe21a28abb2198b45cf/html5/thumbnails/3.jpg)
Design Considerations
![Page 4: Hawaii Tech Day- DC Switching Design](https://reader034.vdocuments.net/reader034/viewer/2022052418/58f0efe21a28abb2198b45cf/html5/thumbnails/4.jpg)
Data Center Design: Scale Up vs Scale Out
Small Spine/Leaf
VXLAN
Single Layer DC
VXLAN
Dual Tier DC
Scale Up
Scale Out
![Page 5: Hawaii Tech Day- DC Switching Design](https://reader034.vdocuments.net/reader034/viewer/2022052418/58f0efe21a28abb2198b45cf/html5/thumbnails/5.jpg)
Connectivity & Features Drive Design Choices
Connectivity Model
– 25 or 10 or 1-GigE Server ports
– NIC/HBA Interfaces per-server
– Copper vs Fiber cabling to ToR
Virtual Networking Requirements
– vSwitch/DVS/OVS/Nexus1Kv/AVS
Programmabiltiy/Automation/Orchestration
– Complete abstraction
– Device or fabric level programmability
Computing Form Factor
– Unified Computing Fabric
– 3rd Party Blade Servers
– Rack Servers (Non-UCS Managed)
Storage & Storage Protocols
– Native Fibre Channel
– Unified Ports, FCoE
– IP-based storage (iSCSI, NAS)
VM VM VM
iSCSI
FCoE
FC
NFS/
CIFS
VM VM VM
![Page 6: Hawaii Tech Day- DC Switching Design](https://reader034.vdocuments.net/reader034/viewer/2022052418/58f0efe21a28abb2198b45cf/html5/thumbnails/6.jpg)
Data Center Network Needs
• Server virtualization is here
• Oversubscription ratio from 20:1 to 1:1
• Horizontal workload mobility
• Any VLAN everywhere, no physical
boundaries
• Highly distributed applications
• Hadoop and microservices
• More server to server communication
• 80/20 traffic pattern reversed?
• Increasing high availability
requirements
• High throughput, low latency
• More speed, less human errorsEAST – WESTTRAFFIC
NO
RT
H-
SO
UT
HT
RA
FF
IC
FC
FCoE iSCSI /
NAS
Server/Compute
Site B
Enterprise
Network
Internet
DATA CENTER NETWORK
Public
Cloud
Mobile
Services
Storage
Orchestration/
Monitoring
OffsiteDC
API
![Page 7: Hawaii Tech Day- DC Switching Design](https://reader034.vdocuments.net/reader034/viewer/2022052418/58f0efe21a28abb2198b45cf/html5/thumbnails/7.jpg)
Why Data Center Network Fabrics?
• Flexibility: allows workload mobility, VLANs everywhere
• Robustness: reduce failure domains, L2/L3 boundaryon leafs, anycast gateway
• (Virtual) Network services move out to border leafs,policy-based service chaining
• Performance: full cross sectional bandwidth (any-to-any) with ECMP, avoid oversubscription
• Latency: deterministic at scale, single hop away
• Scalability: add end nodes, maintain oversubscription
• Cost: fixed switches vs modular switches
EAST – WEST TRAFFIC
NO
RT
H-
SO
UT
HT
RA
FF
IC
FC
FCoE iSCSI /
NAS
Server/Compute
Site B
Enterprise
Network
Internet
DATA CENTER FABRIC
Cloud
Mobile
Services
Storage
Orchestration/
Monitoring
OffsiteDC
API
Definition: ensemble of switches that behaveand get configured like a single giant switch
![Page 8: Hawaii Tech Day- DC Switching Design](https://reader034.vdocuments.net/reader034/viewer/2022052418/58f0efe21a28abb2198b45cf/html5/thumbnails/8.jpg)
Evolution of Data Center Ethernet Networks
![Page 9: Hawaii Tech Day- DC Switching Design](https://reader034.vdocuments.net/reader034/viewer/2022052418/58f0efe21a28abb2198b45cf/html5/thumbnails/9.jpg)
![Page 10: Hawaii Tech Day- DC Switching Design](https://reader034.vdocuments.net/reader034/viewer/2022052418/58f0efe21a28abb2198b45cf/html5/thumbnails/10.jpg)
![Page 11: Hawaii Tech Day- DC Switching Design](https://reader034.vdocuments.net/reader034/viewer/2022052418/58f0efe21a28abb2198b45cf/html5/thumbnails/11.jpg)
![Page 12: Hawaii Tech Day- DC Switching Design](https://reader034.vdocuments.net/reader034/viewer/2022052418/58f0efe21a28abb2198b45cf/html5/thumbnails/12.jpg)
![Page 13: Hawaii Tech Day- DC Switching Design](https://reader034.vdocuments.net/reader034/viewer/2022052418/58f0efe21a28abb2198b45cf/html5/thumbnails/13.jpg)
![Page 14: Hawaii Tech Day- DC Switching Design](https://reader034.vdocuments.net/reader034/viewer/2022052418/58f0efe21a28abb2198b45cf/html5/thumbnails/14.jpg)
![Page 15: Hawaii Tech Day- DC Switching Design](https://reader034.vdocuments.net/reader034/viewer/2022052418/58f0efe21a28abb2198b45cf/html5/thumbnails/15.jpg)
![Page 16: Hawaii Tech Day- DC Switching Design](https://reader034.vdocuments.net/reader034/viewer/2022052418/58f0efe21a28abb2198b45cf/html5/thumbnails/16.jpg)
![Page 17: Hawaii Tech Day- DC Switching Design](https://reader034.vdocuments.net/reader034/viewer/2022052418/58f0efe21a28abb2198b45cf/html5/thumbnails/17.jpg)
![Page 18: Hawaii Tech Day- DC Switching Design](https://reader034.vdocuments.net/reader034/viewer/2022052418/58f0efe21a28abb2198b45cf/html5/thumbnails/18.jpg)
![Page 19: Hawaii Tech Day- DC Switching Design](https://reader034.vdocuments.net/reader034/viewer/2022052418/58f0efe21a28abb2198b45cf/html5/thumbnails/19.jpg)
![Page 20: Hawaii Tech Day- DC Switching Design](https://reader034.vdocuments.net/reader034/viewer/2022052418/58f0efe21a28abb2198b45cf/html5/thumbnails/20.jpg)
![Page 21: Hawaii Tech Day- DC Switching Design](https://reader034.vdocuments.net/reader034/viewer/2022052418/58f0efe21a28abb2198b45cf/html5/thumbnails/21.jpg)
![Page 22: Hawaii Tech Day- DC Switching Design](https://reader034.vdocuments.net/reader034/viewer/2022052418/58f0efe21a28abb2198b45cf/html5/thumbnails/22.jpg)
![Page 23: Hawaii Tech Day- DC Switching Design](https://reader034.vdocuments.net/reader034/viewer/2022052418/58f0efe21a28abb2198b45cf/html5/thumbnails/23.jpg)
![Page 24: Hawaii Tech Day- DC Switching Design](https://reader034.vdocuments.net/reader034/viewer/2022052418/58f0efe21a28abb2198b45cf/html5/thumbnails/24.jpg)
![Page 25: Hawaii Tech Day- DC Switching Design](https://reader034.vdocuments.net/reader034/viewer/2022052418/58f0efe21a28abb2198b45cf/html5/thumbnails/25.jpg)
![Page 26: Hawaii Tech Day- DC Switching Design](https://reader034.vdocuments.net/reader034/viewer/2022052418/58f0efe21a28abb2198b45cf/html5/thumbnails/26.jpg)
![Page 27: Hawaii Tech Day- DC Switching Design](https://reader034.vdocuments.net/reader034/viewer/2022052418/58f0efe21a28abb2198b45cf/html5/thumbnails/27.jpg)
![Page 28: Hawaii Tech Day- DC Switching Design](https://reader034.vdocuments.net/reader034/viewer/2022052418/58f0efe21a28abb2198b45cf/html5/thumbnails/28.jpg)
![Page 29: Hawaii Tech Day- DC Switching Design](https://reader034.vdocuments.net/reader034/viewer/2022052418/58f0efe21a28abb2198b45cf/html5/thumbnails/29.jpg)
![Page 30: Hawaii Tech Day- DC Switching Design](https://reader034.vdocuments.net/reader034/viewer/2022052418/58f0efe21a28abb2198b45cf/html5/thumbnails/30.jpg)
Scaling Data Center Ethernet
![Page 31: Hawaii Tech Day- DC Switching Design](https://reader034.vdocuments.net/reader034/viewer/2022052418/58f0efe21a28abb2198b45cf/html5/thumbnails/31.jpg)
![Page 32: Hawaii Tech Day- DC Switching Design](https://reader034.vdocuments.net/reader034/viewer/2022052418/58f0efe21a28abb2198b45cf/html5/thumbnails/32.jpg)
![Page 33: Hawaii Tech Day- DC Switching Design](https://reader034.vdocuments.net/reader034/viewer/2022052418/58f0efe21a28abb2198b45cf/html5/thumbnails/33.jpg)
![Page 34: Hawaii Tech Day- DC Switching Design](https://reader034.vdocuments.net/reader034/viewer/2022052418/58f0efe21a28abb2198b45cf/html5/thumbnails/34.jpg)
![Page 35: Hawaii Tech Day- DC Switching Design](https://reader034.vdocuments.net/reader034/viewer/2022052418/58f0efe21a28abb2198b45cf/html5/thumbnails/35.jpg)
![Page 36: Hawaii Tech Day- DC Switching Design](https://reader034.vdocuments.net/reader034/viewer/2022052418/58f0efe21a28abb2198b45cf/html5/thumbnails/36.jpg)
![Page 37: Hawaii Tech Day- DC Switching Design](https://reader034.vdocuments.net/reader034/viewer/2022052418/58f0efe21a28abb2198b45cf/html5/thumbnails/37.jpg)
![Page 38: Hawaii Tech Day- DC Switching Design](https://reader034.vdocuments.net/reader034/viewer/2022052418/58f0efe21a28abb2198b45cf/html5/thumbnails/38.jpg)
![Page 39: Hawaii Tech Day- DC Switching Design](https://reader034.vdocuments.net/reader034/viewer/2022052418/58f0efe21a28abb2198b45cf/html5/thumbnails/39.jpg)
![Page 40: Hawaii Tech Day- DC Switching Design](https://reader034.vdocuments.net/reader034/viewer/2022052418/58f0efe21a28abb2198b45cf/html5/thumbnails/40.jpg)
![Page 41: Hawaii Tech Day- DC Switching Design](https://reader034.vdocuments.net/reader034/viewer/2022052418/58f0efe21a28abb2198b45cf/html5/thumbnails/41.jpg)
![Page 42: Hawaii Tech Day- DC Switching Design](https://reader034.vdocuments.net/reader034/viewer/2022052418/58f0efe21a28abb2198b45cf/html5/thumbnails/42.jpg)
Data Center Interconnect (DCI):
just a teaser
![Page 43: Hawaii Tech Day- DC Switching Design](https://reader034.vdocuments.net/reader034/viewer/2022052418/58f0efe21a28abb2198b45cf/html5/thumbnails/43.jpg)
![Page 44: Hawaii Tech Day- DC Switching Design](https://reader034.vdocuments.net/reader034/viewer/2022052418/58f0efe21a28abb2198b45cf/html5/thumbnails/44.jpg)
![Page 45: Hawaii Tech Day- DC Switching Design](https://reader034.vdocuments.net/reader034/viewer/2022052418/58f0efe21a28abb2198b45cf/html5/thumbnails/45.jpg)
![Page 46: Hawaii Tech Day- DC Switching Design](https://reader034.vdocuments.net/reader034/viewer/2022052418/58f0efe21a28abb2198b45cf/html5/thumbnails/46.jpg)
![Page 47: Hawaii Tech Day- DC Switching Design](https://reader034.vdocuments.net/reader034/viewer/2022052418/58f0efe21a28abb2198b45cf/html5/thumbnails/47.jpg)
![Page 48: Hawaii Tech Day- DC Switching Design](https://reader034.vdocuments.net/reader034/viewer/2022052418/58f0efe21a28abb2198b45cf/html5/thumbnails/48.jpg)
![Page 49: Hawaii Tech Day- DC Switching Design](https://reader034.vdocuments.net/reader034/viewer/2022052418/58f0efe21a28abb2198b45cf/html5/thumbnails/49.jpg)
Fibre Channel & Ethernet Convergence
Based on Cisco Live presentation BRKDCN-1902
![Page 50: Hawaii Tech Day- DC Switching Design](https://reader034.vdocuments.net/reader034/viewer/2022052418/58f0efe21a28abb2198b45cf/html5/thumbnails/50.jpg)
Traditional Data Center Design
Ethernet LAN and Fibre Channel SAN
• Physical and Logical separation of LANand SAN traffic
• Additional Physical and Logicalseparation of SAN fabrics
Isolation Convergence
Fabric ‘B’
HBA
L2
L3
NIC
Fabric ‘A’
FC
Nexus
7000
Nexus
5000
MDS 9000
Ethernet
FC
![Page 51: Hawaii Tech Day- DC Switching Design](https://reader034.vdocuments.net/reader034/viewer/2022052418/58f0efe21a28abb2198b45cf/html5/thumbnails/51.jpg)
Data Center Design with E-SAN
Ethernet LAN and Ethernet SAN
• Same topologies as existing networks, but using NexusUnified Fabric Ethernet switches for SANs
• Physical and Logical separation of LAN and SAN traffic
• Additional Physical and Logical separation of SANfabrics
• Ethernet SAN Fabric carries FC/FCoE & IP basedstorage (iSCSI, NAS, …)
• Common components: Ethernet Capacity and Cost
Isolation Convergence
Fabric ‘B’
CNA
L2
L3
NIC
or
CNA
Fabric ‘A’
FCoE
Nexus
7000
Nexus
5000
Nexus
7000
Nexus
5000
Ethernet
FC
![Page 52: Hawaii Tech Day- DC Switching Design](https://reader034.vdocuments.net/reader034/viewer/2022052418/58f0efe21a28abb2198b45cf/html5/thumbnails/52.jpg)
Converged Access with vPCSharing Access Layer for LAN and SAN
• Shared Physical, Separate Logical LAN and SAN traffic atAccess Layer
• Physical and Logical separation of LAN and SAN traffic at Aggregation Layer
• Additional Physical and Logical separation of SAN fabrics
• Storage VDC on Nexus 7000 for additional management /operation separation
Isolation Convergence
Fabric ‘B’
Ethernet
FC
Converged FCoE link
DedicatedFCoE link
L2L3
CNA
Fabric ‘A’
FCFCoE
Nexus
7000
Nexus
5000
MDS
9000
![Page 53: Hawaii Tech Day- DC Switching Design](https://reader034.vdocuments.net/reader034/viewer/2022052418/58f0efe21a28abb2198b45cf/html5/thumbnails/53.jpg)
Agg BW: 40G
FCoE:
Ethernet:
40G
40G
One wire for all traffic types
QoS guarantees minimum bandwidth
allocation
No Clear Port ownership
Desirable for DCI Connections
Dedicated wire for a traffic type
No Extra output feature processing
Distinct Port ownership
Complete Storage Traffic Separation
HA: 4LinksAvailable
Different methods, Producing the same aggregate bandwidth
Dedicated Links provide additional isolation of Storage Traffic
Dedicated vs. Converged ISLs
Why support Dedicated ISLs as opposed to Converged?
Agg BW: 40G
FCoE:
Ethernet:
20G
20G
![Page 54: Hawaii Tech Day- DC Switching Design](https://reader034.vdocuments.net/reader034/viewer/2022052418/58f0efe21a28abb2198b45cf/html5/thumbnails/54.jpg)
Converged Network – Dual Fabrics with Dedicated Links
Maintaining Dual SAN fabrics with Overlay
• LAN and SAN traffic share physical switches
• LAN and SAN traffic use dedicated links between switches
• All Access and Aggregation switches are FCoE FCFswitches
• Dedicated links between switches are VE_Ports
• Storage VDC for additional management / operationseparation
Isolation Convergence
Nexus 7000
Nexus 5000
Ethernet
FC
Converged FCoE link
DedicatedFCoE link
…
…
L2
L3
CNA FCFCoE
FCFFCF
FCF
VE
Fabric ‘A’
Fabric ‘B’
LAN/SAN
![Page 55: Hawaii Tech Day- DC Switching Design](https://reader034.vdocuments.net/reader034/viewer/2022052418/58f0efe21a28abb2198b45cf/html5/thumbnails/55.jpg)
Converged Network – Dual Fabrics with Dedicated Links
Maintaining Dual SAN fabrics with Overlay
• LAN and SAN traffic share physical switches
• LAN and SAN traffic use dedicated links between switches
• All Access and Aggregation switches are FCoE FCFswitches
• Dedicated links between switches are VE_Ports
• Storage VDC for additional management / operation separation
Isolation Convergence
Nexus 7000
Nexus 5000
Ethernet
FC
Converged FCoE link
Dedicated FCoE link
FabricPath
L2
L3
CNA FCFCoE
Fabric ‘A’
Fabric ‘B’
FCF
FCF
FCF
FCF
VE
![Page 56: Hawaii Tech Day- DC Switching Design](https://reader034.vdocuments.net/reader034/viewer/2022052418/58f0efe21a28abb2198b45cf/html5/thumbnails/56.jpg)
Cisco Nexus & MDS Platforms
Operational Simplicity
Architectural Flexibility
Open/ Programmable
Resilience and Scale
Investment Protection
One Operating System - NX-OS
CiscoNexus 2000
Cisco Nexus 9000
Cisco Nexus 3000
Cisco Nexus 1000VL4-7 vServices
Cisco Nexus 7000/7700
Cisco Nexus 5000
Cisco MDS 9200 Multiservice Switch
Cisco MDS 9700 Series
Cisco MDS 9100 Series
Cisco MDS 9500 Series
![Page 57: Hawaii Tech Day- DC Switching Design](https://reader034.vdocuments.net/reader034/viewer/2022052418/58f0efe21a28abb2198b45cf/html5/thumbnails/57.jpg)