1. Architecting Data CenterNetworks in the era of Big Dataand CloudBrad HedlundSpring InteropMay 2012 VIDEO of this session: http://bradhedlund.com/?p=3912Global Marketing

2. Two approaches to DC NetworkingTHE SAME OLD Centralized, Scale-up Layer 2 networks Monstrous chassis switchesTRILL OpenFlow VEPASPB Or a Different ApproachDistributed, Scale-out Layer 3 fabricsEfficient fixed switchesOpen, industry standard protocols2 Brad Hedlund Global Marketing 3. Networks that suck for Cloud & Big DataCoreNetwork Dist TopologyAccess VMCapacityTopologyPARTITIONEDCAPACITYData center networks are in my way -James Hamilton, AWS3 Brad Hedlund Global Marketing 4. Networks that Dont suck for Cloud & Big Data Spine Network TopologyLeafVM Capacity TopologyUNIFORM CAPACITY All points equidistant4Brad HedlundGlobal Marketing 5. Big DataTCP switch switch TCP switch switch switch switchswitchName NodeJob TrackerSecondary NNNode Node NodeNode NodeNodeNode NodeNode NodeNodeNode NodeNode NodeNodeNode NodeNode NodeNodeClient ClientTCPTCPRack 1 Rack 2 Rack 3 Rack 4Rack NWorld Inverse Virtualization Workloads orchestrated like cattle L2 or L3 network. Does it matter?5Brad HedlundGlobal Marketing 6. Basic requirements of Cloud (IaaS) switchswitch Physical Network switchswitch switch switch FW LBWorldVirtual Network VM VM VMVM Secure, Scalable Multi Tenancy Location independence On Demand virtual networks6Brad HedlundGlobal Marketing 7. Blend the Virtual and Physical NetworksVLAN 20 switchVLAN 10 switchswitchVM VMvSwitch vSwitchVM VM VM VM VM VM HostHost Tenant subnet = Network VLAN7Brad HedlundGlobal Marketing 8. Abstract the Virtual Network from Physical Segment ID 20switch Segment ID 10 switchswitchVM VMvSwitchvSwitchVM VM VM VM VM VM Host Host Network Virtualization Overlay Tenant subnet = Software VLAN8Brad HedlundGlobal Marketing 9. Scale-up centralized Layer 2L3 2-post Rooted ArchitectureL2 Centralized L2/L3 L2/L3/ARP table scale? Scale w/ Bigger Boxes Precious Pets VLAN Provisioning? vSwitch vSwitch Broadcasts VM VMVM VM9 Brad Hedlund Global Marketing 10. Scale-out Layer 3 Leaf/Spine Fabric (16) (8) (2)(128) L3 (64)(16) L21980 Server ports3072 Server ports 7686144 Server ports Mesh from Leaf to Spine Non-blocking Spine OSPF, ISIS, BGP, TRILL 3:1 @ ToR ToR w/ 16 uplinks (ECMP) 128 port 2RU Spine10 Brad HedlundGlobal Marketing 11. Uniform fabric for Cloud & Big Data(16)(8) (2) Storage AccessHadoop(128)L3 (64)(16)L2 Name NodeJob TrackerSecondary NN Node Database ports6144 ServerNode NodeNode Node Node Node Node Node vSwitch vSwitch Node NodeBlock I/O Node Node Node Node Node Node Node NodeNAS Node Node VM VMVM VM Client ClientObject Node Node Rack 1 Rack 2 Rack 3 Rack N11Brad Hedlund Global Marketing 12. Attaching Services & North/South (16)(8) (2)(128)L3 (64)(16)L2LB Firewallx86 GatewaysName NodeNodeJob TrackerSecondary NNLB FirewallNodeNodeNodeNodeNodeNodeNodeNodevswitch vswitch vswitchWorld VM VM VMVM VM VMVM VM VM Client Clientvswitch vswitch vswitch Rack 1 Rack N VM VMVM VM VMVM VM VMVM12Brad Hedlund Global Marketing 13. Generic Logical Architecture 1World Overlay based L2 Fabric DC router Physical/Static FWL3L3FWNAT FWNATL2L2 LB LB L3 VMVM L2 L2VMVMVMBig DataGreen Co.Orange Co.13Brad Hedlund Global Marketing 14. Generic Logical Architecture 2 World Overlay based L2 Virtual/Mobile FW Fabric DC router Overlay GatewayPub DMZ L3L3FW NAT FWNAT L2 L2 LBLB L3VMVML2 L2VMVM VMBig Data Green Co.Orange Co.14 Brad HedlundGlobal Marketing 15. Generic Logical Architecture 3World No Overlays TRILL based L2 Fabric DC router Virtual/Mobile FW Pub DMZTRILLL3L3 FW NAT FWNATL2L2LBLB L3 VMVM L2 L2 VMVM VMBig DataGreen Co.Orange Co.15Brad Hedlund Global Marketing 16. Density: Fixed vs. Chassis 10G per RU @ Line Rate (L3) 140 120 100 80 Chassis 60Fixed 40 20020082010 2012 201416Brad Hedlund Global Marketing 17. Power: Fixed vs. Chassis 18 Max Watts / Line Rate 10G (L3) 16 14 12 10Chassis 8Fixed 6 4 2 0 20102012 201417 Brad HedlundGlobal Marketing 18. What are the Challenges? (16)(8)(2) Dell Fabric Manager Design Templates(128) Validate deployment Automate fabric configurationL3(16) Monitoring & OperationsL2 Deployment & Cabling Layer 2 (TRILL?) Configuration & Policy Design Best Practices Monitor & Troubleshoot Dell Fabric Manager18Brad Hedlund Global Marketing 19. Webinar: CLOS Fabrics Explained http://closfabric.eventbrite.com/ Wednesday, June 20, 2012 from 10:00 AM to 1:00 PM (ET)HOSTCO-HOSTBrad HedlundIvan Pepelnjak19Brad Hedlund Global Marketing 20. Three Stage Layer 3 Leaf/Spine Fabric (64) (8) (2) /26(128)0/0/26 /26(512) 0/0 L3 L224,576 Server ports Non-blocking @ top tiers Leaf+ToR mesh groups Default route @ ToR & Leaf ~8usec worst case20 Brad HedlundGlobal Marketing 21. The case for 40G QSFP switch ports 10G 10G 10G 10G10G10G10G 10G VSSFP+ SFP+ SFP+SFP+ QSFP$1K$1K$1K $1K $1,800 32 ToR $230K $512K21 Brad HedlundGlobal Marketing 22. Comparing Fabric efficienciesof Fixed vs. Chassis designs Supplemental SectionBrad HedlundMay 2012Global Marketing 23. Total Fabric Power: Chassis vs. Fixed KWnon-blocking 250 200 150Chassis 100Fixed 500384 20484096 8192 Fabric size23 Brad HedlundGlobal Marketing 24. Total Fabric RU: Chassis vs. Fixed RU non-blocking 700 600 500 400 Chassis 300 Fixed 200 100 0 384 20484096 8192 Fabric size24Brad Hedlund Global Marketing 25. 8192 non-blocking Fabric (64) (8) (2)(128)8192 non-blocking 384RU 153.6KW 8192 ISL 192 switches25 Brad HedlundGlobal Marketing 26. 8192 non-blocking Fabric(32)(256) 8192 non-blocking 608RU 216.3KW 8192 ISL 288 switches26Brad Hedlund Global Marketing 27. 4096 non-blocking Fabric(32) (2) (64) 4096 non-blocking 192RU 76.8KW 4096 ISL 96 switches27Brad Hedlund Global Marketing 28. 4096 non-blocking Fabric(16)(128) 4096 non-blocking 304RU 108.1KW 4096 ISL 144 switches28Brad Hedlund Global Marketing 29. 2048 non-blocking Fabric(16) (2) (32) 2048 non-blocking 96RU 38.4KW 512 ISL 48 switches29Brad Hedlund Global Marketing 30. 2048 non-blocking Fabric(8)(64)2048 non-blocking 152RU 54KW 2048 ISL 72 switches 30Brad HedlundGlobal Marketing 31. 384 non-blocking Fabric (4) (2) (6) 384 non-blocking 20RU 8KW 96 ISL 10 switches31 Brad HedlundGlobal Marketing 32. 384 non-blocking Fabric (2) (12) 384 non-blocking 26RU 18KW 384 ISL 14 switches32 Brad HedlundGlobal Marketing 33. 256 non-blocking Fabric (2) (4) 256 non-blocking 12RU 4.8KW 64 ISL 6 switches33 Brad HedlundGlobal Marketing 34. 256 non-blocking Fabric(2) (8)256 non-blocking 22RU 6.7KW 256 ISL 10 switches34Brad Hedlund Global Marketing 35. Total Fabric Power: Chassis vs. Fixed KW 3:1 oversubscribed 80 70 60 50 40Chassis Fixed 30 20 10 0 768 15363072 6144 Fabric size35Brad Hedlund Global Marketing 36. Total Fabric RU: Chassis vs. Fixed RU 3:1 oversubscribed 250 200 150 Chassis 100 Fixed 500768 1536 3072 6144 Fabric size36Brad Hedlund Global Marketing 37. 768 @ 3:1 oversubscribed Fabric (2)(16) 768 @ 3:1 20RU 7.2KW 64 ISL 18 switches37Brad Hedlund Global Marketing 38. 768 @ 3:1 non-blocking Fabric (2) (16) 256 non-blocking 30RU 7.5KW 256 ISL 18 switches38 Brad HedlundGlobal Marketing 39. 1536 @ 3:1 oversubscribed Fabric(4) (2) (32) 1536 @ 3:1 40RU 14.4KW 128 ISL 36 switches39 Brad HedlundGlobal Marketing 40. 1536 @ 3:1 oversubscribed Fabric(2) (32)1536 @ 3:1 54RU 17KW 512 ISL 34 switches40 Brad HedlundGlobal Marketing 41. 3072 @ 3:1 oversubscribed Fabric(8) (2) (64) 3072 @ 3:1 80RU 28.8KW 1024 ISL 72 switches41 Brad HedlundGlobal Marketing 42. 3072 @ 3:1 oversubscribed Fabric(4) (64)3072 @ 3:1 108RU 34KW 1024 ISL 68 switches42 Brad HedlundGlobal Marketing 43. 6144 @ 3:1 oversubscribed Fabric (16) (8) (2)(128) 6144 @ 3:1 160RU 57.6KW 2048 ISL 144 switches43Brad Hedlund Global Marketing 44. 6144 @ 3:1 oversubscribed Fabric(8)(128)6144 @ 3:1 216RU 68KW 2048 ISL 136 switches44Brad Hedlund Global Marketing 45. Total Fabric Power: Chassis vs. Fixed KW 3:1 oversubscribed - Massive 1400 1200 1000 800 Chassis 600 Fixed 400 200 0 12288245764915298304 Fabric size45Brad Hedlund Global Marketing 46. Total Fabric RU: Chassis vs. Fixed RU 3:1 oversubscribed - Massive 5000 4500 4000 3500 3000 2500Chassis 2000Fixed 1500 1000 500 0 12288245764915298304 Fabric size46Brad Hedlund Global Marketing 47. 12,288 @ 3:1 oversubscribed Fabric(32) (64) (8)(2)(256) 12,288 @ 3:1 448RU 166.4KW 5120 ISL 352 switches47 Brad HedlundGlobal Marketing 48. 12,288 @ 3:1 oversubscribed Fabric(16)(256) 12,288 @ 3:1 432RU 136.3KW 4096 ISL 272 switches48Brad Hedlund Global Marketing 49. 24,576 @ 3:1 oversubscribed Fabric (64)(128) (8) (2)(512)24,576 @ 3:1 896RU 332.8KW 10,240 ISL 704 switches49Brad Hedlund Global Marketing 50. 24,576 @ 3:1 oversubscribed Fabric (32) (256)(512) 24,576 @ 3:1 1120RU 328.9KW 16,384 ISL 800 switches50Brad Hedlund Global Marketing 51. 49,152 @ 3:1 oversubscribed Fabric (128)(256) (1024)49,152 @ 3:1 1792RU 665.6KW 20480 ISL 1408 switches51 Brad HedlundGlobal Marketing 52. 49,152 @ 3:1 oversubscribed Fabric (64)(512)(1024)49,152 @ 3:1 2240RU 657.9KW 32,768 ISL 1600 switches 52 Brad Hedlund Global Marketing 53. 98,304 @ 3:1 oversubscribed Fabric (256)(512) (2048)98,304 @ 3:1 3584RU 1331.2KW 40960 ISL 2816 switches53 Brad HedlundGlobal Marketing 54. 98,304 @ 3:1 oversubscribed Fabric(128)(1024) (2048) 98,304 @ 3:1 4480RU 1315.8KW 65,536 ISL 3200 switches54 Brad HedlundGlobal Marketing 55. The power to do more55