walk through a software defined everything poc

Walk Through a Software DefinedEverything PoC

Chris JaniszewskiOpenStack Architect – Red [email protected]

Objectives

• To enable data center services through an abstraction of compute,network, and storage functionality into a pool of resources simply consumedas a service.

• Enable users to simply and expeditiously provision or decommission anapplication

• Automatically balance applications across data center resources to optimizeefficiency

• Elastically scale out data center resources to meet application demands • Provide complete isolation to prevent unintended resource sharing/intrusion• Be available all the time, tolerant to failure, and maintain data consistent • Elegantly metered, managed, monitored, and configured • Leverage universal on and off premises computing standards

Proof of concept

• SDN distributed architecture that is builtto scale with enhanced security.

• Leading SDS OpenStack solutions

• Integration of Software DefinedTechnologies with EnterpriseOpenStack

• Hardware Management and Monitoring

Concept

Midonet Topology

Infrastructure

Hardware

4x controller nodes:

– CPU: 2 socket Intel Haswell (12 core)

– Memory: 64 GB

– Storage: 4x 4 TB HDD (RAID-10 Virtualization)

8x Ceph OSD nodes:


– Memory: 64 GB

– Storage: 8x 6 TB HDD (OSD) ; 2x 200 GB SSD (Journal)

16x compute nodes:


– Memory: 384 GB

All nodes:

2x 32 GB Class 10 SD Cards (Operating System)

2x 10 Gb and 1Gb NIC

Midonet Configuration

• BGP from gateways to TOR Switches – dynamic routing• BGP timers were shortened:

# bgpd

bgp_connect_retry=10

bgp_holdtime=15

bgp_keepalive=5

• Large Receive Offload (LRO) turned off (packets cannot be larger than the MTUof the destination VM)

• Created share state, port groups for gateway uplinks (tracking of asymmetrictraffic flows for gateways)

• Increased the number of client connections for ZooKeeper instances•

Ceph Configuration

• Replication 3x• 64 OSDs spread equally between 8 nodes• SSD to HDD ratio 1:4• 4096 Placement Groups based on formula:

Total PGs = (OSDs * 100) / # of replicas (increase to next power of 2)• Aggregated 10Gb NICs with VLAN isolation for Public and Cluster

network

OpenStack Services

Extreme Cloud Administration Toolkit

• OpenSource• Lights-out management• Automation and Deploy – bare metal and service VMs• Hardware monitoring and management• Firmware update• Parallel shell commands• RPM Repo, • DNS, HTTP, DHCP, TFTP, NFS, • NAT• SOL Console

Midonet vs OVS – Lessons Learned

Midonet:• Better resiliency – Active/Active configuration• Scalability – distributed architecture• Performance• Security – Firewall bound at each vPort• Advanced features

OVS:• Robust Link Aggregation (balance-tcp with failover)• Operational shortcuts (configuration backup, automated deployment)

OpenStack - Lessons Learned

• “Split Brain” for odd number of database nodes• Restarting RabbitMQ is not a simple task• Power Outages happens • VXLAN offload only works on single UDP port• OpenStack manual deployment is still a complex process

Ceph – Lessons Learned

• Throughput - 5x Read vs Write• IOPS read (seq) BS 4K - more then 80K (no SSD)• SSD for journaling – double performance• No SSD for journaling – partition outer edge of the disk• 20Gbps network bottleneck• OVS balance-tcp > Linux bond• Isolate Cluster and Public networks • SSD to HDD ratio and SSD size

Streamlined Workflow

Business Benefits

• Rapid deployments and scale up of new applications• Reduced management cost and reduced complexity• Management tools that can support the management of thousands of

physical servers• Ability to scale to thousands of VMs per cloud administrator• Reduced cost per VM• Advanced and agile networking that uses Network Virtualization

Overlays• Tenant isolation over shared infrastructure• Simplified underlying Network infrastructure that uses open standards

L3 routing protocols• Improved IT productivity with reduced time to deploy resources

THANK YOU

• Whitepaper - https://lenovopress.com/redp5233

Learn More

walk through a software defined everything poc

Technology