the software-defined datacenter, vms, and containers: a ......containers & vmware nsx •...
TRANSCRIPT
The Software-defined Datacenter, VMs, and Containers:A “Better Together” Story
SDDC3327
Kit Colbert, VMware, Inc
Disclaimer• This presentation may contain product features that are currently under development.
• This overview of new technology represents no commitment from VMware to deliver these features in any generally available product.
• Features are subject to change, and must not be included in contracts, purchase orders, or sales agreements of any kind.
• Technical feasibility and market demand will affect final delivery.
• Pricing and packaging for any new technologies or features discussed or presented have not been determined.
CONFIDENTIAL 2
Agenda
CONFIDENTIAL 3
1 Context
2 Unified Infrastructure Fabric
3 Unified Cloud Management
4 3rd Platform Application Stack
5 Summary
Section 1: Context
HardwareHardware
OS Kernel
OS File system
Use
rspa
ce
ContainerContainer
App
pro
cess
App
pro
cess
App
pro
cess
App
pro
cess
App
pro
cess
App
pro
cess
App
pro
cess
App
pro
cess
App
pro
cess
App
pro
cess
ContainerContainer
App
pro
cess
App
pro
cess
App
pro
cess
App
pro
cess
Linux Containers
55
OS-level Isolation• Isolation at individual kernel subsystem
level (e.g. filesystem, process table, etc)
• User-level process (LXC, libcontainer) orchestrates these subsystems to create a container
Existed for Many Years• Solaris Zones, FreeBSD Jails, OpenVZ
Why?• Process isolation
• Reproducible environment
• Enables management at scale
The Problem in 2014
Static website
Web frontend
User DB Queue Analytics DB
Background workers API endpoint
nginx 1.5 + modsecurity + openssl + bootstrap 2
postgresql + pgv8 + v8 hadoop + hive + thrift + OpenJDK
Ruby + Rails + sass + Unicorn
Redis + redis‐sentinel
Python 3.0 + celery + pyredis + libcurl + ffmpeg + libopencv + nodejs + phantomjs
Python 2.7 + Flask + pyredis + celery + psycopg+ postgresql‐client
Development VM
QA ServerPublic Cloud
Disaster Recovery
Contributor’s LaptopProduction Servers
Multiplicityof Stacks
Multiplicityof hardware
environments
Production VM Cluster
Customer Data Center
Do services and apps interact
appropriately?
Can Imigrate
smoothlyand quickly?
66
Let’s create a shipping container system for applications
Multiplicityof Stacks
Multiplicityof hardware
environments
Do services and apps interact
appropriately?
Can Imigrate
smoothlyand quickly?
Static website Web frontend User DB Queue Analytics DB
Development VM QA Server Public Cloud Contributor’s Laptop
Production VM Cluster
Customer Data Center
An engine that enables any payload to be encapsulated as a lightweight, portable, self-sufficient container…
…that can be manipulated using standard operations and run consistently on virtually any hardware platform
77
Container Fits Well with DevOps Lifecycle
8
Development
Package & RepositoryPackage & Repository
Test Automation
Test Automation
Integrated Dev. Env.
Integrated Dev. Env.
Continuous Integration
UAT
Continuous Delivery Platform
ProductionSys. Int. Test
Code Dev & Check-in
Code Dev & Check-in
Build, Integration and Testing
Build, Integration and Testing
Repository Mgmt
Repository Mgmt
Deployment & Testing
Deployment & Testing
Promotion & GovernancePromotion & Governance
ProductionDeploymentProductionDeployment
Build & Integration
Build & Integration
is a “Shipping Container” for Code
9
Ops ♥ Consistent operations on codeUniform start, stop, logging, monitoring
Devs ♥ Consistent environmentOS, libs, layering on other containers
9
On-premise
Client-server, stateful, scaleup
Tier 1/Converged HW
Classic NAS & SAN
Relies on infrastructure availability
Human-driven
The Rise of Third Platform Applications
10
On/Off premise
Elastic, stateless, scale-out
Commodity/disaggregated HW
DAS, HDFS, Object, Flash, NVM
Built-in application resiliency
API-Driven/DevOps infrastructure
One School of Thought: Containers or VMs?
11
VMsVMs ContainersContainers
Implication: Separate Stacks, Higher CapEx & OpEx
12
ManagementManagement ManagementManagement
InfrastructureInfrastructure InfrastructureInfrastructure
VMsVMs ContainersContainers
Instead, Containers AND VMs!
13
Unified Cloud ManagementUnified Cloud Management
Unified Infrastructure Fabric Unified Infrastructure Fabric
VMsVMs ContainersContainers
Unified Infrastructure Fabric(ex. ESX, NSX, SDS)
Unified Cloud Management(ex. vCAC, vCOps, Log Insight)
ContainersVMs
Containers Without Compromise
14
Open Containers API
Single Platform for VMs and Containers
Consistent developer & deployment experience
Common management, monitoring, compliance across all applications
• ‘Better-than-physical’ compute layer• Network & security controls for containers • SDS: data persistence, backup, SLA
management
Enable 2-tier scheduler model; integration with Kubernetes, Pivotal CF, and other schedulers
Section 2: Unified Infrastructure Fabric
15
Unified Cloud Management
Unified Infrastructure Fabric
VMsVMs ContainersContainers
VM and Container Isolation are Better Together
16
VMs ContainersContainers
• Hardware level isolation• Focused on security and
multi-tenancy• 15 years in production,
battle tested
• OS level isolation• Focused on environmental
consistency• Emerging, still maturing
Great for security Great for reproducibility
VMsVMsrsrs
Best of both worlds
VMs are Lightweight and Efficient
17
Forking
FastSub-second VMprovisioning time
Ready to GoClone a running container in warmed up state
EfficientLower resource usage through sharing
Binaries& Libraries
App A
OS
VM Debunk the Myth• VM overhead < 5%• VM is lightweight• OS tends to be heavierLooking ahead• Thinner OS emerging• Project Fargo
Containers & VMware NSX
• Unified operational model for VMs & containers
• Programmable, datacenter-wide connectivity
• Enterprise-grade security with micro-segmentation.
• Native Open vSwitch support for containers
18
Any Application(without modification)
Virtual Networks
VMware NSX Network Virtualization PlatformVMware NSX Network Virtualization Platform
Logical L2
Any Network Hardware
Any Cloud Management Platform
LogicalFirewall
LogicalLoad Balancer
Logical L3
LogicalVPN
Any Hypervisor
Distributed and Reliable Storage for Container
HOSTHOST
StatelessContainerStatelessContainer
HOSTHOST HOSTHOST HOSTHOST
StatelessContainerStatelessContainer
HOSTHOST
VSAN Distribute, Reliable StorageSnapshots, Clones, QoS, Remote Replication
VSAN Distribute, Reliable StorageSnapshots, Clones, QoS, Remote Replication
BootImage
BootImage
BootImage
Container PROVISIONING AND MANAGEMENT • Simple data persistence
• Easy deployment of containers on cluster
• Reliable, high performance storage
• Tolerant of host/disk failures
• Fast container create leveraging snapshots and clones in VSAN
• Quality of Service Controls
StatelessContainerStatelessContainer
BootImage
StatelessContainerStatelessContainer
BootImage
DBsTraditional Apps
Sharing Infrastructure Efficiently
20
Container cluster Container cluster
• Unified platform to run all your apps• Dynamically allocate resource based on demands and SLA• Strong security and performance isolation
Database clusterDatabase cluster Traditional AppsTraditional Apps
Silo’ed cluster leads to server/cluster sprawling, increases cost
Scenario 1: Multiple workloads Scenario 2: Multiple tenants
Containerized apps Tenant/LOB 1
Tenant 1Tenant 1 Tenant 2Tenant 2 Tenant 3Tenant 3
Data Center VirtualizationSDDC Platform
Hybrid Platform
21
vCloud AirData Centers
On-premisesData Centers
Data
vCloud Plug-in
Security
Apps, Tools, ServicesMulti-tenant
Secure Connectivity
Management
Dedicated
Software-Defined Data Center
Section 3: Unified Cloud Management
22
Unified Cloud Management
Unified Infrastructure Fabric
VMsVMs ContainersContainers
Container Fits Well with Devops Lifecycle
23
Development
Package & RepositoryPackage & Repository
Test Automation
Test Automation
Integrated Dev. Env.
Integrated Dev. Env.
Continuous Integration
UAT
Continuous Delivery Platform
ProductionSys. Int. Test
Code Dev & Check-in
Code Dev & Check-in
Build, Integration and Testing
Build, Integration and Testing
Repository Mgmt
Repository Mgmt
Deployment & Testing
Deployment & Testing
Promotion & GovernancePromotion & Governance
ProductionDeploymentProductionDeployment
Build & Integration
Build & Integration
Manage VMs and Containers at Scale is Key
24
On-premise
Client-server, stateful, scaleup
Tier 1/Converged HW
Classic NAS & SAN
Relies on infrastructure availability
Human-driven
On/Off premise
Elastic, stateless, scale-out
Commodity/disaggregated HW
DAS, HDFS, Object, Flash, NVM
Built-in application resiliency
API-Driven/DevOps infrastructure
Web tier
App tier
DB tier
LoadBalancer
LoadBalancer AuthenticationAuthenticationSession
StoreSession
Store LicensingLicensing
MonitoringMonitoringProvisioningProvisioning
DNSDNSContentContentDatabasex3
Databasex3
WebServer
x3
WebServer
x3
……
Separation of Infrastructure and Apps Concerns
25
Developers“Write code, not tickets”
Infrastructure Team“IT as a service provider”
FocusDeliver IT resources to rest of company
ChallengeAgility for devs, while maintaining control
RoleEnable rapid delivery of dev sandboxes, pre-provision
3rd Platform Services (Kubernetes, Pivotal CF, etc)
FocusFrictionless development, rapid innovation
ChallengeWrite code, without worrying about infrastructure details
RoleSelf-service access to new resources (i.e. new cluster),
comply with company policies and regulations
Separation of Infrastructure and Apps Concerns
Developers“Write code, not tickets”
Infrastructure Team“IT as a service provider”
FocusDeliver IT resources to rest of company
ChallengeAgility for devs, while maintaining control
RoleEnable rapid delivery of dev sandboxes, pre-provision
3rd Platform Services (Kubernetes, Pivotal CF, etc)
FocusFrictionless development, rapid innovation
ChallengeWrite code, without worrying about infrastructure details
RoleSelf-service access to new resources (i.e. new cluster),
comply with company policies and regulations
Architecturally, makes sense to separate infrastructure and app management
Infrastructure Management
Infrequent/no access by developers; devs
shouldn’t care
Application Management
Lightweight, fast; call infrastructure manager
when needed
26
Lifecycle: Self-service, Governance, Automation
27
Data Center VirtualizationSDDC Platform
Benefit: Common portal, catalog, permissions for developers and LOBSelf-Service
Benefit: Compliance consistently enforced across entire datacenterGovernance
Benefit: Same tools for automating traditional and new app lifecyclesAutomation
DBsTraditional Apps Containerized apps Tenant/LOB 1
Operations: Service Availability and Traceability
28
HardwareHardware
vSphere, NSX, vSAN/vVOL
OS
App
Virtual HW
OSVirtual HW
App
Lib
App
Lib
App
Lib
App
Lib
Performance monitoring
Performance monitoring
Capacity management
Capacity management
Log management
Log management…
Instrument all layers of stack
Inputs: Metricsand log data
Delivering better service levels,
availability, root cause analysis, …
Linux OS
App
Lib
App
Lib
App
Lib
Section 4: 3rd Platform Application Stack
29
Unified Cloud Management
Unified Infrastructure Fabric
VMsVMs ContainersContainers
The Rise of Third Platform Applications
30
On-premise
Client-server, stateful, scaleup
Tier 1/Converged HW
Classic NAS & SAN
Relies on infrastructure availability
Human-driven
On/Off premise
Elastic, stateless, scale-out
Commodity/disaggregated HW
DAS, HDFS, Object, Flash, NVM
Built-in application resiliency
API-Driven/DevOps infrastructure
ManagementvCloud Automation Center,
vCenter Operations, Log Insight
ManagementvCloud Automation Center,
vCenter Operations, Log Insight
ContainerRepositories
ContainerRepositories
3rd Platform Apps Stack & DevOps Process
31
Container-optimized LinuxContainer-optimized Linux
Container PackagingContainer Packaging
Container Cluster SchedulerContainer Cluster Scheduler
App Definition, Policies, and ProvisioningApp Definition, Policies, and Provisioning
InfrastructureESXi, NSX, Virtual SANvCloud Hybrid Service
InfrastructureESXi, NSX, Virtual SANvCloud Hybrid Service
Developer Production
Dev’s LaptopDev’s Laptop
Optional: Type 2 HypervisorOptional: Type 2 Hypervisor
LinuxLinux
Developer ToolsDeveloper Tools
Container PackagingContainer Packaging
Open Ecosystem: 3rd Platform Developer Stack
32
ManagementvCloud Automation Center,
vCenter Operations, Log Insight
ManagementvCloud Automation Center,
vCenter Operations, Log Insight
Container-optimized LinuxContainer-optimized Linux
Container PackagingContainer Packaging
Container Cluster SchedulerContainer Cluster Scheduler
App Definition, Policies, and ProvisioningApp Definition, Policies, and Provisioning
InfrastructureESXi, NSX, Virtual SANvCloud Hybrid Service
InfrastructureESXi, NSX, Virtual SANvCloud Hybrid Service
Developer Production
ContainerRepositories
ContainerRepositories
Dev’s LaptopDev’s Laptop
Optional: Type 2 HypervisorOptional: Type 2 Hypervisor
LinuxLinux
Developer ToolsDeveloper Tools
Container PackagingContainer Packaging
E.g. Hashicorp Vagrant, Jenkins, github, etc
E.g. Docker/Docker Hub
E.g. RedHat, Ubuntu, Boot2Docker
E.g. Fusion, Workstation, Player, VirtualBox
ContainerRepositories
ContainerRepositories
Developer Production
Dev’s LaptopDev’s Laptop
Optional: Type 2 HypervisorOptional: Type 2 Hypervisor
LinuxLinux
Developer ToolsDeveloper Tools
Container PackagingContainer Packaging
Open Ecosystem: 3rd Platform Production Stack
E.g. Pivotal CF, FigTerraform, Shipyard
E.g. Kubernetes, libswarm, Mesos, Fleet
E.g. CoreOS, Atomic, Ubuntu
ManagementvCloud Automation Center,
vCenter Operations, Log Insight
ManagementvCloud Automation Center,
vCenter Operations, Log Insight
Container-optimized LinuxContainer-optimized Linux
Container PackagingContainer Packaging
Container Cluster SchedulerContainer Cluster Scheduler
App Definition, Policies, and ProvisioningApp Definition, Policies, and Provisioning
InfrastructureESXi, NSX, Virtual SANvCloud Hybrid Service
InfrastructureESXi, NSX, Virtual SANvCloud Hybrid Service
E.g. Docker
33
Craig McluckieGoogle
Containers at Google
• Everything at Google runs in Linux application containers• A decade of production container experience• We start more than 2 billion a week
• Containers have changed the game • Separation of infra and applications ops • Increased efficiency
35
A few lessons learned...1: Declarative trumps imperative
Imperative: run this container on this serverDeclarative: run between 2 and 100 copies; keep latency < 2ms
Pros• Repeatable and eventually consistent deployment and update• Fire-and-forget app management (self scaling, self healing)• Dynamic scheduling yields better efficiency
Cons• Tracing action/reaction can be hard (“is it done?”)• Diagnostics can be tough (“what happened?”)
So• We need a cluster manager• Strong integration with container metrics, logging, etc helps
36
A few things we have learned...2: Prepare for more production services
The system known as Borg made it easier to run production services at scale...so our engineers wrote a lot more
Pros• Strong shift to dev and away from ops• Radically simpler infrastructure operations
But…• Governance gets harder as service number increases• Managing, finding, versioning
So…• We need a cluster manager• It needs mechanism to deal with large numbers of services
37
So we created Kubernetes...
• OSS project created by Google, but owned by the community
• Google style cluster management • Move from static containers to dynamic management
lightweight modular/ extensible portable
:38
And where do VMs fit in?
• Needed to run untrusted and unconstrained workloads • Linux syscall layer is large and difficult to defend• VMs surface can be aggressively defended• VMware has been doing this for 15 years• Critical for multi-tenant cloud use with untrusted tenants• E.g. VMware vCloud Air
• In Google Cloud Platform• VMs create ‘idealized’ infrastructure• Containers package and run applications
• Kubernetes stitched together VMs to create a mini-Google cluster
39
What is next?
• Make it work everywhere• Operationalize• Extend services for distributed systems development
40
James WattersPivotal
42© Copyright 2014 Pivotal. All rights reserved.
Core Application Patterns Are Changing
43© Copyright 2014 Pivotal. All rights reserved.
To Do List Application and Data Services Centric Platform
– Transform human centric data center processes into a software factory
Move towards real time deployment scaling and operations
Focus on ease of deployment, but deliver exceptional operational benefits
44© Copyright 2014 Pivotal. All rights reserved.
Why Containers are Essential Speed: seconds vs. minutes
– Seconds to deployment – Seconds to scaling – Seconds to network configuration – Seconds to health management
Units of currency– Leverage Docker popularity and simplicity for apps and data
services– Push and application artifact (.WAR) or a Docker image
45© Copyright 2014 Pivotal. All rights reserved.
Value of VMware Integration Being deeply integrated into vSphere APIs
allows automated platform set up and scaling
Mixed VM/container model ideal blend of speed and isolation
Enterprise customers leverage existing infrastructure operations process
46© Copyright 2014 Pivotal. All rights reserved.
Containers Alone Aren’t Enough
Enterprises do not want app development groups each writing their own platforms
Value is unlocked when standard common services are built into each managed container
47© Copyright 2014 Pivotal. All rights reserved.
From Data Center to Software FactoryContainers surrounded with services
Application Containerization & Cluster Scheduling
Native and Extended Data
Services
Automatic App Server & OS Configuration with
Buildpacks
Policy, Identity and Roles Management
App Health Management, Load Balancing, Rapid Scaling, Availability Zones
IaaS Provisioning, Scaling & Configuration
Application Network Security
Groups
Application to Services Binding and
Access
Logging as a service,Application metrics & performance,Metric based scaling
48© Copyright 2014 Pivotal. All rights reserved.
Already Strong in the Enterprise
49© Copyright 2014 Pivotal. All rights reserved.
Demo VideoDiego Sneak Peak
Summary
Data Center VirtualizationSDDC Platform
Case Study: ITBM Leveraging Containers on SDDC & vCloud Air
52
fleet & etcdfleet & etcd
ClusterCluster
MesosMesos
HDFS
Registry
ClusterCluster
DNS
Registry
IT Benchmarking Service (ITBM)SaaS application to measure IT process against peers or common recognized patterns• Build and content generation on private cloud (SDDC), Customer-facing modules on vCloud Air• All services running in Dockers on CoreOS VMs
vCloud Air
Data Center VirtualizationSDDC Platform
Case Study: ITBM Leveraging Containers on SDDC & vCloud Air
52
fleet & etcdfleet & etcd
ClusterCluster
MesosMesos
HDFS
Registry
ClusterCluster
DNS
Registry
IT Benchmarking Service (ITBM)SaaS application to measure IT process against peers or common recognized patterns• Build and content generation on private cloud (SDDC), Customer-facing modules on vCloud Air• All services running in Dockers on CoreOS VMs
vCloud Air
Software-Defined Data Center
• Single platform for running and managing traditional + modern apps
• Enterprise grade: security, performance, operational efficiency
• Ability to extend applications to the hybrid cloud
• Support for community-led projects (Big Data, OpenStack, containers)
53
The Open Platform for Modern Applications
VMware Bridges These Two Worlds
ResilienceSecurity
QoS
ResilienceSecurity
QoS
OpennessPortability
Agility
OpennessPortability
Agility
Traditional Apps Modern Apps
OSOSApp
Virt. HWVirt. HWOSOSApp
Virt. HWVirt. HWOSOSApp
Virt. HWVirt. HWOSOSApp
Virt. HWVirt. HW
AppApp
Container OSContainer OS
AppApp
OpenStack API Open Container API
Software-Defined Data Center
ESXi, NSX, VSAN
On-premise Off-premise
IT Faces Conflicting Demands
In Summary• VMware is focused on helping companies run and manage their applications,
whether they are packaged in VMs or containers
• A software-defined datacenter is the best place to run and manage all application types
• Docker, Google, Pivotal, VMware are working together to help companies efficiently run and operationalize containerized applications
55
Q & A
Unified Infrastructure Fabric(ex. ESXi, NSX, Software-Defined Storage)
Unified Infrastructure Fabric(ex. ESXi, NSX, Software-Defined Storage)
Unified Cloud Management(ex. vCloud Automation Center, vCenter Operations, Log Insight)
Unified Cloud Management(ex. vCloud Automation Center, vCenter Operations, Log Insight)
Extends to management of
containers running on physical hosts
2nd Platform App StackInfrastructure Control
• vSphere (VCenter/HA/DRS/…)• SRM (DR)• 3rd-party integrations w.
vSphere
3rd Platform App StackInfrastructure Control
• Kubernetes, Yarn, Mesos, …• Pivotal, BOSH
3rd Platform App StackInfrastructure Control
• Kubernetes, Yarn, Mesos, …• Pivotal, BOSH
2nd vs 3rd Platform Apps: The Value Prop Changes
57
Open Containers APIOpen Containers API2nd Platform App Stack Infrastructure
Control Plane• Scheduling &
placement (DRS)• Resource controls
(SIOC, NIOC)• High availability (HA,
FT) • Mobility (vMotion)• Disaster recovery
(SRM)• Authentication• Logging/Audit• Etc…
3rd Platform App Stack
Infrastructure Control• Scheduling• Resource Controls• Load balancing,
routing• Service registration• Service discovery• Availability• Authentication• Logging/auditing• Data persistence
Thank You
Fill out a surveyEvery completed survey is entered
into a drawing for a $25 VMware company store gift certificate
The Software-defined Datacenter, VMs, and Containers: A “Better Together” Story
SDDC3327
Kit Colbert, VMware, Inc