for cloud-native applications not for · 2018-09-05 · docker engine. tool for composing...

#vmworld

HCI: The IdealOperational Environment

for Cloud-Native ApplicationsChristos Karamanolis, VMware, Inc.

Cormac Hogan, VMware, Inc.

HCI1338BU

#HCI1338BUVMworld 2018 Content: Not for publication or distribution

Disclaimer

2©2018 VMware, Inc.

This presentation may contain product features orfunctionality that are currently under development.

This overview of new technology represents no commitment from VMware to deliver these features in any generally available product.

Features are subject to change, and must not be included in contracts, purchase orders, or sales agreements of any kind.

Technical feasibility and market demand will affect final delivery.

Pricing and packaging for any new features/functionality/technology discussed or presented, have not been determined.

VMworld 2018 Content: Not for publication or distribution

Agenda


What is HCI and vSAN?

Next-gen stateful applications

Why & how customers run next-gen apps on vSphere/vSAN

Challenges when managing next-gen apps on vSphere/vSAN

Strategy: a control plane for next-gen apps storage



What is HCI?An overview of hyper converged infrastructure

• Building block approach

• Greater agility and scale

• Simplified management

• Data Services through software

Traditional 3-Tiered ArchitectureComplex and Separate Silos

Servers and Blades

External Storage

NetworkingHardware

Hyper-ConvergedInfrastructure

Unified Management

VirtualizationCompute | Storage | Network

Server + Storage Network

Built on Industry-Standard Servers and Switches

Virtualization

Confidential │ ©2018 VMware, Inc. 4



What is vSAN?vSphere HCI - Aggregating local ESXi storage into a single shared vSAN datastore

On each host, local disks form disk group(s)

Each disk group has cache and capacity

A host can have multiple disk groups

Disk groups contribute capacity to a single shared vSAN datastore

The vSAN datastore is accessible by all hosts in the cluster automatically!

Consumed via policiesDisk GroupDisk Group Disk GroupDisk Group Disk GroupDisk Group

VMware vSphere & VMware vSAN



Data Services Consumed via Policies

Define storage related settings for protection and performance

Apply per VM, or even per VMDK level to meet business goals!

Key to software defined storage (SDS) architecture and management at scale!

The Storage Policy Based Management framework used by vSAN

vSphere and vSAN

SPBM Framework

vSAN Storage Policies



vSAN Built-in Operational InsightsHealth/Performance/Capacity/Devices are all critical insights when managing infrastructure



Next-Gen ApplicationsHow are Applications Evolving

Next-gen Apps on vSAN

Why

Benefits

ChallengesVMworld 2018 Content: Not for publication or distribution


Past: Off-the-shelf software in a VM Now: Home grown, distributed software

Evolution of Enterprise Applications



What is a Container?A quick overview of containers

Light-weight, stand-alone, executable package of a piece of software

Abstracts execution environment, facilitates portability of software

Isolating different application components

Containers• Small footprint, fast start-up• App-level abstraction• Process-level isolation• Share OS (and some kernel resources)

vs.

Virtual Machines• Hardware-level abstraction• Tool to manage physical infrastructure• Larger footprint• Strong isolation (performance, security)



Docker Engine

Tool for composing applications out of containers

Initially for Linux, now Windows too

Docker Engine – runtime for containers

What is Docker?Popularized the use of containers by App developers

dockerd

containerd

runc

Docker client



What is Kubernetes (K8s)?Orchestration and clustering of containerized applications

Master

Node

Kubelet

Pod

Pod

Node

Kubelet

Node

Kubelet

kubectl

Namespace

APIserver etcd Ctl mgr

Scheduler

K8s: Container orchestrationdeploy, scale, manage apps

Master: control plane

Node: host where workloads deployed

Pod: unit of scheduling

Namespace: scope per tenant/app

Docker Docker Docker

Local storageor NFS



ESXi ESXi ESXi

Kubernetes with vSphere?Many advantages over bare metal

Operational simplicityUnified infrastructure managementAutomation: vRealize AutomationForecasting, analytics: vRealize OpsLog aggregation: vRealize Log Insight

Resource consolidationTraditional + new gen applications

Robust isolation, resource sharing

VM VM VM VM VM

kubectl

Docker

Kubelet

Node

Pod

Pod

Enterprise Storagee.g., vSAN

Docker

Kubelet

Docker

Kubelet

Master


Scheduler


©2018 VMware, Inc. 14

Not a walk in the park



Challenge #1Statefulness

How to provide persistent storage for cloud-native apps


© 2016 VMware Inc. All rights reserved.

“Docker is for stateless applications!” — Uninformed



Stateful Cloud-Native Applications

7 out of the top 10 cloud native applications (as of June 2018) require persistent storage

Developers may use services on clouds, e.g. DBaaS, but this locks them into a cloud vendor

In many cases, developers want to deploy stateful applications on-premises, or when independence from cloud vendors matters

What they are and why they matter



Kubernetes on vSphere with vSANBut how does the stateful application consume vSphere storage?

Applications can leverage HCI benefits outlined previously –capacity, performance, device and health monitoring.

Provide protection and data services, consumed via policies, to ensure cloud native applications have the right level of availability.

ESXi ESXi ESXi

Kubernetes

Persistent Volumes

vSAN

Kubernetes deployed as a set of VMs

Applications deployed as a set of containers on Kubernetes

Persistent storage provided to applications via vSphere Cloud Provider/Project Hatchway



Persistent Storage for Cloud Native AppsvSphere Cloud Provider for Kubernetes: Project Hatchwaykind: StorageClassapiVersion: storage.k8s.io/v1metadata:

name: demo-storageclassprovisioner: kubernetes.io/vsphere-volumeparameters:

diskformat: thinstoragePolicyName: RAID-5datastore: vsanDatastore

kind: PersistentVolumeClaimapiVersion: v1metadata:

name: demo-pvc-claimannotations:volume.beta.kubernetes.io/storage-class: demo-storageclass

spec:accessModes:

- ReadWriteOncerequests:

storage: 2Gi

ESXi ESXi ESXi

Kubernetes

Persistent Volumes

vSAN

Cloud native applications deployed on Kubernetes

communicate to the vSphere layer for

provisioning of persistent storage



Challenge #2Fault Isolation/Placement Decisions

Ensure that different parts of an application do not reside on the same physical infrastructure



Running Hadoop on vSANThe basic components of Hadoop



ESXi

K8s Node 1

K8s Node 5

K8s Node 9

K8s Node 13

DRS VM/Host Groups and Rules will pin compute to a single host – enables us to separate K8s nodes onto different hosts

Through K8s label selectors, which are placed in the application’s YAML file, you can constrain a pod to only run on particular nodes

POD A

POD B

POD C

POD D



vSAN policy using FTT=0 and Data Locality rules will pin data to a single host – ensures a single failure does not impact 2 copies of the application’s data

ESXi 1

K8s Node 1

vSAN Datastore

Data Node 1

vSAN Datastore

ESXi 2

K8s Node 2

Data Node 2

ESXi 3

K8s Node 3

Data Node 3

Hadoop: Self-protecting components that do not need vSAN protection



Challenge #3Strengthen Applications

Combine data services and protection features from applications AND from HCI in production environments



vSAN FTT=1 and FTM=RAID-1/mirroring places data across different hosts – ensures a failure does not impact 2 copies of the application’s data

ESXi

K8s Node

ESXi

vSAN Datastore

NameNode

Hadoop: vSAN Protecting Components with No Built-in Protection



vSAN offers the following:• Erasure Coding• Checksum• Encryption• Deduplication/Compression• Data Protection• QoS

Choosing application vs. platform features• As seen with Hadoop, applications can also provide data services• Optimize resource utilization while meeting SLOs, e.g., replicate data once• May choose to use platform feature for operational consistency, e.g., data protection

Your platform should allow to “mix and match” features

What Other Data Services can vSAN Offer?With persistent storage, we may also want data services



vSAN already used as a Cloud-Native Platform today

Several examples of stateful containerized applications on vSAN, in production. Examples: Hadoop, Cassandra, MongoDB, etc. Often with Kubernetes as orchestrator

Currently, there is a number of operational considerations to take into account, as discussed.

• Data Persistence• Placement for performance and availability• Affinity/Anti-Affinity• Host/Data Locality

Let’s see our future plans on how to make vSAN (and vSphere) the Cloud Native Platform of choice for all our customers

vSAN as a Cloud Native PlatformKey takeaways from where we are today



StrategyFuture looking enhancements VMware is considering in this space



vmware.github.io/hatchway/



End-to-end provisioning workflows• APIs and UI extension to vSphere

Policy-Based Management for any storage• Mapping K8s storage classes to vSphere

policies

HCI operations for vSAN• Policy enforcement• Monitor, report, remediation• Health checks and analytics• Sizing and capacity planning

Storage Control Plane for PKSEnterprise storage management for Kubernetes


ESXi ESXi ESXi

PKS – Kubernetes

Persistent Volumes

vSAN BetaFall 2018



Operations for Container Volumes

Policy Compliance Status

Rich Filtering CapabilitiesUnified File & Block

volume listing

Console view of vSAN, VMFS, NFS

datastores

Single Console view for

Orchestrators



Kubernetes as a Workload Control PlaneAnd the challenges of deploying next-gen applications today

Kubernetes Workload

Pods Volumes Services …

Kubernetes Cluster

Kubelets Master …ETCD

Developer

Cluster Admin

VI Admin

This is cool!

This is hard!

Wish I could help!Virtual Infrastructure

VMs Disks …Hosts



Simplified Lifecycle for Kubernetes WorkloadsWorkload-centric management with PKS + vSphere

DeveloperDevops

Provision Workload Resources

Web App

Ticket

Automation

CLI / API

Namespace

Consume Workload Resources

Pod

Service

Volume

Any k8s object… VMworld 2018 Content: Not for publication or distribution


Orchestrate K8s Clusters on vSphere vSphere: the best PKS platform

Master

ESXi

Kubelet’

PodVM

Pod VM

ESXi

Kubelet’

ESXi

Kubelet’

Namespace


Scheduler

Use actual Kubernetes

Map K8s concepts to vSphereNamespace = Workload (new)Pod = Pod VMNode = ESXi with special kubelet

TBD: VCSA with K8s Ctl interface

kubectl

VM VM VM VM VM




Single HCI platform, same operational model• vSAN + vSphere control plane • PKS integrated• Policy-based storage management

Scale-out, highly available data path

Multiple protocols • Block (traditional vSAN)• NFS• SMB• TBD: Native file driver• TBD: Object

Application developer’s choice of protocol

vSAN: Multiple Storage Consumption ModelsThe power of choice

PKS – Kubernetes

ESXi ESXi ESXi

vSAN

Block File (native) Object



Automated deployment of share-nothing Apps• App = vSphere workload in PKS

Support popular stateful Apps• Feedback welcome!

Codify best practices:• App vs. vSAN data redundancy• Compute node – data affinity• Compute node isolation• Enterprise data services• Protect Mgmt components• Generic Backup / DR

Stateful Next-gen App OrchestrationPKS + Enterprise Storage + Codify good practices

PKS – Kubernetes

ESXi ESXi ESXi

vSAN

Application

Orchestration

DRS+

StoragePoliciesVMworld 2018 Content: Not for publication or distribution


Tools for offering Data-as-a-Service• VMware admin: operates platform, service instances• Developers: consume data services

Embrace trusted open source Apps• Work closely with community

Enterprise features and operational maturity• Enterprise-grade storage and compute features• Unified infrastructure management (HCI)

Integration with NSX AllSpark (Istio / Envoy)

vSphere evolving to become an Apps platform

Vision: A Platform for Data-as-a-ServiceOpen source IP, offer enterprise features and operations

PKS – Kubernetes

ESXi ESXi ESXi

vSAN

Workloads


PLEASE FILL OUTYOUR SURVEY.Take a survey and enter a drawingfor a VMware company store gift card.

#vmworld #HCI1338BUVMworld 2018 Content: Not for publication or distribution

THANK YOU!

#vmworld #HCI1338BUVMworld 2018 Content: Not for publication or distribution

for cloud-native applications not for · 2018-09-05 · docker engine. tool for composing...

Documents