hawaii tech day- aci, vxlan, n9k overview

ACI, DC Programmability, VXLAN, and Tetration

Chris Breece, CCIE 25075 DC & RS, VMware VCP

Federal Data Center Consulting Systems Engineer

2/28/2016

Cisco Confidential 2© 2013-2014 Cisco and/or its affiliates. All rights reserved.

Agenda

• Intro

• VXLAN and EVPN

• Programmability

• Application Centric Infrastructure

• Tetration

• Nexus 9000 Overview

• Nexus 9500

• Nexus 9300


Data Center SDN Highlights Providing Choice in Automation and Programmability

VxLAN-BGP EVPN standard-based

3rd party controller support

Modern NX-OS with enhanced NX-APIs

Automation Ecosystem (Puppet, Chef, Ansible, etc.)

Common NX-API across N2K-N9K

Turnkey integrated solution

Embedded security, centralized management, and

scale

Automated application centric-policy model

Broad and deep ecosystem

DB DB

Web Web App Web App


VXLAN & EVPNNext Gen Data Center Fabric


VXLAN Overview


VXLAN & EVPNMP-BGP Control Plane


NX-OSProgrammability


Data Center SDN Highlights Providing Choice in Automation and Programmability

VxLAN-BGP EVPN standard-based

3rd party controller support

Modern NX-OS with enhanced NX-APIs

Automation Ecosystem (Puppet, Chef, Ansible, etc.)

Common NX-API across N2K-N9K

Turnkey integrated solution

Embedded security, centralized management, and

scale

Automated application centric-policy model

Broad and deep ecosystem

DB DB

Web Web App Web App


SDN - Openflow & Open Daylight


SDN Forwarding


What is OpenFlow?


What is Open Daylight?


Application Centric Infrastructure (ACI)


Web Servers

vLAN 666

L3

FW

SLBSSL

DB Servers

vLAN 111

vLAN 222

WWW WWW WWW

vLAN 444

App Servers

FW

SLB

APP APP

FW

DB DB

switch1(config)#switch1(config)# int eth 1/1

switch1(config)# switch mode acc

switch1(config)# switch acc vlan 666

switch1(config)# no shut

router(config)#router(config)# int eth 1

router(config)# ip add 6.6.6.1 255.255.255.0

router(config)# not shut

router(config)# int eth 2

router(config)# ip addr 1.1.1.1 255.255.255.0

router(config)# no shut

router(config)# router eigrp 100

router(config)# network 6.6.6.0 mask 255.255.255.0

router(config)# network 1.1.1.0 mask 255.255.255.0

router(config)# ip route 0.0.0.0 0.0.0.0 6.6.6.254

switch2(config)#switch2(config)# int eth 1/2 - 3




fw1(config)#fw1(config)# int eth 0/1

fw1(config)# nameif outside 0

fw1(config)# int eth 0/2

fw1(config)# nameif webfront 20

fw1(config)# object network webfront_vip

fw1(config)# host 6.6.6.6

fw1(config)# static (webfront,outside) 1.1.1.6

fw1(config)# access-list outside_web permit tcp any host 6.6.6.6 eq 80

fw1(config)# access-list outside_web permit tcp any host 6.6.6.6 eq 443

fw1(config)# access-group outside_web in interface outside





vLAN 333

switch4(config)#switch4(config)# int eth 1/6




switch4(config)# int eth 1/7 - 9




IDS/IPS

vLAN 555

IDS/IPS

vLAN 777





switch5(config)# int eth 1/11 - 15




switch5(config)# monitor session 1 source vlan 555

switch5(config)# monitor session 1 dest eth 1/16





switch6(config)# monitor session 1 source vlan 777

switch6(config)# monitor session 1 dest eth 1/20

slb1 (CONFIG) probe http http-probe

interval 30

expect status 200 200

rserver host websrvr1

description foo web server

ip address 3.3.3.1

inservice



ip address 3.3.3.2

inservice



ip address 3.3.3.3

inservice

serverfarm host FOOWEBFARM

probe http-probe

rserver websrvr1 80

inservice

rserver websrvr2 80

inservice

rserver websrvr3 80

inservice

crypto generate key 1024 fooyou.key

crypto csr-params testparms

country US

state California

locality San Jose

organization-name foo

organization-unit you

common-name www.fooyou.com

serial-number crisco123

crypto generate csr testparms fooyou.key

crypto import ftp 12.13.14.15 anonymous fooyou.cer

parameter-map type ssl SSL_PARAMETERS

cipher RSA_WITH_RC4_128_MD5

version TLS1

ssl-proxy service FOOWEB_SSL

key fooyou.key

cert fooyou.cer

class-map match-all FOOSSL_VIP_CLASS

2 match virtual-address 2.2.2.22 tcp eq https

policy-map type loadbalance first-match L7-SSL-MATCH

class L7_WEB

sticky-serverfarm sn_cookie

policy-map multi-match FOOWEB-VIP

class FOOWEB_VIP_CLASS

loadbalance vip inservice

loadbalance policy FOOWEB-MATCH

loadbalance vip icmp-reply

loadbalance vip advertise active

class FOOSSL_VIP_CLASS


loadbalance policy FOOSSL-MATCH




fw2(config)# nameif webfront 20


fw2(config)# nameif appfront 50

fw2(config)# object network appfarm_vip


fw2(config)# nat (appfront,webfront) static 4.4.4.4

fw2(config)# access-list web_to_app permit tcp any host 4.4.4.4 eq 8081slb2 (CONFIG)rserver host appsrvr1

description foo app server

ip address 5.5.5.1

inservice

rserver host appsrvr2


ip address 5.5.5.2

inservice

rserver host appsrvr3


ip address 5.5.5.3

inservice

serverfarm host FOOAPPFARM

probe http-probe

rserver appsrvr1 8081

inservice


inservice


inservice

class-map type http loadbalance match-any FOO_APP

2 match http virtual-address 4.4.4.44 tcp eq 8081

class-map match-all FOO_APP_VIP_CLASS

policy-map type loadbalance first-match FOO_APP-MATCH

class FOO_APP

sticky-serverfarm sn_cookie

policy-map multi-match FOO_APP-VIP

class FOO_APP_VIP_CLASS


loadbalance policy FOO_APP-MATCH




fw3(config)# nameif appfront 70


fw3(config)# nameif dbfront 90

fw3(config)# object network db_cluster


fw3(config)# nat (dbfront,appfront) static 5.5.5.50

fw3(config)# access-list web_to_app permit tcp any host 5.5.5.50 eq 1433


Application Centric Infrastructure

NEXUS 9500, 9300, 2000 and AVS

Embedded Stateless L4 Firewall (zero trust)

Tenant Isolation

Group-based Security Policy* (3rd party included)

Whitelist Policy Enforcement

Fabric High-Availability

ACI

Application Profiles

GROUP BASED POLICES

Declarative Policy Model

Application Centric Desired State

POLICY CONTROLLER

Centralized

Management

Role-Based Access

Audit Logs

Health Monitoring

Open REST APIs


Outside

(Tenant VRF)

Web App DBQoS Policy QoS Policy

FW Service

Policy

QoS Policy

Access PolicyLB Service

Policy

APIC

Decouple Application

from Infrastructure Decouple Application

from Infrastructure

Application Centric InfrastructureService Graph Abstraction from the Network


Service Mgmt Console

Service Automation Through Device PackageDevice PackageDevice Specification

<dev type= “f5”>

<service type= “slb”>

<param name= “vip”>

<dev ident=“210.1.1.1”

<validator=“ip”

<hidden=“no”>

<locked=“yes”>

Cisco APIC – Policy Element Device Model

Device-Specific Python Scripts

Cisco APIC Script Interface

Script Engine

APIC Node

• Service automation requires a vendor

device package. It is a zip file

containing• Device specification (XML file)

• Device scripts (Python)

• Cisco® APIC interfaces with the

device using device Python scripts

• Cisco APIC uses the device

configuration model provided in the

package to pass appropriate

configurations to the device scripts

• Device script handlers interface with

the device using its REST or CLI

interface

Device Interface: REST/CLI

Service Device

Service automation

requires a vendor device

package. It is a zip file

containing

Device specification

(XML file)

Device scripts (Python)


TENANT AND APPLICATION AWARE

READ / WRITEALL FABRIC INFO

PUBLISHED DATA MODEL OPEN SOURCE

Open ecosystem, open APIs, Open SourceComprehensive access to underlying information model

ASA

and CertifiedIndustry Standard Compliant


Application Centric Infrastructure Tenant Model


Defining EPG Relationships Via Contracts

53

EPG Web

EP 1

EP 2

EPG App

EP 1

EP 2

Contract

Subject 1 Filter and Action

EPG communication is defined by mapping EPGs to one another via contracts.

Subject 2


EPG (End-Point) Classification

Server

Virtual Machines & Containers

Storage

Client

• Endpoint == Workload unit connected to network directly or

indirectly

• An endpoint has address (identity), location, attributes (version,

patch level)

• Can be physical or virtual or container

• End Point Group (EPG) membership defined by:

• Ingress physical port (Leaf or FEX)

• Ingress logical port (VM port group)

• VLAN ID

• VXLAN (VNID)

• IP Prefix/Subnet (so far only applicable to external/border leaf

connectivity)

• VM-based attributes

• IP address and subnet

• MAC address

What is Micro-Segmentation?

Broad Security

Zones

Small Micro-

segment Zones

Security Zone

Security

ZoneSecurity

Zone

FW

Separation

Traditional Approach Micro-segmentation Approach


Micro-SegmentationFlexible, Granular, Consistent – Virtual and Physical

Attributes Based Intra-EPG BasedEPG Based

ACI Benefits

Application

Segmentation

WEB

APP

DB

FW

OS

‘Linux’

IP

‘10.1.1.1’

FW

Name

‘Video’

All Workloads Can Communicate

Application Tier Policy Group

Isolate Workloads within Application

Tier

Application Tier Policy Group

VMware VDS Microsoft Hyper-V KVM* Cisco AVS

Policy Driven Micro-Segmentation for Any Workload

Physical*Future


ACI (Application Centric Infrastructure)

ACI SPINE Nodes

ACI FabricVXLAN Overlay

ACI LEAF Nodes

ACI Fabric provides:

• Decoupling of endpoint identity, location, and associated policy, all of which are independent from the underlying topology

• Full normalization of the ingress encapsulation mechanism used: 802.1Q VLAN, IETF VXLAN, IETF NVGRE

• Distributed Layer 3 gateway to ensure optimal forwarding for Layers 3 and 2

• Support for standard bridging and routing semantics without standard location constraints (any IP address anywhere)

• Service insertion and redirection

• Removal of flooding requirements for IP control plane (ARP, GARP)

• ACI Fabric is based on an IP fabric supporting routing to the edge with an integrated overlay for host routing

All end-host (tenant) traffic within the fabric is carried through the overlay

• The fabric is capable of supporting an arbitrary number of tiers and/or partial mesh if required

Why choose an integrated overlay?

• Mobility, scale, multi-tenancy, and integration with emerging

hypervisor designs

• Data traffic can now carry explicit meta data that allows for

distributed policy (flow-level control without requiring flow-

level programming)

Nexus 9500 or 9336

Nexus 9500 or 9300


10.1.1.10 10.1.3.1

1

10.6.3.2

Distributed Default Gateway

• ACI Fabric supports full layer 2 and layer 3 forwarding semantics, no changes required to applications or end point IP

stacks

• ACI Fabric provides optimal forwarding for layer 2 and layer 3

• Fabric provides a pervasive SVI which allows for a distributed default gateway

• Layer 2 and layer 3 traffic is directly forwarded to destination end point

• IP ARP/GARP packets are forwarded directly to target end point address contained within ARP/GARP header

(elimination of flooding)

10.1.3.35 10.1.1.1010.1.3.1

110.6.3.2

Directed ARP Forwarding

10.1.3.35

Location Independent ForwardingLayer 2 and Layer 3


Types of VXLAN Overlays

• Virtual end-points only

• Single admin domain

• VXLAN, NVGRE, STT

• Physical and Virtual

• Resiliency + Scale

• x-organizations/federation

• Open Standards

Network Overlays Integrated Overlays

AppOS

AppOS

Virtual Physical

Fabric DB

VMOS

VMOS

Virtual Virtual

VMOS

VMOS

Host Overlays

Physical Physical

• Router/switch end-points

• Protocols for resiliency/loops

• Traditional VPNs

• OTV, VXLAN, VPLS, LISP


ACI Fabric – Integrated OverlayDecoupled Identity, Location & Policy

PayloadEth

IPVXLAN

Outer

IP

PayloadIPNVGR

E

Outer

IP

PayloadIP802.1Q

PayloadEth

IP

PayloadEth

MAC

ACI VXLAN (VXLAN) header identifies the

attributes of the application end point within

the fabric

Policy attributes are carried by every packet

VTEP

• All Tenant traffic within the Fabric is

tagged with an ACI VXLAN (VXLAN)

header which identifies the policy

attributes of the application end point

within the fabric

• Policy Group (source group)

• Forwarding Group (Tenant, VRF,

Bridge Domain)

• Load Balancing Policy

• Telemetry Policy

• At the ingress port the Fabric translates

an external identifier which can be used

to distinguish different application end

points via the internal VXLAN tagging

format

PayloadMAC

IPVNIDFlagsVTEP

SRC

Group


Additional Features


FCoEUse Case

Host - CNA

VFVF

VF

VNP

VNPFCOE

NPV

N5K

N7K

MDSFCF

FC

Storage

Supported

Hardware

N93180YC-EX

N93108TC-EX

Physical Port carrying both LAN &

SAN Traffic

VF

FCoE• Single Fabric for LAN and SAN Connectivity

• FCOE traffic from Host CNA Port to SAN

Switch through ACI Leaf

• FCOE NPV is enable on leaf switches


Cisco ACI Multi-PoD SolutionSupported Topologies

Multiple sites interconnected by a

generic L3 network

L3 Network

POD 1 POD n

Web/AppDB Web/AppAPIC Cluster

Intra-DC


APIC - Application Centric InfrastructureGraphical Interface to configure, manage and monitor

Not to mention open API to REST, JSON, XML, Puppet, Chef, Opflex…


UCS Director for Application Network Automation with ACI

Automates the configuration of Application Profiles within the ACI fabric

Tenants, Private Networks, Bridge Domains, App Profiles, EPGs, etc.

Automated through APIC REST API

Create

Tenant

Create

Private

Network

Create

Bridge

Domain

Create

Application

Profiles

And More

REST

API

UCS

Director

185+ Tasks for ACIOut-of-the-Box

1 2 3 4 N

Orchestration Workflow

Cisco App Center

• APIC App Center is a digital

distribution platform, developed and

maintained by Cisco for APIC apps.

• The App Center allows users to

browse and download applications

that are developed for APIC’s app

center infrastructure.

• APIC App Center is similar to Apple

App Store designed for iOS devices

but different in the sense that App

Center does not charge the users to

download the apps.

Stateless App

• Simple HTML/CSS/Java Script based Apps that is run as part of APIC UI

• App can be inserted as a separate tab in any part of APIC UI. For ex: under Tenant tab.

• App executes when tenant/admin user clicks on the UI tab that launches APPs JS.

• App stops functioning when tenant/admin navigates away from the UI tab

• L4-L7 vendor specific configuration apps that offer sleek UI

Stateful App

• App has a backend component that runs continuously on APIC

• App may store state in the backend for the app specific functions

Examples –

• Visualization APPs that can plot graphs for the historical data for a specific time interval

• Alerts apps that can send alerts based on certain events (that are not supported natively in APIC)

• Monitoring apps that can monitor APIC’s events, faults and stats and analyze it for anomalies.

• Network monitoring / configuration apps that used to require separate system can now run on the APIC

UI/RESTAPIC

Web

Service

APIC

DB

DOCKER

App


Cisco Tetration Analytics


Cisco Tetration AnalyticsFocus Areas

Action


Cisco Tetration AnalyticsFocus Areas

Cisco Tetration

Analytics™

Visibility and

Forensics

Application

InsightPolicy

Compliance

Application

Segmentation

(Automated Policy

Enforcement)

ActionTETRATION ANALYTICS 1.0

(Policy Recommendation)

TETRATION ANALYTICS 2.0(Application Segmentation)


Cisco Tetration Analytics Architecture Overview

Data Collection

Software Sensor and

Enforcement

Embedded

Network Sensors(Telemetry Only)

Third Party Sources(Configuration Data)

Analytics Engine

Cisco

Tetration

Analytics

Cluster

Open Access

Web GUI

REST API

Event Notification

Tetration Apps


Tetration Analytics Data Sources

• New! Enforcement Point (Software agents)

• Low CPU Overhead (SLA enforced)

• Low Network Overhead (SLA enforced)

• Highly Secure (Code Signed, Authenticated)

• Every Flow (No sampling), NO PAYLOAD

*Note: No per-packet Telemetry, Not an enforcement point

Software Sensors

Universal*(Basic Sensor for other OS)

Linux VM

Windows Server VM

Bare Metal(Linux and Windows Server)

Available Now

Nexus 9200-X

Nexus 9300-EX

Network SensorsNext Generation 9K switches

Third Party Sources

Asset Tagging

Load Balancers

IP Address Management

CMDB

…

3rd party Data Sources


Application Dependency and Cluster Grouping

Cisco Tetration

Analytics™

Platform

BM VM VM VM BMBare-metal, VM,

and switch telemetry

VM telemetry (AMI …)

Bare-metal and VM telemetry

Network-only sensors, host-only sensors, or both (preferred)

Bare metal and VM

On-premises and cloud workloads (AWS)

Unsupervised machine learning

Behavior analysis

Cisco Nexus® 9000 Series

BM VM VM BM

Brownfield

BM VM VM BM

BM VM

BMVM

VM BM

VMVM

VM BM

BMVM

BM


What Is Really Running on My Network?Cisco Tetration Analytics Application Insight—Dependency Map

Use Cisco

Tetration Analytics™

outcome to generate

white-list policies


Application clusters

conversation viewsConversation details

including process bindings

Application Conversation View


Cisco Tetration AnalyticsPolicy Compliance, Simulation and Enforcement


Application DiscoveryWhitelist Policy

Recommendation(Available in JSON, XML, and YAML)

Whitelist Policy Recommendation

Application Profiles

GROUP BASED POLICES

Application Policy

Desired State


Policy Creation Flow

Export Clusters and Policies in JSON/XML format

Import Policy using ACI Toolkit

Automatic creation of EPGs and Contracts

APIC

DataNetwork

Policy

Application Policy

TetrationAnalytics

UCS UCS

Push

Policy Enforced by ACI


Should I allow this flow?

In-Place Policy Decisions


• Validating policy impact assessment in real time

• Simulating policy changes over historic traffic

• View traffic “outliers” for quick intelligence

• Audit becomes a function of continuous machine learning

Cisco Tetration

Analytics™

PlatformVM BM

VMVM

BM VM

VMVM

BM

VM

VM

VM

VM

Real-Time and Historical Policy Simulation


Nexus 9000 Overview


Nexus 9500


Cisco Nexus 9500 Platform SwitchesDensity in DC Optimized Footprint

21 R

U

8-Slot

Cisco Nexus® 9500

4-Slot

16

57.6 Tbps

512

1024

576

2304/2048

8

28.8 Tbps

256

512

288

1152/1024

4

14.4 Tbps

128

256

144

576/512

Nexus 9516

16-Slot

Nexus 9508 Nexus 9504

Payload Slots

Capacity

100G Density

50G Density

40G Density

10/25G Density

13 R

U

7 R

U

Fabric Bandwidth:

3.2 Tbps/ slot

Cloud Deployment Options:

ACI Multi-Site, Telemetry,

Cloud Scale

Multi-Generation

Investment Protection:

No Mid-plane, Power Supply Headroom

for 100/400G and integrated optics/

encryption

2nd Generation of Insieme ASICs –

16nm Technology

CY16+


True Front-To-Back AirflowStreamlined Operations For Next-Generation Data Center Designs

No Mid Plane and Perforated Faceplate for

True Front-to-Back Cooling

• Save Cooling and Power

• Same chassis for multiple generations of

line cards and fabric modules


• Supervisor Module• Redundant half-width supervisor engine

• Performance- and scale-focused

• Range of management interfaces

• External clock input (PPS)

Supervisor Module

Processor Romley, 1.8 GHz, 4 core

System Memory 16 GB, upgradable to 48 GB

RS-232 Serial Ports One (RJ-45)

10/100/1000 Management Ports One (RJ-45)

USB 2.0 Interface Two

SSD Storage 64 GB

64 GB SSD

16 GB DRAM

(Upgradable to 48 GB)

Console port

Dual USB ports

10/100/1000 mgmt port PPS clock input

Latest quad-core Intel Sandy Bridge

Processor

Redundant paths to system controllers

Nexus 9500 Platform


• System Controller Module• Redundant half-width system controller

• Offloads supervisor from device management tasks

Increased system resiliency

Increased scale

• Performance- and scale-focused

Dual core ARM processor, 1.3 GHz

• Central point-of-chassis control

• Ethernet Out of Band Channel (EOBC) switch:

1 Gbps switch for intra-node control plane communication

(device management)

• Ethernet Protocol Channel (EPC) switch:

1 Gbps switch for intra-node data plane communication

(protocol packets)

• Power supplies through system management bus (SMB)

• Fan trays

Nexus 9500 Platform


Price Optimized ACI Spine

DATA CENTER ARCHITECTURAL OPTIONS

Nexus 9500 Series Module PortfolioDeployment Options: Spine, Aggregation, EoR

94xx Series 97xx Series

32p 40G QSFP28

8p 100G CFP2

32p 40G QSFP+

48p 10G SFP+ and 4p 40G

48p 10GT and 4p 40G

32p 100G QSFP28

36p 40G QSFP+

Line Rate Agg

96xx Series

36p 40G QSFP+

Enhanced Access/AggVxLAN

95xx Series

36p 40G QSFP+

48p 10G SFP+ and 4p 40G

48p 10GT and 4p 40G

(ACI & NX-OS)

NewNew


Nexus 9300


Nexus 9K Fixed Access SwitchesACI Leaf and NX-OS Switches

9372PX-E

(48x10G+6x40G)

93180YC-X

(48p 10/25G + 6p 100G/40G)

+ Analytics

9372TX-E

(48x10GT6x40G)

93108TC-X

(48p 10GT + 6p 100G/40G)

+ Analytics

93120TX

(96x10G+6x40G

)

2nd Gen

2nd Gen

9332 QSFP+

(32x40G)

Fixed ACI Spine

9336 (36x40GB)