cisco live 2017 cap

Introduction to ACI for Network Admins

Steve SharmanBRKACI-1002

© 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public

AbstractACI for the Network Administrator takes the attendee through building an ACI network through the eyes of the network administrator.

The session will focus on logical and concrete models, how to use bridge domains and VLANs, how to configure external connectivity from the fabric, and how to integrate third party devices.

BRKACI-1002 3


Session Objectives• Understand ACI through the eyes of the network administrator• Understand ACI building blocks• Understand external and services integration• Consuming ACI with Automation• Getting started with ACI

BRKACI-1002 4

Before We Start, Let’s Get to Know Each Other …

• How do we sell ACI?• Understanding ACI Building Blocks• VMware Integration• External Connectivity• Service Graph Integration• Consuming ACI with Automation• Getting Started with ACI

Agenda

How Do We Sell ACI?


Let me talk to you about Cisco ACI…

ACI is all about applications and I

don’t know applications…

Are all applications based on three

tiers…?

BRKACI-1002

In Reality ACI is all About Networking and How You Deploy Applications Onto the Network!

At a Very Basic Level ACI is Really Just a Clos Network of Nexus 9k Switches with a

Management Platform

Charles Clos – 1952 https://en.wikipedia.org/wiki/Clos_network

The Network Management Platform (APIC) Provides You With a Single Place From Which to Manage the

Network

© 2017 Cisco and/or its affiliates. All rights reserved. Cisco PublicBRKACI-1002

Is ACI an Overlay or Underlay Network?

12

ACI is a Software Defined Network Which Uses VXLAN to Transport Packets Between

Switches Across an Automated IP Fabric with End to End Header Visibility


IETF Draft

BRKACI-1002 14

ACI Can Transport Any IP (and non IP) Traffic Including “Overlay” Networks

Based on VXLAN*, NVGRE* etc.

* ACI has visibility of the outer header

To Help Understand ACI, Let’s Look at a Real Customer Example


OSPF Area 30 OSPF Area 20

OSPF Area 10 (stub)

BRKACI-1002

CPoC – Large Financial Organisation

APIC

APIC

APIC

Spirent TestCentre

Spirent TestCentre

Spirent TestCentre

ESX-01ESX-02

c3850

n7706 n9504n7706-01 n7706-02

n5672-01 n5672-02

L2

L3

OSPF Area 0

e1/3

e1/1 e1/2 e1/1 e1/2

e1/7 e1/8

e1/15 e1/15 e1/15e1/5 e1/6 e1/11 e1/12

17


“ACI Has to be Operationally Simple. Our Ops Team are

Used to Using the CLI, if They’re Not Comfortable with

Troubleshooting ACI it Won’t be Accepted!”

18

Step 1 – Building the Network and Provisioning Interfaces


Physically Building the ACI Network

BRKACI-1002

APIC

APIC

APIC

Management options:• GUI (basic/advanced)• CLI• XML/JSON• Scripting• Open API• Automation

Benefits:• Distributed, Centralised Management• Full traffic visibility*• Self documenting• Integrated virtual and physical network• Integrated L4-7 device management• Policy defined network

20


Network Provisioning

Manual setup

Quick Start wizard

21


Switch PoliciesLeaf Profiles

Leafs_101_and_102

Which switches should be

configured?

Interface PoliciesPolicies

CDP_enabledLACP_Active

Interface PoliciesLeaf Policy GroupsvPC_to_UCS_FI_A

SVI_to_outside

AAEP(Allowed VLANs)

vCenter-01-DVS-01UCS-phys-svrsOutside-Fabric

PoolsVLAN/VXLAN

vCenter-01-DVS-01UCS-phys-svrsOutside-Fabric

Virtual Machine Domains

(vSwitches)vCenter-01-DVS-01

Phy/Out Domains(VLAN mgmt)UCS-phys-svrsOutside-Fabric

What interface settings do I want

to configure?

What type of interface do I want

to configure?

Group my VLANstogether to allow

them on an interface

What “function” doI want to allocate

VLANs for?

Which DVS do I want to configure?

Where do I wantto use my VLANs?

BRKACI-1002

Policy Defined Network

Concrete ModelLogical Model

APIC

APIC

APIC

Interface Selector1/21

Leaf ProfilevPC_to_UCS_FI_A

Security Domain(optional)

Interface PoliciesLeaf Profiles

Which interfacesshould be configured?

22

A Consistent Naming Convention is Critical for Simple Troubleshooting


Example Rack Layout

24


Example Naming Approach• VLAN Pool

• Domains (L2, L3, Phys)

• AAEP (allowed VLANs)

• Interface Polices (settings)

• Leaf Policy Groups (aggregated settings)

• Leaf Profiles (settings mapped to interfaces)

• Switch Profiles (interfaces mapped to switches)

• Customer_A_01

• Customer_A_L3_01

• Customer_A_01

• 10G, CDP_enabled

• 10G_access_c3850-01

• 101_to_c3850-01

• A1_101

• Tenant_Name

• Tenant_Name

• Tenant_Name

• Enabled/Disabled

• PortSpeed_PortType_Usage

• Rack_ID/Switch_ID_to_ConnectedDevice

• Rack_ID or Rack_ID_SwitchID

25


Example Rack DetailsLegend TenantName TenantName Tenant Comment TenantName Settings PortSpeed_PortType_Usage Rack_PortSpeed_PortType_Tenant_ConnectedDevice InterfaceNumber RackID(vPC)

Tenant RackID_Switch(singleconnection)Tenant

Tenant_vDS_Number

ConnectedDeviceType Tenant(Consumer) VLANPool Domain DomainType AAEP InterfacePolicies LeafPolicyGroups LeafProfiles AccessPortSelector SwitchID(s)(allowedVLANs) (InterfaceSettings) (InterfaceType) (Interfacenumber) (SwitchProfiles)

1Gbpscdp_enabled

10Gbpscdp_enabled

1Gbpscdp_enabled

10Gbpscdp_enabled

10Gbpscdp_enabledLCAP_active10Gbps

cdp_enabledLCAP_active10Gbps



cdp_enabled

10Gbpscdp_enabled

B3_10Gbps_acc_Linux 1/21-30 B3

10Gbps_acc_Linux

A2_10Gbps_acc_Linux 1/21-30 A2

B2_10Gbps_acc_Linux 1/21-30 B2

A3_10Gbps_acc_Linux 1/21-30 A3LinuxHost Tenant_01 Tenant_01 Tenant_01 Physical Tenant_01

ESXHost Tenant_01 Tenant_01 Tenant_01 Physical Tenant_01 10Gbps_acc_ESXA3_10Gbps_acc_ESX 1/1-20 A3

B3_10Gbps_acc_ESX 1/1-20 B3

A2_10Gbps_acc_ESX 1/1-20 A2

B2_10Gbps_acc_ESX 1/1-20 B2

F5IO Tenant_01 Tenant_01 Tenant_01 OutsideRouted Tenant_01 10Gbps_acc_F5_io

OutsideRouted

F5Management Tenant_01 Tenant_01 Tenant_01 Physical

1Gbps_acc_ASA_mgmtASAFirewallManagement Tenant_01 Tenant_01 Tenant_01 Physical Tenant_01

1/2

A1_101

B1_121

A1_101

B1_121

B1_1Gbps_acc_F5_mgmt

B1_10Gbps_acc_ASA_io 1/3 B1

A110Gbps_acc_ASA_ioTenant_01ASAIO Tenant_01 Tenant_01 Tenant_01

A1_10Gbps_acc_ASA_io

B1_10Gbps_acc_F5_io 1/4 B1

A1_10Gbps_acc_F5_io 1/4 A1

A1_1Gbps_acc_F5_mgmt 1/2Tenant_01 1Gbps_acc_F5_mgmt

B1_10Gbps_vPC_to_N5k_02 1/10

Tenant_01 10Gbps_vPC_N7k_02 B1_10Gbps_vPC_to_N7k_02 1/11 B1

Tenant_01 10Gbps_vPC_N7k_01 A1_10Gbps_vPC_to_N7k_01 1/11 A1

OutsideBridged Tenant_01

Nexus7k Tenant_01 Tenant_01 Tenant_01 OutsideRouted

Tenant_01 10Gbps_vPC_N5k_02

1/3

B1_1Gbps_acc_ASA_mgmt 1/1

A1_1Gbps_acc_ASA_mgmt 1/1

B1

Nexus7k Tenant_01 Tenant_01 Tenant_01 OutsideRouted

10Gbps_vPC_N5k_01 A1_10Gbps_vPC_to_N5k_01 1/10 A1

Nexus5k Tenant_01 Tenant_01 Tenant_01 OutsideBridged

Nexus5k Tenant_01 Tenant_01 Tenant_01

26

How Does it Look When we Apply the Naming Convention?



OSPF Area 10 (stub)

BRKACI-1002


APIC

APIC

APIC

c3850

n7706 n9504n7706-01 n7706-02

n5672-01 n5672-02

L2

L3

OSPF Area 0

e1/3

e1/1 e1/2 e1/1 e1/2

e1/7 e1/8

e1/15 e1/15 e1/15e1/5 e1/6 e1/11 e1/12

Spirent TestCentre

Spirent TestCentre

Spirent TestCentre

ESX-01ESX-02

28


Interface PoliciesCDP_enabled

VLAN PoolCustomer_A_01

External Routed Domain

Customer_A_L3_01

AAEPCustomer_A_01

Leaf Policy Group10G_acc_c3850

Interface Policies10G

Leaf Profileli07_to_

ld04-c3850-01

Leaf ProfileLeafs_101_and_102


BRKACI-1002

10G_acc_c3850


Rack/Switch to connected device

Interface setting group

29



BRKACI-1002

10G_acc_n7706



Customer_A_L3_01

AAEPCustomer_A_01

Leaf Policy Group10G_acc_n7706



lg05-n7706-01






30



BRKACI-1002

10G_acc_n9504



Customer_A_L3_01

AAEPCustomer_A_01




lg11-n9504-01






31



BRKACI-1002

10G_acc_Spirent_Test_Center

Leaf ProfileLeaf_101


Physical DomainCustomer_A_Phys_01

AAEPCustomer_A_01

Leaf Policy Group10G_acc_Spirent_

Test_Center


Leaf Profileli08_104_to_

Spirent_Test_Center


Spirent_Test_Center


Spirent_Test_Center









32


Interface PoliciesLLDP_enabled

BRKACI-1002

10G_vPC_esx_li07-c220m4-01



AAEPCustomer_A_01

Leaf Policy Group10G_vPC_esx_li07-c220m4-01



li07-c220m4-01



Interface PoliciesLACP_active



Unique Interface setting group

33


Interface PoliciesLLDP_enabled

BRKACI-1002

10G_vPC_esx_li07-c220m4-02



AAEPCustomer_A_01

Leaf Policy Group10G_vPC_esx_li07-c220m4-02



li07-c220m4-02



Interface PoliciesLACP_active



Unique Interface setting group

34

Couldn’t we Reduce the Number of Leaf Policy Groups?

Yes – Provided That They Are “Access” Policy Groups With The

Same Interface Policies



BRKACI-1002

10G_acc_ c3850 | n7706 | n9504




Customer_A_L3_01

AAEPCustomer_A_01



lg11-n9504-01


lg05-n7706-01


ld04-c3850-01






Leaf Policy Group10G_acc_c3850



All Leaf Policy Groups use the same Interface Policies

(Settings and allowed VLANs)


37



BRKACI-1002

10G_acc_to_external_L3_switch




Customer_A_L3_01

AAEPCustomer_A_01



lg11-n9504-01


lg05-n7706-01


ld04-c3850-01






Leaf Policy Group10G_acc_to_external_

L3_switch

Consolidated Leaf Policy Group for Interfaces which use the

same Interface Policies (Settings and allowed VLANs)


38

Couldn’t We Reduce The Number of Leaf Profiles?

Yes – Provided That They Use The Same Interfaces On The Physical

Switch(es)



BRKACI-1002





Customer_A_L3_01

AAEPCustomer_A_01



lg11-n9504-01


lg05-n7706-01


ld04-c3850-01







L3_switch

Multiple Leaf Profiles / Interface Selectors consume the same

Leaf Policy Group(Settings and allowed VLANs)


41



BRKACI-1002




Customer_A_L3_01

AAEPCustomer_A_01


L3_switch


Leaf Profileli07_to_external

L3_switch


Interface Selector1/3, 1/7, 1/8

Consolidated Leaf Profiles / Interface Selectors consume the same Leaf Policy Group

(Settings and allowed VLANs)


42


Automating “Access Policies” Abstracts the Naming Rules Away From APIC Thus Ensuring Configuration Conformance

43


In Large Organisations Having an Automated Approach to Interface Configuration Could Allow the “rack/stack” Team to Configure the Switches From a Simple IT Services Catalogue

44


Notes to Remember:

• Interface Policies can be reused across any interface type

• Leaf Policy Groups for “Access” ports can be used by different Leaf Profiles

• Leaf Policy Groups for PC/vPC cannot be used by different Leaf Profiles

• Leaf Profiles can be used by different Switch Profiles

45

Step 2 – VRF, SVI, Bridge Domain Configuration



OSPF Area 10 (stub)

BRKACI-1002


APIC

APIC

APIC

Spirent TestCentre

Spirent TestCentre

Spirent TestCentre

ESX-01ESX-02

c3850

n7706 n9504n7706-01 n7706-02

n5672-01 n5672-02

L2

L3

OSPF Area 0

e1/3

e1/1 e1/2 e1/1 e1/2

e1/7 e1/8

e1/15 e1/15 e1/15e1/5 e1/6 e1/11 e1/12

47


Network Consumption

Quick Start wizard

Tenants

48


ACI Tenants are Network Wide Administrative Containers

BRKACI-1002

Tenant: Common

Tenant: Production Tenant: Pre-Production

Objects created in “Common” can be consumed by other Tenants

BD: 01 BD: 02 BD: 03

VRF: A VRF: B VRF: C

AD DHCPDNS

APIC

APIC

APIC

Tenant: ESX-Hosts

BD: 01 BD: 02 BD: 03

VRF: A

49


Looking Under the Covers at Tenants

BRKACI-1002

apic1# show tenantTenant Tag Description--------------- --------------- ----------------------------------------avankercommonfgandolahyper-vinframgmtnickmartnvermandnvermand-vRA-01 openstackrobvandrwhitearssharmanvmware

apic1#

50


ACI VRFs (aka Private Networks, aka Contexts) Provide the Routing Function Within a Given Tenant

BRKACI-1002

VRF: VRF-01 (Anycast gateway)

Tenant: Common

APIC

APIC

APIC

51


Multiple VRFs Allow Overlapping IP Address Space and Integration with External Devices

BRKACI-1002

VRF: VRF-01 (Anycast gateway) VRF: VRF-02 (Anycast gateway)

Tenant: Common

APIC

APIC

APIC

52


Looking Under the Covers at VRFs

BRKACI-1002

apic1# show vrf

Tenant Vrf---------- ----------

common defaultcommon inside_enforcedcommon inside_unenforcedcommon outside_ospfcommon outside_staticcommon outside_vlansfgandola VRF-01 mgmt inbmgmt oobnickmart nickmartnvermand VRF-01 nvermand VRF-02 nvermand VRF-AVS

53


ACI Bridge Domains are Pervasive Layer 2 Boundaries with Defined Forwarding Characteristics

BRKACI-1002

VRF: VRF-01 (Anycast gateway)Bridge Domain: BD-01

APIC

APIC

APIC

Tenant: Common

BD: 03Hardware Proxy: NoARP Flooding: Yes

Unknown Unicast Flooding: YesIP Routing: No





The Bridge Domain to VRF association is always required, even if the VRF is not routing

54


Display Details of a Single Bridge Domain

BRKACI-1002

apic1# show bridge-domain outside_infra-ssharmanTenant : commonInterface : outside_infra-ssharmanMAC Address : 00:22:BD:F8:19:FFMTU : inheritDescription :Multi-Destination Action : bd-floodUnknown Multicast Action : floodUnknown MAC Unicast Action : flood

Tenant : ssharmanInterface : Internal_Fabric_02MAC Address : 00:22:BD:F8:19:FFMTU : inheritDescription :Multi-Destination Action : bd-floodUnknown Multicast Action : opt-floodUnknown MAC Unicast Action : proxy

55


A Bridge Domain Uses a Locally Significant VLAN ID on Each Leaf which Dynamically Maps to a VXLAN ID

BRKACI-1002

VRF: VRF-01 (Anycast gateway)Bridge Domain: outside_infra-ssharman

APIC

APIC

APIC

Tenant: Common

Leaf 101Tenant: Common

BD: outside_infra-ssharman

Leaf 102Tenant: Common

BD: outside_infra-ssharman

The Bridge Domain to VRF association is always required, even if the VRF is not routing

Layer 2 Bridge Domain carried over VXLAN

56


A Bridge Domain Uses a Locally Significant VLAN ID Underneath

BRKACI-1002

apic1# fabric 101 show vlan----------------------------------------------------------------Node 101 (Leaf-1)----------------------------------------------------------------VLAN Name Status Ports---- -------------------------------- --------- -------------------------------9 infra:default active Eth1/1, Eth1/21, Eth1/22, Po3, Po411 common:outside_infra-robvand active Eth1/11, Eth1/21, Eth1/22, Po3,14 fgandola:www-zone1 active Eth1/33, Po215 ssharman:192.168.66.0 active Eth1/21, Eth1/22, Po3, Po426 common:outside_infra-ssharman active Eth1/11, Eth1/21, Eth1/22, Po3, Po4, Po8

apic1# fabric 102 show vlan----------------------------------------------------------------Node 102 (Leaf-2)----------------------------------------------------------------VLAN Name Status Ports---- -------------------------------- --------- -------------------------------9 infra:default active Eth1/1, Eth1/21, Eth1/22, Po1, Po211 ssharman:L2-to-outside:Group-05 active Eth1/21, Eth1/22, Po1, Po214 fgandola:app-zone2 active Eth1/33, Po815 -- active Eth1/69, Po735 common:outside_infra-ssharman active Eth1/11, Eth1/21, Eth1/22, Po1, Po2, Po4

Leaf 101

Leaf 102

57


VXLANs Require VTEPs

BRKACI-1002

VRF: 01 (Anycast gateway)

BD: 01Hardware Proxy: Yes

ARP Flooding: NoUnknown Unicast Flooding: No

IP Routing: Yes

APIC

APIC

APIC

Known unicast traffic forwarded directly between Leaf VTEP’s

Unknown unicast traffic is forwarded to anycast spine proxy VTEP’s

Logical vPC switch is represented by anycast Leaf vPC VTEP’s

Multicast and any allowed broadcast traffic is forwarded to a Group VTEP that exists on any leaf with membership for that specific group

VTEP’s may exist in physical or virtual switches

VTEP VTEP VTEP VTEP

VTEP VTEP VTEP VTEP VTEP VTEP

Tenant: Common

VTEPs are dynamically created as required

58


A Bridge Domain Uses a VXLAN to Transport Data Between Leaf Switches

BRKACI-1002

apic1# fabric 101 show vlan id 26 extended----------------------------------------------------------------Node 101 (Leaf-1)----------------------------------------------------------------VLAN Name Status Ports---- -------------------------------- --------- -------------------------------26 common:outside_infra-ssharman active Eth1/11, Eth1/21, Eth1/22, Po3,

Po4, Po8VLAN Type Vlan-mode Encap---- ----- ---------- -------------------------------26 enet CE vxlan-15433637

apic1# fabric 102 show vlan id 35 extended----------------------------------------------------------------Node 102 (Leaf-2)----------------------------------------------------------------VLAN Name Status Ports---- -------------------------------- --------- -------------------------------35 common:outside_infra-ssharman active Eth1/11, Eth1/21, Eth1/22, Po1,

Po2, Po4VLAN Type Vlan-mode Encap---- ----- ---------- -------------------------------35 enet CE vxlan-15433637

Leaf 101

Leaf 102

59


ACI SVIs are Configured on a Given Bridge Domain and Instantiated on the Associated VRF

BRKACI-1002


APIC

APIC

APIC

Tenant: Common



IP Routing: 192.168.10.1/24

60


ACI Bridge Domains can be Configured with Multiple Subnets/Default Gateways (Secondary)

BRKACI-1002


APIC

APIC

APIC

Tenant: Common



IP Routing: 192.168.10.1/24: 192.168.20.1/24

61


apic1# show bridge-domain outside_infra-ssharmanTenant : commonInterface : outside_infra-ssharmanMAC Address : 00:22:BD:F8:19:FFMTU : inheritDescription :Multi-Destination Action : bd-floodUnknown Multicast Action : floodUnknown MAC Unicast Action : flood

Tenant : ssharmanInterface : Internal_Fabric_02MAC Address : 00:22:BD:F8:19:FFMTU : inheritDescription :Multi-Destination Action : bd-floodUnknown Multicast Action : opt-floodUnknown MAC Unicast Action : proxy

Display Details of a Single Bridge Domain

BRKACI-1002

apic1# show ip interface bridge-domain outside_infra-ssharman----- IPv4 Bridge-Domain Information: -----Tenant : commonInterface : outside_infra-ssharmanVRF Member : outside_vlansIP Addresses : 192.168.29.254/24

192.168.30.254/24

Bridge Domain + SVIBridge Domain + SVI

VRF name

62


ACI Nomenclature• A Tenant is just an Administrative boundary

• A VRF is a VRF as you know it today

• A Bridge Domain is a L2 segment where flooding rules apply – think VLAN but without a VLAN ID

• A Bridge Domain is the scope of one or more subnets – think SVI and IP Secondary

BRKACI-1002 63

Step 3 – Consume the Configured Interfaces


Leaf Profiles(Target Switches)

Leafs_101_and_102

AAEP(Allowed VLANs)

UCS-phys-svrs

Interface PoliciesCDP_enabledLACP_Active

VLAN/VXLAN(Pools)

UCS-phys-svrs

VLAN mgmt(Phy/Out Domain)

UCS-phys-svrs

BRKACI-1002

Network Interfaces Must be Configured First!


APIC

APIC

APIC



ANP: My_App

EPG: WebDomain: UCS-phys-svrs

Path: vPC_to_UCS_FI_AVLAN_10

Path: vPC_to_UCS_FI_BVLAN_10


Leaf Policy GroupvPC_to_UCS_FI_A

Leaf Policy GroupvPC_to_UCS_FI_B


Leaf ProfilevPC_to_UCS_FI_B

Leaf Profiles

65

Application Network Profiles – a Collection of Endpoint Groups

Endpoint Groups – a Collection of Interfaces and VLANs


EPG Tag: DB (VLAN 12)Security Zone

EPG Tag: App (VLAN 11)Security Zone

EPG Tag: Web (VLAN 10)Security Zone

BRKACI-1002

Option 1: Single EPG on a Single BD with a Single Subnet – “Standard Networking”

ANP: My_App

APIC

APIC

APIC

Tenant: My_TenantCommunication allowed within EPG Communication allowed within EPGCommunication allowed within EPG

BD:192.168.30.xHardware Proxy: Yes


IP Routing: Yes

BD: 192.168.10.XHardware Proxy: Yes


IP Routing: Yes


192.168.20.11/24 192.168.20.12/24 192.168.30.11/24 192.168.30.12/24192.168.10.11/24 192.168.10.12/24

BD: 192.168.20.xHardware Proxy: Yes


IP Routing: Yes Endpoints in EPG identified by Switch/Interface and VLAN ID

68


Display the Mac Addresses Contained in the EPG

BRKACI-1002

apic1# fabric 101 show mac address-table vlan 37Legend:

* - primary entry, G - Gateway MAC, (R) - Routed MAC, O - Overlay MACage - seconds since last seen,+ - primary entry using vPC Peer-Link,(T) - True, (F) - False

VLAN MAC Address Type age Secure NTFY Ports/SWID.SSID.LID---------+-----------------+--------+---------+------+----+------------------* 37 0000.0c07.ac08 dynamic - F F po2* 37 001a.a2d5.c080 dynamic - F F po2* 37 02a0.981c.b2be dynamic - F F po2* 37 0026.0bf1.f002 dynamic - F F po2* 37 0014.384e.26e1 dynamic - F F po2* 37 0016.355b.ddda dynamic - F F po2* 37 0060.1646.97da dynamic - F F po2* 37 0010.18cf.c318 dynamic - F F po2* 37 0018.74e2.1540 dynamic - F F po2* 37 0004.02f6.1f13 dynamic - F F po2* 37 0025.b506.006d dynamic - F F po2* 37 001b.21be.fa68 dynamic - F F po2* 37 0025.b501.04af dynamic - F F po2* 37 0025.b501.049f dynamic - F F po2* 37 0025.b501.04bf dynamic - F F po2* 37 0025.b506.007c dynamic - F F po2* 37 0025.b501.04df dynamic - F F po2* 37 0025.b506.0027 dynamic - F F po2* 37 0025.b506.0068 dynamic - F F po2

69


Displaying the Endpoints on the Network

BRKACI-1002

apic1# show endpointsTenant Application AEPg End Point MAC IP Address Node Interface Encap---------- ----------------- ---------------------------------------- ---------- ------------------------------ ----------

vmware ESXi- Host-mgmt 00:25:B5:06:00:1F 192.168.29.43 101 102 vpc 1Gbps_vPC_to_ucs-02-b vlan-8ssharman

vmware ESXi- Host-mgmt 00:25:B5:06:00:3E 192.168.29.44 101 102 vpc 1Gbps_vPC_to_ucs-02-b vlan-8ssharman

vmware ESXi- Host-mgmt 00:25:B5:06:00:47 192.168.29.46 101 102 vpc 1Gbps_vPC_to_ucs-02-b vlan-8ssharman

vmware ESXi- Host-mgmt 00:50:56:86:81:1D 192.168.29.102 101 102 vpc 1Gbps_vPC_to_ucs-02-b vlan-8ssharman

vmware ESXi- Host-mgmt 00:50:56:86:F7:6A 192.168.29.106 101 102 vpc 1Gbps_vPC_to_ucs-02-b vlan-8ssharman

70


Displaying the Endpoints on a Leaf

BRKACI-1002

apic1# fabric 101 show endpointLegend:O - peer-attached H - vtep a - locally-aged S - staticV - vpc-attached p - peer-aged L - local M - spans - static-arp B - bounce+-----------------------------------+---------------+-----------------+--------------+-------------+

VLAN/ Encap MAC Address MAC Info/ InterfaceDomain VLAN IP Address IP Info

+-----------------------------------+---------------+-----------------+--------------+-------------+common:outside_ospf 101.1.1.1 L44/common:outside_ospf vxlan-15302582 0000.0c07.ac30 L eth1/9644/common:outside_ospf vxlan-15302582 0018.74e2.1540 L eth1/9644/common:outside_ospf vxlan-15302582 001a.a2d5.c080 L eth1/9613 vlan-2022 0025.b506.0062 LV po3common:outside_vlans vlan-2022 192.168.22.14 LV13 vlan-2022 0025.b506.0002 LV po3common:outside_vlans vlan-2022 192.168.22.15 LVcommon:outside_vlans vlan-2022 192.168.22.17 LV32 vlan-22 0000.0c07.ac16 LV po2common:outside_vlans vlan-22 192.168.22.1 LV32 vlan-22 001a.a2d5.c080 LV po2common:outside_vlans vlan-22 192.168.22.3 LV32/common:outside_vlans vlan-22 0018.74e2.1540 LV po232 vlan-22 0050.5699.9099 LV po2common:outside_vlans vlan-22 192.168.22.16 LV32 vlan-22 0050.5699.7e05 LV po2

71





BRKACI-1002

Option 2: Multiple EPGs on a Single BD with a Single Subnet – µSegmentation in IP space

VRF: 01 (Anycast gateway)Bridge Domain: 192.168.10.X_24Gateway: 192.168.10.1

ANP: My_App

Bridge DomainHardware Proxy: Yes


IP Routing: Yes

APIC

APIC

APIC

Tenant: My_Tenant

192.168.10.11/24 192.168.10.12/24 192.168.10.13/24 192.168.10.14/24 192.168.10.15/24 192.168.10.16/24

Endpoints in EPG identified by Switch/Interface and VLAN ID

Communication allowed within EPG Communication allowed within EPGCommunication allowed within EPG

Layer 2 Segment

72

Just Because You Can Doesn't Always Mean You Should





BRKACI-1002

Option 3a: Multiple EPGs on a Single BD with Multiple Subnets – IP Secondary

VRF: 01 (Anycast gateway)Bridge Domain: multiple_subnetsGateway: 192.168.10.1

192.168.20.1192.168.30.1

ANP: My_App



IP Routing: Yes

APIC

APIC

APIC

Tenant: My_Tenant

192.168.10.11/24 192.168.10.12/24 192.168.20.11/24 192.168.20.12/24 192.168.30.11/24 192.168.30.12/24



74





BRKACI-1002

Option 3b: Multiple EPGs on a Single BD with Multiple Subnets – IP Secondary

VRF: 01 (Anycast gateway)Bridge Domain: multiple_subnetsGateway: 192.168.10.1

192.168.20.1

ANP: My_App



IP Routing: Yes

APIC

APIC

APIC

Tenant: My_Tenant

192.168.10.11/24 192.168.20.11/24 192.168.10.12/24 192.168.20.12/24 192.168.10.15/24 192.168.10.16/24



75

What About Segmenting Inside an EPG?



BRKACI-1002

Options 1, 2, and 3 – µSegmentation within an EPG/Port Group (no East/West traffic flows)


ANP: My_App



IP Routing: Yes

APIC

APIC

APIC

Tenant: My_Tenant

192.168.10.11/24 192.168.10.12/24 192.168.10.13/24 192.168.10.14/24 192.168.10.15/24 192.168.10.16/24


Communication not allowed within EPG

Layer 2 Segment

77


EPG Tag: All_Web_Servers (VLAN 10)Security Zone

BRKACI-1002

Options 1, 2, and 3 – µSegmentation within an EPG/Port Group Based on Machine Attribute


ANP: My_App



IP Routing: Yes

APIC

APIC

APIC

Tenant: My_Tenant

192.168.10.11/24 192.168.10.12/24 192.168.10.13/24 192.168.10.14/24 192.168.10.15/24 192.168.10.16/24


Layer 2 Segment

Name Contains:Web_1

Name Contains:Web_2

Name Contains:Web_3

Communication allowed within uSeg EPG

78


Advanced Query: How to Find If/Where any VLAN has Been Used

BRKACI-1002

apic1# moquery -c fvIfConn | grep dn | grep common | grep vlan

dn: uni/epp/br-[uni/tn-common/l2out-outside_infra-robvand/instP-EPG_outside_infra-robvand]/node-102/stpathatt-[1Gbps_vPC_to_n5548]/conndef/conn-[vlan-47]-[0.0.0.0]dn: uni/epp/br-[uni/tn-common/l2out-outside_infra-robvand/instP-EPG_outside_infra-robvand]/node-101/stpathatt-[1Gbps_vPC_to_n5548]/conndef/conn-[vlan-47]-[0.0.0.0]

dn: uni/epp/br-[uni/tn-common/l2out-outside_infra-anvanker/instP-EPG_outside_infra-anvanker]/node-102/stpathatt-[1Gbps_vPC_to_n5548]/conndef/conn-[vlan-13]-[0.0.0.0]dn: uni/epp/br-[uni/tn-common/l2out-outside_infra-anvanker/instP-EPG_outside_infra-anvanker]/node-101/stpathatt-[1Gbps_vPC_to_n5548]/conndef/conn-[vlan-13]-[0.0.0.0]

Interface Connection

Distinguished Name

Tenant Name VLANManaged

Object Class

79

High Level Packet Walk


Where are IP/Mac Addresses Stored?

BRKACI-1002




IP Routing: Yes

APIC

APIC

APIC

Proxy Proxy Proxy Proxy

FIB FIB FIB FIB FIB FIB

Tenant: Common

Leaf Local Station Table contains addresses of ‘all’

hosts attached directly to the Leaf

10.1.3.11 Port 9

Leaf Global Station Table contains a local cache of the

fabric endpoints 10.1.3.35 Leaf 3

Proxy A*

10.1.3.35 Leaf 310.1.3.11 Leaf 1

Leaf 4Leaf 6

fe80::8e5efe80::5b1a

Spine Proxy Station Table contains addresses of ‘all’ hosts attached to the

fabric

81


High Level Packet Walk

BRKACI-1002


ANP: ESXi-Hosts

BD: ESXiHardware Proxy: Yes


IP Routing: Yes

EPG: Host-MgmtSecurity Zone

Leaf-101/1/10vlan-8

Leaf-102/1/10vlan-8

APIC

APIC

APIC

Tenant: ESXi-HostsEndpoints identified by Interface and VLAN ID

PayloadIP

Packet Sourced from physical server1

PayloadIPVXLANL1 VTEP

Leaf swaps ingress encapsulation with VXLAN (EPG) ID and performs any required policy functions2

Leaf-103/1/10vlan-8

Leaf-104/1/10vlan-8

Leaf-105/1/10vlan-8

Leaf-106/1/10vlan-8

3aIf the ingress Leaf has learned the destination IP to egress VTEP binding it will set required destination VTEP address and forward


If the ingress Leaf has NOT learned the destination IP to egress VTEP binding it will set required destination VTEP to the Spine Proxy VTEP

3b

PayloadIPVXLANS1 VTEP

PayloadIP

Packet Delivered to physical server5

Communication allowed within EPG


Leaf removes ingress VXLAN (EPG) ID and performs any required policy functions4

There is no requirement to use the same VLAN on every Leaf

82


Host-mgmt EPG –Access Encap VLAN 8

Alternate command:show vlan extended

Remember for troubleshooting use the Internal VLAN ID not the

Access Encap VLAN ID

apic1# fabric 101 show system internal epm vlan all

+----------+---------+-----------------+----------+------+----------+---------VLAN ID Type Access Encap Fabric H/W id BD VLAN Endpoint

(Type Value) Encap Count+----------+---------+-----------------+----------+------+----------+---------9 Infra BD 802.1Q 3967 16777209 11 9 310 Ext. BD 802.1Q 2050 15269816 12 10 011 Ext. BD 802.1Q 49 15531935 111 11 212 Tenant BD NONE 0 15662984 14 12 013 FD vlan 802.1Q 2022 8814 15 12 214 Ext. BD 802.1Q 2020 14909414 16 14 015 Tenant BD NONE 0 15171524 17 15 016 FD vlan 802.1Q 33 8324 19 15 117 FD vlan 802.1Q 2131 9023 20 15 018 Tenant BD NONE 0 15138760 18 18 019 FD vlan 802.1Q 2125 9017 21 18 020 FD vlan 802.1Q 47 8338 22 18 434 Tenant BD NONE 0 15302581 29 34 035 FD vlan 802.1Q 14 8305 40 34 436 Tenant BD NONE 0 15400873 30 36 037 FD vlan 802.1Q 8 8299 41 36 1938 Ext. BD 802.1Q 115 15269817 31 38 1

Lets Look at Which VLANs/VXLANs Have Been Used by Bridge Domains and EPGs on a Given Leaf

BRKACI-1002

BD_CTRL_VLAN: The infrastructure vlan which was configured during the APIC setup script.

BD_EXT_VLAN: Bridge Domain to represent external VLAN

BD_VLAN: An internal Bridge Domain construct which is represented by the grouping of multiple FD_VLANs/VXLANs – i.e many FD_VLANs can map to one BD_VLAN

FD_VLAN: A VLAN backed EPG identified by the “Access encap” VLAN ID mapped to the Bridge Domain – a FD_VLAN can only map to a single BD_VLAN

FD_VXLAN: Used to communicate with hosts behind hypervisors using VXLAN

Access encap: The Access_enc is significant outside the ACI network as it is the VLAN that is programmed on a front panel port mapping inbound frames to an EPG (FD_VLAN)

Fabric Encap: The VXLAN ID for a given EPG/BD

HW_VlanId: The VLAN used to encapsulate incoming traffic from Access_enc to send to the ALE

VlanId: The VlanId is significant for troubleshooting, most (if not all) show commands use the VlanId not the Access_enc VLAN ID

83

External VLANs – L2 Connection to Legacy Networks


Option 1: Same VLANs Outside/Inside (No Contract Required)

ANP: Outside_VLANs

Bridge DomainHardware Proxy: NoARP Flooding: Yes

Unknown Unicast Flooding: YesIP Routing: Yes

vPC_to_UCS_avlan-10

vPC_to_UCS_bvlan-10

EPG: Host-Mgmt192.168.10.11 192.168.10.10

vPC_to_n5ksvlan-10

vlan-10

APIC

APIC

APIC

Tenant: My_Tenant

VRF: 01 (Anycast gateway)Bridge Domain: outside_vlan_10Gateway: 192.168.10.1


85


Option 2: Different VLANs Outside/Inside(Contract Required)

ANP: Outside_VLANs

EPG

Bridge DomainHardware Proxy: NoARP Flooding: Yes


L2outvPC_to_n5ks

vlan-10

vlan-10

APIC

APIC

APIC

Tenant: My_Tenant

VRF: 01 (Anycast gateway)Bridge Domain: outside_vlan_10Gateway: 192.168.10.1

vPC_to_UCS_avlan-100

vPC_to_UCS_bvlan-100

EPG: Host-Mgmt192.168.10.10 192.168.10.11


Communication allowed to External EPG

86

External Subnets


External Routed Connections



IP Routing: Yes

L3out: Area0101/1/96: 192.168.30.1/30102/1/96: 192.168.30.5/30

Outside

Security Import Subnet*i.e which external subnets can be accessed through this EPG

APIC

APIC

APIC

EPG0.0.0.0/0

OSPF Configuration



ANP: My_App

192.168.10.11/22 192.168.10.12/22 192.168.10.21/22 192.168.10.22/24

Communication allowed to 10.1.1.0/24

VRF: 01 (Anycast gateway)Bridge Domain: 192.168.10.x_22Gateway: 192.168.10.1

Permit access to all remote subnets:

0.0.0.0/0Tenant: My_Tenant

Communication allowed to all External Subnets

EPG10.1.1.0/24

Permit access to remote subnet:

10.1.1.0/24

88



OSPF Area 10 (stub)

BRKACI-1002


APIC

APIC

APIC

Spirent TestCentre

Spirent TestCentre

Spirent TestCentre

ESX-01ESX-02

c3850

n7706 n9504n7706-01 n7706-02

n5672-01 n5672-02

L2

L3

OSPF Area 0

e1/3

e1/1 e1/2 e1/1 e1/2

e1/7 e1/8

e1/15 e1/15 e1/15e1/5 e1/6 e1/11 e1/12

89


Transit Routing – Multiple L3 Out per VRF

BRKACI-1002

Outside Outside

MP BGP

L3outArea 10

EPG EPG

L3outArea 20

Contract = Allow Communication

Use a 0.0.0.0/0 subnet with the ‘aggregate export’ option checked to export all routes

VRF: Production

70.1.1.0/24

80.1.1.0/24

60.1.1.0/24

Tenant: Common

BD: InsideHardware Proxy: Yes


IP Routing: 192.168.10.1/24

90


Let’s consider the consumers of a cloud provider. The consumers don’t concern themselves with server connectivity…

91


They simply concern themselves with the IP addresses/gateway for their applications, and the security rules which allow access to those applications

92


Automating “Tenant” configuration allows teams other than the network team to consume network services

93


ACI Nomenclature• An EPG is just a logical grouping of devices – think interfaces and VLANs

• An EPG is a Port Group in VMware

• An EPG can contain different VLANs, e.g. when mixing dynamic Virtual Port Groups and Physical machines – think hardware VTEP

• Devices in an EPG are allowed to communicate (by default)

• Isolated EPGs block communication within the EPG – think PVLAN

• Micro Segmentation (µSeg) EPGs are used to dynamically move devices from a “base” EPG into a more specific EPG

• An Application Network Profile is a group of one or more EPGs – remember an EPG can only be a member of one ANP

• Communication between EPGs and/or from devices off the ACI fabric require Contracts (ACLs)

BRKACI-1002 94

Step 4 – Allow Communication Between EPGs with Contracts


Filter: Any-TrafficFilter: 80, 443 etc EPG:Clients

Contract: Any-to-Any

Contracts are “directional” Access Lists between Provider and Consumer EPGs. They comprise of one or more Filters (ACEs) to identify traffic, e.g:

• Contract: Any-to-Any | Filter: Any-Traffic

• Contract: Web | Filter: 80, 443, 8000

• Contract: DNS | Filter: 53

Contracts (ACLs)

BRKACI-1002

Provider Consumer

Contract: Clients-to-WebFilter: none

Flags:

• Apply in both directions (single contract which allows return traffic)

• Reverse filter ports (dynamically permits return flow based on src/dst ports)

Flags:

• IP Protocol

• Ports

• Stateful

• Etc.

EPG: Web

ExternalSubnet

ANP: My-Web-App

L3out: Clients

96


Contracts Permit Communication Between EPGs

Tenant: My_Tenant

VRF: 01

ANP: DB

EPG: DB_1192.168.10.11/24 192.168.10.12/24

EPG: Web_1192.168.10.11/24 192.168.10.12/24

EPG: App_1192.168.20.11/24 192.168.20.12/24

ANP: MyApp_2

BD: 192.168.10.X

BD: 192.168.20.x

EPG: App_1 BD:192.168.30.xEPG: Web_1192.168.10.11/24 192.168.10.12/24 192.168.10.11/24 192.168.10.12/24

ANP: MyApp_1

97


Contracts are “scoped” at:

• Global

• Tenant

• Context (aka Private Network, aka VRF)

Web_to_App

• Application Profile

App_to_DB

Contracts Scope

BRKACI-1002

ANP: 01

EPG: Web

EPG: App

EPG: DB

ANP: 02

EPG: Web

EPG: App

EPG: DB

VRF: 01Tenant: Web_Hosting


IP Routing: Yes

98

What Happens If I Don’t Know The Required Filter Ports?


• Ask the Application Owner – it’s their application, they will (ok should) know

• Ask the Security Admin for the firewall rules

• Use an “any-any” Filter between EPGs ß Most customers start here

• Use Wireshark

• Configure “Unenforced” mode on the VRF

Filter Discovery

BRKACI-1002 100

Once the ACI Fabric is Up and Running How Does it Integrate with VMware’s Virtual Switches?


Firstly, why should you care about integrating with VMware’s Virtual Switches?

102


A perceived barrier to timely delivery of new services (from Virtualisation Teams) is that it takes too long to provision Network Services i.e. VLANs, Subnets, and L4-7 Devices

103


The reality was that until the release of Cisco ACI there was no turnkey SDN solution for both Physical Machines, Virtual Machines, and L4-7 Devices

104


1. Manually configure the vSwitch/vDS as you do today2. Dynamically configure the vDS (VMware) by pushing Port Groups

(VLAN) from APIC to vCentre3. Dynamically configure the vDS (Cisco AVS) by pushing Port Groups

(VLAN/VXLAN) from APIC to vCentre4. Build NSX overlay networks (VXLAN) between different hosts –

requires additional (costly) NSX licenses from VMware

There are Four Integration Options with VMware

BRKACI-1002 105


Traditional NetworkingSVI | VLAN | Port Group Relationship

BRKACI-1002

Layer 2 VLAN: VLAN10

VRF: VRF-01 (HSRP gateway)Interface VLAN10IP Address 192.168.10.1/24

vDS-01VM VM VM VM VM VM VM VM VM VM VM VM VM VM VM VM VM VM

Port Group: Web(VLAN 10)

Host-01 Host-02 Host-03 Host-04

106


Single EPG on a Single BD with a Single Subnet –“Standard Networking”

BRKACI-1002

ANP: My-App-01

vCentre

Service Request:Create Application Create vDS Port Groups

Tenant: Tenant-01

APIC

APIC

BD: AppsIP Routing: 192.168.10.1/24

Outside

EPG: Web (Dynamic VLAN 2001)



APIC

Port Group: VMware|My-App-01|Web (Dynamic VLAN 2001)


107


Multiple EPGs on a Single BD with a Single Subnet –µSegmentation in IP space

BRKACI-1002

ANP: My-App-01No Contract = No Communication

vCentre

Service Request:Create Application Create vDS Port Groups

Tenant: Tenant-01

APIC

APIC

BD: AppsIP Routing: 192.168.10.1/24

Outside

EPG: App (Dynamic VLAN 2002)EPG: Web (Dynamic VLAN 2001) EPG: DB (Dynamic VLAN 2003)


Contract = Allow Communication Contract = Allow Communication


APIC

Port Group: VMware|My-App-01|Web (Dynamic VLAN 2001)

Port Group: VMware|My-App-01|App (Dynamic VLAN 2002)

Port Group: VMware|My-App-01|DB (Dynamic VLAN 2003)


PS PS

(Eth1/50, 51 VLAN 3600)

108


NSX Overlay

BRKACI-1002

ANP: Overlay_Network

vCentre

Tenant: Tenant-01

APIC

APIC

Outside

EPG: NSX_Transport (VLAN 1000)

APIC

vDS-01 (not managed by APIC)

VLAN 1000 VTEP 10.0.0.4VTEP 10.0.0.3VTEP 10.0.0.2VTEP 10.0.0.1

VM VM VM VM VM VM VM

Dedicated Hosts for “Edge” Functionality

NSX Logical Switch:Layer 2 segment carried over

VXLAN, carried over a dedicated VLAN

DLR DLR B/U

NSX ESG Routers Peer with the Physical Network

NSX Manager

APIC Configures fabric with an NSX Transport EPG (VLAN) across all hosts

ESG ESG B/U

NSX DLR informs controllers of learnt routes

VRF: VRF-01

EPG

VM VM VM VM VM

BD: NSXIP Routing: Yes

Controllers push routes to Hosts

L3outInterface: VLAN 2000

IP: 192.168.30.1IP: 192.168.30.2

NSX Controller Cluster

109


Virtual Switching Comparison

BRKACI-1002

Feature / Requirement StandardvSwitch

VMware NSX APIC ManagedvDS (VMware)

APIC ManagedvDS (Cisco)

Manual port group / EPG Configuration � N/A � �Automated port group / EPG configuration pushed from APIC � N/A � �VLAN backed port groups � � � �VXLAN backed port groups � � � �Integrated Physical and Virtual Machine security (inc FW, SLB) � � � �Micro-segmentation – VM/VM/Physical separation within the same IP address space � � � �Micro-segmentation – VM to VM separation within a port group (attribute based) � � � �No requirement for dedicated ESX hosts to provide L2/L3 Controllers/Gateways between Virtual and Physical environments � � � �Traffic visibility between Virtual and Physical Environments � � � �Simple Troubleshooting � � � �

110


Cisco AVS is a Partner Supported VIB• Let’s look at vSphere 6.0 Official Documentation about kernel

Virtual Installation Bundles (VIB) - http://vmw.re/1Ta1Zz0

111


• Cisco AVS Statement of Support

Customers Call Cisco for AVS Support

BRKACI-1002

OpFlex

VM VM VM

VMware ESXi Server

VM VM VM

VMware ESXi Server

OpFlex

VMware vCentreCisco APICVMM Domain

AVS AVS

http://www.cisco.com/c/dam/en/us/products/collateral/switches/application-virtual-switch/avs-support-statement-an.pdf

112

Adding L4-7 Devices to the Network –Service Graphs and Service Chains

Service Graph Contracts Connect two EPGs and Optionally Provide Configuration Parameters to the FW and SLB Which Sit Between the EPGsNote: Normal L2/L3 rules still apply, you still have to direct the traffic to the FW/SLB

In “Managed” Mode the APIC Pushes the Required VLANs and Configuration to the FW/SLBNote: Normal L2/L3 rules still apply, you still have to direct the traffic to the FW/SLB

In “Unmanaged” Mode the APIC Only Pushes the Required VLANs to the EPG

Note: Normal L2/L3 rules still apply, you still have to direct the traffic to the FW/SLB

Service Chains are Two L4-7 Devices Linked in a SeriesNote: Normal L2/L3 rules still apply, you still have to direct the traffic to the FW/SLB

It is Possible to use L4-7 Devices Without Service Graphs, in this Mode the Fabric Only Provides L2 Connectivity


Transparent Firewall – Server’s Default Gateway is the Bridge Domain on the ACI Fabric

BRKACI-1002

EPG: Servers_Inside

L3out

EPG: Servers_Outside

Stan

dard

_Con

tract

ANP: My-App-01 Service_Graph_Contract

BD: OutsideHardware Proxy: NoARP Flooding: Yes


BD: InsideHardware Proxy: NoARP Flooding: Yes


Connector type must be specified as L2


Tenant: Common

192.168.10.x/24192.168.10.x/24

Servers_Outside can communicate externally via

the contract to the L3out

Servers_Outside can communicate with Servers_Inside via the Service

Graph Contract

VRF not used

Server default gateway

VRF: 01 VRF: 02

119


Transparent Firewall – Server’s Default Gateway is the Bridge Domain on the ACI Fabric

BRKACI-1002

EPG: Servers_InsideANP: My-App-01

L3out

BD: OutsideHardware Proxy: NoARP Flooding: Yes


BD: InsideHardware Proxy: NoARP Flooding: Yes


Service_Graph_Contract

Tenant: Common

192.168.10.x/24




Servers_Inside can communicate to the “outside world” via the Service

Graph Contract to the L3out

192.168.10.x/24

VRF not used

VRF: 01 VRF: 02

120


Routed Firewall – Server’s Default Gateway is the Firewall Attached to the ACI Fabric

BRKACI-1002

EPG: Servers_InsideANP: My-App-01


ARP Flooding: YesUnknown Unicast Flooding: No

IP Routing: No

L3out

L3out

Tenant: Common



Servers_Inside can communicate to the “outside world” via the Service



Static route to firewall “inside” subnet via L3out ot Firewall

VRF has Static route to firewall “inside” subnet

via L3out to Firewall

192.168.10.x/24

10.1.1.0/30

VRF not used


VRF: 01 VRF: 02

121


Routed Firewall – Server’s Default Gateway is the Bridge Domain on the ACI Fabric

BRKACI-1002

EPG: Servers_Inside


ANP: My-App-01



IP Routing: Yes

L3out

L3out VRF: 01

L3out

VRF: 02



Tenant: Common Servers_Inside can communicate to the “outside world” via the Service


10.1.1.0/30 10.1.2.0/30

192.168.10.x/24


Static route to firewall “inside” subnet via L3out ot Firewall

VRFs peer with Firewallvia L3out

122


Install a L4-7 device once (e.g the ASA firewall) and deploy it multiple times in different logical topologies

The benefits of the service graph are:• Reusable configuration templates• Automatic management of VLAN assignments• Health score collection from the L4-7 device• Statistics collection from the L4-7 device• Automatic ACLs and Pools configuration with endpoint discovery

Service Graph Benefits

BRKACI-1002 123


ADC Device Package Status (as of 09/02/2016)

BRKACI-1002

Device Package Status

Virtual andphysical

Mode FunctionProfile

HA Multi-context on physical appliance Dynamic Routing

Dynamic EPG

IPv6 Feature Operational model

Citrix NetScaler

FCS Yes Go-To(one-arm and two-arm)

Yes No(manual OOB)

YesCreate Virtual instance on SDX manually

Yes Yesmember of pool for VIP

Yes ADC Everything via APIC

F5BIG-IP LTM


Yes Yes YesCreate route-domain on physical LTM automatically or create vCMPmanually (no HA)

No Yesmember of pool for VIP

No ADC Everything via APICor BIG-IQ

F5Big-IQ cloud

Q1CY16 Yes - - - - - - - - -

A10Thunder


No No(manual OOB)

No No No No ADC Everything via APIC

RadwareAlteon

FCS Physical Go-To No No No No No No ADC Everything via APIC

Avi Networks FCS Virtual only

Go-To Yes Yes - No No No ADC Avi controller is required.

124


FW Device Package Status (as of 09/02/2016)

BRKACI-1002

Device Package Status

Virtual andphysical

Mode FunctionProfile

HA Multi-context on physical appliance Dynamic Routing

Dynamic EPG IPv6 Feature Operational model

CiscoASA

FCS Yes Go-ToGo-Through

Yes Yes YesCreate context on ASA5500X manuallyallocate-interface to each context is done by APIC

Yes Yesobject-group for ACE

Yes FW, ACL,NAT

Everythingvia APIC

Palo Alto CA Yes Go-To Yes No No No1HCY16planning

No No FW Panorama is required

CiscoFirePOWER

FCS Oct 2015, in controlled introduction

Yes Go-Through Yes No No - - - IPS Everythingvia APIC

Checkpoint Q2CY16 Yes Go-ToGo-Through

Yes Yes(manual OOB)

Yes No No Yes FW Everythingvia APIC

Fortinet Q2CY16 Yes Go-ToGo-Through

Yes Yes Yes No No Yes FW Everythingvia APIC

125


Three Tier Application

BRKACI-1002

Bridge Domain:Clients

192.168.14.x

Bridge Domain:Web_

192.168.30.x

Bridge Domain:Service_chain_clients_

to_web

EPG:WebServers

Tenant: VMware_AVS

Tenant: Common

EPG:Clients

IP: 192.168.14.254Zone: external

IP: 192.168.100.254Zone: internal

Ext SIP: 192.168.100.2

Int SIP: 192.168.30.254

VM

IP: 192.168.14.11GW: 192.168.14.254

VM

IP: 192.168.30.13GW: 192.168.30.1

VM

IP: 192.168.30.14GW: 192.168.30.1

VM

IP: 192.168.30.15GW: 192.168.30.1

PA-FW

Service_chain_clients_to_web

PA-VM-01

Bridge Domain:Service_chain_web_to_a

pp



Bridge Domain:Application_192.168.40.x

EPG:AppServersService_chain_web_to_app

Ext SIP: 192.168.150.2

Int SIP: 192.168.40.254

VM

IP: 192.168.40.11GW: 192.168.40.1

VM

IP: 192.168.40.12GW: 192.168.40.1

vIP: 192.168.150.150

vIP: 192.168.100.100 I06-vCMP-01 I06-vCMP-02 PA-VM-02

Bridge Domain:Service_chain_app_to_d

b



Bridge Domain:Database_

192.168.50.x

Ext SIP: 192.168.200.2

Int SIP: 192.168.50.254

vIP: 192.168.200.200

I06-vCMP-03

EPG:DBServers

VM

IP: 192.168.50.11GW: 192.168.50.1

VM

IP: 192.168.50.12GW: 192.168.50.1

Service_chain_app_to_db

https://cisco.box.com/s/fn47le5r5um091fynbds43r32kwdcrxf

126

Now That We Have a Better Understanding of ACI, Lets Consider How Customers Can Consume ACI With Automation


Customer Use CasesCredit Services• Multi-Tier application

Deployments• Tenants• VRFs• Bridge Domains• Endpoint Groups• Contracts• Load Balancing (Citrix)• VM creation

Media• Tenants• VRFs• Bridge Domains• Endpoint Groups• Contracts• Switch Interfaces

Banking• VRFs• Bridge Domains• Endpoint Groups• Contracts• Switch Interfaces• VM creation• OS Installation

128


What Should You Look to do First?A. Automate the building of networking infrastructure

B. Automate the consumption of networking resources• Blueprints for Tenants, L2 (EPG/VLAN/VXLAN), L3, L4-7 services• IP Address Management (IPAM)• Summary routes into the fabric• Virtual machine creation• Containers• Application Provisioning• Self service offering

C. Automate both infrastructure and consumption

D. Automate application deployment

BRKACI-1002 129

Take a Step Back, Most Customers Actually Require a Number of Pre-Defined Functional “Blueprints”


Sample Network Blueprints

Clients

ACI Gateway

(not used)

External Routerto WAN

Gateway192.168.10.1

L2 Fabric (external g/w)

Clients

ACI Gateway


L3 Fabric

Clients

ACI Gateway


L3 Fabric with external firewall

131


Sample Network Blueprints

Clients

ACI Internal Gateway


L3 Fabric with firewall on fabric

ACI External Gateway

Clients

ACI Internal Gateway


L3 Fabric with SLB on fabric

ACI External Gateway

SLB

Clients

ACI Gateway


L3 Fabric with firewall and SLB

SLB

132

If We Now Understand The “Why”…

We Next Need To Understand The “How”…


How Many of You....• Are already scripting and automating common tasks?

• In my experience, most of us are not

• Are really good at copy and paste?• That’s me that is!!

BRKACI-1002 135


Congratulations!

136


Being Serious For A Moment• We talk to a lot of partner and customer engineers all over the world

• It is clear that some knowledge of programming concepts is quite valuable these days

• The top question is always “Do I need to learn programming to keep doing my job?”

• I’ve got some good news for you...• In a nutshell, the answer is No....• But only if you learn to consume the easy-to-use tools and processes out there

BRKACI-1002 137

ACI and the API


What is ACI?It is all about the API and Object Model

BRKACI-1002

APIC

APIC

APIC

139


ACI and REST API• REST is fundamental to APIC interaction

• All other tools are built around it

• Understand REST, understand ACI automation

• The second time you need to do something, think about automating it instead!!

BRKACI-1002 140


Using REST• HTTP(S) to the URL or Address of an object• Select an Action to perform (GET, POST etc)• Send the Payload (in XML or JSON format)

BRKACI-1002 141


Common (Free) Tools For The Network EngineerUse these to automate things in ACI

• Postman Plugin for Google Chrome

• API Inspector

• APIC GUI

• COBRA SDK

• Python IDE (Pycharm, Atom, others)

• Git / Github• ARYA• ACI Toolkit• Many Others

BRKACI-1002 142


Different Engineers, Different Tools

APIC CLI

APIC GUI

REST APISDK

BRKACI-1002

Powerful/Complex

Simple/Rigid

143


API Inspector – a REST API Sniffer• Record your GUI interaction as JSON• Modify and replay with tools like Postman

BRKACI-1002 144


Postman Plugin for Google Chrome

BRKACI-1002 145


Python SDK (aka “Cobra”) + ARYA• Full featured access to entire APIC

REST API

• Native ACI language – configure in GUI and turn into Cobra SDK

• Contributors include: Business Unit Engineers, Technical Services Engineers, Advanced Services Engineers

• Complete user use cases all possible

• http://github.com/datacenter/cobra

• http://github.com/datacenter/arya

BRKACI-1002

XML/JSON

arya.py

Python code

{"fvTenant":{"attributes":{"dn":"uni/tn-Cisco","name":"Cisco","rn":"tn-Cisco","status":"created"},"children":[{"fvBD":{"attributes":{"dn":"uni/tn-Cisco/BD-CiscoBd","mac":"00:22:BD:F8:19:FF","name":"CiscoBd","rn":"BD-CiscoBd","status":"created"},"children":[{"fvRsCtx":{"attributes":{"tnFvCtxName":"CiscoNetwork","status":"created,modified"},"children":[]}},{"fvSubnet":{"attributes":{"dn":"uni/tn-Cisco/BD-CiscoBd/subnet-[10.0.0.1/8]","ip":"10.0.0.1/8","rn":"subnet-[10.0.0.1/8]","status":"created"},"children":[]}}]}},{"fvCtx":{"attributes":{"dn":"uni/tn-Cisco/ctx-CiscoNetwork","name":"CiscoNetwork","rn":"ctx-CiscoNetwork","status":"created"},"children":[]}}]}}

fvTenant = cobra.model.fv.Tenant(topMo, name='Cisco')

fvCtx = cobra.model.fv.Ctx(fvTenant, name='CiscoNetwork')

fvBD = cobra.model.fv.BD(fvTenant, mac='00:22:BD:F8:19:FF', name='CiscoBd')

fvRsCtx = cobra.model.fv.RsCtx(fvBD, tnFvCtxName=fvCtx.name)

fvSubnet = cobra.model.fv.Subnet(fvBD, ip='10.0.0.1/8')

146


Practical example of tool usage

BRKACI-1002 147


Cisco on Github

• https://github.com/datacenter

• https://github.com/datacenter/ACI

• https://github.com/datacenter/aci-examples

• https://github.com/datacenter/sparci

• https://github.com/datacenter/acitoolkit

148

Customer Demo

How Should I Get Started with ACI?


Choose Your Management Method(s)

BRKACI-1002 152


Connect the Old to the New

BRKACI-1002

APIC

APIC

APIC

Layer 2 vPC to existing network

Layer 3 (OSPF etc) to existing network

Connect new workloads to the ACI fabric and

route out

Separate “border leafs” shown for clarity

vDS-02vDS-01vDS-01

Separate “border leafs” shown for clarity

153

Key Takeaways


Leaf Profiles(Target Switches)

Leafs_101_and_102

AAEP(Allowed VLANs)

UCS-phys-svrs

Interface PoliciesCDP_enabledLACP_Active

VLAN/VXLAN(Pools)

UCS-phys-svrs

VLAN mgmt(Phy/Out Domain)

UCS-phys-svrs

BRKACI-1002

Understand the Interface Policies


APIC

APIC

APIC



ANP: My_App

EPG: WebDomain: UCS-phys-svrs

Path: vPC_to_UCS_FI_AVLAN_10

Path: vPC_to_UCS_FI_BVLAN_10


Leaf Policy GroupvPC_to_UCS_FI_A

Leaf Policy GroupvPC_to_UCS_FI_B


Leaf ProfilevPC_to_UCS_FI_B

Leaf Profiles

155


Understand the Managed Object Hierarchy

BRKACI-1002

EP EP

EPGEPG

EP EP

Bridge Domain(Flood)

EP EP

EPGEPG

EP EP EP EP

EPGEPG

EP EP

Bridge Domain(Hardware Proxy)

Tenant “Private”

Private Network (VRF)

Private Network (VRF)

Tenant “Common”

Bridge Domain(Hardware Proxy)

Application Network Profile

OutsideOutside

156


Requirements Hardware Proxy no ARP flooding IP Routing Subnet Check

Routed traffic, no silent hosts Yes Yes Yes Yes

Routed traffic, silent hosts Yes ARP flooding (optionalsince Subnet is present) (*)

Yes Yes

non-IP switched traffic, silent hosts No N/A No No

non-IP switched traffic, no silent hosts Yes N/A No No

IP L2 switched traffic, silent hosts Yes ARP flooding (optional if Subnet is present) (*)

Yes (for advancedfunctions and aging)

Yes (for aging and ARP gleaning)

IP L2 switched traffic, no silent hosts Yes no ARP flooding (if hosts send DHCP requests or gratuitous ARP)

Yes (for advancedfunctions and aging)

Yes (for aging and ARP gleaning)

Bridge Domain Options

BRKACI-1002

(*) if the Subnet is configured ACI can do ARP gleaning so ARP flooding is not strictly needed

157


1. You must have at least one Tenant or use the Common Tenant

2. VRFs are constrained within Tenants

3. VRFs provide external L3 connectivity (with a contract)

4. You must have at least one Bridge Domain

5. Bridge Domains determine the L2 forwarding characteristics

6. Bridge Domains provide internal L3 connectivity (default gateways)

7. Bridge Domains to outside VLANs must be mapped 1:1

8. Endpoint Groups map to a single Bridge Domain

9. Endpoint Groups are security zones where communication is allowed

10. Communication between Endpoint Groups is allowed through contracts (ACLs)

11. Endpoint Groups must be bound to a virtual, physical, or outside domain

12. Endpoint Groups allow you to mix and match VLANs/VXLANs/interfaces (access, port channel, virtual port channel)

13. Endpoints can only be a member of a single Endpoint Group

14. AAEP’s allow VLANs on interfaces or VMM domains

ACI Networking Rules!

BRKACI-1002 158


Cisco Spark Ask Questions, Get Answers, Continue the Experience

Use Cisco Spark to communicate with the Speaker and fellow participants after the session

Download the Cisco Spark app from iTunes or Google Play

1. Go to the Cisco Live Melbourne 2017 Mobile app 2. Find this session3. Click the Spark button under Speakers in the session description 4. Enter the room, room name = BRKACI-10025. Join the conversation!

The Spark Room will be open for 2 weeks after Cisco Live

160BRKACI-1002


Other Sessions of Interest• BRKACI-2603 – ACI Operation and Troubleshooting

• BRKACI-2016 – ACI L4-7 Integration

• BRKACI-3502 – ACI Multisite Deployment

• BRKACI-2004 – How to Setup an ACI Fabric from Scratch• LABDC-1011 – ACI with VMware Integration

BRKACI-1002 161


Complete Your Online Session Evaluation

162BRKACI-1002

Learn online with Cisco Live! Visit us online after the conference for full access to session videos and presentations. www.CiscoLiveAPAC.com

Give us your feedback and receive a Cisco Live 2017 Cap by completing the overall event evaluation and 5 session evaluations.

All evaluations can be completed via the Cisco Live Mobile App.

Caps can be collected Friday 10 March at Registration.

Thank you

My Favourite Show Commands


• fabric <#> show system internal epm vlan all ßalways use this command first

• fabric <#> show interface vlan <#>

• fabric <#> show vlan brief

• fabric <#> show vlan extended

• fabric <#> show interface trunk

• fabric <#> show interface ethernet <#/#>

• fabric <#> show port-channel summary

• fabric <#> show cdp neighbors

• fabric <#> show lldp neighbors

Layer 2 Commands

BRKACI-1002 166



• show endpoints vpc context <#> <#> interface vpc <#>

Layer 2 Commands

BRKACI-1002 167



• fabric <#> show ip interface brief

• fabric <#> show ip interface brief vrf <tenant>:<vrf>

• fabric <#> show ip route vrf <tenant>:<vrf>

• fabric <#> show ip route vrf <tenant>:<vrf> <route>

• fabric <#> show ip route ospf vrf <tenant>:<vrf>

• fabric <#> show ip ospf neighbors vrf <tenant>:<vrf>

• fabric <#> show ip ospf neighbors detail vrf <tenant>:<vrf>

• fabric <#> show bgp ipv4 unicast vrf <tenant>:<vrf>

L3 Commands

BRKACI-1002 168



• fabric <#> show ip igmp interface brief vrf <tenant>:<vrf>

• fabric <#> show ip igmp group vrf <tenant>:<vrf>

• fabric <#> show ip mroute vrf <tenant>:<vrf>

• fabric <#> show ip pim vrf <tenant>:<vrf>

• fabric 101 show ip pim neighbor vrf Production:VRF-01

Multicast Commands

BRKACI-1002 169


• show running-config leaf <#> interface ethernet <#/#>

• show running-config template policy-group <#>

• show running-config template port-channel <#>

• show running-config leaf-interface-profile <#>

• show running-config leaf-profile <#>

• show running-config leaf <#> vrf context tenant <#> vrf <#>

• show running-config leaf <#> router ospf

Show Run Commands

BRKACI-1002 170


• show running-config tenant <#> vrf context <#>

• show running-config tenant <#> interface bridge-domain <#>

• show running-config tenant <#> external-l3

• show running-config tenant <#> application <#>

• show running-config tenant <#> application <#> epg <#>

Show Run Tenant Commands

BRKACI-1002 171


• show tenant <#> detail

• show tenant <#> vrf <#> detail

• show tenant <#> bridge-domain <#> detail

• show tenant <#> epg <#> detail

• show tenant <#> contract <#>

• show tenant <#> access-list <#>

Show Tenant Commands

BRKACI-1002 172


li08-apic-svr-01# sh run leaf 101 interface e 1/15

leaf 101

interface ethernet 1/15

# Policy-group configured from leaf-profile ['Leaf_101'], leaf-interface-profile li07_101_to_Spirent_Test_Center

# policy-group 10G_acc_Spirent_Test_Center

switchport trunk allowed vlan 10 tenant Production application ANP-01 epg vlan-10__10.161.10.x_24

exit

exitl

i08-apic-svr-01#

How To Find What EPG Is On An Interface

BRKACI-1002 173


How To Find All Interfaces For An EPG

li08-apic-svr-01# show epg vlan-18__10.181.18.x_24 detail

[snip]

Static Paths:

Encap: (P):Primary VLAN, (S):Secondary VLAN

Node Interface Encap

---------- ------------------------------ -------------------------

101 eth1/30 unknown(P),vlan-18(S)

101 102 vpc 10G_vPC_esx_li07-c220m4-02 unknown(P),vlan-18(S)

103 104 vpc 10G_vPC_esx_li07-c220m4-01 unknown(P),vlan-18(S)

[snip]

Untagged EPG

BRKACI-1002 174


• moquery -c fvLocale | grep dn | grep <epg name> - finds which node an epg is applied

• moquery -c fvIfConn | grep dn | grep vlan-<#> - finds where a VLAN has been applied

Advanced Commands

BRKACI-1002 175


li08-apic-svr-01# configure

tenant <#>

application <#>

epg <#>

bridge-domain member <#>

contract consumer <#>

contract provider <#>

exit

exit

exit

Configure: Tenant, Application, EPG

BRKACI-1002 176

cisco live 2017 cap

Documents