brkdct 2049 otv

BRKDCT-2049

Natale Ruello – [email protected] Nexus 7000 and NX-OS TME

Overlay Transport Virtualization

© 2011 Cisco and/or its affiliates. All rights reserved. Cisco Public BRKDCT-2049 2

Agenda

Distributed Data Centers: Goals and Challenges

OTV Architecture Principles

Control Plane and Data Plane

Failure Isolation

Multi-homing

Mobility

L2 Multicast Forwarding

QoS

Path Optimization

OTV at the Aggregation Layer


Agenda





Distributed Data Centers Building the Data Center Cloud

Seamless workload mobility between multiple datacenters

Distributed applications closer to end users

Pool and maximize global compute resources

Ensure business continuity with workload mobility and distributed deployments

Distributed Data Center Goals:


Traditional Layer 2 VPNs

EoMPLS

VPLS

Dark Fiber


Flooding Behavior

x2

Site A

Site B

Site C

MAC 1

propagation MAC 1

Traditional Layer 2 VPN technologies rely on flooding to propagate

MAC reachability

The flooding behavior causes failures to propagate to every site in the

Layer 2 VPN

Our goal…

providing layer 2 connectivity, yet restrict the reach of the unknown unicast

flooding domain in order to contain failures and preserve the resiliency


Pseudo-Wires Maintenance

Before any learning can happen a full mesh of pseudo-wires/

tunnels must be in place

For N sites, there will be N*(N-1)/2 pseudo-wires. Complex to add

and remove sites

Head-end replication for multicast and broadcast. Sub-optimal

BW utilization

Our goal… providing point-to-cloud provisioning and optimal bandwidth

utilization in order to reduce cost


Multi-homing

L2 Site L2 Site L2 VPN

Active Active

Require additional protocols to support Multi-homing

STP is often extended across the sites of the Layer 2 VPN. Very

difficult to manage as the number of sites grows

Malfunctions on one site will likely impact all sites on the VPN

Our goal… natively providing automatic detection of multi-homing without the

need of extending the STP domains, together with a more efficient load-

balancing


Overlay Transport Virtualization (OTV)

O

V

Overlay - A solution that is independent of the

infrastructure technology and services, flexible

over various inter-connect facilities

Transport - Transporting services for layer 2

and layer 3 Ethernet and IP traffic

Virtualization - Provides virtual stateless multi-

access connections, which are virtualized and

partitioned into VPNs, VRFs, VLANs

T

OTV delivers Layer 2 extensions over any type of transport infrastructure


OTV changes the game…

Flooding Based Learning Control-Plane Based Learning

Move to a Control Plane protocol that proactively advertises MAC addresses and their reachability instead of the current flooding mechanism

Pseudo-wires and Tunnels Dynamic Encapsulation

Not require static tunnel or pseudo-wire configuration

Offer optimal replication of traffic done closer to the destination, which translates into much more efficient bandwidth utilization in the core

Complex Dual-homing Native Automated Multi-homing

Allow load balancing of flows within a single VLAN across the active devices in the same site, while preserving the independence of the sites. STP confined within the site (each site with its own STP Root bridge)


Agenda




Failure Isolation

Multi-homing

Mobility


QoS

Path Optimization



Overlay Transport Virtualization

OTV is a “MAC in IP” technique to extend Layer 2 domains

OVER ANY TRANSPORT

Technology Pillars

Protocol Learning

Built-in Loop Prevention

Preserve Failure

Boundary

Site Independence

Automated Multi-homing

Dynamic Encapsulation

No Pseudo-Wire State

Maintenance

Optimal Multicast

Replication

Multipoint Connectivity

Point-to-Cloud Model

First platform to support OTV!

Nexus 7000


L2

L3

Transport Infrastructure*

OTV OTV

Terminology: “Edge Device” The Edge Device is responsible for performing all the OTV functionality

The Edge Device can be located at the Aggregation Layer as well as at the Core Layer depending on the network topology of the site

A given site can have multiple OTV Edge Devices (multi-homing)

* It can be owned by the Enterprise

or by the Service Provider

OTV Edge Device OTV Edge Device


L2

L3

Transport Infrastructure

OTV OTV

Terminology: “Internal Interfaces” The Internal Interfaces are those interfaces of the Edge Devices that face

the site and carry at least one of the VLANs extended through OTV

Internal Interfaces behave as regular layer 2 interfaces. No OTV configuration is needed on the OTV Internal Interfaces

OTV Internal Interface =

OTV Internal

Interfaces

OTV Internal

Interfaces


L2

L3


OTV OTV

Terminology: “Join Interface” The Join interface is one of the uplink interfaces of the Edge Device

The Join Interface is a point-to-point routed interface and it can be a single physical interface as well as a port-channel (higher resiliency)

The Join Interface is used to physically “join” the Overlay network

OTV Join Interface OTV Join Interface


Terminology: “Overlay Interface”

The Overlay Interface is a new virtual interface where all the OTV configuration is placed

It’s a logical multi-access multicast-capable interface

The Overlay Interface encapsulates the site Layer 2 frames in IP unicast or multicast packets that are then sent to the other sites

L2

L3


OTV OTV Overlay Interface Overlay Interface


OTV Data Plane: Intra-Site Packet Flow

OTV OTV OTV OTV

MAC TABLE

VLAN MAC IF

100 MAC 1 Eth 2

100 MAC 2 Eth 1

Layer 2

Lookup

2

West

Site

MAC 1 East

Site

MAC 2

MAC 1 MAC 2

Transport

Infrastructure

1


Transport

Infrastructure

OTV Data Plane: Inter-Site Packet Flow

OTV OTV OTV OTV

MAC TABLE

VLAN MAC IF

100 MAC 1 Eth 2

100 MAC 2 Eth 1

100 MAC 3 IP B

100 MAC 4 IP B

MAC 1 MAC 3

IP A IP B MAC 1 MAC 3

MAC TABLE

VLAN MAC IF

100 MAC 1 IP A

100 MAC 2 IP A

100 MAC 3 Eth 3

100 MAC 4 Eth 4

Layer 2

Lookup

6

IP A IP B MAC 1 MAC 3 MAC 1 MAC 3 Layer 2

Lookup

2 Encap

3

Decap

5

MAC 1 MAC 3 West

Site MAC 1

MAC 3 East

Site

4

7

IP A IP B

1


20B + 8B + 14B* = 42Byte

of total overhead

OTV encapsulation adds 42 Bytes to the packet IP MTU size

Outer IP Header and OTV Shim Header in addition to original L2 Header stripped off of the .1Q header

The outer OTV shim header contains information about the overlay (VLAN, overlay number)

The 802.1Q header is removed from the original frame and the VLAN field copied over into the OTV shim header

6B 6B 2B 20B 8B

DMAC SMAC Ether

Type IP Header

Payload 4B

CRC OTV Shim

802.1Q DMAC SMAC

Ether

Type

802.1Q

OTV Data Plane Encapsulation

14B*

Original L2 Frame

L2

Header

802.1Q header removed

* The 4Bytes of .1Q header have

already been removed


West

OTV

Building the MAC tables The OTV Control Plane

OTV proactively advertises MAC reachability (control-plane learning)

MAC addresses advertised in the background once OTV has been configured

No specific configuration is required

IS-IS is the OTV Control Protocol running between the Edge Devices. No need to learn how IS-IS works

IP A IP B

IP C

East

South

MAC Addresses

Advertisements OTV

OTV


OTV Control Plane Neighbor Discovery and Adjacency Formation

Before any MAC address can be advertised the OTV Edge Devices must:

Discover each other

Build a neighbor relationship with each other

The neighbor relationship can be built over a transport infrastructure, that can be:

multicast-enabled

unicast-only

Technology Benefit: OTV can leverage any networking capability provided by the transport infrastructure (multicast, fast-reroute, ECMP, etc.)


OTV Control Plane Neighbor Discovery (over Multicast Transport)

The end result

Adjacencies are maintained over the multicast group

A single update reaches all neighbors

The mechanism

Edge Devices (EDs) join an multicast group in the transport, as they were hosts (no PIM on EDs)

OTV hellos and updates are encapsulated in the multicast group

West

OTV OTV Control Plane

IP A East

OTV

OTV Control Plane

IP B

Multicast-enable

Transport


West

OTV

South

East

OTV

OTV

OTV Control Plane

OTV Control Plane OTV Control Plane

IP A IP B

IP C

Encap Decap

Decap


OTV Hello

OTV Hello

OTV Hello

IGMP Join G

IGMP Join G

IGMP Join G Multicast state for group G

established throughout transport

Transport natively replicates

multicast to all OIFs

All edge devices join

OTV control-group G

1

2

3

4

5

6

6

7

7

IP A G OTV Hello

IP A G OTV Hello

IP A G OTV Hello

OTV Hello IP A G OTV Hello

OTV Hello IP A G OTV Hello

Neighbor IP Addr

West IP A

Neighbor IP Addr

West IP A

Neighbor IP Addr

Multicast-enabled

Transport


South

East West

OTV

OTV

OTV

OTV Control Plane

OTV Control Plane OTV Control Plane

IP A IP B

IP C

Decap Decap

Encap


OTV Hello

IP C G OTV Hello

IP C G OTV Hello IP C G OTV Hello

OTV Hello OTV Hello

OTV Hello IP C G OTV Hello OTV Hello IP C G OTV Hello

Neighbor IP Addr

West IP A

Neighbor IP Addr

West IP A

South IP C

Neighbor IP Addr

South IP C

1

2

3

4 4

5 5

Bidirectional

adjacency formed

The South Site creates its

hello with West’s address

in the TLV

Multicast-enabled

Transport


South

East West

OTV

OTV

OTV

VLAN MAC IF

100 MAC A IP A

100 MAC B IP A

100 MAC C IP A

VLAN MAC IF

100 MAC A IP A

100 MAC B IP A

100 MAC C IP A

VLAN MAC IF

100 MAC A e1/1

100 MAC B e1/1

100 MAC C e1/1

VLAN MAC IF

100 MAC A e1/1

100 MAC B e1/1

100 MAC C e1/1

OTV Control Plane Route (MAC) Advertisements (over Multicast Transport)

Update A

VLAN MAC IF

100 MAC A IP A

100 MAC B IP A

100 MAC C IP A

VLAN MAC IF

100 MAC A IP A

100 MAC B IP A

100 MAC C IP A

New MACs

learned in

OTV VLAN

Craft OTV

update with

new MACs

IP A G Update A

Update A

Update A

IP A G Update A

IP A G Update A

Update A IP A G Update A

Update A IP A G Update A

Encap Decap

Decap 1

2

3

4

5

5

6

6

VLAN MAC IF

100 MAC A IP A

100 MAC B IP A

100 MAC C IP A

7

7

Add MACs

learned

through OTV

Add MACs

learned

through OTV

MAC Table

MAC Table

MAC Table

Multicast-enabled

Transport


OTV Control Plane Route (MAC) Advertisements

Every time an Edge Device learns a new MAC address, the OTV control plane will advertise it together with its associated VLAN IDs and IP next hop

The IP next hops are the addresses of Edge Devices’ join-interfaces*

A single OTV update can contain multiple MAC addresses for different VLANs

A single update reaches all OTV Edge Devices with the same mechanism used for the neighbor discovery

* In a future release the “IP next hops” can be loopback addresses


OTV can leverage the benefits of a multicast-enabled transport for both control and data planes

The following summarizes the requirements for a multicast transport:

Control group – Single PIM-SM or PIM-Bidir group used to form adjacencies and exchange MAC reachability information

Data groups – … discussed later in this presentation

Multicast Groups in the Core Summary


OTV

A Packet Walk is Worth a Million Words Establishing OTV Unicast Communication

OTV

MAC 1

ED1

ED2

MAC 2

(MAC 1, e1/1)

(MAC 1, IP A)

( MAC 2, IP B )

1

2

3cp

4cp

3dp

4dp

MAC 2

5 7cp

8cp

6

7dp

8dp

1 – Server 1 sends a broadcast ARP for MAC 2

2 – ARP broadcast is received by ED1, which

learns MAC 1 on its internal interface

3cp – ED1 advertises MAC 1 in an OTV Update sent

via the multicast control group

4cp – ED2 receives the update and stores MAC1 in

MAC table, next-hop is ED1

3dp – ED1 encapsulates broadcast in the core IP

multicast group so all the EDs in the overlay

receive it

4dp – ED2 decapsulates the frame and forwards the

ARP broadcast request into the site

5 – Server 2 receives the ARP and replies with a

unicast ARP reply to MAC 1

6 – ED2 learns MAC 2 on its internal interface

7cp – ED2 advertises MAC 2 in IS-IS LSP sent via

the multicast control group

8cp – ED1 receives the update and stores MAC2 in

MAC table, next-hop is ED2

7dp – ED2 knows that MAC 1 is reachable via IP A

so encapsulates the packet and sends it unicast

to ED1’s IP address (IP A)

8dp – Core delivers packet to ED1, ED1

decapsulates and forwards it into the site to

MAC 1

Control Plane Data Plane

IPA

IPB


OTV Control Plane CLI Verification

dc1-agg-7k1# show otv adjacency

Overlay Adjacency database

Overlay-Interface Overlay100 :

Hostname System-ID Dest Addr Up Time

Adj-State

dc2-agg-7k1 001b.54c2.efc2 20.11.23.2 15:08:53 UP

dc1-agg-7k2 001b.54c2.e1c3 20.12.23.2 15:43:27 UP

dc2-agg-7k2 001b.54c2.e142 20.22.23.2 14:49:11 UP

dc1-agg-7k1# show otv route

OTV Unicast MAC Routing Table For Overlay100

VLAN MAC-Address Metric Uptime Owner Next-hop(s)

---- -------------- ------ -------- --------- -----------

2001 0000.0c07.ac01 1 3d15h site Ethernet1/1

2001 0000.1641.d70e 1 3d15h site Ethernet1/2

2001 0000.49f3.88ff 42 2d22h overlay dc2-agg-7k1

2001 0000.49f3.8900 42 2d22h overlay dc2-agg-7k2

Establishment of control plane adjacencies between

OTV Edge Devices:

Unicast MAC reachability information:

Remote Site

MAC

Local Site

MAC


West

OTV

Configuration OTV over a Multicast Transport

Minimal configuration required to get OTV up and running

IP A IP B

IP C

East

South

OTV

OTV

feature otv

otv site-vlan 99

interface Overlay1

description WEST-DC

otv join-interface e1/1

otv control-group 239.1.1.1

otv data-group 232.192.1.0/24

otv extend-vlan 100-150

feature otv

otv site-vlan 99

interface Overlay1

description EAST-DC

otv join-interface e1/1.10




feature otv

otv site-vlan 99

interface Overlay1

description SOUTH-DC

otv join-interface Po16





OTV Control Plane Neighbor Discovery (Unicast-Only Transport)

The end result

Neighbor Discovery is automated by the “Adjacency Server”

All signaling must be replicated for each neighbor

Data traffic must also be replicated at the head-end

The mechanism

Edge Devices (EDs) register with an “Adjacency Server” ED

EDs receive a full list of Neighbors (oNL) from the AS

OTV hellos and updates are encapsulated in IP and unicast to each neighbor

West

OTV OTV Control Plane

IP A East

OTV

OTV Control Plane

IP B

Unicast-only

Transport

Target

2QCY11

Ideal for connecting two or three sites

With a higher number of sites a multicast transport is the best choice



OTV “adjacency server” provides support over unicast core

Adjacency server is “just an OTV edge device” (i.e. Nexus 7000) Not a separate server or other device

Advertises IP of each Edge Device (ED) to all other EDs (OTV neighbor list – oNL)

IP A

Site 1

Site 2 Site 3

Site 4 Site 5

Unicast-Only

Transport

IP B IP C

IP D IP E Adjacency

Server Mode

oNL

Site 1, IP A

Site 2, IP B

Site 3, IP C

Site 4, IP D

Site 5, IP E

* A redundant pair may be configured

Target

2QCY11


Unicast-Only

Transport

East

South

OTV

OTV

Control Plane

OTV

Control Plane

OTV

Control Plane

OTV OTV

IP A IP B

IP C


West Encap

3

OTV Hello

1

The West Site sends

a “hello”

oNL

South , IP C

East , IP B

2 Head-End

Replication

OTV Hello IP A IP C OTV Hello

IP A IP B OTV Hello

Target

2QCY11


OTV Control Plane MAC Advertisements (Unicast-Only Transport)

A single update needs to be created for each destination Edge Device present on the Overlay

Same for the sites’ multicast and broadcast packets to be sent to the other sites

Core

IP A

West

East

3 New MACs are

learned on VLAN 100

Vlan 100 MAC A

Vlan 100 MAC B

Vlan 100 MAC C

South-East

VLAN MAC IF

100 MAC A IP A

100 MAC B IP A

100 MAC C IP A

4

OTV update is replicated

at the head-end

3

3

2

VLAN MAC IF

100 MAC A IP A

100 MAC B IP A

100 MAC C IP A

4

3 New MACs are

learned on VLAN 100

1

oNL

East, IP B

Sout-East, IP C

Target

2QCY11


West

OTV

Configuration OTV over an unicast-only transport

Establishing a DCI has never been this simple

IP A IP B

IP C

East

South

OTV

OTV

feature otv

otv site-vlan 99

interface Overlay1

description WEST-DC


otv adjacency-server local


feature otv

otv site-vlan 99

interface Overlay1

description EAST-DC

otv join-interface e1/1.10

otv adjacency-server 10.1.1.1


feature otv

otv site-vlan 99

interface Overlay1

description SOUTH-DC

otv join-interface Po16

otv adjacency-server 10.1.1.1


Target

2QCY11


Agenda




Failure Isolation

Multi-homing

Mobility


QoS

Path Optimization



L2

L3

OTV OTV

Spanning Tree and OTV Site Independence

OTV is site transparent: no changes to the STP topology

Each site keeps its own STP domain

This functionality is built-in into OTV and no additional configuration is required

An Edge Device will send and receive BPDUs ONLY on the OTV Internal Interfaces

The BPDUs

stop here

The BPDUs

stop here


L2

L3

OTV OTV

Unknown Unicast and OTV No longer unknown unicast storms across the DCI

No requirements to forward unknown unicast frames

OTV does not forward unknown unicast frames to the overlay. This is achieved without any additional configuration

The assumption here is that the end-points connected to the network are not silent or uni-directional

MAC TABLE

VLAN MAC IF

100 MAC 1 Eth1

100 MAC 2 IP B

- - -

MAC 1 MAC 3

No MAC 3 in the

MAC Table


Controlling ARP traffic ARP Neighbor-Discovery (ND) Cache

An ARP cache is maintained by every OTV edge device and is populated by snooping ARP replies

Initial ARP requests are broadcasted to all sites, but subsequent ARP requests are suppressed at the Edge Device and answered locally

ARP traffic spanning multiple sites can thus be significantly reduced

Transport

Network

OTV

OTV

ARP Cache

MAC 1 IP A

MAC 2 IP B

ARP reply

2

First ARP

request (IP A)

1

Snoop & cache ARP reply

3

Subsequent ARP requests

(IP A)

4 ARP reply on behalf of

remote server (IP A)

5


Agenda




Failure Isolation

Multi-homing

Mobility


QoS

Path Optimization



OTV Automated Multi-homing

The detection of the multi-homing is fully automated and it does not require additional protocols and configuration

The Edge Devices within a site discover each other over the “otv site-vlan”

OTV elects one of the Edge Devices to be the Authoritative Edge Device (AED) for a subset of the extended VLANs

The AED is responsible for:

MAC addresses advertisement for its VLANs

Forwarding its VLANs’ traffic inside and outside the site

OTV OTV

Internal peering for

AED election

AED


OTV and Multi-homing VLAN Splitting between Edge Devices

VLANs are split between the OTV Edge Devices belonging to the same site

Achieved via a very deterministic algorithm (not configurable). In a dual-homed site:

Lower IS-IS System-ID (Ordinal 0) = EVEN VLANs

Higher IS-IS System-ID (Ordinal 1) = ODD VLANs

Future functionality will allow to tune the behavior

OTV-ED# show otv site

Site Adjacency Information (Site-VLAN: 1999)

(* - this device)

Overlay100 Site-Local Adjacencies (Count: 2)

Hostname System-ID Ordinal

---------------- ---------------- -------

dc2a-agg-7k2-otv 001b.54c2.e142 0

* dc2a-agg-7k1-otv 0022.5579.0f42 1

OTV OTV

Internal peering for

AED election

AED

ODD VLANs

AED

EVEN VLANs


OTV Automated Multi-homing AED and Broadcast Handling

Broadcast reaches all the Edge Devices within the site

Only the AED for that VLAN forwards the traffic to the overlay

All the Edge Devices at the other sites receive the broadcast

Only the AED at the remote sites will forward the packet from the overlay into the site

Core

OTV

OTV

OTV

AED AED

Bcast

pkt

Broadcast

stops here

Broadcast

stops here

OTV


OTV Automated Multi-homing vPC Active-Active OTV Edge Devices

Within a single VLAN different flows can reach different OTV Edge Devices (vPC peers)

The traffic will go over the vPC peer-link if it reaches the non-AED vPC peer

Future Release: Both Edge Devices advertise MAC addresses and forward traffic for all the extended VLANs. No need to go over the vPC peer-link

Core

OTV

OTV

AED

AED

MAC TABLE

VLAN MAC IF

100 MAC 1 IP A

IP B IP A

IP B

OTV

OTV

vPC


Agenda




Failure Isolation

Multi-homing

Mobility


QoS

Path Optimization



OTV wins Best of VMworld 2010 Gold Award

We are thrilled to announce that Cisco Nexus 7000 won the prestigious

Best of VMworld 2010 award in the Hardware for Virtualization category

for Overlay Transport Protocol (OTV).

The panel of judges for the Best of VMworld 2010 awards was a mix of industry experts,

IT consultants and TechTarget editors. 200 entrants were scored on innovation, the value

provided by the product, performance, reliability and ease of use.

What the judges said: “Cisco Nexus 7000 Overlay Transport Virtualization lets you extend data

networks across data centers, which has tremendous benefits for multi-site disaster recovery.”

Go here for more information about the award

Best of VMworld 2010 Awards

http://searchservervirtualization.techtarget.com/news/article/0,289142,sid94_gci1519569,00.html


OTV Use Case: Vmotion Live migration of VMs from one data center to another

Data Center A Data Center B

OTV Ethernet Extension

Any Transport

LD VMotion

“Moving workloads between data centers has typically involved complex and time-consuming network design and configurations. VMware VMotion™ can now leverage Cisco OTV to easily and cost-effectively move data center workloads across long distances, providing customers with resource flexibility and workload portability that span across geographically dispersed data centers.”

“This represents a significant advancement for virtualized environments by simplifying and accelerating long-distance workload migrations.”

Ben Matheson, senior director, global partner marketing, VMware.


OTV and Long Distance Live Vmotion

Cisco and VMWare have performed jointly tests with NetApp and EMC to validate OTV for Long Distance Live Vmotion

These tests are described here:

Cisco, VMware and Netapp: http://www.cisco.com/en/US/prod/collateral/switches/ps9441/ps9402/white_paper_c11-591960.pdf

Cisco, VMware and EMC: http://media.vceportal.com/documents/WhitePaper_Application_Mobility.pdf

More testing with Microsoft Hyper-V and other virtualization technologies are in progress

http://www.cisco.com/en/US/prod/collateral/switches/ps9441/ps9402/white_paper_c11-591960.pdf




http://media.vceportal.com/documents/WhitePaper_Application_Mobility.pdf

http://media.vceportal.com/documents/WhitePaper_Application_Mobility.pdf


OTV and MAC Mobility

OTV

AED

AED

OTV

OTV

OTV

MAC X

MAC X

MAC X

VM Moves

MAC X

OTV

MAC X

MAC X

AED

OTV West

West East

OTV

OTV East

1

Server originates a

Gratuitous ARP

(GARP) frame

AED advertises MAC X

with a metric of zero MAC X

AED detects

MAC X is now

local

MAC X

MAC X

MAC X

ESX

MAC X

ESX

ESX ESX

MAC X

MAC X

2.3

2.2 2.1


OTV

AED

OTV West

AED

MAC X

AED

OTV

OTV East

MAC X

AED in site East

forwards the

GARP broadcast

frame across the

overlay

AED

MAC X

MAC X

MAC X ESX

AED in site West

forwards the GARP

into the site and the

L2 switches update

their CAM tables

ESX

MAC X

MAC X

OTV and MAC Mobility

MAC X

AED

OTV

OTV West

AED

OTV

OTV

MAC X

MAC X

East MAC X

MAC X

ESX

ESX

MAC X

MAC X

EDs in site West see MAC X advertisement with a

better metric from site East and change them to

remote MAC address.

2.4

3.1

3.2

MAC X


Agenda




Failure Isolation

Multi-homing

Mobility


QoS

Path Optimization



Multicast Traffic For Extended VLANs

OTV can leverage the multicast support available in the transport network to optimize the delivery of the multicast traffic for the VLANs stretched across sites.

Three steps:

1. Automated mapping of the sites’ multicast groups to a range of multicast groups in the transport network

2. Creation of the Multicast state information at the OTV Edge Devices

3. Sites’ Multicast traffic delivered over the Overlay


S1

OTV OTV

IP A IP B

West East

Mcast Stream 1

1. The Mcast source starts sending traffic to the group Gs1

2. The West ED maps (S1,Gs1) to a delivery group Gd1 (from the SSM group in the core)

3. The West ED communicates the mapping information (including the source VLAN) to the other EDs

4. Same process happens once source S2 is enabled (sending to a different group Gs2)

The site multicast groups are mapped to a SSM group range in the core

Each (Si,Gsi) maps to a different SSM group in round-robin fashion

S1 Gs1

IP C

South

OTV

The Mapping is communicated to

the other EDs 3

Mapping to a Delivery Group

2

Multicast-enabled

Transport

Multicast Traffic Step 1 – Mapping of the Site Multicast Groups

Mcast Group Mapping

Site Group Core Group

Gs1 Gd1

S2

S2 Gs2

4

Mcast Group Mapping

Site Group Core Group

Gs1 Gd1

Gs2 Gd2


OTV

Receiver

(for GS1)

OTV

IP A

IP B

West East

OIL-List

Group IF

Gs1 Gd1 Overlay

Client IGMP snoop

2

Client IGMP report to join

Gs1

1

IGMPv3 report to join (IP A, Gd1) , the SSM group in

the Core.

3.2

Receive GM-Update Update OIL

4

SSM Tree for Gd

From Right to Left

1. A multicast receiver on the East site sends an IGMP report to join the multicast group Gs1

2. The Edge Device (ED) snoops these IGMP reports (without forwarding it)

3. Upon snooping the IGMP reports, the ED performs two actions:

1. Announces the receivers in a Group-Membership Update (GM-Update) to all other EDs

2. Sends an IGMPv3 report to join the (IP A, Gd1) SSM group in the core

4. The source ED adds the Overlay interface to the Outbound Interface List (OIL). An SSM tree for Gd1 (rooted at the source ED) is built in the core

5. The same process happens if a second receiver requests to join a different group Gs2

A different SSM tree for Gd2 is built in the core (still rooted on the same source ED)

It is important to clarify that the edge devices join the core multicast groups as hosts, not as routers!

GM-Update 3.1

Multicast-enabled Transport

Multicast Traffic Step 2 – Multicast State Creation

S1

S1 Gs1

S2

S2 Gs2 Receiver

(for GS2)

OIL-List

Group IF

Gs1 Gd1 Overlay

Gs2 Gd2 Overlay


Receiver

(for Gs2)

Receiver

(for Gs1)

OTV

IP A

IP B

West East IP C

Receiver

(for Gs1)

South

OTV

Encap

2

IP A Gd1 s1 Gs1

Transport

Replication

3

IP A Gd1 S1 Gs1 IP A Gd1 S1 Gs1

4

4

IP A Gd1 S1 Gs1

S1 Gs1

S1 Gs1

Decap 5

Decap 5

Multicast-enabled

Transport

Multicast Traffic Step 3 – Multicast Packet Flow

S1

S1 Gs1

S2

OTV

OIF-List

Group IF

Gs1 Gd1 Overlay Lookup

1


Receiver

(for Gs1)

Receiver

(for Gs2)

OTV

IP A

IP B

West East IP C

Receiver

(for Gs1)

South

OTV

Encap

2

IP A Gd2 s2 Gs2

Transport

Replication

3

IP A Gd2 S2 Gs2 IP A Gd2 S2 Gs2

4

S2 Gs2

Decap 5

Multicast-enabled

Transport

Multicast Traffic Step 3 – Multicast Packet Flow

S1

S2

S2 Gs2

OTV

OIL-List

Group IF

Gs1 Gd1 Overlay

Gs2 Gd2 Overlay Lookup

1


OTV can leverage the benefits of a multicast-enabled transport for both control and data planes

The following summarizes the requirements for a multicast transport:

Control group – Single PIM-SM or PIM-Bidir group used to form adjacencies and exchange MAC reachability information

Data groups – Range of SSM groups used to carry multicast data traffic generated by the sites

The right number of SSM groups to be used depends on a tradeoff between the amount of multicast state to be maintained in the core and the optimization of Layer 2 multicast traffic delivery

Multicast Groups in the Core Summary

interface Overlay1






Agenda




Failure Isolation

Multi-homing

Mobility


QoS

Path Optimization



Control Plane Traffic

Statically marked by default with CoS = 6/DSCP = 48

QoS and OTV Traffic Marking (Default Behavior)

OTV

Overlay

OTV

Outer

DSCP

IP ETHER

TYPE DMAC SMAC

Encap

= 48

Inbound Data Plane Traffic (Decap Side)

The outer DSCP value is used to populate the CoS field

The new CoS could differ from the original CoS

802.1Q Overlay

802.1Q ETHER

TYPE

ETHERTYPE

0x8100

CoS

802.1p CFI VLANID

DMAC SMAC IP

Decap OTV

OTV

Outer

DSCP

IP ETHER

TYPE DMAC SMAC

Outbound Data Plane Traffic (Encap Side)

The CoS value is copied to the “outer” DSCP field

802.1Q

Overlay

Outer

DSCP

802.1Q ETHER

TYPE

ETHERTYPE

0x8100

CoS

802.1p CFI VLANID

DMAC SMAC IP IP ETHER

TYPE DMAC SMAC

Encap

OTV

OTV


OTV and QoS for IP Traffic Modifying Outbound DSCP Mapping

CoS DSCP

0-7 56

802.1Q Overlay

It may be desirable to modify the DSCP marking of the OTV encapsulated frames

For that, it is possible to configure Cos-to-DSCP maps on traffic belonging to the OTV extended VLANs

802.1Q ETHER

TYPE IP OTV

ETHERTYPE

0x8100

CoS

802.1p CFI VLANID

DMAC SMAC DMAC SMAC

Outer

DSCP

IP

class a

match cos 0-7

!

policy-map otv-cos-2-dscp

class a

set dscp 56

!applying to extended VLANs

vlan x-y

service-policy input otv-cos-2-dscp

Encap

OTV


Agenda




Failure Isolation

Multi-homing

Mobility


QoS

Path Optimization



Path Optimization Optimal Routing Challenges

Server-Server Egress Routing Localization:

Server-Client

Egress Routing Localization:

Server-Client

Layer 2 extensions represent a challenge for optimal routing

Challenging placement of gateway and advertisement of routing prefix/subnet

Hypervisor Hypervisor

Ingress Routing

Localization: Clients-

Server


Extended VLAN typically has associated HSRP group

Only one HSRP router active, with all servers pointing to HSRP VIP as default gateway

Result: sub-optimal (trombone) routing

Egress Routing with VLAN Extension

HSRP

Active

HSRP

Standby

HSRP

Listen

HSRP

Listen

HSRP Hellos

VLAN

20

VLAN

10






HSRP

Active

HSRP

Standby

HSRP

Listen

HSRP

Listen

ARP

reply

ARP for

HSRP VIP

VLAN

20

VLAN

10






HSRP

Active

HSRP

Standby

HSRP

Listen

HSRP

Listen

VLAN

20

VLAN

10

Packet from

Vlan 10 to Vlan 20

DMAC = DGW

Routing

Packet from

Vlan 10 to Vlan 20

DMAC = Host Vlan 20


HSRP

Active HSRP

Standby

ARP for

HSRP VIP

ARP

reply

HSRP Filtering

Egress Routing Localization FHRP Filtering Solution

Filter FHRP with combination of VACL and MAC route filter

Result: Still have one HSRP group with one VIP, but now have active router at each site for optimal first-hop routing

HSRP

Active HSRP

Standby

HSRP Hellos HSRP Hellos

VLAN

20

VLAN

10


Sample Cluster - Primary Service in Left DC FHRP Localization – Egress Path Optimization

HA cluster Node B

Layer 3 Core

ISP A ISP B

HA cluster Node A

Access

Agg

Cluster VIP = 10.1.1.100 Preempt

Default GW = 10.1.1.1

Node A

Data Center

A Data Center

B

VLAN A

Public Network

Asymmetrical flows No Stateful device

Low ingress traffic

HSRP

Active

HSRP

Standby HSRP

Active

HSRP

Standby HSRP Filtering


Challenge

Subnets are spread across locations

Subnet information in the routing tables is

not specific enough

Routing doesn’t know if a server has moved

between locations

Traffic may be sent to the wrong location

Options

DNS Based

DNS redirection with ACE/GSS

Routing Based

Host Route Injection

LISP-VM

Ingress Routing Localization

West East

Ingress Traffic

Localization: Client

to Server Traffic

DCI LAN Extension

Hypervisor Hypervisor


DNS Based Ingress Optimization Option 1 – Detection of movement of VM using ACE probes

Layer 3

WAN

ISP A ISP B

Access

Agg

Access

VM= 10.1.1.100


Data Center

A Data Center

B

VLAN A

144.254.1.100

KAL-AP Changes

AppIP

144.254.200.100

VIP = 144.254.200.100 VIP = 144.254.1.100

SNAT SNAT

L2 Links (GE or 10GE)

L3 Links (GE or 10GE)


144.254.1.100

Layer 3 WAN

VM= 10.1.1.100


VLAN A

Public Network

MAC moved

Change the IP@

144.254.200.100

Access

Agg

ISP A

Data Center A

144.254.1.100 144.254.200.100

Access

Agg

ISP B

Data Center B

DNS Based Ingress Optimization Option 2 – Movement of VM announced via VCenter


Layer 3 Core

Intranet

Access

Agg Agg

DC A DC B

VLAN A

Public Network

IS 10.1.1.100 OK?

144.254.100.0/24

Backup for Data Center A 144.254.100.0/25 & 144.254.100.128/25

App VM= 10.1.1.100


144.254.100.100/32 is advertised

into L3 using RHI

Routing Based Ingress Optimization Route Injection using ACE Probes

RHI

The VM moves.

Probe to 10.1.1.100

is now OK


Access

Agg

VM= 10.10.10.1

Default GW = 10.10.10.100

ISP A ISP B

Access

Agg

Data Center A

LAN Extension

Prefix

(EID)

Route Locator

(RLOC)

10.10.10.1 A, B

10.10.10.2 A, B

… …

10.10.10.5 C, D

10.10.10.6 C, D

Ingress Tunnel

Router (ITR)

Moved to C, D

Decap

3

IP_DA = 10.10.10.1

1

ETR

Routing Based Ingress Optimization LISP

A B C D

IP_DA = B IP_DA = 10.10.10.1

IP_DA = 10.10.10.1

4

5

Decap

7

IP_DA = C IP_DA = 10.10.10.1

6 Encap

2

Data Center B

ETR

VM= 10.10.10.1

Default GW = 10.10.10.100

IP_DA = 10.10.10.1

VM IP Address

10.10.10.1


Agenda






No universal response where to place the OTV Edge Device

Main Options:

OTV at the Core Layer

OTV at the Aggregation Layer (most common – discussed in this presentation)

OTV Design Options

For more details on Deployment Models see: BRKDCT-3060

OTV Edge Device

L2

L3


OTV OTV


Guideline: The current OTV implementation on the Nexus 7000 enforces the separation between SVI routing and OTV encapsulation for a given VLAN

This separation can be achieved with having two separate devices to perform these two functions

An alternative, cleaner and less intrusive solution is the use of Virtual Device Contexts (VDCs) available with Nexus 7000 platform:

A dedicated OTV VDC to perform the OTV functionalities

The Aggregation-VDC used to provide SVI routing support

Aggregation OTV

VDC

OTV

VDC L2

L3

OTV and SVI Separation


Two different deployment models:

OTV Appliance on a Stick

Inline OTV Appliance

Join Interface

Internal Interface

OTV Appliance on a Stick

OTV

VDC

Common Uplinks

for Layer3 and DCI

L2

L3 SVIs

Inline OTV Appliance

Uplinks to the

Layer3 Transport

Dedicated

Uplink for DCI

OTV

VDC

L2

L3 SVIs

No difference in OTV functionality between the two models

The Inline OTV Appliance requires availability of Core downstream links

OTV and SVI Separation VDC Models


Aggregation

Core

Access


DC Core performs only Layer 3 role

STP and unknown unicast domains isolated between PODs

Intra-DC and inter-DC LAN extension provided by OTV

Ideal for single aggregation block topology

OTV at the Aggregation with L3 Boundary

VPC OTV VDC OTV VDC

VPC OTV VDC OTV VDC

SVIs SVIs SVIs SVIs

Recommended for Greenfield

Join Interface

Internal Interface

Virtual Overlay

Interface



Case Study for Mobile Company in South-America (CPOC SJ)


OTV at the Aggregation Layer OTV in Point-to-Point Deployments

VPC OTV VDC OTV VDC

VPC OTV VDC OTV VDC

Data Centers directly connected at the Aggregation

OTV (virtual) Appliance on a Stick

Advantages over VSS-vPC:

Provision of Layer 2 and Layer 3 connectivity leveraging the same dark fiber connections

Native STP isolation: no need to explicitly configure BPDU filtering

ARP Optimization with the OTV ARP Cache

Simplified provisioning of FHRP isolation

Easy Addition of Sites



Case Study for an European Financial Institution (ECATS Team)

Data Center B

VPC OTV VDC OTV VDC

VPC OTV VDC OTV VDC

VPC OTV VDC OTV VDC

Data Center A

Branch

Office

CTS Encrypted


OTV at the Aggregation with L3 boundary on the Firewalls

Aggregation

Core

Def GWY

Firewall Firewall

OTV OTV

Def GWY

The Firewalls host the Default Gateway

No SVIs at the Aggregation Layer

No Need for the OTV VDC


L2

L3



Case Study for European SP (Customer Lab)

Extending

VLAN 500-600

OTV OTV

STP Domain

Same Physical

Device

Core

La

ye

r 2

La

ye

r 3

OTV VDC

OTV Join Interface

La

ye

r 2

La

ye

r 3

OTV VDC

OTV OTV

Internet Internet


OTV: LAN extension made easy Real Problems solved by OTV

Extensions over any transport (IP, MPLS)

Failure boundary preservation

Site independence

Optimal BW utilization (no head-end replication)

Automated Built-in Multihoming

End-to-End loop prevention

Scalability

Sites, VLANs, MACs

Operations simplicity

South

Data

Center

North

Data

Center Fault

Domain

Fault

Domain

Fault

Domain

Fault

Domain

OTV

Only 5 CLI commands


Now that you know how it works…

Make sure to learn and follow the Cisco design guidelines to deploy OTV successfully

First Step:

Check out our DCI page on cisco.com: http://www.cisco.com/en/US/netsol/ns975/index.html#~in_depth

Read the following OTV whitepaper:

http://www.cisco.com/en/US/docs/solutions/Enterprise/Data_Center/DCI/whitepaper/DCI3_OTV_Intro.html

http://www.cisco.com/en/US/netsol/ns975/index.html



brkdct 2049 otv

Documents

ha cluster

aggregation

default gateway

isp bdata

isp adata

otv control

otv control

ip addr ip