solution guide infrastructure as a service: evpn and vxlan · fabric1 10.0.0.1 65001 fabric2...

Solution Guide

Infrastructure as a Service: EVPN and VXLAN

Modified: 2016-10-16

Copyright © 2016, Juniper Networks, Inc.

Juniper Networks, Inc.1133 InnovationWaySunnyvale, California 94089USA408-745-2000www.juniper.net

Copyright © 2016, Juniper Networks, Inc. All rights reserved.

Juniper Networks, Junos, Steel-Belted Radius, NetScreen, and ScreenOS are registered trademarks of Juniper Networks, Inc. in the UnitedStates and other countries. The Juniper Networks Logo, the Junos logo, and JunosE are trademarks of Juniper Networks, Inc. All othertrademarks, service marks, registered trademarks, or registered service marks are the property of their respective owners.

Juniper Networks assumes no responsibility for any inaccuracies in this document. Juniper Networks reserves the right to change, modify,transfer, or otherwise revise this publication without notice.

Solution Guide Infrastructure as a Service: EVPN and VXLANCopyright © 2016, Juniper Networks, Inc.All rights reserved.

The information in this document is current as of the date on the title page.

YEAR 2000 NOTICE

Juniper Networks hardware and software products are Year 2000 compliant. Junos OS has no known time-related limitations through theyear 2038. However, the NTP application is known to have some difficulty in the year 2036.

ENDUSER LICENSE AGREEMENT

The Juniper Networks product that is the subject of this technical documentation consists of (or is intended for use with) Juniper Networkssoftware. Use of such software is subject to the terms and conditions of the End User License Agreement (“EULA”) posted athttp://www.juniper.net/support/eula.html. By downloading, installing or using such software, you agree to the terms and conditions ofthat EULA.

Copyright © 2016, Juniper Networks, Inc.ii

http://www.juniper.net/support/eula.html

Table of Contents

Chapter 1 Infrastructure as a Service: EVPN and VXLAN . . . . . . . . . . . . . . . . . . . . . . . . . . 5

About This Solution Guide . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

Understanding the IaaS: EVPN and VXLAN Solution . . . . . . . . . . . . . . . . . . . . . . . . 5

Market Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

Solution Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

Solution Elements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

Design Considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

Solution Implementation Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

Example: Configuring the IaaS: EVPN and VXLAN Solution . . . . . . . . . . . . . . . . . . 18

iiiCopyright © 2016, Juniper Networks, Inc.

CHAPTER 1

Infrastructure as a Service: EVPN andVXLAN

• About This Solution Guide on page 5

• Understanding the IaaS: EVPN and VXLAN Solution on page 5

• Example: Configuring the IaaS: EVPN and VXLAN Solution on page 18

About This Solution Guide

This Infrastructureasaservice (IaaS) solution focuseson theuseofEthernetVPN(EVPN)

and Virtual Extensible VLAN (VXLAN) over a bare-metal server (BMS)-based network.

Such a network offers data center operators a way to create an external BGP

(EBGP)-based IP fabric underlay, which provides a solid foundation for the EVPN and

VXLAN overlay. By implementing this solution, telcos and data center operators can

scale their cloud-enabled business, migrate legacy architectures to more flexible and

modernarchitectures, competewithemergingWebservicesproviders, andmanagecosts

all at the same time.

This guide provides an overview of the IaaS: EVPN and VXLAN solution, the solution

requirements, design considerations, and how the solution was implemented by the

Juniper Networks solutions team. It also provides an example of how to configure the

network and verify that the solution is working as expected.

Understanding the IaaS: EVPN and VXLAN Solution

• Market Overview on page 5

• Solution Overview on page 6

• Solution Elements on page 7

• Design Considerations on page 11

• Solution Implementation Summary on page 17

Market Overview

In addition to owning their transport infrastructure, service providers are also in the

business of offering managed IT andmanaged data center services to a large variety of

customers. Because service providers own the infrastructure, they have the ability to

offer higher service-level agreements (SLAs), quality of service (QoS), and security, as

5Copyright © 2016, Juniper Networks, Inc.

these services are often provided over dedicated circuits. However, the cost structure of

these services can be relatively high, especially in comparison to the nimble and

fast-executingWeb services companies, for whom the cost structure is very lean and

low.

As service providers increasingly feel this competitive pressure, there is a need for them

to innovate their business models and adopt cloud computing architectures in order to

lower costs, increase efficiency, andmaintain their competitiveness in Infrastructure as

aService (IaaS) offerings.While they continue to useSLAs, flexibility of deployment, and

choice of topologies as a way to differentiate themselves fromWeb services providers,

service providers also need to invest significantly in building highly automated networks.

These improvements will help to cut operating expenses, and enable them to find new

sources of revenue by offering new services, in order to compete more effectively.

Service providers vary widely in how they build traditional networks, and there is not one

specific standardor topology that is followed.However, as theymove forwardandextend

their networks to offer cloud services,many providers are converging around two general

topologies based on some high-level requirements:

• A large percentage of standalone bare-metal servers (BMSs), with some part of the

network dedicated to offering virtualized compute services. This type of design keeps

the “intelligence” in the traditional physical network.

• Largely virtualized services,with some small amount of BMS-based services. This type

of design moves the “intelligence” out of the physical network and into the virtual

network, and generally requires a software-defined network (SDN) controller.

This solution guide focuses on the first use case, with a particular focus on the BMS

environment. This guide will help you understand the requirements for an IaaS network,

the architecture required to build the network, how to configure each layer, and how to

verify its operational state.

Solution Overview

Traditionally, data centers haveusedLayer 2 technologies suchasSpanningTreeProtocol

(STP) andmultichassis link aggregation groups (MC-LAG) to connect compute and

storage resources. As the design of these data centers evolves to scale out multitenant

networks, anewdatacenterarchitecture isneeded thatdecouples theunderlay (physical)

network from a tenant overlay network. Using a Layer 3 IP-based underlay coupled with

a VXLAN-Ethernet VPN (EVPN) overlay, data center and cloud operators can deploy

much larger networks thanareotherwisepossiblewith traditional Layer 2Ethernet-based

architectures.Withoverlays, endpoints (servers or virtualmachines [VMs]) canbeplaced

anywhere in the network and remain connected to the same logical Layer 2 network,

enabling the virtual topology to be decoupled from the physical topology.

For the reasons of scale and operational efficiency outlined above, virtual networking is

beingwidely deployed in data centers. Also, the role of bare-metal compute has become

more relevant forhigh-performance, scaleout, or container-drivenworkloads.This solution

guidedescribeshowstandards-basedcontrol and forwardingplaneprotocols canenable

interconnectivity by leveraging control-plane learning. In particular, this guide describes

how using EVPN for control plane learning can facilitate BMS interconnection within

Copyright © 2016, Juniper Networks, Inc.6


VXLAN virtual networks (VNs), and between VNs using a gateway such as a Juniper

Networks QFX Series switch.

Solution Elements

Underlay Network

In data center environments, the role of the physical underlay network is to provide an

IP fabric. Also known as a Clos network, its responsibility is to provide unicast IP

connectivity from any physical device (server, storage device, router, or switch) to any

other physical device. An ideal underlay network provides low-latency, nonblocking,

high-bandwidth connectivity from any point in the network to any other point in the

network.

At the underlay layer, devices maintain and share reachability information about the

physical network itself. However, this layer does not contain any “per-tenant” state; that

is, devices do not maintain and share reachability information about virtual or physical

endpoints. This is a task for the overlay layer.

IP fabrics can vary in size and scale. A typical solution uses two layers—spine and leaf—to

formwhat is known as a three-stage Clos network, where each leaf device is connected

to each spine device, as shown in shown in Figure 1 on page 7. A spine and leaf fabric is

sometimes referred to as a folded, three-stage Clos network, because the first and third

stages—the ingress and egress nodes—are folded back on top of each other. In this

configuration, spine devices are typically Layer 3 switches that provide connectivity

between leaf devices, and leaf devices are top-of-rack (TOR) switches that provide

connectivity to the servers.

Figure 1: Three-Stage Clos-Based IP Fabric

g043

017

Spine Spine Spine Spine

Leaf Leaf Leaf Leaf Leaf

As the scale of the fabric increases, it can be necessary to expand to a five-stage Clos

network, as shown in Figure 2 on page 8. This scenario adds a fabric layer to provide

inter-POD, or inter-data center, connectivity.


Chapter 1: Infrastructure as a Service: EVPN and VXLAN

Figure 2: Five-Stage Clos-Based IP Fabric

A key benefit of a Clos-based fabric is natural resiliency. High availability mechanisms,

such as MC-LAG or Virtual Chassis, are not required as the IP fabric uses multiple links

at each layer and device; resiliency and redundancy are provided by the physical network

infrastructure itself.

Building an IP fabric is very straightforward and serves as a great foundation for overlay

technologies such as EVPN and VXLAN.

NOTE: For more information about Clos-based IP fabrics, see Clos IP Fabrics

with QFX5100 Switches.

Overlay

Using an overlay architecture in the data center allows you to decouple physical network

devices fromtheendpoints in thenetwork.Thisdecouplingallows thedatacenternetwork

to be programmatically provisioned at a per-tenant level. Overlay networking generally

supports both Layer 2 and Layer 3 transport between servers or VMs. It also supports a

much larger scale: a traditional network using VLANs for separation can support a

maximum of about 4,000 tenants, while an overlay protocol such as VXLAN supports

over 16 million.

NOTE: At the time of this writing, QFX5100 and QFX10000 Series switchessupport 4000 virtual network identifiers (VNIs) per device.

Virtual networks (VNs) are a key concept in an overlay environment. VNs are logical

constructs implemented on top of the physical networks that replace VLAN-based

isolation and provide multitenancy in a virtualized data center. Each VN is isolated from

other VNs unless explicitly allowed by security policy. VNs can be interconnected within

a data center, and between data centers.

In data center networks, tunneling protocols such as VXLAN are used to create the data

plane for the overlay layer. For devices using VXLAN, each entity that performs the

encapsulation and decapsulation of packets is called a VXLAN tunnel endpoint (VTEP).

VTEPs typically reside within the hypervisor of virtualized hosts, but can also reside in

network devices to support BMS endpoints.



http://www.juniper.net/us/en/local/pdf/whitepapers/2000565-en.pdf


Figure 3 on page 9 shows a typical overlay architecture.

Figure 3: Overlay Architecture

In the diagram, server to the left of the IP fabric has been virtualized with a hypervisor.

The hypervisor contains a VTEP that handles the encapsulation of data-plane traffic

between VMs, as well as MAC address learning, provisioning of new virtual networks,

and other configuration changes. The physical servers above and to the right of the IP

fabric do not have any VTEP capabilities of their own. In order for these servers to

participate in the overlay architecture and communicate with other endpoints (physical

or virtual), theyneedhelp toencapsulate thedata-plane traffic andperformMACaddress

learning. In this case, that help comes from the attached network device, typically a

top-of-rack (TOR) switch or a leaf device in the IP fabric. Supporting the VTEP role in a

network device simplifies the overlay architecture; now any device with physical servers

connected to it can simply perform theoverlay encapsulation and control-plane function

on their behalf. Fromthepointof viewofaphysical server, thenetwork functionsasusual.

NOTE: For more information on VXLAN and VTEPs in overlay networks, seeLearn About: VXLAN in Virtualized Data Center Networks.

To support the scale of data center networks, the overlay layer typically requires a

control-plane protocol to facilitate learning and sharing of endpoints. EVPN is a popular

choice for this function.

EVPN is a control-plane technology that usesMultiprotocol BGP (MP-BGP) forMACand

IP address (endpoint) distribution, with MAC addresses being treated as “routes.” Route

entries can contain just aMACaddress, or aMACaddress plus an IP address (ARPentry).

Asused indatacenter environments, EVPNenablesdevicesactingasVTEPs toexchange

reachability information with each other about their endpoints.

To support its range of capabilities, EVPN introduces several new concepts, including

new route types and BGP communities. It also defines a new BGP network layer

reachability information (NLRI), called the EVPN NLRI.

Tor this solution, two route types are of particular note:

• EVPN Route Type 2: MAC/IP Advertisement route—Extends BGP to advertise MAC

and IP addresses in the EVPNNLRI. Key uses of this route type include advertising host



http://www.juniper.net/techpubs/en_US/learn-about/LA_VXLANinDCs.pdf

MAC and IP reachability, allowing control plane-based MAC learning for remote PE

devices, minimizing flooding across aWAN, and allowing PE devices to perform

proxy-ARP locally for remote hosts. Typically, the Type 2 route is used to support Layer

2 (intra-VXLAN) traffic, though it can also support Layer 3 (inter-VXLAN) traffic.

• EVPN Route Type 5: IP Prefix route—Extends EVPNwith a route type for the

advertisement of IP prefixes. This route type decouples the advertisement of IP

information from the advertisement of MAC addresses. The ability to advertise an

entire IP prefix provides improved scaling (versus advertising MAC/IP information for

every host), as well as increased efficiency in advertising and withdrawing routes.

Typically, the Type 5 route is used to support Layer 3 (inter-VXLAN) traffic.

NOTE: For more information on EVPN in a data center context, see Improve

Data Center Interconnect, L2 Services with Juniper’s EVPN.

Moving to anoverlay architecture shifts the “intelligence” of thedata center. Traditionally,

servers and VMs each consume aMAC address and host route entry in the physical

(underlay) network. However, with an overlay architecture, only the VTEPs consume a

MAC address and host route entry in the physical network. All host-to-host traffic is now

encapsulated between VTEPs, and the MAC address and host route of each server or

VM aren’t visible to the underlying networking equipment. The MAC address and host

route scale have beenmoved from the underlay environment into the overlay.

Gateways

A gateway in a virtualized network environment typically refers to physical routers or

switches that connect the tenant virtual networks to physical networks such as the

Internet, a customer VPN, another data center, or to nonvirtualized servers. This solution

uses multiple types of gateways.

A Layer 2 VXLAN gateway, also known as a VTEP gateway, maps VLANs to VXLANs and

handles VXLAN encapsulation and decapsulation so that non-virtualized resources do

not need to support the VXLAN protocol. This permits the VXLAN and VLAN segments

to act as one forwarding domain.

In data center environments, a VTEP gateway often runs in software as a virtual switch

or virtual router instance on a virtualized server. However, switches and routers can also

function as VTEP gateways, encapsulating and decapsulating VXLAN packets on behalf

of bare-metal servers, as shown earlier in Figure 3 on page 9. This setup is referred to

as a hardware VTEP gateway. In this solution, the QFX5100 (leaf) devices act as Layer

2 gateways to support intra-VXLAN traffic.

To forward traffic between VXLANs, a Layer 3 gateway is required. In this solution, the

QFX10002 (spine) devices act as Layer 3 gateways to support inter-VXLAN traffic.



https://www.juniper.net/assets/us/en/local/pdf/whitepapers/2000596-en.pdf


NOTE: For more information on Layer 3 gateways in a data center context,see Day One: Using Ethernet VPNs for Data Center Interconnect and Juniper

Networks EVPN Implementation for Next-Generation Data Center Architectures.

Design Considerations

There are several design considerations when implementing an IaaS network.

Fabric Connectivity

Data center fabrics can be based on Layer 2 or Layer 3 technologies. Ethernet fabrics,

such as Juniper Networks Virtual Chassis Fabric, are simple tomanage and provide scale

and equal-costmultipath (ECMP) capabilities to a certain degree. However, as the fabric

increases in size, the scale of the network eventually becomes toomuch for an Ethernet

fabric to handle. Tenant separation is another issue; as Ethernet fabrics have no overlay

network, VLANsmustbeused, addinganother limitation to the scalability of thenetwork.

An IaaS data center network requires Layer 3 protocols to provide the ECMP and scale

capabilities for a network of this size. While IGPs provide excellent ECMP capabilities,

BGP is the ideal option to provide the proper scaling and performance required by this

solution. BGP was designed to handle the scale of the global Internet, and can be

repurposed to support the needs of top-tier service provider data centers.

BGPDesign (Underlay)

WithBGPdecideduponas the routingprotocol for the fabric, thenextdecision iswhether

touse internalBGP(IBGP)or externalBGP(EBGP).Theverynatureofan IP fabric requires

having multiple, equal-cost paths; therefore, the key factor to consider here is how IBGP

and EBGP implement ECMP functionality.

IBGP requires that all devices peer with one another. In an IaaS network, BGP route

reflectors typically would be implemented in the spine layer of the network to help with

scaling. However, standard BGP route reflection only reflects the best (single) prefix to

clients. In order to enable full ECMP, you need to configure the BGP AddPath feature to

provide additional ECMP paths into the BGP route reflection advertisements to clients.

Alternatively, EBGP supports ECMPwithout enabling additional features. It is easy to

configure, and also facilitates traffic engineering if desired through standard EBGP

techniques such as autonomous system (AS) padding.

With EBGP, each device in the IP fabric uses a different AS number. It is also a good

practice to align the AS numbers within each layer. As an example, Figure 4 on page 12

shows the spine layer with AS numbering in the 651xx range, and the leaf layer with AS

numbering in the 652xx range.



http://www.juniper.net/us/en/training/jnbooks/day-one/proof-concept-labs/using-ethernet-vpns/



Figure 4: AS Numbering in an IP Fabric Underlay

ASN 65101 ASN 65102

ASN 65201 ASN 65202 ASN 65203 ASN 65204

SPINE

LEAF

g043

020

Because EBGP supports ECMP in amore straightforward fashion, an EBGP-based IP

fabric is typically used at the underlay layer.

NOTE: For information on Juniper Networks validated Clos-based Layer 3 IPfabric solution, see Solution Guide: Software as a Service.

BGPDesign (Overlay)

At the overlay layer, similar decisionsmust bemade. Again the very nature of an IP fabric

requires having multiple, equal-cost paths. In addition, youmust consider the overlay

protocolbeingused.This solutionusesEVPNas thecontrol-planeprotocol for theoverlay;

given that EVPN uses MP-BGP for communication (signaling), BGP is again a logical

choice to be used in the overlay.

There is more than one way to design the overlay environment. Because this solution is

“controllerless,”meaning there is noSDNcontroller in use, thenetwork itselfmustperform

both the underlay and overlay functions. This solution uses an IBGP overlay design with

route reflection, as shown in Figure 5 on page 12. With this design, leaf devices within a

given point of delivery (POD) share endpoint information upstream as EVPN routes to

thespinedevices,whichareactingas route reflectors. Thespinedevices reflect the routes

downstream to the other leaf devices.

Figure 5: BGP (EVPN) Overlay Design—Single POD

The spine devices can also advertise the EVPN routes to other PODs. As shown in

Figure 6 on page 13, the spine devices use an MP-IBGP full mesh to share EVPN routes

and provide inter-POD communication.



http://www.juniper.net/techpubs/en_US/release-independent/solutions/information-products/pathway-pages/sg-001-software-as-a-service.pdf

Figure 6: BGP (EVPN) Overlay Design—Multiple PODs

NOTE: For more information about Clos-based IP fabric design, see Clos IP

Fabrics with QFX5100 Switches.

EVPNDesign

As noted above, this solution uses EVPN as the control-plane protocol for the overlay.

EVPN runs between VXLAN gateways, and removes the need for VXLAN to handle the

advertisement of MAC and IP reachability information in the data plane by enabling this

functionality in the control plane.

A multitenant data center environment requires mechanisms to support traffic flows

bothwithin and betweenVNs. For this solution, intra-VXLAN traffic is handled at the leaf

layer, with theQFX5100 switches acting as VXLAN Layer 2 gateways. Inter-VXLAN traffic

is handled at the spine layer, with the QFX10002 switches acting as VXLAN Layer 3

gateways. Spine devices are configured with integrated routing and bridging (IRB)

interfaces, which endpoints use as a default gateway for non-local traffic.

Intra-VXLAN forwarding is typically performed with the help of EVPN route Type 2

announcements, which advertise MAC addresses (along with their related IP address).

Inter-VXLAN routing can also be performed using EVPN route Type 2 announcements,

though it is increasingly performed with the help of EVPN route Type 5 announcements,

which advertise entire IP prefixes.

Inter-VXLAN routing supports two operating modes: asymmetric and symmetric. These

terms relate to the number of lookups performed by the devices at each end of a VXLAN

tunnel. The following describes the twomodes:

Asymmetric mode

• The sending device maintains explicit reachability to all remote endpoints.

• Benefit: just a single lookup is required on the receiving device (since the endpoint was

already known by the sending device).

• Drawback: large environments can cause very large lookup tables.





Symmetric mode

• The sending device does not maintain explicit reachability to all remote endpoints;

rather, it puts remote traffic into a single “routing” VXLAN tunnel and lets the receiving

device perform the endpoint lookup locally.

• Benefit: reduces lookup table size.

• Drawback: an additional lookup is required by the receiving device (since the endpoint

was not explicitly known by the sending device).

This solution uses symmetric mode for inter-VXLAN routing. This mode is generally

preferred, as current Junos OS platforms can performmultiple lookups in hardware with

no impact to line-rate performance.

NOTE: At the time of this writing, the QFX10002 andMX Series routerssupport asymmetricmodewithEVPN routeType2.QFX10002also supportssymmetric mode with EVPN route Type 5.

NOTE: Formoredetailed informationon inter-VXLANrouting, seeConfiguring

EVPN Type 5 for QFX10000 Series Switches.

EVPN supports “all-active” (multipath) forwarding for endpoints, allowing them to be

connected to two or more leaf devices for redundant connectivity, as shown in

Figure 7 on page 14.

Figure 7: EVPN Server Multihoming

In EVPNterms, the links toamultihomedserver aredefinedasa singleEthernet segment.

Each Ethernet segment is identified using a unique Ethernet segment identifier (ESI).

NOTE: Formoredetailed informationaboutEVPNESIs, seeEVPNMultihoming

Overview.

VXLANDesign

VXLAN in the overlay has the following design characteristics:



http://www.juniper.net/techpubs/en_US/release-independent/solutions/information-products/pathway-pages/solutions/evpn-type5-configuring.pdf

http://www.juniper.net/techpubs/en_US/release-independent/solutions/information-products/pathway-pages/solutions/evpn-type5-configuring.pdf

http://www.juniper.net/documentation/en_US/junos/topics/concept/evpn-bgp-multihoming-overview.html

http://www.juniper.net/documentation/en_US/junos/topics/concept/evpn-bgp-multihoming-overview.html

• Each bridge domain / VXLAN network identifier (VNI) must have a VXLAN tunnel to

each spine and leaf in a full mesh, that is, any-to-any connectivity.

• VXLAN is the data plane encapsulation between servers.

• EVPN is used as the control plane for MAC address learning.

An example of the VXLAN design for this solution is shown in Figure 8 on page 15.

Figure 8: VXLANDesign

Tenant Design

This solution provides tenant separation and connectivity at the spine and leaf layers.

Tenant design in the spine devices has the following design characteristics:

• Each tenant gets its own VRF.

• Each tenant VRF can havemultiple bridge domains.

• Bridge domains within a VRF can switch and route freely.

• Bridge domains between VRFsmust not switch and route.

• Each bridge domain must provide VXLAN Layer 2 gateway functionality.

• Each bridge domain will have a routed Layer 3 interface.

• IRB interfaces must be able to perform inter-VXLAN routing.

• Each spine device in the PODmust be configured with identical VRF, bridge domain,

and IRB components.

An example of the spine tenant design for this solution is shown in Figure 9 on page 16.



Figure 9: Tenant Design in Spine Devices

By comparison, tenant design in the leaf devices is very simple, with the following design

characteristics:

• Leaf devices are Layer 2 only (no VRF or IRB interfaces).

• By default, all traffic is isolated per bridge domain.

• Although a given tenant might own BD1, BD2, and BD3, there are no VRFs on the leaf

device.

An example of the leaf tenant design for this solution is shown in Figure 10 on page 16.

Figure 10: Tenant Design in Leaf Devices

IRB Design

Inter-VXLAN gateway functionality is implemented in this solution at the spine layer,

using IRB interfaces. These interfaces have the following design characteristics:

• Every bridge domain must have an Layer 3 / routed interface that is associated with

an IRB interface.

• Each bridge domain’s IRB interface can use IPv4 addressing, IPv6 addressing, or both.

• Each spine device must use the same IPv4 and IPv6 IRB interface addresses (this

reduces the public IP addresses wasted at scale).

• Each spine must implement EVPN anycast gateway.

An example of the leaf tenant design setup is shown in Figure 11 on page 16.

Figure 11: IRB Interface Design on Spine Devices



Solution Implementation Summary

The following hardware equipment and key software features were used to create the

IaaS solution described in the upcoming example:

Fabric

• Four QFX5100-24Q switches

• Underlay network

• EBGP peering with the downstream (spine) devices using two-byte AS numbers

• BFD for all BGP sessions

• Traffic load balancing

• EBGPmultipath

• Resilient hashing

• Per-packet load balancing

Spine

• Four QFX10002-72Q switches


• EBGP peering with the upstream (fabric) devices using two-byte AS numbers

• EBGP peering with the downstream (leaf) devices using two-byte AS numbers

• Overlay network

• EVPN / IBGP full mesh between all spine devices

• EVPN / IBGP route reflection to leaf devices

• Each spine device is a route reflector for leaf devices in its POD

• Each POD is a separate cluster



• EBGPmultipath



• Nine VLANs (100 to 108) to illustrate intra-VLAN and inter-VLAN traffic using EVPN

route Type 2

• Two VLANs (999 on Spine 1 and Spine 2, 888 on Spine 3 and Spine 4) to illustrate

inter-VLAN traffic using EVPN route Type 5

Leaf



• Four QFX5100-48S switches


• EBGP peering with the upstream (spine) devices using two-byte AS numbers

• Overlay network

• EVPN / IBGPpeeringwith the upstream (spine) devices using two-byte ASnumbers



• EBGPmultipath



• Nine VLANs (100 to 108) to illustrate intra-VLAN and inter-VLAN traffic using EVPN

route Type 2

• TwoVLANs(999onLeaf 1andLeaf2,888onLeaf3andLeaf4) to illustrate inter-VLAN

traffic using EVPN route Type 5

Servers / End hosts

• Bare-metal servers attached to leaf devices

• Traffic generator simulating BMS hosts, sending intra- and inter-VLAN traffic

RelatedDocumentation

Example: Configuring the IaaS: EVPN and VXLAN Solution on page 18•

Example: Configuring the IaaS: EVPN and VXLAN Solution

This example describes how to build, configure, and verify a bare metal server (BMS)

network containing a BGP-based IP fabric underlay, supported by an EVPN and VXLAN

overlay.

• Requirements on page 18

• Overview and Topology on page 19

• Configuring the IaaS: EVPN and VXLAN Solution on page 26

• Configuring Additional Features for the IaaS: EVPN and VXLAN Solution on page 51

• Verification on page 60

Requirements

Table 1 on page 19 lists the hardware and software components used in this example.



Table 1: Solution Hardware and Software Requirements

SoftwareHardwareDevice

Junos OS Release 14.1X53-D30.3QFX5100-24QFabric devices

Junos OS Release 15.1X53-D60.4QFX10002-72QSpine devices

Junos OS Release 14.1X53-D35.3QFX5100-48SLeaf devices

Traffic GeneratorHost emulation

Overview and Topology

The topology used in this example consists of a series of QFX5100 and QFX10002

switches, as shown in Figure 12 on page 19.

Figure 12: IaaS: EVPN and VXLAN Solution - Underlay Topology

In this example, the fabric layer has four QFX5100-24Q switches, the spine layer has four

QFX10002-72Q switches, and the leaf layer uses four QFX5100-48S switches. Leaf 1,

Leaf 2, Spine 1, and Spine 2 are included in a single point of delivery (POD) named POD



1; and Leaf 3, Leaf 4, Spine 3, and Spine 4 are included in POD 2. Both data center PODs

connect to the fabric layer, which provides inter-POD connectivity.

NOTE: This topology simulates conditions for PODs contained either in thesame data center or PODs located in different data centers.

Two hosts per leaf device are connected to Leaf 1, Leaf 2, and Leaf 3. One host is

dual-homed to Leaf 3 and Leaf 4 through Switch 5, and one host is single-homed to Leaf

4.

This first diagramalso represents theEBGPunderlay for the solution, utilizingan individual

autonomous system number for each device and a unique loopback address for each

device for easy monitoring and troubleshooting of the network.

The topology for the overlay is shown in Figure 13 on page 20.

Figure 13: IaaS: EVPN and VXLAN Solution - Overlay Topology

A full mesh IBGP configuration connects the spine devices together, and all spine and

leaf devices belong to a single autonomous system (65200). A route reflector cluster is

assigned to each POD and enables the leaf devices within the POD to have redundant

connections to the spine layer.

The example included in this solution explores the use of both Type 2 and Type 5 EVPN

routes and contains configuration excerpts to enable you to select either option. In

Figure 14 on page 21, Type 2 routes are distributed within the same VLAN.



Figure 14: IaaS: EVPN and VXLAN Solution - Type 2 Intra-VLAN Traffic

As shown, when traffic flows between hosts that are connected to the same leaf (1.1),

the traffic stays locally on the leaf and does not need to be sent to the upper layers.

To reach hosts connected to other leaf devices in the same POD (1.2), traffic travels

between the leaf devices and spine devices across the IP fabric. Host traffic is switched

using a VXLAN tunnel established between the leaf devices. The ingress leaf device

encapsulates the host traffic with a VXLANheader, the traffic is switched using the outer

header, and it travels over the spine layer to reach the other leaf device. The egress leaf

device de-encapsulates the VXLAN header and switches the frame to the destination

host.

To reach hosts located in another POD (1.3), the traffic must be sent up through the leaf,

spine, and fabric layers and then down through the spine and leaf layers in the second

POD to reach the destination host. The VXLAN tunnel established between the leaf

devices in the different PODs enables traffic to travel from the ingress leaf device, across

the spine layer in the first POD, through the fabric layer, to the spine layer in the second

POD, and to the egress leaf device. The egress leaf device de-encapsulates the VXLAN

header and switches the frame to the destination host.

Figure 15 on page 22 shows how Type 2 routes are handled between different VLANs.



Figure 15: IaaS: EVPN and VXLAN Solution - Type 2 Inter-VLAN Traffic

As shown, the process is the same for all three cases of inter-VLAN traffic because they

each require Layer 3 routing (1.1, 1.2, and 1.3). Host traffic containing an inner header is

encapsulated with a VXLAN header and an outer header that lists the local spine device

as the destination. The spine device strips the outer header, de-encapsulates the VXLAN

header, performs a route lookup for the inner header, and forwards the traffic across an

EVPN routing instance to the respective host using a VXLAN tunnel that references the

appropriate leaf device. The egress leaf device de-encapsulates the VXLAN header and

switches the frame to the desired host. In this example, VLANs 100 to 108 illustrate

intra-VLAN and inter-VLAN traffic using EVPN route Type 2.

As a final option, Figure 16 on page 23 shows how Type 5 routes are handled between

VLANs.



Figure 16: IaaS:EVPNandVXLANSolution-Type2andType5 Inter-VLANTraffic

For the first two cases (1.1 and 1.2), inter-VLAN traffic is handled the same as shown in

Figure 15 onpage 22. However, when sendingType 5 inter-VLAN traffic betweendifferent

data centers (1.3), the host traffic is encapsulated with a VXLAN header and an outer

header that lists the local spine device as the destination. The local spine device

de-encapsulates the VXLAN header, performs a route lookup for the inner header, and

forwards the traffic across an EVPN routing instance to the remote spine device in the

second POD by using a VXLAN header. The remote spine device de-encapsulates the

packet and performs a route lookup for the respective routing instance based on the VNI

number. The spine device then encapsulates the traffic and sends it across a VXLAN

tunnel to the respective leaf device. The egress leaf device de-encapsulates the VXLAN

header and switches the frame to the destination host. In this example, VLANs 999

(Spine 1 and Spine 2) and 888 (Spine 3 and Spine 4) illustrate inter-VLAN traffic using

EVPN route Type 5.

NOTE: At the time this guide was written, Type 5 can only be used forinter-VLAN topologies. To support intra-VLAN topologies, use Type 2.



Table 2 on page 24 lists the IPv4 addresses used in this example, Table 3 on page 24

displays the IPv6 addresses used in this example, and Table 4 on page 25 lists the

loopback addresses and autonomous system numbers for the fabric, spine, and leaf

devices.

Table 2: IPv4 Addressing

NetworkIPv4 Network Prefixes

172.16.0.0/24Fabric to spine point-to-point links

172.16.0.0/24Spine to leaf point-to-point links

10.0.0.0/24Loopback IP addresses (for all devices)

A set of nine addresses that increment the third octet and use .1 for thefourth octet:

• 10.1.100.1/16

• 10.1.101.1/16

• 10.1.102.1/16

• 10.1.103.1/16

• 10.1.104.1/16

• 10.1.105.1/16

• 10.1.106.1/16

• 10.1.107.1/16

• 10.1.108.1/16

Anycast IPv4 addresses

A rangeof fiveaddresses (0 -4)per host,with thehost number representedin the tens place. For example,Host 7has the following rangeof addresses:10.1.100.70/16 - 10.1.100.74/16.

Server/traffic generator IPv4 host devices

Table 3: IPv6 Addressing


A set of nine addresses that increment the fifth double-octet and use :1 for the finaldouble-octet:

• 2001:db8:10:1:100::1/80

• 2001:db8:10:1:101::1/80

• 2001:db8:10:1:102::1/80

• 2001:db8:10:1:103::1/80

• 2001:db8:10:1:104::1/80

• 2001:db8:10:1:105::1/80

• 2001:db8:10:1:106::1/80

• 2001:db8:10:1:107::1/80

• 2001:db8:10:1:108::1/80

Anycast IPv6 addresses



Table 3: IPv6 Addressing (continued)


A set of addresses that increment the fifth double-octet and use :<210 + spine-number> forthe final double-octet. For example, for Spine 1, 210 + 1 equals 211, so the corresponding IPv6addresses are as follows:

• 2001:db8:10:1:100::211/80

• 2001:db8:10:1:101::211/80

• 2001:db8:10:1:102::211/80

• 2001:db8:10:1:103::211/80

• 2001:db8:10:1:104::211/80

• 2001:db8:10:1:105::211/80

• 2001:db8:10:1:106::211/80

• 2001:db8:10:1:107::211/80

• 2001:db8:10:1:108::211/80

Server/traffic generator IPv6host devices

Table 4: Loopback Addresses and Underlay ASNs for Fabric Devices, Spine Devices, and LeafDevices

ASNLoopback Address

6500110.0.0.1Fabric 1

6500210.0.0.2Fabric 2

6500310.0.0.3Fabric 3

6500410.0.0.4Fabric 4

65011 (underlay)

65200 (overlay)

10.0.0.11Spine 2

65012 (underlay)

65200 (overlay)

10.0.0.12Spine 3

65013 (underlay)

65200 (overlay)

10.0.0.13Spine 4

65014 (underlay)

65200 (overlay)

10.0.0.14Spine 4

65021 (underlay)

65200 (overlay)

10.0.0.21Leaf 1

65022 (underlay)

65200 (overlay)

10.0.0.22Leaf 2



Table 4: Loopback Addresses and Underlay ASNs for Fabric Devices, Spine Devices, and LeafDevices (continued)

ASNLoopback Address

65023 (underlay)

65200 (overlay)

10.0.0.23Leaf 3

65024 (underlay)

65200 (overlay)

10.0.0.24Leaf 4

Configuring the IaaS: EVPN and VXLAN Solution

NOTE: You can useAnsible scripts to generate a large portion of the IP fabricand EVPN VXLAN configurations. For more information, see: Ansible Junos

Configuration for EVPN/VXLAN.

This sectionexplainshowtobuildout the leaf, spine, and fabric layerswithanEBGP-based

IP fabricunderlayandan IBGP-basedEVPNandVXLANoverlay for thesolution. It includes

the following sections:

• Configuring Leaf Devices for the IaaS: EVPN and VXLAN Solution on page 26

• Configuring Spine Devices for the IaaS: EVPN and VXLAN Solution on page 34

• Configuring Fabric Devices for the IaaS: EVPN and VXLAN Solution on page 47

• Configuring Host Multihoming on page 50

Configuring Leaf Devices for the IaaS: EVPN and VXLAN Solution

CLI QuickConfiguration

To quickly configure the leaf devices, enter the following representative configuration

statements on each device:

NOTE: The configuration shown here applies to Leaf 1.

[edit]set interfaces xe-0/0/12 description "To Host 1"set interfaces xe-0/0/12 unit 0 family ethernet-switching interface-mode trunkset interfaces xe-0/0/12 unit 0 family ethernet-switching vlanmembers 100-108set interfaces xe-0/0/12 unit 0 family ethernet-switching vlanmembers 999set interfaces xe-0/0/13 description "To Host 5"set interfaces xe-0/0/13 unit 0 family ethernet-switching interface-mode trunkset interfaces xe-0/0/13 unit 0 family ethernet-switching vlanmembers 100-108set interfaces xe-0/0/13 unit 0 family ethernet-switching vlanmembers 999set interfaces et-0/0/50 description "To Spine 1"set interfaces et-0/0/50mtu 9192set interfaces et-0/0/50 unit 0 family inetmtu 9000set interfaces et-0/0/50 unit 0 family inet address 172.16.0.33/31



https://github.com/JNPRAutomate/ansible-junos-evpn-vxlan

https://github.com/JNPRAutomate/ansible-junos-evpn-vxlan

set interfaces et-0/0/51 description "To Spine 2"set interfaces et-0/0/51mtu 9192set interfaces et-0/0/51 unit 0 family inetmtu 9000set interfaces et-0/0/51 unit 0 family inet address 172.16.0.37/31set interfaces lo0 unit 0 family inet address 10.0.0.21/32set routing-options forwarding-table export pfe-ecmpset routing-options router-id 10.0.0.21set protocols bgp group underlay-ipfabric type externalset protocols bgp group underlay-ipfabric mtu-discoveryset protocols bgp group underlay-ipfabric import bgp-ipclos-inset protocols bgp group underlay-ipfabric export bgp-ipclos-outset protocols bgp group underlay-ipfabric local-as 65021set protocols bgp group underlay-ipfabric bfd-liveness-detectionminimum-interval 350set protocols bgp group underlay-ipfabric bfd-liveness-detectionmultiplier 3setprotocolsbgpgroupunderlay-ipfabricbfd-liveness-detectionsession-modeautomaticset protocols bgp group underlay-ipfabric multipathmultiple-asset protocols bgp group underlay-ipfabric neighbor 172.16.0.32 peer-as 65011set protocols bgp group underlay-ipfabric neighbor 172.16.0.36 peer-as 65012set protocols bgp log-updownset protocols bgp graceful-restartset protocols bgp group overlay-evpn type internalset protocols bgp group overlay-evpn local-address 10.0.0.21set protocols bgp group overlay-evpn import OVERLAY-INset protocols bgp group overlay-evpn family evpn signalingset protocols bgp group overlay-evpn local-as 65200set protocols bgp group overlay-evpn bfd-liveness-detectionminimum-interval 350set protocols bgp group overlay-evpn bfd-liveness-detectionmultiplier 3set protocols bgp group overlay-evpn bfd-liveness-detection session-mode automaticset protocols bgp group overlay-evpnmultipathset protocols bgp group overlay-evpn neighbor 10.0.0.11set protocols bgp group overlay-evpn neighbor 10.0.0.12set protocols evpn vni-options vni 1000 vrf-target export target:1:1000set protocols evpn vni-options vni 1001 vrf-target export target:1:1001set protocols evpn vni-options vni 1002 vrf-target export target:1:1002set protocols evpn vni-options vni 1003 vrf-target export target:1:1003set protocols evpn vni-options vni 1004 vrf-target export target:1:1004set protocols evpn vni-options vni 1005 vrf-target export target:1:1005set protocols evpn vni-options vni 1006 vrf-target export target:1:1006set protocols evpn vni-options vni 1007 vrf-target export target:1:1007set protocols evpn vni-options vni 1008 vrf-target export target:1:1008set protocols evpn vni-options vni 1999 vrf-target export target:1:1999set protocols evpn encapsulation vxlanset protocols evpn extended-vni-list 1000set protocols evpn extended-vni-list 1001set protocols evpn extended-vni-list 1002set protocols evpn extended-vni-list 1003set protocols evpn extended-vni-list 1004set protocols evpn extended-vni-list 1005set protocols evpn extended-vni-list 1006set protocols evpn extended-vni-list 1007set protocols evpn extended-vni-list 1008set protocols evpn extended-vni-list 1999set protocols evpnmulticast-mode ingress-replicationset protocols lldp interface allset policy-options community com1000members target:1:1000set policy-options community com1001members target:1:1001



set policy-options community com1002members target:1:1002set policy-options community com1003members target:1:1003set policy-options community com1004members target:1:1004set policy-options community com1005members target:1:1005set policy-options community com1006members target:1:1006set policy-options community com1007members target:1:1007set policy-options community com1008members target:1:1008set policy-options community com1999members target:1:1999set policy-options community comm-leaf_esi members target:9999:9999set policy-options policy-statement bgp-ipclos-in term loopbacks from route-filter10.0.0.0/16 orlonger

set policy-options policy-statement bgp-ipclos-in term loopbacks then acceptset policy-options policy-statement bgp-ipclos-out term loopback from protocol directset policy-options policy-statement bgp-ipclos-out term loopback from route-filter10.0.0.21/32 orlonger

set policy-options policy-statement bgp-ipclos-out term loopback then next-hop selfset policy-options policy-statement bgp-ipclos-out term loopback then acceptset policy-options policy-statement bgp-ipclos-out term reject then rejectset policy-options policy-statement LEAF-IN term import_leaf_esi from communitycomm-leaf_esi

set policy-options policy-statement LEAF-IN term import_leaf_esi then acceptset policy-options policy-statement LEAF-IN term import_vni1000 from communitycom1000

set policy-options policy-statement LEAF-IN term import_vni1000 then acceptset policy-options policy-statement LEAF-IN term import_vni1001 from communitycom1001









set policy-options policy-statement LEAF-IN term import_vni1999 then acceptset policy-options policy-statement LEAF-IN term default then rejectset policy-options policy-statement OVERLAY-IN term reject-remote-gw from familyevpn

set policy-options policy-statement OVERLAY-IN term reject-remote-gw from next-hop10.0.0.13



set policy-options policy-statement OVERLAY-IN term reject-remote-gw from next-hop10.0.0.14

set policy-options policy-statement OVERLAY-IN term reject-remote-gw fromnlri-route-type 1

set policy-options policy-statement OVERLAY-IN term reject-remote-gw fromnlri-route-type 2

set policy-options policy-statement OVERLAY-IN term reject-remote-gw then rejectset policy-options policy-statement OVERLAY-IN term accept-all then acceptset policy-options policy-statement pfe-ecmp then load-balance per-packetset switch-options route-distinguisher 10.0.0.21:1set switch-options vrf-import LEAF-INset switch-options vrf-target target:9999:9999set switch-options vtep-source-interface lo0.0set vlans v1000 vlan-id 100set vlans v1000 vxlan vni 1000set vlans v1000 vxlan ingress-node-replicationset vlans v1001 vlan-id 101set vlans v1001 vxlan vni 1001set vlans v1001 vxlan ingress-node-replicationset vlans v1002 vlan-id 102set vlans v1002 vxlan vni 1002set vlans v1002 vxlan ingress-node-replicationset vlans v1003 vlan-id 103set vlans v1003 vxlan vni 1003set vlans v1003 vxlan ingress-node-replicationset vlans v1004 vlan-id 104set vlans v1004 vxlan vni 1004set vlans v1004 vxlan ingress-node-replicationset vlans v1005 vlan-id 105set vlans v1005 vxlan vni 1005set vlans v1005 vxlan ingress-node-replicationset vlans v1006 vlan-id 106set vlans v1006 vxlan vni 1006set vlans v1006 vxlan ingress-node-replicationset vlans v1007 vlan-id 107set vlans v1007 vxlan vni 1007set vlans v1007 vxlan ingress-node-replicationset vlans v1008 vlan-id 108set vlans v1008 vxlan vni 1008set vlans v1008 vxlan ingress-node-replication

Step-by-StepProcedure

To configure the leaf devices:

Configure Ethernet interfaces to reach the hosts :1.

[edit]user@leaf-1# set interfaces xe-0/0/12 description "To Host 1"user@leaf-1# set interfaces xe-0/0/12 unit 0 family ethernet-switchinginterface-mode trunk

user@leaf-1#set interfacesxe-0/0/12unit0familyethernet-switchingvlanmembers100-108

user@leaf-1#set interfacesxe-0/0/12unit0familyethernet-switchingvlanmembers999

user@leaf-1# set interfaces xe-0/0/13 description "To Host 5"user@leaf-1# set interfaces xe-0/0/13 unit 0 family ethernet-switchinginterface-mode trunk



user@leaf-1#set interfacesxe-0/0/13unit0familyethernet-switchingvlanmembers100-108

user@leaf-1#set interfacesxe-0/0/13unit0familyethernet-switchingvlanmembers999

2. Configure the interfaces connecting the leaf device to the spine devices:

[edit]user@leaf-1# set interfaces et-0/0/50 description "To Spine 1"user@leaf-1# set interfaces et-0/0/50mtu 9192user@leaf-1# set interfaces et-0/0/50 unit 0 family inetmtu 9000user@leaf-1# set interfaces et-0/0/50 unit 0 family inet address 172.16.0.33/31user@leaf-1# set interfaces et-0/0/51 description "To Spine 2"user@leaf-1# set interfaces et-0/0/51mtu 9192user@leaf-1# set interfaces et-0/0/51 unit 0 family inetmtu 9000user@leaf-1# set interfaces et-0/0/51 unit 0 family inet address 172.16.0.37/31

3. Configure the loopback interface with a reachable IPv4 address. This loopback

address is the tunnel source address.

[edit]user@leaf-1# set interfaces lo0 unit 0 family inet address 10.0.0.21/32

4. Configure the router ID for the leaf device:

[edit]user@leaf-1# set routing-options router-id 10.0.0.21

5. Configure an EBGP-based underlay between the leaf and spine devices and enable

BFD and LLDP:

[edit]user@leaf-1# set protocols bgp group underlay-ipfabric type externaluser@leaf-1# set protocols bgp group underlay-ipfabric mtu-discoveryuser@leaf-1# set protocols bgp group underlay-ipfabric import bgp-ipclos-inuser@leaf-1# set protocols bgp group underlay-ipfabric export bgp-ipclos-outuser@leaf-1# set protocols bgp group underlay-ipfabric local-as 65021user@leaf-1# set protocols bgp group underlay-ipfabric bfd-liveness-detectionminimum-interval 350

user@leaf-1# set protocols bgp group underlay-ipfabric bfd-liveness-detectionmultiplier 3

user@leaf-1# set protocols bgp group underlay-ipfabric bfd-liveness-detectionsession-mode automatic

user@leaf-1# set protocols bgp group underlay-ipfabric multipathmultiple-asuser@leaf-1#setprotocolsbgpgroupunderlay-ipfabricneighbor 172.16.0.32peer-as65011

user@leaf-1#setprotocolsbgpgroupunderlay-ipfabricneighbor 172.16.0.36peer-as65012

user@leaf-1# set protocols lldp interface all

6. Create a routing policy that only advertises and receives loopback addresses from

the IP fabric and EBGP underlay:

[edit]user@leaf-1# set policy-options policy-statement bgp-ipclos-in term loopbacksfrom route-filter 10.0.0.0/16 orlonger

user@leaf-1# set policy-options policy-statement bgp-ipclos-in term loopbacksthen accept



user@leaf-1# set policy-options policy-statement bgp-ipclos-out term loopbackfrom protocol direct

user@leaf-1# set policy-options policy-statement bgp-ipclos-out term loopbackfrom route-filter 10.0.0.21/32 orlonger

user@leaf-1# set policy-options policy-statement bgp-ipclos-out term loopbackthen next-hop self

user@leaf-1# set policy-options policy-statement bgp-ipclos-out term loopbackthen accept

user@leaf-1# set policy-options policy-statement bgp-ipclos-out term reject thenreject

7. Configure an IBGP overlay between the leaf and spine devices, enable BFD and

BMP, and include the EVPN signaling network layer reachability information (NLRI)

in the IBGP group:

[edit]user@leaf-1# set protocols bgp log-updownuser@leaf-1# set protocols bgp graceful-restartuser@leaf-1# set protocols bgp group overlay-evpn type internaluser@leaf-1# set protocols bgp group overlay-evpn local-address 10.0.0.21user@leaf-1# set protocols bgp group overlay-evpn import OVERLAY-INuser@leaf-1# set protocols bgp group overlay-evpn family evpn signalinguser@leaf-1# set protocols bgp group overlay-evpn local-as 65200user@leaf-1# set protocols bgp group overlay-evpn bfd-liveness-detectionminimum-interval 350

user@leaf-1#setprotocolsbgpgroupoverlay-evpnbfd-liveness-detectionmultiplier3

user@leaf-1# set protocols bgp group overlay-evpn bfd-liveness-detectionsession-mode automatic

user@leaf-1# set protocols bgp group overlay-evpnmultipathuser@leaf-1# set protocols bgp group overlay-evpn neighbor 10.0.0.11user@leaf-1# set protocols bgp group overlay-evpn neighbor 10.0.0.12

8. Configure load balancing:

[edit]user@leaf-1# set routing-options forwarding-table export pfe-ecmpuser@leaf-1# set policy-options policy-statement pfe-ecmp then load-balanceper-packet

9. Configurea routingpolicy to reject EVPNType 1 andType2 routes fromspinedevices

in the other POD (this facilitates optimal path selection):

[edit]user@leaf-1# set policy-options policy-statement OVERLAY-IN termreject-remote-gw from family evpn

user@leaf-1# set policy-options policy-statement OVERLAY-IN termreject-remote-gw from next-hop 10.0.0.13

user@leaf-1# set policy-options policy-statement OVERLAY-IN termreject-remote-gw from next-hop 10.0.0.14

user@leaf-1# set policy-options policy-statement OVERLAY-IN termreject-remote-gw from nlri-route-type 1

user@leaf-1# set policy-options policy-statement OVERLAY-IN termreject-remote-gw from nlri-route-type 2

user@leaf-1# set policy-options policy-statement OVERLAY-IN termreject-remote-gw then reject

user@leaf-1# set policy-options policy-statement OVERLAY-IN term accept-allthen accept



10. Configure EVPN:

[edit]user@leaf-1# set protocols evpn encapsulation vxlanuser@leaf-1# set protocols evpn extended-vni-list 1000user@leaf-1# set protocols evpn extended-vni-list 1001user@leaf-1# set protocols evpn extended-vni-list 1002user@leaf-1# set protocols evpn extended-vni-list 1003user@leaf-1# set protocols evpn extended-vni-list 1004user@leaf-1# set protocols evpn extended-vni-list 1005user@leaf-1# set protocols evpn extended-vni-list 1006user@leaf-1# set protocols evpn extended-vni-list 1007user@leaf-1# set protocols evpn extended-vni-list 1008user@leaf-1# set protocols evpn extended-vni-list 1999user@leaf-1# set protocols evpnmulticast-mode ingress-replicationuser@leaf-1#setprotocolsevpnvni-optionsvni 1000vrf-targetexport target:1:1000user@leaf-1# set protocols evpn vni-options vni 1001 vrf-target export target:1:1001user@leaf-1#setprotocolsevpnvni-optionsvni 1002vrf-targetexport target:1:1002user@leaf-1#setprotocolsevpnvni-optionsvni 1003vrf-targetexport target:1:1003user@leaf-1#setprotocolsevpnvni-optionsvni 1004vrf-targetexport target:1:1004user@leaf-1#setprotocolsevpnvni-optionsvni 1005vrf-targetexport target:1:1005user@leaf-1#setprotocolsevpnvni-optionsvni 1006vrf-targetexport target:1:1006user@leaf-1# setprotocols evpnvni-optionsvni 1007vrf-target export target:1:1007user@leaf-1#setprotocolsevpnvni-optionsvni 1008vrf-targetexport target:1:1008user@leaf-1#setprotocolsevpnvni-optionsvni 1999vrf-targetexport target:1:1999

11. Configure a routing policy to import EVPN routes into the switching table and

establish BGP communities:

[edit]user@leaf-1# set policy-options community com1000members target:1:1000user@leaf-1# set policy-options community com1001members target:1:1001user@leaf-1# set policy-options community com1002members target:1:1002user@leaf-1# set policy-options community com1003members target:1:1003user@leaf-1# set policy-options community com1004members target:1:1004user@leaf-1# set policy-options community com1005members target:1:1005user@leaf-1# set policy-options community com1006members target:1:1006user@leaf-1# set policy-options community com1007members target:1:1007user@leaf-1# set policy-options community com1008members target:1:1008user@leaf-1# set policy-options community com1999members target:1:1999user@leaf-1# set policy-options community comm-leaf_esi memberstarget:9999:9999

user@leaf-1# set policy-options policy-statement LEAF-IN term import_leaf_esifrom community comm-leaf_esi

user@leaf-1# set policy-options policy-statement LEAF-IN term import_leaf_esithen accept

user@leaf-1# set policy-options policy-statement LEAF-IN term import_vni1000from community com1000

user@leaf-1# set policy-options policy-statement LEAF-IN term import_vni1000then accept





















user@leaf-1# set policy-options policy-statement LEAF-IN termdefault then reject

12. Configure switch options to set a route distinguisher and VRF target for the EVPN

routing instance, apply the EVPN routing policy, and associate interface lo0 with

the VTEP:

[edit]user@leaf-1# set switch-options route-distinguisher 10.0.0.21:1user@leaf-1# set switch-options vrf-import LEAF-INuser@leaf-1# set switch-options vrf-target target:9999:9999user@leaf-1# set switch-options vtep-source-interface lo0.0

13. Configure VLANs and VXLAN VNIs:

[edit]user@leaf-1# set vlans v1000 vlan-id 100user@leaf-1# set vlans v1000 vxlan vni 1000user@leaf-1# set vlans v1000 vxlan ingress-node-replicationuser@leaf-1# set vlans v1001 vlan-id 101user@leaf-1# set vlans v1001 vxlan vni 1001user@leaf-1# set vlans v1001 vxlan ingress-node-replicationuser@leaf-1# set vlans v1002 vlan-id 102user@leaf-1# set vlans v1002 vxlan vni 1002user@leaf-1# set vlans v1002 vxlan ingress-node-replicationuser@leaf-1# set vlans v1003 vlan-id 103user@leaf-1# set vlans v1003 vxlan vni 1003user@leaf-1# set vlans v1003 vxlan ingress-node-replication



user@leaf-1# set vlans v1004 vlan-id 104user@leaf-1# set vlans v1004 vxlan vni 1004user@leaf-1# set vlans v1004 vxlan ingress-node-replicationuser@leaf-1# set vlans v1005 vlan-id 105user@leaf-1# set vlans v1005 vxlan vni 1005user@leaf-1# set vlans v1005 vxlan ingress-node-replicationuser@leaf-1# set vlans v1006 vlan-id 106user@leaf-1# set vlans v1006 vxlan vni 1006user@leaf-1# set vlans v1006 vxlan ingress-node-replicationuser@leaf-1# set vlans v1007 vlan-id 107user@leaf-1# set vlans v1007 vxlan vni 1007user@leaf-1# set vlans v1007 vxlan ingress-node-replicationuser@leaf-1# set vlans v1008 vlan-id 108user@leaf-1# set vlans v1008 vxlan vni 1008user@leaf-1# set vlans v1008 vxlan ingress-node-replication

Configuring Spine Devices for the IaaS: EVPN and VXLAN Solution


To quickly configure the spine devices, enter the following representative configuration


NOTE: The configuration shown here applies to Spine 1.

[edit]set interfaces et-0/0/58 description "To Fabric 1"set interfaces et-0/0/58mtu 9192set interfaces et-0/0/58 unit 0 family inetmtu 9000set interfaces et-0/0/58 unit 0 family inet address 172.16.0.1/31set interfaces et-0/0/59 description "To Fabric 2"set interfaces et-0/0/59mtu 9192set interfaces et-0/0/59 unit 0 family inetmtu 9000set interfaces et-0/0/59 unit 0 family inet address 172.16.0.9/31set interfaces et-0/0/60 description "To Fabric 4"set interfaces et-0/0/60mtu 9192set interfaces et-0/0/60 unit 0 family inetmtu 9000set interfaces et-0/0/60 unit 0 family inet address 172.16.0.25/31set interfaces et-0/0/61 description "To Fabric 3"set interfaces et-0/0/61mtu 9192set interfaces et-0/0/61 unit 0 family inetmtu 9000set interfaces et-0/0/61 unit 0 family inet address 172.16.0.17/31set interfaces et-0/0/66 description "To Leaf 1"set interfaces et-0/0/66mtu 9192set interfaces et-0/0/66 unit 0 family inetmtu 9000set interfaces et-0/0/66 unit 0 family inet address 172.16.0.32/31set interfaces et-0/0/67 description "To Leaf 2"set interfaces et-0/0/67mtu 9192set interfaces et-0/0/67 unit 0 family inetmtu 9000set interfaces et-0/0/67 unit 0 family inet address 172.16.0.34/31set interfaces irb unit 100 description "Tenant 10 - VLAN 100 - VNI 1000"set interfaces irb unit 100 family inet address 10.1.100.211/24 virtual-gateway-address10.1.100.1



set interfaces irb unit 100 family inet6 address 2001:db8:10:1:100::211/112virtual-gateway-address 2001:db8:10:1:100::1

set interfaces irb unit 100 proxy-macip-advertisementset interfaces irb unit 101 description "Tenant 10 - VLAN 101 - VNI 1001"set interfaces irb unit 101 family inet address 10.1.101.211/24 virtual-gateway-address10.1.101.1
















set interfaces irb unit 108 proxy-macip-advertisementset interfaces irb unit 999 description "EVPN Type 5 - VLAN 999 - VNI 1999"set interfaces irb unit 999 family inet address 10.255.99.211/24 virtual-gateway-address10.255.99.1




set interfaces lo0 unit 0 family inet address 10.0.0.11/32set interfaces lo0 unit 10 family inet address 10.10.0.11/32set interfaces lo0 unit 20 family inet address 10.20.0.11/32set interfaces lo0 unit 999 family inet address 10.9.9.9/32set routing-options router-id 10.0.0.11set routing-options forwarding-table export pfe-ecmpset routing-options forwarding-table ecmp-fast-rerouteset protocols bgp bfd-liveness-detectionminimum-interval 350set protocols bgp bfd-liveness-detectionmultiplier 3set protocols bgp bfd-liveness-detection session-mode automaticset protocols bgp group overlay-evpn type internalset protocols bgp group overlay-evpn local-address 10.0.0.11set protocols bgp group overlay-evpn family evpn signalingset protocols bgp group overlay-evpn local-as 65200set protocols bgp group overlay-evpnmultipathset protocols bgp group overlay-evpn neighbor 10.0.0.12set protocols bgp group overlay-evpn neighbor 10.0.0.13set protocols bgp group overlay-evpn neighbor 10.0.0.14set protocols bgp group overlay-evpn-rr type internalset protocols bgp group overlay-evpn-rr local-address 10.0.0.11set protocols bgp group overlay-evpn-rr family evpn signalingset protocols bgp group overlay-evpn-rr cluster 10.2.2.2set protocols bgp group overlay-evpn-rr local-as 65200set protocols bgp group overlay-evpn-rr multipathset protocols bgp group overlay-evpn-rr neighbor 10.0.0.21set protocols bgp group overlay-evpn-rr neighbor 10.0.0.22set protocols bgp group overlay-evpn-rr export no-type5set protocols bgp group overlay-evpn-rr vpn-apply-exportset protocols bgp group underlay-ipfabric type externalset protocols bgp group underlay-ipfabric mtu-discoveryset protocols bgp group underlay-ipfabric import bgp-ipclos-inset protocols bgp group underlay-ipfabric export bgp-ipclos-outset protocols bgp group underlay-ipfabric local-as 65011set protocols bgp group underlay-ipfabric bfd-liveness-detectionminimum-interval 350set protocols bgp group underlay-ipfabric bfd-liveness-detectionmultiplier 3setprotocolsbgpgroupunderlay-ipfabricbfd-liveness-detectionsession-modeautomaticset protocols bgp group underlay-ipfabric multipathmultiple-asset protocols bgp group underlay-ipfabric neighbor 172.16.0.0 peer-as 65001set protocols bgp group underlay-ipfabric neighbor 172.16.0.8 peer-as 65002set protocols bgp group underlay-ipfabric neighbor 172.16.0.16 peer-as 65003set protocols bgp group underlay-ipfabric neighbor 172.16.0.24 peer-as 65004set protocols bgp group underlay-ipfabric neighbor 172.16.0.33 peer-as 65021set protocols bgp group underlay-ipfabric neighbor 172.16.0.35 peer-as 65022set protocols evpn default-gateway no-gateway-communityset protocols evpn encapsulation vxlanset protocols evpn extended-vni-list 1000set protocols evpn extended-vni-list 1001set protocols evpn extended-vni-list 1002set protocols evpn extended-vni-list 1003set protocols evpn extended-vni-list 1004set protocols evpn extended-vni-list 1005set protocols evpn extended-vni-list 1006set protocols evpn extended-vni-list 1007set protocols evpn extended-vni-list 1008set protocols evpn extended-vni-list 1999set protocols evpnmulticast-mode ingress-replication



set protocols evpn vni-options vni 1000 vrf-target export target:1:1000set protocols evpn vni-options vni 1001 vrf-target export target:1:1001set protocols evpn vni-options vni 1002 vrf-target export target:1:1002set protocols evpn vni-options vni 1003 vrf-target export target:1:1003set protocols evpn vni-options vni 1004 vrf-target export target:1:1004set protocols evpn vni-options vni 1005 vrf-target export target:1:1005set protocols evpn vni-options vni 1006 vrf-target export target:1:1006set protocols evpn vni-options vni 1007 vrf-target export target:1:1007set protocols evpn vni-options vni 1008 vrf-target export target:1:1008set protocols evpn vni-options vni 1999 vrf-target export target:1:1999set protocols lldp interface allset policy-options as-path asPathLength2 ".{2,}"set policy-options community com1000members target:1:1000set policy-options community com1001members target:1:1001set policy-options community com1002members target:1:1002set policy-options community com1003members target:1:1003set policy-options community com1004members target:1:1004set policy-options community com1005members target:1:1005set policy-options community com1006members target:1:1006set policy-options community com1007members target:1:1007set policy-options community com1008members target:1:1008set policy-options community com1999members target:1:1999set policy-options community comm-leaf_esi members target:9999:9999set policy-options community MYCOMMUNITYmembers target:12345:111set policy-options policy-statement bgp-ipclos-in term loopbacks from route-filter10.0.0.0/16 orlonger


set policy-options policy-statement bgp-ipclos-out term loopback then community addMYCOMMUNITY

set policy-options policy-statement bgp-ipclos-out term loopback then next-hop selfset policy-options policy-statement bgp-ipclos-out term loopback then acceptset policy-options policy-statement bgp-ipclos-out term as-path from as-pathasPathLength2

set policy-options policy-statement bgp-ipclos-out term as-path from communityMYCOMMUNITY

set policy-options policy-statement bgp-ipclos-out term as-path then rejectset policy-options policy-statement LEAF-IN term import_vni1000 from communitycom1000











set policy-options policy-statement LEAF-IN term import_vni1008 then acceptset policy-options policy-statement LEAF-IN term import_leaf_esi from communitycomm-leaf_esi

set policy-options policy-statement LEAF-IN term import_leaf_esi then acceptset policy-options policy-statement LEAF-IN term import_vni1999 from communitycom1999

set policy-options policy-statement LEAF-IN term import_vni1999 then acceptset policy-options policy-statement LEAF-IN term default then rejectset policy-options policy-statement no-type5 term t1 from family evpnset policy-options policy-statement no-type5 term t1 from nlri-route-type 5set policy-options policy-statement no-type5 term t1 then rejectset policy-options policy-statement no-type5 term t3 then acceptset policy-options policy-statement pfe-ecmp then load-balance per-packetset policy-options policy-statement TYPE-5-POLICY term v4 from route-filter10.255.99.0/24 orlonger

set policy-options policy-statement TYPE-5-POLICY term v4 then acceptset policy-options policy-statement TYPE-5-POLICY term v6 from route-filter2001:db8:10:255:99::0/112 orlonger

set policy-options policy-statement TYPE-5-POLICY term v6 then acceptset routing-instances TYPE-5 instance-type vrfset routing-instances TYPE-5 interface irb.999set routing-instances TYPE-5 interface lo0.999set routing-instances TYPE-5 route-distinguisher 10.0.0.11:999set routing-instances TYPE-5 vrf-target target:10:999set routing-instances TYPE-5 protocols evpn ip-prefix-routes advertise direct-nexthopset routing-instances TYPE-5 protocols evpn ip-prefix-routes encapsulation vxlanset routing-instances TYPE-5 protocols evpn ip-prefix-routes vni 999set routing-instances TYPE-5 protocols evpn ip-prefix-routes export TYPE-5-POLICYset routing-instances VRF_TENANT_10 instance-type vrfset routing-instances VRF_TENANT_10 interface irb.100set routing-instances VRF_TENANT_10 interface irb.101set routing-instances VRF_TENANT_10 interface irb.102set routing-instances VRF_TENANT_10 interface irb.103set routing-instances VRF_TENANT_10 interface irb.104set routing-instances VRF_TENANT_10 interface lo0.10set routing-instances VRF_TENANT_10 route-distinguisher 10.0.0.11:10set routing-instances VRF_TENANT_10 vrf-target target:10:10set routing-instances VRF_TENANT_20 instance-type vrfset routing-instances VRF_TENANT_20 interface irb.105set routing-instances VRF_TENANT_20 interface irb.106set routing-instances VRF_TENANT_20 interface irb.107set routing-instances VRF_TENANT_20 interface irb.108set routing-instances VRF_TENANT_20 interface lo0.20set routing-instances VRF_TENANT_20 route-distinguisher 10.0.0.11:20set routing-instances VRF_TENANT_20 vrf-target target:10:20set switch-options route-distinguisher 10.0.0.11:1set switch-options vrf-import LEAF-IN



set switch-options vrf-target target:9999:9999set switch-options vtep-source-interface lo0.0set vlans TYPE-5-VLAN vlan-id 999set vlans TYPE-5-VLAN l3-interface irb.999set vlans TYPE-5-VLAN vxlan vni 1999set vlans TYPE-5-VLAN vxlan ingress-node-replicationset vlans v1000 vlan-id 100set vlans v1000 l3-interface irb.100set vlans v1000 vxlan vni 1000set vlans v1000 vxlan ingress-node-replicationset vlans v1001 vlan-id 101set vlans v1001 l3-interface irb.101set vlans v1001 vxlan vni 1001set vlans v1001 vxlan ingress-node-replicationset vlans v1002 vlan-id 102set vlans v1002 l3-interface irb.102set vlans v1002 vxlan vni 1002set vlans v1002 vxlan ingress-node-replicationset vlans v1003 vlan-id 103set vlans v1003 l3-interface irb.103set vlans v1003 vxlan vni 1003set vlans v1003 vxlan ingress-node-replicationset vlans v1004 vlan-id 104set vlans v1004 l3-interface irb.104set vlans v1004 vxlan vni 1004set vlans v1004 vxlan ingress-node-replicationset vlans v1005 vlan-id 105set vlans v1005 l3-interface irb.105set vlans v1005 vxlan vni 1005set vlans v1005 vxlan ingress-node-replicationset vlans v1006 vlan-id 106set vlans v1006 l3-interface irb.106set vlans v1006 vxlan vni 1006set vlans v1006 vxlan ingress-node-replicationset vlans v1007 vlan-id 107set vlans v1007 l3-interface irb.107set vlans v1007 vxlan vni 1007set vlans v1007 vxlan ingress-node-replicationset vlans v1008 vlan-id 108set vlans v1008 l3-interface irb.108set vlans v1008 vxlan vni 1008set vlans v1008 vxlan ingress-node-replication


To configure interfaces on the spine devices:

Configure interfaces to connect to the leaf and fabric devices:1.

[edit]user@spine-1# set interfaces et-0/0/58 description "To Fabric 1"user@spine-1# set interfaces et-0/0/58mtu 9192user@spine-1# set interfaces et-0/0/58 unit 0 family inetmtu 9000user@spine-1# set interfaces et-0/0/58 unit 0 family inet address 172.16.0.1/31user@spine-1# set interfaces et-0/0/59 description "To Fabric 2"user@spine-1# set interfaces et-0/0/59mtu 9192user@spine-1# set interfaces et-0/0/59 unit 0 family inetmtu 9000user@spine-1# set interfaces et-0/0/59 unit 0 family inet address 172.16.0.9/31



user@spine-1# set interfaces et-0/0/60 description "To Fabric 4"user@spine-1# set interfaces et-0/0/60mtu 9192user@spine-1# set interfaces et-0/0/60 unit 0 family inetmtu 9000user@spine-1# set interfaces et-0/0/60 unit 0 family inet address 172.16.0.25/31user@spine-1# set interfaces et-0/0/61 description "To Fabric 3"user@spine-1# set interfaces et-0/0/61mtu 9192user@spine-1# set interfaces et-0/0/61 unit 0 family inetmtu 9000user@spine-1# set interfaces et-0/0/61 unit 0 family inet address 172.16.0.17/31user@spine-1# set interfaces et-0/0/66 description "To Leaf 1"user@spine-1# set interfaces et-0/0/66mtu 9192user@spine-1# set interfaces et-0/0/66 unit 0 family inetmtu 9000user@spine-1# set interfaces et-0/0/66 unit 0 family inet address 172.16.0.32/31user@spine-1# set interfaces et-0/0/67 description "To Leaf 2"user@spine-1# set interfaces et-0/0/67mtu 9192user@spine-1# set interfaces et-0/0/67 unit 0 family inetmtu 9000user@spine-1# set interfaces et-0/0/67 unit 0 family inet address 172.16.0.34/31

2. Configure IRB interfaces with both IPv4 and IPv6 addresses for each VLAN. This

dual stack configuration provides a gateway for both IPv4 and IPv6 hosts:

NOTE: By including the proxy-macip-advertisement statement at the

[edit interfaces irb unit logical-unit-number] hierarchy level, the spine

device generates an EVPN Type 2 proxy advertisement that containsboth the MAC address and the IP route.

[edit]user@spine-1# set interfaces irb unit 100 description "Tenant 10 - VLAN 100 - VNI1000"

user@spine-1# set interfaces irb unit 100 family inet address 10.1.100.211/24virtual-gateway-address 10.1.100.1

user@spine-1# set interfaces irb unit 100 family inet6 address2001:db8:10:1:100::211/112 virtual-gateway-address 2001:db8:10:1:100::1

user@spine-1# set interfaces irb unit 100 proxy-macip-advertisementuser@spine-1# set interfaces irb unit 101 description "Tenant 10 - VLAN 101 - VNI1001"


























user@spine-1# set interfaces irb unit 108 proxy-macip-advertisementuser@spine-1# set interfaces irb unit 999 description "EVPN Type 5 - VLAN 999 -VNI 1999"



3. Configure a loopback interface for the device (lo0) and logical loopback addresses

(lo0.x) for each routing instance:

[edit]user@spine-1# set interfaces lo0 unit 0 family inet address 10.0.0.11/32user@spine-1# set interfaces lo0 unit 10 family inet address 10.10.0.11/32user@spine-1# set interfaces lo0 unit 20 family inet address 10.20.0.11/32user@spine-1# set interfaces lo0 unit 999 family inet address 10.9.9.9/32

4. Configure the router ID for the spine device:

[edit]user@spine-1# set routing-options router-id 10.0.0.11




[edit]user@spine-1# set routing-options forwarding-table export pfe-ecmpuser@spine-1# set routing-options forwarding-table ecmp-fast-rerouteuser@spine-1# set policy-options policy-statement pfe-ecmp then load-balanceper-packet

6. ConfigureanEBGP-basedunderlaybetweenthespineand leafdevices, andbetween

the spine and fabric devices, and enable BFD and LLDP:

[edit]user@spine-1# set protocols bgp group underlay-ipfabric type externaluser@spine-1# set protocols bgp group underlay-ipfabric mtu-discoveryuser@spine-1# set protocols bgp group underlay-ipfabric import bgp-ipclos-inuser@spine-1# set protocols bgp group underlay-ipfabric export bgp-ipclos-outuser@spine-1# set protocols bgp group underlay-ipfabric local-as 65011user@spine-1# set protocols bgp group underlay-ipfabric bfd-liveness-detectionminimum-interval 350

user@spine-1# set protocols bgp group underlay-ipfabric bfd-liveness-detectionmultiplier 3

user@spine-1# set protocols bgp group underlay-ipfabric bfd-liveness-detectionsession-mode automatic

user@spine-1# set protocols bgp group underlay-ipfabric multipathmultiple-asuser@spine-1#setprotocolsbgpgroupunderlay-ipfabricneighbor 172.16.0.0peer-as65001

user@spine-1#setprotocolsbgpgroupunderlay-ipfabricneighbor 172.16.0.8peer-as65002





user@spine-1# set protocols lldp interface all

7. Create a routing policy that only advertises and receives loopback addresses from

the IP fabric and the EBGP underlay:

NOTE: Thispolicyalsosuppressesadvertisements toother spinedeviceloopback interfaces in the same POD and enables optimal routing.

[edit]user@spine-1# set policy-options community MYCOMMUNITYmemberstarget:12345:111

user@spine-1# set policy-options as-path asPathLength2 ".{2,}"user@spine-1# set policy-options policy-statement bgp-ipclos-in term loopbacksfrom route-filter 10.0.0.0/16 orlonger

user@spine-1# set policy-options policy-statement bgp-ipclos-in term loopbacksthen accept

user@spine-1# set policy-options policy-statement bgp-ipclos-out term loopbackfrom protocol direct



user@spine-1# set policy-options policy-statement bgp-ipclos-out term loopbackfrom route-filter 10.0.0.11/32 orlonger

user@spine-1# set policy-options policy-statement bgp-ipclos-out term loopbackthen community addMYCOMMUNITY

user@spine-1# set policy-options policy-statement bgp-ipclos-out term loopbackthen next-hop self

user@spine-1# set policy-options policy-statement bgp-ipclos-out term loopbackthen accept

user@spine-1# set policy-options policy-statement bgp-ipclos-out term as-pathfrom as-path asPathLength2

user@spine-1# set policy-options policy-statement bgp-ipclos-out term as-pathfrom community MYCOMMUNITY

user@spine-1# set policy-options policy-statement bgp-ipclos-out term as-paththen reject

8. Configure an IBGP overlay between the spine and leaf devices, enable a BGP route

reflector cluster andBMP,and include theEVPNsignalingnetwork layer reachability

information (NLRI) in the IBGP group:

[edit]user@spine-1# set protocols bgp group overlay-evpn-rr type internaluser@spine-1# set protocols bgp group overlay-evpn-rr local-address 10.0.0.11user@spine-1# set protocols bgp group overlay-evpn-rr family evpn signalinguser@spine-1# set protocols bgp group overlay-evpn-rr cluster 10.2.2.2user@spine-1# set protocols bgp group overlay-evpn-rr local-as 65200user@spine-1# set protocols bgp group overlay-evpn-rr multipathuser@spine-1# set protocols bgp group overlay-evpn-rr neighbor 10.0.0.21user@spine-1# set protocols bgp group overlay-evpn-rr neighbor 10.0.0.22user@spine-1# set protocols bgp group overlay-evpn-rr export no-type5user@spine-1# set protocols bgp group overlay-evpn-rr vpn-apply-export

9. Configurea routingpolicy that suppressesEVPNType5 routes frombeingadvertised

to the leaf devices (that are using Type 2 routes instead):

[edit]user@spine-1# set policy-options policy-statement no-type5 term t1 from familyevpn

user@spine-1# set policy-options policy-statement no-type5 term t1 fromnlri-route-type 5

user@spine-1# set policy-options policy-statement no-type5 term t1 then rejectuser@spine-1# set policy-options policy-statement no-type5 term t3 then accept

10. Configure a second IBGP overlay to connect the spine devices to each other and

include the EVPN signaling network layer reachability information (NLRI) in the

IBGP group:

[edit]user@spine-1# set protocols bgp group overlay-evpn type internaluser@spine-1# set protocols bgp group overlay-evpn local-address 10.0.0.11user@spine-1# set protocols bgp group overlay-evpn family evpn signalinguser@spine-1# set protocols bgp group overlay-evpn local-as 65200user@spine-1# set protocols bgp group overlay-evpnmultipathuser@spine-1# set protocols bgp group overlay-evpn neighbor 10.0.0.12user@spine-1# set protocols bgp group overlay-evpn neighbor 10.0.0.13user@spine-1# set protocols bgp group overlay-evpn neighbor 10.0.0.14

11. Configure BFD for all BGP sessions:



[edit]user@spine-1# set protocols bgp bfd-liveness-detectionminimum-interval 350user@spine-1# set protocols bgp bfd-liveness-detectionmultiplier 3user@spine-1# set protocols bgp bfd-liveness-detection session-mode automatic

12. Configure EVPN:

NOTE: By including the no-gateway-community statement at the [edit

protocols evpn default-gateway] hierarchy level, the spine device

advertises the MAC address of the IRB interface without the defaultgateway community.

[edit]user@spine-1# set protocols evpn default-gateway no-gateway-communityuser@spine-1# set protocols evpn encapsulation vxlanuser@spine-1# set protocols evpn extended-vni-list 1000user@spine-1# set protocols evpn extended-vni-list 1001user@spine-1# set protocols evpn extended-vni-list 1002user@spine-1# set protocols evpn extended-vni-list 1003user@spine-1# set protocols evpn extended-vni-list 1004user@spine-1# set protocols evpn extended-vni-list 1005user@spine-1# set protocols evpn extended-vni-list 1006user@spine-1# set protocols evpn extended-vni-list 1007user@spine-1# set protocols evpn extended-vni-list 1008user@spine-1# set protocols evpn extended-vni-list 1999user@spine-1# set protocols evpnmulticast-mode ingress-replicationuser@spine-1#setprotocolsevpnvni-optionsvni 1000vrf-targetexport target:1:1000user@spine-1#setprotocolsevpnvni-optionsvni 1001vrf-targetexport target:1:1001user@spine-1#setprotocolsevpnvni-optionsvni 1002vrf-targetexport target:1:1002user@spine-1#setprotocolsevpnvni-optionsvni 1003vrf-targetexport target:1:1003user@spine-1#setprotocolsevpnvni-optionsvni 1004vrf-targetexport target:1:1004user@spine-1#setprotocolsevpnvni-optionsvni 1005vrf-targetexport target:1:1005user@spine-1#setprotocolsevpnvni-optionsvni 1006vrf-targetexport target:1:1006user@spine-1#setprotocolsevpnvni-optionsvni 1007vrf-targetexport target:1:1007user@spine-1#setprotocolsevpnvni-optionsvni 1008vrf-targetexport target:1:1008user@spine-1#setprotocolsevpnvni-optionsvni 1999vrf-targetexport target:1:1999

13. Configure switch options to set a route distinguisher and VRF target for the EVPN

routing instance, apply the EVPN routing policy, and associate interface lo0 with

the VTEP:

[edit]user@spine-1# set switch-options route-distinguisher 10.0.0.11:1user@spine-1# set switch-options vrf-import LEAF-INuser@spine-1# set switch-options vrf-target target:9999:9999user@spine-1# set switch-options vtep-source-interface lo0.0

14. Configure a routing policy to import EVPN routes to the switching table:

[edit]user@spine-1# set policy-options policy-statement LEAF-IN term import_vni1000from community com1000

user@spine-1# set policy-options policy-statement LEAF-IN term import_vni1000then accept



user@spine-1# set policy-options policy-statement LEAF-IN term import_vni1001from community com1001
















user@spine-1# set policy-options policy-statement LEAF-IN term import_leaf_esifrom community comm-leaf_esi

user@spine-1# set policy-options policy-statement LEAF-IN term import_leaf_esithen accept



user@spine-1#setpolicy-optionspolicy-statementLEAF-INtermdefault then rejectuser@spine-1# set policy-options community com1000members target:1:1000user@spine-1# set policy-options community com1001members target:1:1001user@spine-1# set policy-options community com1002members target:1:1002user@spine-1# set policy-options community com1003members target:1:1003user@spine-1# set policy-options community com1004members target:1:1004user@spine-1# set policy-options community com1005members target:1:1005user@spine-1# set policy-options community com1006members target:1:1006user@spine-1# set policy-options community com1007members target:1:1007user@spine-1# set policy-options community com1008members target:1:1008user@spine-1# set policy-options community com1999members target:1:1999user@spine-1# set policy-options community comm-leaf_esi memberstarget:9999:9999

15. Create a policy to export IPv4 and IPv6 network addresses for Type 5 routes:



[edit]user@spine-1# set policy-options policy-statement TYPE-5-POLICY term v4 fromroute-filter 10.255.99.0/24 orlonger

user@spine-1# set policy-options policy-statement TYPE-5-POLICY term v4 thenaccept

user@spine-1# set policy-options policy-statement TYPE-5-POLICY term v6 fromroute-filter 2001:db8:10:255:99::0/112 orlonger

user@spine-1# set policy-options policy-statement TYPE-5-POLICY term v6 thenaccept

16. Configure VLANs and VXLAN VNIs:

[edit]user@spine-1# set vlans TYPE-5-VLAN vlan-id 999user@spine-1# set vlans TYPE-5-VLAN l3-interface irb.999user@spine-1# set vlans TYPE-5-VLAN vxlan vni 1999user@spine-1# set vlans TYPE-5-VLAN vxlan ingress-node-replicationuser@spine-1# set vlans v1000 vlan-id 100user@spine-1# set vlans v1000 l3-interface irb.100user@spine-1# set vlans v1000 vxlan vni 1000user@spine-1# set vlans v1000 vxlan ingress-node-replicationuser@spine-1# set vlans v1001 vlan-id 101user@spine-1# set vlans v1001 l3-interface irb.101user@spine-1# set vlans v1001 vxlan vni 1001user@spine-1# set vlans v1001 vxlan ingress-node-replicationuser@spine-1# set vlans v1002 vlan-id 102user@spine-1# set vlans v1002 l3-interface irb.102user@spine-1# set vlans v1002 vxlan vni 1002user@spine-1# set vlans v1002 vxlan ingress-node-replicationuser@spine-1# set vlans v1003 vlan-id 103user@spine-1# set vlans v1003 l3-interface irb.103user@spine-1# set vlans v1003 vxlan vni 1003user@spine-1# set vlans v1003 vxlan ingress-node-replicationuser@spine-1# set vlans v1004 vlan-id 104user@spine-1# set vlans v1004 l3-interface irb.104user@spine-1# set vlans v1004 vxlan vni 1004user@spine-1# set vlans v1004 vxlan ingress-node-replicationuser@spine-1# set vlans v1005 vlan-id 105user@spine-1# set vlans v1005 l3-interface irb.105user@spine-1# set vlans v1005 vxlan vni 1005user@spine-1# set vlans v1005 vxlan ingress-node-replicationuser@spine-1# set vlans v1006 vlan-id 106user@spine-1# set vlans v1006 l3-interface irb.106user@spine-1# set vlans v1006 vxlan vni 1006user@spine-1# set vlans v1006 vxlan ingress-node-replicationuser@spine-1# set vlans v1007 vlan-id 107user@spine-1# set vlans v1007 l3-interface irb.107user@spine-1# set vlans v1007 vxlan vni 1007user@spine-1# set vlans v1007 vxlan ingress-node-replicationuser@spine-1# set vlans v1008 vlan-id 108user@spine-1# set vlans v1008 l3-interface irb.108user@spine-1# set vlans v1008 vxlan vni 1008user@spine-1# set vlans v1008 vxlan ingress-node-replication

17. Configure three routing instances—one for a tenant that uses EVPN Type 5 within

its data center, and two for tenants that use EVPN Type 2:



EVPN Type 5 Routing Instance

[edit]user@spine-1# set routing-instances TYPE-5 instance-type vrfuser@spine-1# set routing-instances TYPE-5 interface irb.999user@spine-1# set routing-instances TYPE-5 interface lo0.999user@spine-1# set routing-instances TYPE-5 route-distinguisher 10.0.0.11:999user@spine-1# set routing-instances TYPE-5 vrf-target target:10:999user@spine-1# set routing-instances TYPE-5 protocols evpn ip-prefix-routesadvertisedirect-nexthop##The ip-prefix-routesstatementhelpscreate theType5 routes.

user@spine-1# set routing-instances TYPE-5 protocols evpn ip-prefix-routesencapsulation vxlan

user@spine-1# set routing-instances TYPE-5 protocols evpn ip-prefix-routes vni999

user@spine-1#set routing-instancesTYPE-5protocolsevpn ip-prefix-routesexportTYPE-5-POLICY ##Apply the Type 5 routing policy.

EVPN Type 2 Routing Instance (VRF Tenant 10)

[edit]user@spine-1# set routing-instances VRF_TENANT_10 instance-type vrfuser@spine-1# set routing-instances VRF_TENANT_10 interface irb.100user@spine-1# set routing-instances VRF_TENANT_10 interface irb.101user@spine-1# set routing-instances VRF_TENANT_10 interface irb.102user@spine-1# set routing-instances VRF_TENANT_10 interface irb.103user@spine-1# set routing-instances VRF_TENANT_10 interface irb.104user@spine-1# set routing-instances VRF_TENANT_10 interface lo0.10user@spine-1#set routing-instancesVRF_TENANT_10route-distinguisher 10.0.0.11:10user@spine-1# set routing-instances VRF_TENANT_10 vrf-target target:10:10

EVPN Type 2 Routing Instance (VRF Tenant 20)

[edit]user@spine-1# set routing-instances VRF_TENANT_20 instance-type vrfuser@spine-1# set routing-instances VRF_TENANT_20 interface irb.105user@spine-1# set routing-instances VRF_TENANT_20 interface irb.106user@spine-1# set routing-instances VRF_TENANT_20 interface irb.107user@spine-1# set routing-instances VRF_TENANT_20 interface irb.108user@spine-1# set routing-instances VRF_TENANT_20 interface lo0.20user@spine-1# set routing-instances VRF_TENANT_20 route-distinguisher10.0.0.11:20

user@spine-1# set routing-instances VRF_TENANT_20 vrf-target target:10:20

Configuring Fabric Devices for the IaaS: EVPN and VXLAN Solution


To quickly configure the fabric devices, enter the following representative configuration


NOTE: The configuration shown here applies to Fabric 1.

[edit]set interfaces et-0/0/12 description "To Spine 4"



set interfaces et-0/0/12mtu 9192set interfaces et-0/0/12 unit 0 family inetmtu 9000set interfaces et-0/0/12 unit 0 family inet address 172.16.0.6/31set interfaces et-0/0/13 description "To Spine 3"set interfaces et-0/0/13mtu 9192set interfaces et-0/0/13 unit 0 family inetmtu 9000set interfaces et-0/0/13 unit 0 family inet address 172.16.0.4/31set interfaces et-0/0/14 description "To Spine 2"set interfaces et-0/0/14mtu 9192set interfaces et-0/0/14 unit 0 family inetmtu 9000set interfaces et-0/0/14 unit 0 family inet address 172.16.0.2/31set interfaces et-0/0/15 description "To Spine 1"set interfaces et-0/0/15mtu 9192set interfaces et-0/0/15 unit 0 family inetmtu 9000set interfaces et-0/0/15 unit 0 family inet address 172.16.0.0/31set interfaces lo0 unit 0 family inet address 10.0.0.1/32set forwarding-options enhanced-hash-key ecmp-resilient-hashset routing-options forwarding-table export pblbset routing-options router-id 10.0.0.1set protocols bgp log-updownset protocols bgp graceful-restartset protocols bgp group underlay-ipfabric type externalset protocols bgp group underlay-ipfabric mtu-discoveryset protocols bgp group underlay-ipfabric import bgp-ipclos-inset protocols bgp group underlay-ipfabric export bgp-ipclos-outset protocols bgp group underlay-ipfabric local-as 65001set protocols bgp group underlay-ipfabric bfd-liveness-detectionminimum-interval 350set protocols bgp group underlay-ipfabric bfd-liveness-detectionmultiplier 3setprotocolsbgpgroupunderlay-ipfabricbfd-liveness-detectionsession-modeautomaticset protocols bgp group underlay-ipfabric multipathmultiple-asset protocols bgp group underlay-ipfabric neighbor 172.16.0.1 peer-as 65011set protocols bgp group underlay-ipfabric neighbor 172.16.0.3 peer-as 65012set protocols bgp group underlay-ipfabric neighbor 172.16.0.5 peer-as 65013set protocols bgp group underlay-ipfabric neighbor 172.16.0.7 peer-as 65014set protocols lldp interface allset policy-options policy-statement bgp-ipclos-in term loopbacks from route-filter10.0.0.0/16 orlonger


set policy-options policy-statement bgp-ipclos-out term loopback then next-hop selfset policy-options policy-statement bgp-ipclos-out term loopback then acceptset policy-options policy-statement pplb then load-balance per-packet


To configure interfaces on the spine devices:

Configure interfaces to connect to the spine devices in both PODs:1.

[edit]user@fabric-1# set interfaces et-0/0/12 description "To Spine 4"user@fabric-1# set interfaces et-0/0/12mtu 9192user@fabric-1# set interfaces et-0/0/12 unit 0 family inetmtu 9000user@fabric-1# set interfaces et-0/0/12 unit 0 family inet address 172.16.0.6/31user@fabric-1# set interfaces et-0/0/13 description "To Spine 3"user@fabric-1# set interfaces et-0/0/13mtu 9192



user@fabric-1# set interfaces et-0/0/13 unit 0 family inetmtu 9000user@fabric-1# set interfaces et-0/0/13 unit 0 family inet address 172.16.0.4/31user@fabric-1# set interfaces et-0/0/14 description "To Spine 2"user@fabric-1# set interfaces et-0/0/14mtu 9192user@fabric-1# set interfaces et-0/0/14 unit 0 family inetmtu 9000user@fabric-1# set interfaces et-0/0/14 unit 0 family inet address 172.16.0.2/31user@fabric-1# set interfaces et-0/0/15 description "To Spine 1"user@fabric-1# set interfaces et-0/0/15mtu 9192user@fabric-1# set interfaces et-0/0/15 unit 0 family inetmtu 9000user@fabric-1# set interfaces et-0/0/15 unit 0 family inet address 172.16.0.0/31user@fabric-1# set interfaces lo0 unit 0 family inet address 10.0.0.1/32


[edit]user@fabric-1# set forwarding-options enhanced-hash-key ecmp-resilient-hashuser@fabric-1# set routing-options forwarding-table export pblbuser@fabric-1# set policy-options policy-statement pplb then load-balanceper-packet

3. Configure the router ID for the fabric device:

[edit]user@fabric-1# set routing-options router-id 10.0.0.1

4. Complete the underlay by configuring an EBGP session with each spine device,

enabling LLDP on all interfaces, and creating a routing policy that accepts the

loopbackaddresses fromall devices in the IP fabric andadvertises its own loopback

address:

[edit]user@fabric-1# set protocols bgp log-updownuser@fabric-1# set protocols bgp graceful-restartuser@fabric-1# set protocols bgp group underlay-ipfabric type externaluser@fabric-1# set protocols bgp group underlay-ipfabric mtu-discoveryuser@fabric-1# set protocols bgp group underlay-ipfabric import bgp-ipclos-inuser@fabric-1# set protocols bgp group underlay-ipfabric export bgp-ipclos-outuser@fabric-1# set protocols bgp group underlay-ipfabric local-as 65001user@fabric-1# set protocols bgp group underlay-ipfabric bfd-liveness-detectionminimum-interval 350

user@fabric-1# set protocols bgp group underlay-ipfabric bfd-liveness-detectionmultiplier 3

user@fabric-1# set protocols bgp group underlay-ipfabric bfd-liveness-detectionsession-mode automatic

user@fabric-1# set protocols bgp group underlay-ipfabric multipathmultiple-asuser@fabric-1#setprotocolsbgpgroupunderlay-ipfabricneighbor 172.16.0.1peer-as65011

user@fabric-1#setprotocolsbgpgroupunderlay-ipfabricneighbor 172.16.0.3peer-as65012



user@fabric-1# set protocols lldp interface alluser@fabric-1# set policy-options policy-statement bgp-ipclos-in term loopbacksfrom route-filter 10.0.0.0/16 orlonger

user@fabric-1# set policy-options policy-statement bgp-ipclos-in term loopbacksthen accept



user@fabric-1# set policy-options policy-statement bgp-ipclos-out term loopbackfrom protocol direct

user@fabric-1# set policy-options policy-statement bgp-ipclos-out term loopbackfrom route-filter 10.0.0.1/32 exact

user@fabric-1# set policy-options policy-statement bgp-ipclos-out term loopbackthen next-hop self

user@fabric-1# set policy-options policy-statement bgp-ipclos-out term loopbackthen accept

Configuring Host Multihoming


Sometenants require their hosts tobemultihomedtomultiple leafdevices for redundancy

and resiliency. To enable Host 8 to bemultihomed to Leaf 3 and Leaf 4:

1. Configure Switch 5 to permit traffic to flow between Host 8, Leaf 3, and Leaf 4.

Create an aggregated Ethernet interface that includes a link to Leaf 3 and a link to

Leaf 4 in the bundle, establish a trunk port, and enable VLANs 100 through 108.

[edit]user@switch5# set chassis aggregated-devices ethernet device-count 10user@switch5# set chassis fpc 0 pic 0 port 48 channel-speed 10guser@switch5# set interfaces xe-0/0/48:0 unit 0 family ethernet-switchinginterface-mode trunk

user@switch5# set interfaces xe-0/0/48:0 unit 0 family ethernet-switching vlanmembers 100-108

user@switch5# set interfaces et-0/0/52 ether-options 802.3ad ae0user@switch5# set interfaces et-0/0/53 ether-options 802.3ad ae0user@switch5# set interfaces ae0 aggregated-ether-options lacp activeuser@switch5# set interfaces ae0unit 0 family ethernet-switching interface-modetrunk

user@switch5# set interfaces ae0 unit 0 family ethernet-switching vlanmembers100-108

user@switch5# set routing-options autonomous-system 65069user@switch5# set vlans v1000 vlan-id 100user@switch5# set vlans v1001 vlan-id 101user@switch5# set vlans v1002 vlan-id 102user@switch5# set vlans v1003 vlan-id 103user@switch5# set vlans v1004 vlan-id 104user@switch5# set vlans v1005 vlan-id 105user@switch5# set vlans v1006 vlan-id 106user@switch5# set vlans v1007 vlan-id 107user@switch5# set vlans v1008 vlan-id 108

2. Configure Leaf 3 so it can connect with Host 8 through Switch 5. Configure an

aggregated Ethernet interface to connect to Switch 5, establish a trunk port, set an

EVPN Ethernet segment identifier (ESI), and permit VLANs 100 through 108.

[edit]user@leaf-3# set chassis aggregated-devices ethernet device-count 10user@leaf-3# set chassis fpc 0 pic 0 port 52 channel-speed 10guser@leaf-3# set interfaces et-0/0/52 description "To Switch 5 and Host 8"user@leaf-3# set interfaces et-0/0/52 ether-options 802.3ad ae0user@leaf-3# set interfaces ae0 description "To Switch 5 and Host 8"user@leaf-3# set interfaces ae0 esi 00:01:01:01:01:01:01:01:01:01user@leaf-3# set interfaces ae0 esi all-activeuser@leaf-3# set interfaces ae0 aggregated-ether-options lacp active



user@leaf-3# set interfaces ae0 aggregated-ether-options lacp system-id00:00:00:01:01:01

user@leaf-3# set interfaces ae0 unit 0 family ethernet-switching interface-modetrunk

user@leaf-3# set interfaces ae0 unit 0 family ethernet-switching vlanmembers100-108

3. Configure Leaf 4 so it can connect with Host 8 through Switch 5. Configure an

aggregated Ethernet interface to connect to Switch 5, establish a trunk port, set an

ESI, and permit VLANs 100 through 108.

[edit]user@leaf-3# set chassis aggregated-devices ethernet device-count 10user@leaf-3# set chassis fpc 0 pic 0 port 53 channel-speed 10guser@leaf-4# set interfaces et-0/0/53 description "To Switch 5 and Host 8"user@leaf-4# set interfaces et-0/0/53 ether-options 802.3ad ae0user@leaf-4# set interfaces ae0 description "To Switch 5 and Host 8"user@leaf-4# set interfaces ae0 esi 00:01:01:01:01:01:01:01:01:01user@leaf-4# set interfaces ae0 esi all-activeuser@leaf-4# set interfaces ae0 aggregated-ether-options lacp activeuser@leaf-4# set interfaces ae0 aggregated-ether-options lacp system-id00:00:00:01:01:01

user@leaf-4# set interfaces ae0 unit 0 family ethernet-switching interface-modetrunk

user@leaf-4# set interfaces ae0 unit 0 family ethernet-switching vlanmembers100-108

Configuring Additional Features for the IaaS: EVPN and VXLAN Solution

In this section, youconfigureBGPMonitoringProtocol (BMP), distributeddenial of service

(DDoS) protection, storm control, and class of service on all devices to enhance the

capabilities of the network described in this IaaS solution.

• Configuring BMP, DDoS Protection, Storm Control, CoS, and Port Mirroring on page 51

Configuring BMP, DDoS Protection, Storm Control, CoS, and Port Mirroring


To configure BMP, DDoS protection, storm control, CoS, and port mirroring:

NOTE: The following configurations are taken from Leaf 1, so remember toextend this configurationmodel to the other devices in the IP fabric.

1. Configure BMP on all devices in the IP fabric:

[edit]user@leaf-1#setgroupsBMProuting-optionsbmpstationBMP-1 initiation-message"BMPMessage from spine-1"

user@leaf-1#setgroupsBMProuting-optionsbmpstationBMP-1connection-modeactive

user@leaf-1# set groups BMP routing-options bmp station BMP-1 station-address10.94.63.195

user@leaf-1#setgroupsBMProuting-optionsbmpstationBMP-1station-port 11019



user@leaf-1#setgroupsBMProuting-optionsbmpstationBMP-1statistics-timeout60

user@leaf-1# set groups BMP protocols bgp bmpmonitor enableuser@leaf-1# set groups BMP protocols bgp bmp route-monitoring post-policyexclude-non-eligible

user@leaf-1# set apply-groups BMP

2. Configure DDoS to protect the Routing Engine of your device:

[edit]user@leaf-1# set groups RE_Filter interfaces lo0 unit 0 family inet filter inputDOS-Protect

user@leaf-1# set groups RE_Filter policy-options prefix-list bgp-peer-addresses10.0.0.1/32













user@leaf-1# set groups RE_Filter policy-options prefix-list telnet-ssh-addresses172.17.38.0/24




user@leaf-1# set groups RE_Filter policy-options prefix-list snmp-addresses172.17.12.17/32










user@leaf-1# set groups RE_Filter policy-options prefix-list dns-addresses172.17.12.19/32





user@leaf-1# set groups RE_Filter policy-options prefix-list Trusted-Addresses10.0.0.0/8



user@leaf-1# set groups RE_Filter firewall family inet filter DOS-Protect termMartian-Address-Discard from address 0.0.0.0/8







user@leaf-1# set groups RE_Filter firewall family inet filter DOS-Protect termMartian-Address-Discard then discard

user@leaf-1# set groups RE_Filter firewall family inet filter DOS-Protect termallow-trusted-port from interface-set Trusted-Interfaces

user@leaf-1# set groups RE_Filter firewall family inet filter DOS-Protect termallow-trusted-port then accept

user@leaf-1# set groups RE_Filter firewall family inet filter DOS-Protect termip-fragments-limit-1 from first-fragment

user@leaf-1# set groups RE_Filter firewall family inet filter DOS-Protect termip-fragments-limit-1 then policer IP-FRAG-Policer

user@leaf-1# set groups RE_Filter firewall family inet filter DOS-Protect termip-fragments-limit-2 from fragment-offset 64-65535

user@leaf-1# set groups RE_Filter firewall family inet filter DOS-Protect termip-fragments-limit-2 then policer IP-FRAG-Policer

user@leaf-1# set groups RE_Filter firewall family inet filter DOS-Protect termip-option-limit from ip-options any

user@leaf-1# set groups RE_Filter firewall family inet filter DOS-Protect termip-option-limit then policer IP-OPT-Policer

user@leaf-1# set groups RE_Filter firewall family inet filter DOS-Protect termtcp-syn-fin-limit from protocol tcp



user@leaf-1# set groups RE_Filter firewall family inet filter DOS-Protect termtcp-syn-fin-limit from port bgp

user@leaf-1# set groups RE_Filter firewall family inet filter DOS-Protect termtcp-syn-fin-limit from port ldp

user@leaf-1# set groups RE_Filter firewall family inet filter DOS-Protect termtcp-syn-fin-limit from port snmp

user@leaf-1# set groups RE_Filter firewall family inet filter DOS-Protect termtcp-syn-fin-limit then policer TCP-SYN-Policer

user@leaf-1# set groups RE_Filter firewall family inet filter DOS-Protect term bgpfrom source-prefix-list bgp-peer-addresses

user@leaf-1# set groups RE_Filter firewall family inet filter DOS-Protect term bgpfrom protocol tcp

user@leaf-1# set groups RE_Filter firewall family inet filter DOS-Protect term bgpfrom port bgp

user@leaf-1# set groups RE_Filter firewall family inet filter DOS-Protect term bgpfrom interface-set BGP-Interfaces

user@leaf-1# set groups RE_Filter firewall family inet filter DOS-Protect term bgpthen accept

user@leaf-1# set groups RE_Filter firewall family inet filter DOS-Protect termaccept-icmp from source-address 10.0.0.0/8



user@leaf-1# set groups RE_Filter firewall family inet filter DOS-Protect termaccept-icmp from protocol icmp

user@leaf-1# set groups RE_Filter firewall family inet filter DOS-Protect termaccept-icmp from icmp-type echo-request

user@leaf-1# set groups RE_Filter firewall family inet filter DOS-Protect termaccept-icmp from icmp-type echo-reply

user@leaf-1# set groups RE_Filter firewall family inet filter DOS-Protect termaccept-icmp from icmp-type unreachable

user@leaf-1# set groups RE_Filter firewall family inet filter DOS-Protect termaccept-icmp from icmp-type redirect

user@leaf-1# set groups RE_Filter firewall family inet filter DOS-Protect termaccept-icmp from icmp-type parameter-problem

user@leaf-1# set groups RE_Filter firewall family inet filter DOS-Protect termaccept-icmp then count icmp-count

user@leaf-1# set groups RE_Filter firewall family inet filter DOS-Protect termaccept-icmp then accept

user@leaf-1# set groups RE_Filter firewall family inet filter DOS-Protect termtraceroute-limit from source-prefix-list Trusted-Addresses

user@leaf-1# set groups RE_Filter firewall family inet filter DOS-Protect termtraceroute-limit from protocol udp

user@leaf-1# set groups RE_Filter firewall family inet filter DOS-Protect termtraceroute-limit from destination-port 33434-33524

user@leaf-1# set groups RE_Filter firewall family inet filter DOS-Protect termtraceroute-limit then policer Traceroute-Policer

user@leaf-1# set groups RE_Filter firewall family inet filter DOS-Protect termtraceroute-limit then accept

user@leaf-1# set groups RE_Filter firewall family inet filter DOS-Protect termtelnet-ssh from source-prefix-list telnet-ssh-addresses

user@leaf-1# set groups RE_Filter firewall family inet filter DOS-Protect termtelnet-ssh from protocol tcp

user@leaf-1# set groups RE_Filter firewall family inet filter DOS-Protect termtelnet-ssh from port telnet



user@leaf-1# set groups RE_Filter firewall family inet filter DOS-Protect termtelnet-ssh from port ssh

user@leaf-1# set groups RE_Filter firewall family inet filter DOS-Protect termtelnet-ssh then policer telnet-ssh-policer

user@leaf-1# set groups RE_Filter firewall family inet filter DOS-Protect termtelnet-ssh then accept

user@leaf-1# set groups RE_Filter firewall family inet filter DOS-Protect term snmpfrom source-prefix-list snmp-addresses

user@leaf-1# set groups RE_Filter firewall family inet filter DOS-Protect term snmpfrom protocol udp

user@leaf-1# set groups RE_Filter firewall family inet filter DOS-Protect term snmpfrom port snmp

user@leaf-1# set groups RE_Filter firewall family inet filter DOS-Protect term snmpfrom port snmptrap

user@leaf-1# set groups RE_Filter firewall family inet filter DOS-Protect term snmpthen policer snmp-policer

user@leaf-1# set groups RE_Filter firewall family inet filter DOS-Protect term snmpthen accept

user@leaf-1# set groups RE_Filter firewall family inet filter DOS-Protect term dnsfrom source-prefix-list dns-addresses

user@leaf-1# set groups RE_Filter firewall family inet filter DOS-Protect term dnsfrom protocol udp

user@leaf-1# set groups RE_Filter firewall family inet filter DOS-Protect term dnsfrom source-port domain

user@leaf-1# set groups RE_Filter firewall family inet filter DOS-Protect term dnsthen policer dns-policer

user@leaf-1# set groups RE_Filter firewall family inet filter DOS-Protect term dnsthen accept

user@leaf-1#setgroupsRE_Filter firewall family inet filterDOS-Protect termdefaultthen log

user@leaf-1#setgroupsRE_Filter firewall family inet filterDOS-Protect termdefaultthen discard

user@leaf-1# set groups RE_Filter firewall policer dns-policer if-exceedingbandwidth-limit 8k

user@leaf-1# set groups RE_Filter firewall policer dns-policer if-exceedingburst-size-limit 1500

user@leaf-1# set groups RE_Filter firewall policer dns-policer then discarduser@leaf-1# set groups RE_Filter firewall policer snmp-policer if-exceedingbandwidth-limit 12k

user@leaf-1# set groups RE_Filter firewall policer snmp-policer if-exceedingburst-size-limit 1500

user@leaf-1# set groups RE_Filter firewall policer snmp-policer then discarduser@leaf-1# set groups RE_Filter firewall policer telnet-ssh-policer if-exceedingbandwidth-limit 15k

user@leaf-1# set groups RE_Filter firewall policer telnet-ssh-policer if-exceedingburst-size-limit 1500

user@leaf-1# set groups RE_Filter firewall policer telnet-ssh-policer then discarduser@leaf-1# set groups RE_Filter firewall policer ARP-Policer if-exceedingbandwidth-limit 8k

user@leaf-1# set groups RE_Filter firewall policer ARP-Policer if-exceedingburst-size-limit 1500

user@leaf-1# set groups RE_Filter firewall policer ARP-Policer then discarduser@leaf-1# set groups RE_Filter firewall policer IP-FRAG-Policer filter-specificuser@leaf-1# set groups RE_Filter firewall policer IP-FRAG-Policer if-exceedingbandwidth-limit 10k



user@leaf-1# set groups RE_Filter firewall policer IP-FRAG-Policer if-exceedingburst-size-limit 1500

user@leaf-1# set groups RE_Filter firewall policer IP-FRAG-Policer then discarduser@leaf-1# set groups RE_Filter firewall policer IP-OPT-Policer if-exceedingbandwidth-limit 8k

user@leaf-1# set groups RE_Filter firewall policer IP-OPT-Policer if-exceedingburst-size-limit 1500

user@leaf-1# set groups RE_Filter firewall policer IP-OPT-Policer then discarduser@leaf-1# set groups RE_Filter firewall policer Traceroute-Policer if-exceedingbandwidth-limit 8k

user@leaf-1# set groups RE_Filter firewall policer Traceroute-Policer if-exceedingburst-size-limit 1500

user@leaf-1# set groups RE_Filter firewall policer Traceroute-Policer then discarduser@leaf-1# set groups RE_Filter firewall policer TCP-SYN-Policer if-exceedingbandwidth-limit 8k

user@leaf-1# set groups RE_Filter firewall policer TCP-SYN-Policer if-exceedingburst-size-limit 1500

user@leaf-1# set groups RE_Filter firewall policer TCP-SYN-Policer then discarduser@leaf-1# set groups RE_Filter firewall interface-set Trusted-Interfacesxe-0/0/12.0

user@leaf-1# set groups RE_Filter firewall interface-set Trusted-Interfacesxe-0/0/13.0

user@leaf-1# set groups RE_Filter firewall interface-set Trusted-Interfaceset-0/0/50.0

user@leaf-1# set groups RE_Filter firewall interface-set Trusted-Interfaceset-0/0/51.0

user@leaf-1#setgroupsRE_Filter firewall interface-setBGP-Interfaceset-0/0/50.0user@leaf-1#setgroupsRE_Filter firewall interface-setBGP-Interfaceset-0/0/51.0user@leaf-1# set apply-groups RE_Filter

3. Configure storm control on all devices in the IP fabric:

[edit]user@leaf-1# set groups StormControl interfaces xe-0/0/12 unit 0 familyethernet-switching storm-control stm-ctrl

user@leaf-1# set groups StormControl forwarding-options storm-control-profilesstm-ctrl all bandwidth-percentage 40

user@leaf-1# set groups StormControl forwarding-options storm-control-profilesstm-ctrl all no-multicast

user@leaf-1# set apply-groups StormControl

4. Configure CoS on all devices in the IP fabric:

[edit]user@leaf-1# set groups COS_8Q class-of-service classifiers dscp dscp-clforwarding-class CS0 loss-priority low code-points 000000

user@leaf-1# set groups COS_8Q class-of-service classifiers dscp dscp-clforwarding-class CS1 loss-priority low code-points 001000









user@leaf-1# set groups COS_8Q class-of-service forwarding-classes class CS0queue-num0

user@leaf-1# set groups COS_8Q class-of-service forwarding-classes class CS1queue-num 1





user@leaf-1# set groups COS_8Q class-of-service forwarding-classes class CS6queue-num6


user@leaf-1# set groups COS_8Q class-of-service drop-profiles be-dp interpolatefill-level 30

user@leaf-1# set groups COS_8Q class-of-service drop-profiles be-dp interpolatefill-level 50

user@leaf-1# set groups COS_8Q class-of-service drop-profiles be-dp interpolatedrop-probability 0

user@leaf-1# set groups COS_8Q class-of-service drop-profiles be-dp interpolatedrop-probability 80

user@leaf-1# set groups COS_8Q class-of-service drop-profiles dp-be-lowinterpolate fill-level 25

user@leaf-1# set groups COS_8Q class-of-service drop-profiles dp-be-lowinterpolate fill-level 50

user@leaf-1# set groups COS_8Q class-of-service drop-profiles dp-be-lowinterpolate drop-probability 0

user@leaf-1# set groups COS_8Q class-of-service drop-profiles dp-be-lowinterpolate drop-probability 80

user@leaf-1# set groups COS_8Q class-of-service drop-profiles dp-be-highinterpolate fill-level 10

user@leaf-1# set groups COS_8Q class-of-service drop-profiles dp-be-highinterpolate fill-level 40

user@leaf-1# set groups COS_8Q class-of-service drop-profiles dp-be-highinterpolate drop-probability 0

user@leaf-1# set groups COS_8Q class-of-service drop-profiles dp-be-highinterpolate drop-probability 100

user@leaf-1# set groups COS_8Q class-of-service traffic-control-profiles tcp-1scheduler-map smap-1

user@leaf-1# set groups COS_8Q class-of-service traffic-control-profiles tcp-1guaranteed-rate percent 50

user@leaf-1# set groups COS_8Q class-of-service traffic-control-profiles tcp-2scheduler-map smap-2

user@leaf-1# set groups COS_8Q class-of-service traffic-control-profiles tcp-2guaranteed-rate percent 50

user@leaf-1# set groups COS_8Q class-of-service forwarding-class-sets fc-set-1class CS0










user@leaf-1# set groups COS_8Q class-of-service interfaces xe-0/0/12forwarding-class-set fc-set-1 output-traffic-control-profile tcp-1

user@leaf-1# set groups COS_8Q class-of-service interfaces xe-0/0/12forwarding-class-set fc-set-2 output-traffic-control-profile tcp-2

user@leaf-1# set groups COS_8Q class-of-service interfaces et-0/0/50 classifiersdscp dscp-cl

user@leaf-1# set groups COS_8Q class-of-service interfaces et-0/0/51 classifiersdscp dscp-cl

user@leaf-1# set groups COS_8Q class-of-service scheduler-maps smap-1forwarding-class CS0 scheduler be-sched

user@leaf-1# set groups COS_8Q class-of-service scheduler-maps smap-1forwarding-class CS1 scheduler sched-2







user@leaf-1#setgroupsCOS_8Qclass-of-serviceschedulersbe-schedtransmit-rate3g

user@leaf-1#setgroupsCOS_8Qclass-of-serviceschedulersbe-schedshaping-rate5g

user@leaf-1# set groups COS_8Q class-of-service schedulers be-sched priority lowuser@leaf-1# set groups COS_8Q class-of-service schedulers be-scheddrop-profile-map loss-priority low protocol any drop-profile dp-be-low

user@leaf-1# set groups COS_8Q class-of-service schedulers be-scheddrop-profile-map loss-priority high protocol any drop-profile dp-be-high

user@leaf-1# set groups COS_8Q class-of-service schedulers be-schedexplicit-congestion-notification

user@leaf-1# setgroupsCOS_8Qclass-of-service schedulers sched-1 transmit-ratepercent 5

user@leaf-1# set groups COS_8Q class-of-service schedulers sched-1 priority lowuser@leaf-1#setgroupsCOS_8Qclass-of-serviceschedulerssched-2 transmit-rate4g

user@leaf-1# set groupsCOS_8Qclass-of-service schedulers sched-2 shaping-rate5g

user@leaf-1# set groups COS_8Q class-of-service schedulers sched-2 priority low



user@leaf-1# set groups COS_8Q class-of-service schedulers sched-2drop-profile-map loss-priority high protocol any drop-profile dp-be-high

user@leaf-1# set groups COS_8Q class-of-service schedulers sched-2drop-profile-map loss-priority low protocol any drop-profile dp-be-low

user@leaf-1#setgroupsCOS_8Qclass-of-serviceschedulerssched-3 transmit-ratepercent 10

user@leaf-1# set groups COS_8Q class-of-service schedulers sched-3 priority lowuser@leaf-1# set apply-groups COS_8Quser@leaf-1# set firewall familyethernet-switching filterEthFilter interface-specificuser@leaf-1# set firewall family ethernet-switching filter EthFilter term prec0 fromdscp cs0

user@leaf-1# set firewall family ethernet-switching filter EthFilter term prec0 thenforwarding-class CS0

user@leaf-1# set firewall family ethernet-switching filter EthFilter term prec0 thenloss-priority low

user@leaf-1# set firewall family ethernet-switching filter EthFilter term prec0 thencount CS0

user@leaf-1# set firewall family ethernet-switching filter EthFilter term prec1 fromdscp cs1









user@leaf-1# set firewall family ethernet-switching filter EthFilter term prec3 fromsource-port 80


user@leaf-1# set firewall family ethernet-switching filter EthFilter term prec3 thenloss-priority high




















user@leaf-1# set firewall family ethernet-switching filter EthFilter term def thenaccept

user@leaf-1# set firewall family ethernet-switching filter EthFilter term def thencount DEF

user@leaf-1# set interfaces xe-0/0/12 unit 0 family ethernet-switching filter inputEthFilter

user@leaf-1# set interfaces xe-0/0/13 unit 0 family ethernet-switching filter inputEthFilter

5. Configure port mirroring on all devices in the IP fabric:

[edit]user@leaf-1# set groups pm interfaces <et-*> unit 0 family inet filter input PMuser@leaf-1# set interfaces apply-groups pmuser@leaf-1# set forwarding-options port-mirroring instance VXLAN_PM_Instancefamily ethernet-switching output interface xe-0/0/34:1.0

user@leaf-1# set firewall family inet filter PM interface-specificuser@leaf-1# set firewall family inet filter PM term t1 from destination-port 4789user@leaf-1# set firewall family inet filter PM term t1 then count PM_T1user@leaf-1# set firewall family inet filter PM term t1 then port-mirroruser@leaf-1# set firewall family inet filter PM term t1 then acceptuser@leaf-1# set firewall family inet filter PM term def then count DEFuser@leaf-1# set firewall family inet filter PM term def then port-mirroruser@leaf-1# set firewall family inet filter PM term def then accept

Verification

Confirm that the IaaS: EVPN and VXLAN solution configuration is working properly.

• Leaf: Verifying Interfaces on page 61

• Leaf: Verifying IPv4 BGP Sessions on page 62

• Leaf: Verifying BFD on page 63

• Leaf: Verifying EVPN Routes on page 64

• Leaf: Verifying the EVPN Routes in Detail on page 74



• Leaf: Verifying VTEP Interfaces on page 75

• Leaf: Verifying VNI-to-VXLAN Tunnel Mappings on page 78

• Leaf: Verifying MAC Address Learning on page 80

• Leaf: Verifying Multihoming on page 82

• Leaf: Verifying ECMP on page 84

• Leaf: Verifying Remote MAC Address Reachability Through ECMP on page 85

• Leaf: Verifying Local and Remote MAC Address Learning on page 86

• Spine: Verifying Interfaces on page 89

• Spine: Verifying IPv4 BGP Sessions on page 89

• Spine: Verifying BFD on page 91

• Spine: Verifying the IRB Interfaces on page 91

• Spine: Verifying VTEP Interfaces on page 92

• Spine: Verifying VTEP Destination Addresses on page 93

• Spine: Verifying Inter-Spine ECMP on page 95

• Spine: Verifying the Routing Instances on page 96

• Spine: Verifying the Layer 3 Gateway on page 100

• Spine: Verifying the Switching Table on page 107

• Spine: Verifying the Source of the VXLAN Tunnel on page 109

• Spine: Verifying VNI-to-VXLAN Tunnel Mapping on page 110

• Spine: Verifying MAC Address Learning on page 112

• Fabric: Verifying Interfaces on page 114

• Fabric: Verifying IPv4 BGP Sessions on page 115

• Fabric: Verifying BFD on page 115

• All Devices: Verifying Port Mirroring on page 116

Leaf: Verifying Interfaces

Purpose Verify the state of the server-facing and spine-facing interfaces.



Action Verify that the server-facing and spine-facing interfaces are up:

user@leaf-1> show interfaces terseInterface Admin Link Proto Local Remotegr-0/0/0 up uppfe-0/0/0 up uppfe-0/0/0.16383 up up inet inet6 pfh-0/0/0 up uppfh-0/0/0.16383 up up inet xe-0/0/1 up upxe-0/0/1.16386 up up xe-0/0/12 up upxe-0/0/12.0 up up eth-switch ## <<< Connected to Host 1xe-0/0/13 up upxe-0/0/13.0 up up eth-switch ## <<< Connected to Host 5xe-0/0/24 up upxe-0/0/24.0 up up eth-switchxe-0/0/25 up upxe-0/0/25.0 up up eth-switchxe-0/0/26 up upxe-0/0/26.0 up up eth-switchet-0/0/50 up upet-0/0/50.0 up up inet 172.16.0.33/31 ## <<< Connected to Spine 1et-0/0/51 up upet-0/0/51.0 up up inet 172.16.0.37/31 ## <<< Connected to Spine 2

Meaning The server-facing and spine-facing interfaces are connected and operating correctly.

Leaf: Verifying IPv4 BGP Sessions

Purpose Verify the state of underlay (EBGP) and overlay (IBGP) sessions between the leaf and

spine devices.



Action Verify that IPv4 EBGP and IBGP sessions are established with Spine 1 and Spine 2:

user@leaf-1> show bgp summaryGroups: 2 Peers: 4 Down peers: 0Table Tot Paths Act Paths Suppressed History Damp State Pendingbgp.evpn.0 1558 615 0 0 0 0inet.0 44 38 0 0 0 0Peer AS InPkt OutPkt OutQ Flaps Last Up/Dwn State|#Active/Received/Accepted/Damped...10.0.0.11 65200 178053 101798 0 9 6d 3:17:10 Establ bgp.evpn.0: 515/779/615/0 default-switch.evpn.0: 515/779/615/0 __default_evpn__.evpn.0: 0/0/0/010.0.0.12 65200 194908 108744 0 5 6d 15:55:04 Establ bgp.evpn.0: 100/779/615/0 default-switch.evpn.0: 100/779/615/0 __default_evpn__.evpn.0: 0/0/0/0172.16.0.32 65011 19554 19446 0 9 6d 3:17:12 Establ inet.0: 22/22/22/0172.16.0.36 65012 21232 21116 0 5 6d 15:55:12 Establ inet.0: 16/22/22/0

Meaning Because therearepeer connections toAS65011 (underlay toSpine 1), AS65012 (underlay

to Spine 2), and AS 65200 (overlay to both spine devices), both the EBGP and IBGP

sessions are established and functioning correctly.

Leaf: Verifying BFD

Purpose Verify that bidirectional forwarding detection is operating correctly between the leaf and

spine devices.

Action Verify that BFD is operating between Leaf 1, Spine 1, and Spine 2:

user@leaf-1> show bfd session Detect TransmitAddress State Interface Time Interval Multiplier10.0.0.11 Up 1.050 0.350 3 10.0.0.12 Up 1.050 0.350 3 172.16.0.32 Up et-0/0/50.0 1.050 0.350 3 172.16.0.36 Up et-0/0/51.0 1.050 0.350 3

4 sessions, 4 clientsCumulative transmit rate 11.4 pps, cumulative receive rate 11.4 pps

Meaning BFD is operating correctly between the leaf and spine devices for both the underlay and

the overlay.



Leaf: Verifying EVPN Routes

Purpose Verify that the EVPN routes are being learned through the overlay.



Action Issue the show route table bgp.evpn.0 command to display the status of learned EVPN

routes for VNI 1000:

user@leaf-1> show route table bgp.evpn.0 evpn-ethernet-tag-id 1000

bgp.evpn.0: 783 destinations, 1566 routes (619 active, 0 holddown, 328 hidden)+ = Active Route, - = Last Active, * = Both

2:10.0.0.11:1::1000::00:00:5e:00:01:01/304 *[BGP/170] 1w2d 03:18:08, localpref 100, from 10.0.0.11 AS path: I, validation-state: unverified > to 172.16.0.32 via et-0/0/50.0 [BGP/170] 1w2d 03:18:10, localpref 100, from 10.0.0.12 AS path: I, validation-state: unverified > to 172.16.0.32 via et-0/0/50.02:10.0.0.11:1::1000::00:00:5e:00:02:01/304 *[BGP/170] 1w2d 03:18:08, localpref 100, from 10.0.0.11 AS path: I, validation-state: unverified > to 172.16.0.32 via et-0/0/50.0 [BGP/170] 1w2d 03:18:10, localpref 100, from 10.0.0.12 AS path: I, validation-state: unverified > to 172.16.0.32 via et-0/0/50.02:10.0.0.11:1::1000::00:31:46:7b:e1:18/304 *[BGP/170] 1w2d 03:18:08, localpref 100, from 10.0.0.11 AS path: I, validation-state: unverified > to 172.16.0.32 via et-0/0/50.0 [BGP/170] 1w2d 03:18:10, localpref 100, from 10.0.0.12 AS path: I, validation-state: unverified > to 172.16.0.32 via et-0/0/50.02:10.0.0.12:1::1000::00:00:5e:00:01:01/304 *[BGP/170] 1w2d 15:56:02, localpref 100, from 10.0.0.12 AS path: I, validation-state: unverified > to 172.16.0.36 via et-0/0/51.0 [BGP/170] 1w2d 03:18:08, localpref 100, from 10.0.0.11 AS path: I, validation-state: unverified > to 172.16.0.36 via et-0/0/51.02:10.0.0.12:1::1000::00:00:5e:00:02:01/304 *[BGP/170] 1w2d 15:56:02, localpref 100, from 10.0.0.12 AS path: I, validation-state: unverified > to 172.16.0.36 via et-0/0/51.0 [BGP/170] 1w2d 03:18:08, localpref 100, from 10.0.0.11 AS path: I, validation-state: unverified > to 172.16.0.36 via et-0/0/51.02:10.0.0.12:1::1000::ec:3e:f7:89:15:1a/304 *[BGP/170] 1w2d 15:56:02, localpref 100, from 10.0.0.12 AS path: I, validation-state: unverified > to 172.16.0.36 via et-0/0/51.0 [BGP/170] 1w2d 03:18:08, localpref 100, from 10.0.0.11 AS path: I, validation-state: unverified > to 172.16.0.36 via et-0/0/51.02:10.0.0.22:1::1000::00:00:5e:00:54:10/304 *[BGP/170] 00:00:47, localpref 100, from 10.0.0.11 AS path: I, validation-state: unverified > to 172.16.0.32 via et-0/0/50.0 to 172.16.0.36 via et-0/0/51.0 [BGP/170] 00:00:47, localpref 100, from 10.0.0.12 AS path: I, validation-state: unverified > to 172.16.0.32 via et-0/0/50.0 to 172.16.0.36 via et-0/0/51.02:10.0.0.22:1::1000::00:00:5e:00:54:11/304



*[BGP/170] 00:00:47, localpref 100, from 10.0.0.11 AS path: I, validation-state: unverified > to 172.16.0.32 via et-0/0/50.0 to 172.16.0.36 via et-0/0/51.0 [BGP/170] 00:00:47, localpref 100, from 10.0.0.12 AS path: I, validation-state: unverified > to 172.16.0.32 via et-0/0/50.0 to 172.16.0.36 via et-0/0/51.02:10.0.0.22:1::1000::00:00:5e:00:54:12/304 *[BGP/170] 00:00:47, localpref 100, from 10.0.0.11 AS path: I, validation-state: unverified > to 172.16.0.32 via et-0/0/50.0 to 172.16.0.36 via et-0/0/51.0 [BGP/170] 00:00:47, localpref 100, from 10.0.0.12 AS path: I, validation-state: unverified > to 172.16.0.32 via et-0/0/50.0 to 172.16.0.36 via et-0/0/51.02:10.0.0.22:1::1000::00:00:5e:00:54:13/304 *[BGP/170] 00:00:47, localpref 100, from 10.0.0.11 AS path: I, validation-state: unverified > to 172.16.0.32 via et-0/0/50.0 to 172.16.0.36 via et-0/0/51.0 [BGP/170] 00:00:47, localpref 100, from 10.0.0.12 AS path: I, validation-state: unverified > to 172.16.0.32 via et-0/0/50.0 to 172.16.0.36 via et-0/0/51.02:10.0.0.22:1::1000::00:00:5e:00:54:14/304 *[BGP/170] 00:00:47, localpref 100, from 10.0.0.11 AS path: I, validation-state: unverified > to 172.16.0.32 via et-0/0/50.0 to 172.16.0.36 via et-0/0/51.0 [BGP/170] 00:00:47, localpref 100, from 10.0.0.12 AS path: I, validation-state: unverified > to 172.16.0.32 via et-0/0/50.0 to 172.16.0.36 via et-0/0/51.02:10.0.0.22:1::1000::de:ad:be:e1:00:20/304 *[BGP/170] 00:00:47, localpref 100, from 10.0.0.11 AS path: I, validation-state: unverified > to 172.16.0.32 via et-0/0/50.0 to 172.16.0.36 via et-0/0/51.0 [BGP/170] 00:00:47, localpref 100, from 10.0.0.12 AS path: I, validation-state: unverified > to 172.16.0.32 via et-0/0/50.0 to 172.16.0.36 via et-0/0/51.02:10.0.0.22:1::1000::de:ad:be:e1:00:21/304 *[BGP/170] 00:00:47, localpref 100, from 10.0.0.11 AS path: I, validation-state: unverified > to 172.16.0.32 via et-0/0/50.0 to 172.16.0.36 via et-0/0/51.0 [BGP/170] 00:00:47, localpref 100, from 10.0.0.12 AS path: I, validation-state: unverified > to 172.16.0.32 via et-0/0/50.0 to 172.16.0.36 via et-0/0/51.02:10.0.0.22:1::1000::de:ad:be:e1:00:22/304 *[BGP/170] 00:00:47, localpref 100, from 10.0.0.11 AS path: I, validation-state: unverified > to 172.16.0.32 via et-0/0/50.0 to 172.16.0.36 via et-0/0/51.0 [BGP/170] 00:00:47, localpref 100, from 10.0.0.12 AS path: I, validation-state: unverified > to 172.16.0.32 via et-0/0/50.0



to 172.16.0.36 via et-0/0/51.02:10.0.0.22:1::1000::de:ad:be:e1:00:23/304 *[BGP/170] 00:00:47, localpref 100, from 10.0.0.11 AS path: I, validation-state: unverified > to 172.16.0.32 via et-0/0/50.0 to 172.16.0.36 via et-0/0/51.0 [BGP/170] 00:00:47, localpref 100, from 10.0.0.12 AS path: I, validation-state: unverified > to 172.16.0.32 via et-0/0/50.0 to 172.16.0.36 via et-0/0/51.02:10.0.0.22:1::1000::de:ad:be:e1:00:24/304 *[BGP/170] 00:00:47, localpref 100, from 10.0.0.11 AS path: I, validation-state: unverified > to 172.16.0.32 via et-0/0/50.0 to 172.16.0.36 via et-0/0/51.0 [BGP/170] 00:00:47, localpref 100, from 10.0.0.12 AS path: I, validation-state: unverified > to 172.16.0.32 via et-0/0/50.0 to 172.16.0.36 via et-0/0/51.02:10.0.0.22:1::1000::fa:ce:b0:01:00:20/304 *[BGP/170] 00:00:47, localpref 100, from 10.0.0.11 AS path: I, validation-state: unverified to 172.16.0.32 via et-0/0/50.0 > to 172.16.0.36 via et-0/0/51.0 [BGP/170] 00:00:47, localpref 100, from 10.0.0.12 AS path: I, validation-state: unverified to 172.16.0.32 via et-0/0/50.0 > to 172.16.0.36 via et-0/0/51.02:10.0.0.22:1::1000::fa:ce:b0:01:00:21/304 *[BGP/170] 00:00:47, localpref 100, from 10.0.0.11 AS path: I, validation-state: unverified to 172.16.0.32 via et-0/0/50.0 > to 172.16.0.36 via et-0/0/51.0 [BGP/170] 00:00:47, localpref 100, from 10.0.0.12 AS path: I, validation-state: unverified to 172.16.0.32 via et-0/0/50.0 > to 172.16.0.36 via et-0/0/51.02:10.0.0.22:1::1000::fa:ce:b0:01:00:22/304 *[BGP/170] 00:00:47, localpref 100, from 10.0.0.11 AS path: I, validation-state: unverified to 172.16.0.32 via et-0/0/50.0 > to 172.16.0.36 via et-0/0/51.0 [BGP/170] 00:00:47, localpref 100, from 10.0.0.12 AS path: I, validation-state: unverified to 172.16.0.32 via et-0/0/50.0 > to 172.16.0.36 via et-0/0/51.02:10.0.0.22:1::1000::fa:ce:b0:01:00:23/304 *[BGP/170] 00:00:47, localpref 100, from 10.0.0.11 AS path: I, validation-state: unverified to 172.16.0.32 via et-0/0/50.0 > to 172.16.0.36 via et-0/0/51.0 [BGP/170] 00:00:47, localpref 100, from 10.0.0.12 AS path: I, validation-state: unverified to 172.16.0.32 via et-0/0/50.0 > to 172.16.0.36 via et-0/0/51.02:10.0.0.22:1::1000::fa:ce:b0:01:00:24/304 *[BGP/170] 00:00:47, localpref 100, from 10.0.0.11 AS path: I, validation-state: unverified to 172.16.0.32 via et-0/0/50.0 > to 172.16.0.36 via et-0/0/51.0 [BGP/170] 00:00:47, localpref 100, from 10.0.0.12



AS path: I, validation-state: unverified to 172.16.0.32 via et-0/0/50.0 > to 172.16.0.36 via et-0/0/51.02:10.0.0.23:1::1000::00:00:5e:00:55:10/304 *[BGP/170] 00:08:47, localpref 100, from 10.0.0.11 AS path: I, validation-state: unverified > to 172.16.0.32 via et-0/0/50.0 to 172.16.0.36 via et-0/0/51.0 [BGP/170] 00:08:47, localpref 100, from 10.0.0.12 AS path: I, validation-state: unverified > to 172.16.0.32 via et-0/0/50.0 to 172.16.0.36 via et-0/0/51.02:10.0.0.23:1::1000::00:00:5e:00:55:11/304 *[BGP/170] 00:08:47, localpref 100, from 10.0.0.11 AS path: I, validation-state: unverified > to 172.16.0.32 via et-0/0/50.0 to 172.16.0.36 via et-0/0/51.0 [BGP/170] 00:08:47, localpref 100, from 10.0.0.12 AS path: I, validation-state: unverified > to 172.16.0.32 via et-0/0/50.0 to 172.16.0.36 via et-0/0/51.02:10.0.0.23:1::1000::00:00:5e:00:55:12/304 *[BGP/170] 00:08:47, localpref 100, from 10.0.0.11 AS path: I, validation-state: unverified > to 172.16.0.32 via et-0/0/50.0 to 172.16.0.36 via et-0/0/51.0 [BGP/170] 00:08:47, localpref 100, from 10.0.0.12 AS path: I, validation-state: unverified > to 172.16.0.32 via et-0/0/50.0 to 172.16.0.36 via et-0/0/51.02:10.0.0.23:1::1000::00:00:5e:00:55:13/304 *[BGP/170] 00:08:47, localpref 100, from 10.0.0.11 AS path: I, validation-state: unverified > to 172.16.0.32 via et-0/0/50.0 to 172.16.0.36 via et-0/0/51.0 [BGP/170] 00:08:47, localpref 100, from 10.0.0.12 AS path: I, validation-state: unverified > to 172.16.0.32 via et-0/0/50.0 to 172.16.0.36 via et-0/0/51.02:10.0.0.23:1::1000::00:00:5e:00:55:14/304 *[BGP/170] 00:08:47, localpref 100, from 10.0.0.11 AS path: I, validation-state: unverified > to 172.16.0.32 via et-0/0/50.0 to 172.16.0.36 via et-0/0/51.0 [BGP/170] 00:08:47, localpref 100, from 10.0.0.12 AS path: I, validation-state: unverified > to 172.16.0.32 via et-0/0/50.0 to 172.16.0.36 via et-0/0/51.02:10.0.0.23:1::1000::de:ad:be:e1:00:30/304 *[BGP/170] 00:08:47, localpref 100, from 10.0.0.11 AS path: I, validation-state: unverified > to 172.16.0.32 via et-0/0/50.0 to 172.16.0.36 via et-0/0/51.0 [BGP/170] 00:08:47, localpref 100, from 10.0.0.12 AS path: I, validation-state: unverified > to 172.16.0.32 via et-0/0/50.0 to 172.16.0.36 via et-0/0/51.02:10.0.0.23:1::1000::de:ad:be:e1:00:31/304 *[BGP/170] 00:08:47, localpref 100, from 10.0.0.11 AS path: I, validation-state: unverified > to 172.16.0.32 via et-0/0/50.0



to 172.16.0.36 via et-0/0/51.0 [BGP/170] 00:08:47, localpref 100, from 10.0.0.12 AS path: I, validation-state: unverified > to 172.16.0.32 via et-0/0/50.0 to 172.16.0.36 via et-0/0/51.02:10.0.0.23:1::1000::de:ad:be:e1:00:32/304 *[BGP/170] 00:08:47, localpref 100, from 10.0.0.11 AS path: I, validation-state: unverified > to 172.16.0.32 via et-0/0/50.0 to 172.16.0.36 via et-0/0/51.0 [BGP/170] 00:08:47, localpref 100, from 10.0.0.12 AS path: I, validation-state: unverified > to 172.16.0.32 via et-0/0/50.0 to 172.16.0.36 via et-0/0/51.02:10.0.0.23:1::1000::de:ad:be:e1:00:33/304 *[BGP/170] 00:08:47, localpref 100, from 10.0.0.11 AS path: I, validation-state: unverified > to 172.16.0.32 via et-0/0/50.0 to 172.16.0.36 via et-0/0/51.0 [BGP/170] 00:08:47, localpref 100, from 10.0.0.12 AS path: I, validation-state: unverified > to 172.16.0.32 via et-0/0/50.0 to 172.16.0.36 via et-0/0/51.02:10.0.0.23:1::1000::de:ad:be:e1:00:34/304 *[BGP/170] 00:08:47, localpref 100, from 10.0.0.11 AS path: I, validation-state: unverified > to 172.16.0.32 via et-0/0/50.0 to 172.16.0.36 via et-0/0/51.0 [BGP/170] 00:08:47, localpref 100, from 10.0.0.12 AS path: I, validation-state: unverified > to 172.16.0.32 via et-0/0/50.0 to 172.16.0.36 via et-0/0/51.02:10.0.0.23:1::1000::fa:ce:b0:01:00:30/304 *[BGP/170] 00:08:47, localpref 100, from 10.0.0.11 AS path: I, validation-state: unverified to 172.16.0.32 via et-0/0/50.0 > to 172.16.0.36 via et-0/0/51.0 [BGP/170] 00:08:47, localpref 100, from 10.0.0.12 AS path: I, validation-state: unverified to 172.16.0.32 via et-0/0/50.0 > to 172.16.0.36 via et-0/0/51.02:10.0.0.23:1::1000::fa:ce:b0:01:00:31/304 *[BGP/170] 00:08:47, localpref 100, from 10.0.0.11 AS path: I, validation-state: unverified to 172.16.0.32 via et-0/0/50.0 > to 172.16.0.36 via et-0/0/51.0 [BGP/170] 00:08:47, localpref 100, from 10.0.0.12 AS path: I, validation-state: unverified to 172.16.0.32 via et-0/0/50.0 > to 172.16.0.36 via et-0/0/51.02:10.0.0.23:1::1000::fa:ce:b0:01:00:32/304 *[BGP/170] 00:08:47, localpref 100, from 10.0.0.11 AS path: I, validation-state: unverified to 172.16.0.32 via et-0/0/50.0 > to 172.16.0.36 via et-0/0/51.0 [BGP/170] 00:08:47, localpref 100, from 10.0.0.12 AS path: I, validation-state: unverified to 172.16.0.32 via et-0/0/50.0 > to 172.16.0.36 via et-0/0/51.02:10.0.0.23:1::1000::fa:ce:b0:01:00:33/304 *[BGP/170] 00:08:47, localpref 100, from 10.0.0.11



AS path: I, validation-state: unverified to 172.16.0.32 via et-0/0/50.0 > to 172.16.0.36 via et-0/0/51.0 [BGP/170] 00:08:47, localpref 100, from 10.0.0.12 AS path: I, validation-state: unverified to 172.16.0.32 via et-0/0/50.0 > to 172.16.0.36 via et-0/0/51.02:10.0.0.23:1::1000::fa:ce:b0:01:00:34/304 *[BGP/170] 00:08:47, localpref 100, from 10.0.0.11 AS path: I, validation-state: unverified to 172.16.0.32 via et-0/0/50.0 > to 172.16.0.36 via et-0/0/51.0 [BGP/170] 00:08:47, localpref 100, from 10.0.0.12 AS path: I, validation-state: unverified to 172.16.0.32 via et-0/0/50.0 > to 172.16.0.36 via et-0/0/51.02:10.0.0.23:1::1000::fa:ce:b0:01:00:42/304 *[BGP/170] 00:08:47, localpref 100, from 10.0.0.11 AS path: I, validation-state: unverified to 172.16.0.32 via et-0/0/50.0 > to 172.16.0.36 via et-0/0/51.0 [BGP/170] 00:08:47, localpref 100, from 10.0.0.12 AS path: I, validation-state: unverified to 172.16.0.32 via et-0/0/50.0 > to 172.16.0.36 via et-0/0/51.02:10.0.0.24:1::1000::00:00:5e:00:56:10/304 *[BGP/170] 00:00:37, localpref 100, from 10.0.0.11 AS path: I, validation-state: unverified to 172.16.0.32 via et-0/0/50.0 > to 172.16.0.36 via et-0/0/51.0 [BGP/170] 00:00:37, localpref 100, from 10.0.0.12 AS path: I, validation-state: unverified to 172.16.0.32 via et-0/0/50.0 > to 172.16.0.36 via et-0/0/51.02:10.0.0.24:1::1000::00:00:5e:00:56:11/304 *[BGP/170] 00:00:36, localpref 100, from 10.0.0.11 AS path: I, validation-state: unverified to 172.16.0.32 via et-0/0/50.0 > to 172.16.0.36 via et-0/0/51.0 [BGP/170] 00:00:36, localpref 100, from 10.0.0.12 AS path: I, validation-state: unverified to 172.16.0.32 via et-0/0/50.0 > to 172.16.0.36 via et-0/0/51.02:10.0.0.24:1::1000::00:00:5e:00:56:12/304 *[BGP/170] 00:00:36, localpref 100, from 10.0.0.11 AS path: I, validation-state: unverified to 172.16.0.32 via et-0/0/50.0 > to 172.16.0.36 via et-0/0/51.0 [BGP/170] 00:00:36, localpref 100, from 10.0.0.12 AS path: I, validation-state: unverified to 172.16.0.32 via et-0/0/50.0 > to 172.16.0.36 via et-0/0/51.02:10.0.0.24:1::1000::00:00:5e:00:56:13/304 *[BGP/170] 00:00:36, localpref 100, from 10.0.0.11 AS path: I, validation-state: unverified to 172.16.0.32 via et-0/0/50.0 > to 172.16.0.36 via et-0/0/51.0 [BGP/170] 00:00:36, localpref 100, from 10.0.0.12 AS path: I, validation-state: unverified to 172.16.0.32 via et-0/0/50.0 > to 172.16.0.36 via et-0/0/51.0



2:10.0.0.24:1::1000::00:00:5e:00:56:14/304 *[BGP/170] 00:00:36, localpref 100, from 10.0.0.11 AS path: I, validation-state: unverified to 172.16.0.32 via et-0/0/50.0 > to 172.16.0.36 via et-0/0/51.0 [BGP/170] 00:00:36, localpref 100, from 10.0.0.12 AS path: I, validation-state: unverified to 172.16.0.32 via et-0/0/50.0 > to 172.16.0.36 via et-0/0/51.02:10.0.0.24:1::1000::de:ad:be:e1:00:40/304 *[BGP/170] 00:00:36, localpref 100, from 10.0.0.11 AS path: I, validation-state: unverified to 172.16.0.32 via et-0/0/50.0 > to 172.16.0.36 via et-0/0/51.0 [BGP/170] 00:00:36, localpref 100, from 10.0.0.12 AS path: I, validation-state: unverified to 172.16.0.32 via et-0/0/50.0 > to 172.16.0.36 via et-0/0/51.02:10.0.0.24:1::1000::de:ad:be:e1:00:41/304 *[BGP/170] 00:00:36, localpref 100, from 10.0.0.11 AS path: I, validation-state: unverified to 172.16.0.32 via et-0/0/50.0 > to 172.16.0.36 via et-0/0/51.0 [BGP/170] 00:00:36, localpref 100, from 10.0.0.12 AS path: I, validation-state: unverified to 172.16.0.32 via et-0/0/50.0 > to 172.16.0.36 via et-0/0/51.02:10.0.0.24:1::1000::de:ad:be:e1:00:42/304 *[BGP/170] 00:00:36, localpref 100, from 10.0.0.11 AS path: I, validation-state: unverified to 172.16.0.32 via et-0/0/50.0 > to 172.16.0.36 via et-0/0/51.0 [BGP/170] 00:00:36, localpref 100, from 10.0.0.12 AS path: I, validation-state: unverified to 172.16.0.32 via et-0/0/50.0 > to 172.16.0.36 via et-0/0/51.02:10.0.0.24:1::1000::de:ad:be:e1:00:43/304 *[BGP/170] 00:00:36, localpref 100, from 10.0.0.11 AS path: I, validation-state: unverified to 172.16.0.32 via et-0/0/50.0 > to 172.16.0.36 via et-0/0/51.0 [BGP/170] 00:00:36, localpref 100, from 10.0.0.12 AS path: I, validation-state: unverified to 172.16.0.32 via et-0/0/50.0 > to 172.16.0.36 via et-0/0/51.02:10.0.0.24:1::1000::de:ad:be:e1:00:44/304 *[BGP/170] 00:00:36, localpref 100, from 10.0.0.11 AS path: I, validation-state: unverified to 172.16.0.32 via et-0/0/50.0 > to 172.16.0.36 via et-0/0/51.0 [BGP/170] 00:00:36, localpref 100, from 10.0.0.12 AS path: I, validation-state: unverified to 172.16.0.32 via et-0/0/50.0 > to 172.16.0.36 via et-0/0/51.02:10.0.0.11:1::1000::00:00:5e:00:01:01::10.1.100.1/304 *[BGP/170] 1w2d 03:18:08, localpref 100, from 10.0.0.11 AS path: I, validation-state: unverified > to 172.16.0.32 via et-0/0/50.0 [BGP/170] 1w2d 03:18:10, localpref 100, from 10.0.0.12 AS path: I, validation-state: unverified > to 172.16.0.32 via et-0/0/50.0



2:10.0.0.11:1::1000::00:31:46:7b:e1:18::10.1.100.211/304 *[BGP/170] 1w2d 03:18:08, localpref 100, from 10.0.0.11 AS path: I, validation-state: unverified > to 172.16.0.32 via et-0/0/50.0 [BGP/170] 1w2d 03:18:10, localpref 100, from 10.0.0.12 AS path: I, validation-state: unverified > to 172.16.0.32 via et-0/0/50.02:10.0.0.12:1::1000::00:00:5e:00:01:01::10.1.100.1/304 *[BGP/170] 1w2d 15:56:02, localpref 100, from 10.0.0.12 AS path: I, validation-state: unverified > to 172.16.0.36 via et-0/0/51.0 [BGP/170] 1w2d 03:18:08, localpref 100, from 10.0.0.11 AS path: I, validation-state: unverified > to 172.16.0.36 via et-0/0/51.02:10.0.0.12:1::1000::ec:3e:f7:89:15:1a::10.1.100.212/304 *[BGP/170] 1w2d 15:56:02, localpref 100, from 10.0.0.12 AS path: I, validation-state: unverified > to 172.16.0.36 via et-0/0/51.0 [BGP/170] 1w2d 03:18:08, localpref 100, from 10.0.0.11 AS path: I, validation-state: unverified > to 172.16.0.36 via et-0/0/51.02:10.0.0.11:1::1000::00:00:5e:00:02:01::2001:db8:10:1:100::1/304 *[BGP/170] 1w2d 03:18:08, localpref 100, from 10.0.0.11 AS path: I, validation-state: unverified > to 172.16.0.32 via et-0/0/50.0 [BGP/170] 1w2d 03:18:10, localpref 100, from 10.0.0.12 AS path: I, validation-state: unverified > to 172.16.0.32 via et-0/0/50.02:10.0.0.11:1::1000::00:31:46:7b:e1:18::2001:db8:10:1:100::211/304

*[BGP/170] 1w2d 03:18:08, localpref 100, from 10.0.0.11 AS path: I, validation-state: unverified > to 172.16.0.32 via et-0/0/50.0 [BGP/170] 1w2d 03:18:10, localpref 100, from 10.0.0.12 AS path: I, validation-state: unverified > to 172.16.0.32 via et-0/0/50.02:10.0.0.11:1::1000::00:31:46:7b:e1:18::fe80::231:4600:647b:e118/304

*[BGP/170] 1w2d 03:18:08, localpref 100, from 10.0.0.11 AS path: I, validation-state: unverified > to 172.16.0.32 via et-0/0/50.0 [BGP/170] 1w2d 03:18:10, localpref 100, from 10.0.0.12 AS path: I, validation-state: unverified > to 172.16.0.32 via et-0/0/50.02:10.0.0.12:1::1000::00:00:5e:00:02:01::2001:db8:10:1:100::1/304 *[BGP/170] 1w2d 15:56:02, localpref 100, from 10.0.0.12 AS path: I, validation-state: unverified > to 172.16.0.36 via et-0/0/51.0 [BGP/170] 1w2d 03:18:08, localpref 100, from 10.0.0.11 AS path: I, validation-state: unverified > to 172.16.0.36 via et-0/0/51.02:10.0.0.12:1::1000::ec:3e:f7:89:15:1a::2001:db8:10:1:100::212/304

*[BGP/170] 1w2d 15:56:02, localpref 100, from 10.0.0.12 AS path: I, validation-state: unverified > to 172.16.0.36 via et-0/0/51.0 [BGP/170] 1w2d 03:18:08, localpref 100, from 10.0.0.11 AS path: I, validation-state: unverified > to 172.16.0.36 via et-0/0/51.02:10.0.0.12:1::1000::ec:3e:f7:89:15:1a::fe80::ee3e:f700:6489:151a/304



*[BGP/170] 1w2d 15:56:02, localpref 100, from 10.0.0.12 AS path: I, validation-state: unverified > to 172.16.0.36 via et-0/0/51.0 [BGP/170] 1w2d 03:18:08, localpref 100, from 10.0.0.11 AS path: I, validation-state: unverified > to 172.16.0.36 via et-0/0/51.03:10.0.0.11:1::1000::10.0.0.11/304 *[BGP/170] 1w2d 03:18:08, localpref 100, from 10.0.0.11 AS path: I, validation-state: unverified > to 172.16.0.32 via et-0/0/50.0 [BGP/170] 1w2d 03:18:10, localpref 100, from 10.0.0.12 AS path: I, validation-state: unverified > to 172.16.0.32 via et-0/0/50.03:10.0.0.12:1::1000::10.0.0.12/304 *[BGP/170] 1w2d 15:56:02, localpref 100, from 10.0.0.12 AS path: I, validation-state: unverified > to 172.16.0.36 via et-0/0/51.0 [BGP/170] 1w2d 03:18:08, localpref 100, from 10.0.0.11 AS path: I, validation-state: unverified > to 172.16.0.36 via et-0/0/51.03:10.0.0.13:1::1000::10.0.0.13/304 *[BGP/170] 1w2d 03:18:05, localpref 100, from 10.0.0.11 AS path: I, validation-state: unverified to 172.16.0.32 via et-0/0/50.0 > to 172.16.0.36 via et-0/0/51.0 [BGP/170] 1w2d 15:56:02, localpref 100, from 10.0.0.12 AS path: I, validation-state: unverified to 172.16.0.32 via et-0/0/50.0 > to 172.16.0.36 via et-0/0/51.03:10.0.0.14:1::1000::10.0.0.14/304 *[BGP/170] 1w2d 03:18:01, localpref 100, from 10.0.0.11 AS path: I, validation-state: unverified to 172.16.0.32 via et-0/0/50.0 > to 172.16.0.36 via et-0/0/51.0 [BGP/170] 1w2d 15:55:58, localpref 100, from 10.0.0.12 AS path: I, validation-state: unverified to 172.16.0.32 via et-0/0/50.0 > to 172.16.0.36 via et-0/0/51.03:10.0.0.22:1::1000::10.0.0.22/304 *[BGP/170] 1w2d 03:18:04, localpref 100, from 10.0.0.11 AS path: I, validation-state: unverified to 172.16.0.32 via et-0/0/50.0 > to 172.16.0.36 via et-0/0/51.0 [BGP/170] 1w2d 15:56:02, localpref 100, from 10.0.0.12 AS path: I, validation-state: unverified to 172.16.0.32 via et-0/0/50.0 > to 172.16.0.36 via et-0/0/51.03:10.0.0.23:1::1000::10.0.0.23/304 *[BGP/170] 1w2d 00:57:12, localpref 100, from 10.0.0.11 AS path: I, validation-state: unverified to 172.16.0.32 via et-0/0/50.0 > to 172.16.0.36 via et-0/0/51.0 [BGP/170] 1w2d 00:57:12, localpref 100, from 10.0.0.12 AS path: I, validation-state: unverified to 172.16.0.32 via et-0/0/50.0 > to 172.16.0.36 via et-0/0/51.03:10.0.0.24:1::1000::10.0.0.24/304 *[BGP/170] 1w2d 00:55:08, localpref 100, from 10.0.0.11 AS path: I, validation-state: unverified to 172.16.0.32 via et-0/0/50.0 > to 172.16.0.36 via et-0/0/51.0



[BGP/170] 1w2d 00:55:08, localpref 100, from 10.0.0.12 AS path: I, validation-state: unverified to 172.16.0.32 via et-0/0/50.0 > to 172.16.0.36 via et-0/0/51.0

Meaning Because the output contains routes for all spine devices (10.0.0.11, 10.0.0.12, 10.0.0.13,

and 10.0.0.14) and all fabric devices (10.0.0.21, 10.0.0.22, 10.0.0.23, and 10.0.0.24), EVPN

routes are being learned through the overlay.

Leaf: Verifying the EVPNRoutes in Detail

Purpose Verify additional information about the EVPN routes.

Action NOTE: When analyzing EVPN operational command output, the addressformat is as follows:

<route-type>:<route-distinguisher>::<vni>::<mac-address>

Theaddress2:10.0.0.23:1::1000::de:ad:be:e1:00:30/304canbebrokendownas follows:

• EVPN route type—2

• Route distinguisher—10.0.0.23:1

• VNI—1000

• MAC address—de:ad:be:e1:00:30

a. Verify the mapping of EVPN routes and MAC addresses:

user@leaf-1> show route table bgp.evpn.0 evpn-ethernet-tag-id 1000 evpn-mac-addressde:ad:be:e1:00:30

bgp.evpn.0: 783 destinations, 1566 routes (619 active, 0 holddown, 328 hidden)+ = Active Route, - = Last Active, * = Both

2:10.0.0.23:1::1000::de:ad:be:e1:00:30/304 *[BGP/170] 00:01:50, localpref 100, from 10.0.0.11 AS path: I, validation-state: unverified > to 172.16.0.32 via et-0/0/50.0 to 172.16.0.36 via et-0/0/51.0 [BGP/170] 00:01:50, localpref 100, from 10.0.0.12 AS path: I, validation-state: unverified > to 172.16.0.32 via et-0/0/50.0 to 172.16.0.36 via et-0/0/51.0

b. Verify detailed information about the mapping of EVPN routes and MAC addresses:

user@leaf-1> show route table bgp.evpn.0 evpn-ethernet-tag-id 1000 evpn-mac-addressde:ad:be:e1:00:30 detail

bgp.evpn.0: 783 destinations, 1566 routes (619 active, 0 holddown, 328 hidden)2:10.0.0.23:1::1000::de:ad:be:e1:00:30/304 (2 entries, 0 announced) *BGP Preference: 170/-101



Route Distinguisher: 10.0.0.23:1 Next hop type: Indirect Address: 0x9603158 Next-hop reference count: 600 Source: 10.0.0.11 Protocol next hop: 10.0.0.23 Indirect next hop: 0x2 no-forward INH Session ID: 0x0 State: <Active Int Ext> Peer AS: 65200 Age: 1:52 Metric2: 0 Validation State: unverified Task: BGP_65200_65200.10.0.0.11+50113 AS path: I (Originator) Cluster list: 10.2.2.2 10.3.3.3 Originator ID: 10.0.0.23 Communities: target:1:1000 encapsulation0:0:0:0:vxlan Import Accepted Route Label: 993 ESI: 00:00:00:00:00:00:00:00:00:00 Localpref: 100 Router ID: 10.0.0.11 Secondary Tables: default-switch.evpn.0 BGP Preference: 170/-101 Route Distinguisher: 10.0.0.23:1 Next hop type: Indirect Address: 0x9603158 Next-hop reference count: 600 Source: 10.0.0.12 Protocol next hop: 10.0.0.23 Indirect next hop: 0x2 no-forward INH Session ID: 0x0 State: <NotBest Int Ext> Inactive reason: Not Best in its group - Update source Peer AS: 65200 Age: 1:52 Metric2: 0 Validation State: unverified Task: BGP_65200_65200.10.0.0.12+58525 AS path: I (Originator) Cluster list: 10.2.2.2 10.3.3.3 Originator ID: 10.0.0.23 Communities: target:1:1000 encapsulation0:0:0:0:vxlan Import Accepted Route Label: 993 ESI: 00:00:00:00:00:00:00:00:00:00 Localpref: 100 Router ID: 10.0.0.12 Secondary Tables: default-switch.evpn.0

Meaning Themapping of EVPN routes and MAC addresses is functioning correctly.

Leaf: Verifying VTEP Interfaces

Purpose Verify the source and destination address of the VTEP interfaces and their status.

Action Verify source address information for the VTEP interfaces:a.

user@leaf-1> show ethernet-switching vxlan-tunnel-end-point sourceLogical System Name Id SVTEP-IP IFL L3-Idx<default> 0 10.0.0.21 lo0.0 0 L2-RTT Bridge Domain VNID MC-Group-IP



default-switch v1000+100 1000 0.0.0.0

default-switch v1001+101 1001 0.0.0.0

default-switch v1002+102 1002 0.0.0.0

default-switch v1003+103 1003 0.0.0.0

default-switch v1004+104 1004 0.0.0.0

default-switch v1005+105 1005 0.0.0.0

default-switch v1006+106 1006 0.0.0.0

default-switch v1007+107 1007 0.0.0.0

default-switch v1008+108 1008 0.0.0.0

default-switch v1999+999 1999 0.0.0.0

b. Verify the summary status of the VTEP interfaces:

user@leaf-1> show interfaces terse vtepInterface Admin Link Proto Local Remotevtep up upvtep.32768 up up vtep.32769 up up eth-switchvtep.32770 up up eth-switchvtep.32771 up up eth-switchvtep.32772 up up eth-switchvtep.32773 up up eth-switchvtep.32774 up up eth-switchvtep.32775 up up eth-switch

NOTE: There are four leaf devices and four spine devices, so there are atotal of eight VTEP interfaces (one VTEP per device).

c. Verify the full status of the VTEP interfaces:

user@leaf-1> show interfaces vtep*Physical interface: vtep, Enabled, Physical link is Up Interface index: 645, SNMP ifIndex: 527 Type: Software-Pseudo, Link-level type: VxLAN-Tunnel-Endpoint, MTU: Unlimited, Speed: Unlimited Device flags : Present Running Link type : Full-Duplex Link flags : None Last flapped : Never Input packets : 0 Output packets: 0

Logical interface vtep.32768 (Index 552) (SNMP ifIndex 530) Flags: Up SNMP-Traps Encapsulation: ENET2 VXLAN Endpoint Type: Source, VXLAN Endpoint Address: 10.0.0.21, L2 Routing Instance: default-switch, L3 Routing Instance: default Input packets : 0 Output packets: 0



Logical interface vtep.32769 (Index 565) (SNMP ifIndex 531) Flags: Up SNMP-Traps Encapsulation: ENET2 VXLAN Endpoint Type: Remote, VXLAN Endpoint Address: 10.0.0.11, L2 Routing Instance: default-switch, L3 Routing Instance: default Input packets : 0 Output packets: 0 Protocol eth-switch, MTU: Unlimited Flags: Trunk-Mode






Logical interface vtep.32775 (Index 571) (SNMP ifIndex 561) Flags: Up SNMP-Traps Encapsulation: ENET2 VXLAN Endpoint Type: Remote, VXLAN Endpoint Address: 10.0.0.23, L2 Routing Instance: default-switch, L3 Routing Instance: default Input packets : 538109 Output packets: 35368495



Protocol eth-switch, MTU: Unlimited Flags: Trunk-Mode

Meaning Because the VLAN-to-VNI mappings are correct, all eight VTEP interfaces are up, and

eachVTEP terminates remotely at one of the leaf and spine devices, the VTEP interfaces

are functioning normally.

Leaf: Verifying VNI-to-VXLAN Tunnel Mappings

Purpose Verify that each VNI maps properly to each VXLAN tunnel, the leaf device is properly

connected to the remote VTEPs, and has the correct reachability.



Action Verify the mapping of the VNIs to the VXLAN tunnels by displaying the remote VTEP

information:

user@leaf-1> show ethernet-switching vxlan-tunnel-end-point remoteLogical System Name Id SVTEP-IP IFL L3-Idx<default> 0 10.0.0.21 lo0.0 0 RVTEP-IP IFL-Idx NH-Id 10.0.0.11 565 1763 VNID MC-Group-IP 1999 0.0.0.0 1000 0.0.0.0 1001 0.0.0.0 1002 0.0.0.0 1003 0.0.0.0 1004 0.0.0.0 1005 0.0.0.0 1006 0.0.0.0 1007 0.0.0.0 1008 0.0.0.0 RVTEP-IP IFL-Idx NH-Id 10.0.0.12 567 1792 VNID MC-Group-IP 1999 0.0.0.0 1000 0.0.0.0 1001 0.0.0.0 1002 0.0.0.0 1003 0.0.0.0 1004 0.0.0.0 1005 0.0.0.0 1006 0.0.0.0 1007 0.0.0.0 1008 0.0.0.0 RVTEP-IP IFL-Idx NH-Id 10.0.0.13 566 1791 VNID MC-Group-IP 1008 0.0.0.0 1007 0.0.0.0 1006 0.0.0.0 1005 0.0.0.0 1004 0.0.0.0 1003 0.0.0.0 1002 0.0.0.0 1001 0.0.0.0 1000 0.0.0.0 RVTEP-IP IFL-Idx NH-Id 10.0.0.14 569 1795 VNID MC-Group-IP 1008 0.0.0.0 1007 0.0.0.0 1006 0.0.0.0 1005 0.0.0.0 1004 0.0.0.0 1003 0.0.0.0 1002 0.0.0.0 1001 0.0.0.0 1000 0.0.0.0 RVTEP-IP IFL-Idx NH-Id 10.0.0.22 568 1793 VNID MC-Group-IP 1008 0.0.0.0



1007 0.0.0.0 1006 0.0.0.0 1005 0.0.0.0 1004 0.0.0.0 1003 0.0.0.0 1002 0.0.0.0 1001 0.0.0.0 1000 0.0.0.0 1999 0.0.0.0 RVTEP-IP IFL-Idx NH-Id 10.0.0.23 571 1798 VNID MC-Group-IP 1008 0.0.0.0 1006 0.0.0.0 1003 0.0.0.0 1004 0.0.0.0 1005 0.0.0.0 1002 0.0.0.0 1000 0.0.0.0 1001 0.0.0.0 1007 0.0.0.0 RVTEP-IP IFL-Idx NH-Id 10.0.0.24 570 1797 VNID MC-Group-IP 1008 0.0.0.0 1006 0.0.0.0 1005 0.0.0.0 1004 0.0.0.0 1000 0.0.0.0 1001 0.0.0.0 1003 0.0.0.0 1007 0.0.0.0 1002 0.0.0.0

Meaning TheVNIs aremapped to the correct VXLAN tunnels, the leaf device is properly connected

to the remote VTEPs, and has the correct reachability.

NOTE: VNI 1999 only appears in VXLAN tunnels associated with POD 1(10.0.0.11, 10.0.0.12, 10.0.0.21, and 10.0.0.22).

Leaf: VerifyingMACAddress Learning

Purpose Verify that the MAC addresses are learned through the VXLAN tunnels.



Action Display the MAC addresses that are learned through the VXLAN tunnels:

user@leaf-1> show ethernet-switching vxlan-tunnel-end-point remotemac-table

user@leaf-1> show ethernet-switching vxlan-tunnel-end-point remote mac-table

MAC flags (S -static MAC, D -dynamic MAC, L -locally learned, C -Control MAC SE -Statistics enabled, NM -Non configured MAC, R -Remote PE MAC)

Logical system : <default>Routing instance : default-switch Bridging domain : v1000+100, VLAN : 100, VNID : 1000 MAC MAC Logical Remote VTEP address flags interface IP address 00:00:5e:00:01:01 DR esi.1764 10.0.0.11 10.0.0.12 00:00:5e:00:02:01 DR esi.1764 10.0.0.11 10.0.0.12 00:31:46:7b:e1:18 D vtep.32769 10.0.0.11 ec:3e:f7:89:15:1a D vtep.32771 10.0.0.12 00:00:5e:00:54:10 D vtep.32772 10.0.0.22 00:00:5e:00:54:11 D vtep.32772 10.0.0.22 00:00:5e:00:54:12 D vtep.32772 10.0.0.22 00:00:5e:00:54:13 D vtep.32772 10.0.0.22 00:00:5e:00:54:14 D vtep.32772 10.0.0.22 de:ad:be:e1:00:20 D vtep.32772 10.0.0.22 de:ad:be:e1:00:21 D vtep.32772 10.0.0.22 de:ad:be:e1:00:22 D vtep.32772 10.0.0.22 de:ad:be:e1:00:23 D vtep.32772 10.0.0.22 de:ad:be:e1:00:24 D vtep.32772 10.0.0.22 fa:ce:b0:01:00:20 D vtep.32772 10.0.0.22 fa:ce:b0:01:00:21 D vtep.32772 10.0.0.22 fa:ce:b0:01:00:22 D vtep.32772 10.0.0.22 fa:ce:b0:01:00:23 D vtep.32772 10.0.0.22 fa:ce:b0:01:00:24 D vtep.32772 10.0.0.22 00:00:5e:00:56:10 D vtep.32774 10.0.0.24 00:00:5e:00:56:11 D vtep.32774 10.0.0.24 00:00:5e:00:56:12 D vtep.32774 10.0.0.24 00:00:5e:00:56:13 D vtep.32774 10.0.0.24 00:00:5e:00:56:14 D vtep.32774 10.0.0.24 de:ad:be:e1:00:40 D vtep.32774 10.0.0.24 de:ad:be:e1:00:41 D vtep.32774 10.0.0.24 de:ad:be:e1:00:42 D vtep.32774 10.0.0.24 de:ad:be:e1:00:43 D vtep.32774 10.0.0.24 de:ad:be:e1:00:44 D vtep.32774 10.0.0.24 00:00:5e:00:55:10 D vtep.32775 10.0.0.23 00:00:5e:00:55:11 D vtep.32775 10.0.0.23 00:00:5e:00:55:12 D vtep.32775 10.0.0.23 00:00:5e:00:55:13 D vtep.32775 10.0.0.23 00:00:5e:00:55:14 D vtep.32775 10.0.0.23 de:ad:be:e1:00:30 D vtep.32775 10.0.0.23 de:ad:be:e1:00:31 D vtep.32775 10.0.0.23 de:ad:be:e1:00:32 D vtep.32775 10.0.0.23 de:ad:be:e1:00:33 D vtep.32775 10.0.0.23 de:ad:be:e1:00:34 D vtep.32775 10.0.0.23 fa:ce:b0:01:00:30 D vtep.32775 10.0.0.23 fa:ce:b0:01:00:31 D vtep.32775 10.0.0.23 fa:ce:b0:01:00:32 D vtep.32775 10.0.0.23 fa:ce:b0:01:00:33 D vtep.32775 10.0.0.23 fa:ce:b0:01:00:34 D vtep.32775 10.0.0.23

Meaning MAC addresses are being shared across the VXLAN tunnels correctly.



Leaf: VerifyingMultihoming

Purpose Verify that multihoming is working on Leaf 3 and Leaf 4 by reviewing information for the

LAG interfaces in POD 2.



Action Display VTEP ESI information:

user@leaf-1> show ethernet-switching vxlan-tunnel-end-point esiESI RTT VLNBH INH ESI-IFL LOC-IFL #RVTEPs00:01:01:01:01:01:01:01:01:01 default-switch 1799 131087 esi.1799 2 RVTEP-IP RVTEP-IFL VENH MASK-ID FLAGS 10.0.0.24 vtep.32774 1797 0 2 10.0.0.23 vtep.32775 1798 1 2 ESI RTT VLNBH INH ESI-IFL LOC-IFL #RVTEPs05:00:00:00:00:00:00:03:e8:00 default-switch 1764 131072 esi.1764 2 RVTEP-IP RVTEP-IFL VENH MASK-ID FLAGS 10.0.0.11 vtep.32769 1763 0 2 10.0.0.12 vtep.32771 1792 1 2 ESI RTT VLNBH INH ESI-IFL LOC-IFL #RVTEPs05:00:00:00:00:00:00:03:e9:00 default-switch 1765 131073 esi.1765 2 RVTEP-IP RVTEP-IFL VENH MASK-ID FLAGS 10.0.0.11 vtep.32769 1763 0 2 10.0.0.12 vtep.32771 1792 1 2 ESI RTT VLNBH INH ESI-IFL LOC-IFL #RVTEPs05:00:00:00:00:00:00:03:ea:00 default-switch 1766 131074 esi.1766 2 RVTEP-IP RVTEP-IFL VENH MASK-ID FLAGS 10.0.0.11 vtep.32769 1763 0 2 10.0.0.12 vtep.32771 1792 1 2 ESI RTT VLNBH INH ESI-IFL LOC-IFL #RVTEPs05:00:00:00:00:00:00:03:eb:00 default-switch 1767 131075 esi.1767 2 RVTEP-IP RVTEP-IFL VENH MASK-ID FLAGS 10.0.0.11 vtep.32769 1763 0 2 10.0.0.12 vtep.32771 1792 1 2 ESI RTT VLNBH INH ESI-IFL LOC-IFL #RVTEPs05:00:00:00:00:00:00:03:ec:00 default-switch 1768 131076 esi.1768 2 RVTEP-IP RVTEP-IFL VENH MASK-ID FLAGS 10.0.0.11 vtep.32769 1763 0 2 10.0.0.12 vtep.32771 1792 1 2 ESI RTT VLNBH INH ESI-IFL LOC-IFL #RVTEPs05:00:00:00:00:00:00:03:ed:00 default-switch 1769 131077 esi.1769 2 RVTEP-IP RVTEP-IFL VENH MASK-ID FLAGS 10.0.0.11 vtep.32769 1763 0 2 10.0.0.12 vtep.32771 1792 1 2 ESI RTT VLNBH INH ESI-IFL LOC-IFL #RVTEPs05:00:00:00:00:00:00:03:ee:00 default-switch 1770 131078 esi.1770 2 RVTEP-IP RVTEP-IFL VENH MASK-ID FLAGS 10.0.0.11 vtep.32769 1763 0 2 10.0.0.12 vtep.32771 1792 1 2 ESI RTT VLNBH INH ESI-IFL



LOC-IFL #RVTEPs05:00:00:00:00:00:00:03:ef:00 default-switch 1771 131079 esi.1771 2 RVTEP-IP RVTEP-IFL VENH MASK-ID FLAGS 10.0.0.11 vtep.32769 1763 0 2 10.0.0.12 vtep.32771 1792 1 2 ESI RTT VLNBH INH ESI-IFL LOC-IFL #RVTEPs05:00:00:00:00:00:00:03:f0:00 default-switch 1772 131080 esi.1772 2 RVTEP-IP RVTEP-IFL VENH MASK-ID FLAGS 10.0.0.11 vtep.32769 1763 0 2 10.0.0.12 vtep.32771 1792 1 2 ESI RTT VLNBH INH ESI-IFL LOC-IFL #RVTEPs05:00:00:00:00:00:00:07:cf:00 default-switch 1800 131088 esi.1800 2 RVTEP-IP RVTEP-IFL VENH MASK-ID FLAGS 10.0.0.11 vtep.32769 1763 1 2 10.0.0.12 vtep.32771 1792 0 2

NOTE: There are 11 Ethernet segment identifier (ESI) numbers. The first ESInumber that starts with 00: belongs to themultihomed LAG interfaceconnecting Switch 5 to Leaf 3 and Leaf 4. The remaining 10 ESI numbers thatstartwith05:mapto theLayer3gateway for the 10VNIs.TheLayer3gatewayof eachVNI is reachable fromLeaf 1 througheither Spine 1 (10.0.0.11) or Spine2 (10.0.0.12).

Meaning Multihoming is working on Leaf 3 and Leaf 4 as expected.

Leaf: Verifying ECMP

Purpose Verify that the VXLAN tunnels prefer to use ECMP-based underlay paths.



Action Verify that the VXLAN tunnel to Fabric 3 (vtep.32775) prefers the ECMP-based paths

over the underlay paths:

user@leaf-1> show route forwarding-table table default-switch extensive | find vtep.32775

Destination: vtep.32775 Route type: interface Route reference: 0 Route interface-index: 571 Multicast RPF nh index: 0 Flags: sent to PFE Nexthop: Next-hop type: composite Index: 1798 Reference: 150 Next-hop type: indirect Index: 131086 Reference: 3 Next-hop type: unilist Index: 131070 Reference: 27 Nexthop: 172.16.0.32 Next-hop type: unicast Index: 1761 Reference: 7 Next-hop interface: et-0/0/50.0 Weight: 0x0 Nexthop: 172.16.0.36 Next-hop type: unicast Index: 1762 Reference: 7 Next-hop interface: et-0/0/51.0 Weight: 0x0

Meaning The VXLAN tunnel prefers to use ECMP-based underlay paths.

Leaf: Verifying RemoteMACAddress Reachability Through ECMP

Purpose Verify that the remote MAC address is reachable through ECMP.



Action Display extensive forwarding table information for a selected MAC address to verify its

reachability:

user@leaf-1> show route forwarding-table table default-switch extensive destinationde:ad:be:e1:00:21/48Routing table: default-switch.evpn-vxlan [Index 6] Bridging domain: v1000.evpn-vxlan [Index 3] VPLS:

Destination: de:ad:be:e1:00:21/48 Learn VLAN: 0 Route type: user Route reference: 0 Route interface-index: 568 Multicast RPF nh index: 0 IFL generation: 539 Epoch: 0 Sequence Number: 0 Learn Mask: 0x4000000000000000000000000000000000000000 L2 Flags: control_dyn Flags: sent to PFE Nexthop: Next-hop type: composite Index: 1793 Reference: 153 Next-hop type: indirect Index: 131083 Reference: 3 Next-hop type: unilist Index: 131070 Reference: 21 Nexthop: 172.16.0.32 Next-hop type: unicast Index: 1761 Reference: 13 Next-hop interface: et-0/0/50.0 Weight: 0x0 Nexthop: 172.16.0.36 Next-hop type: unicast Index: 1762 Reference: 7 Next-hop interface: et-0/0/51.0 Weight: 0x0

Meaning The remote MAC address is reachable through ECMP.

Leaf: Verifying Local and RemoteMACAddress Learning

Purpose Verify that the switching table learns both local and remote MAC addresses.



Action Display switching table information for VLAN 100:

user@leaf-1> show ethernet-switching table vlan-id 100

MAC flags (S - static MAC, D - dynamic MAC, L - locally learned, P - Persistent static SE - statistics enabled, NM - non configured MAC, R - remote PE MAC, O - ovsdb MAC)

Ethernet switching table : 70 entries, 70 learnedRouting instance : default-switch Vlan MAC MAC Logical Active

name address flags interface source

v1000 00:00:5e:00:01:01 DR esi.1764 05:00:00:00:00:00:00:03:e8:00 v1000 00:00:5e:00:02:01 DR esi.1764 05:00:00:00:00:00:00:03:e8:00 v1000 00:00:5e:00:53:00 D xe-0/0/12.0 v1000 00:00:5e:00:53:01 D xe-0/0/12.0 v1000 00:00:5e:00:53:02 D xe-0/0/12.0 v1000 00:00:5e:00:53:03 D xe-0/0/12.0 v1000 00:00:5e:00:53:04 D xe-0/0/12.0 v1000 00:00:5e:00:54:10 D vtep.32772 10.0.0.22

v1000 00:00:5e:00:54:11 D vtep.32772 10.0.0.22

v1000 00:00:5e:00:54:12 D vtep.32772 10.0.0.22

v1000 00:00:5e:00:54:13 D vtep.32772 10.0.0.22

v1000 00:00:5e:00:54:14 D vtep.32772 10.0.0.22

v1000 00:00:5e:00:55:10 D vtep.32775 10.0.0.23

v1000 00:00:5e:00:55:11 D vtep.32775 10.0.0.23

v1000 00:00:5e:00:55:12 D vtep.32775 10.0.0.23

v1000 00:00:5e:00:55:13 D vtep.32775 10.0.0.23

v1000 00:00:5e:00:55:14 D vtep.32775 10.0.0.23

v1000 00:00:5e:00:56:10 D vtep.32774 10.0.0.24

v1000 00:00:5e:00:56:11 D vtep.32774 10.0.0.24

v1000 00:00:5e:00:56:12 D vtep.32774 10.0.0.24

v1000 00:00:5e:00:56:13 D vtep.32774 10.0.0.24

v1000 00:00:5e:00:56:14 D vtep.32774 10.0.0.24

v1000 00:31:46:7b:e1:18 D vtep.32769 10.0.0.11

v1000 de:ad:be:e1:00:10 D xe-0/0/12.0 v1000 de:ad:be:e1:00:11 D xe-0/0/12.0 v1000 de:ad:be:e1:00:12 D xe-0/0/12.0



v1000 de:ad:be:e1:00:13 D xe-0/0/12.0 v1000 de:ad:be:e1:00:14 D xe-0/0/12.0 v1000 de:ad:be:e1:00:20 D vtep.32772 10.0.0.22

v1000 de:ad:be:e1:00:21 D vtep.32772 10.0.0.22

v1000 de:ad:be:e1:00:22 D vtep.32772 10.0.0.22

v1000 de:ad:be:e1:00:23 D vtep.32772 10.0.0.22

v1000 de:ad:be:e1:00:24 D vtep.32772 10.0.0.22

v1000 de:ad:be:e1:00:30 D vtep.32775 10.0.0.23

v1000 de:ad:be:e1:00:31 D vtep.32775 10.0.0.23

v1000 de:ad:be:e1:00:32 D vtep.32775 10.0.0.23

v1000 de:ad:be:e1:00:33 D vtep.32775 10.0.0.23

v1000 de:ad:be:e1:00:34 D vtep.32775 10.0.0.23

v1000 de:ad:be:e1:00:40 D vtep.32774 10.0.0.24

v1000 de:ad:be:e1:00:41 D vtep.32774 10.0.0.24

v1000 de:ad:be:e1:00:42 D vtep.32774 10.0.0.24

v1000 de:ad:be:e1:00:43 D vtep.32774 10.0.0.24

v1000 de:ad:be:e1:00:44 D vtep.32774 10.0.0.24

v1000 ec:3e:f7:89:15:1a D vtep.32771 10.0.0.12

v1000 fa:ce:b0:01:00:10 D xe-0/0/13.0 v1000 fa:ce:b0:01:00:11 D xe-0/0/13.0 v1000 fa:ce:b0:01:00:12 D xe-0/0/13.0 v1000 fa:ce:b0:01:00:13 D xe-0/0/13.0 v1000 fa:ce:b0:01:00:14 D xe-0/0/13.0 v1000 fa:ce:b0:01:00:20 D vtep.32772 10.0.0.22

v1000 fa:ce:b0:01:00:21 D vtep.32772 10.0.0.22

v1000 fa:ce:b0:01:00:22 D vtep.32772 10.0.0.22

v1000 fa:ce:b0:01:00:23 D vtep.32772 10.0.0.22

v1000 fa:ce:b0:01:00:24 D vtep.32772 10.0.0.22

v1000 fa:ce:b0:01:00:30 D vtep.32775 10.0.0.23

v1000 fa:ce:b0:01:00:31 D vtep.32775 10.0.0.23

v1000 fa:ce:b0:01:00:32 D vtep.32775 10.0.0.23

v1000 fa:ce:b0:01:00:33 D vtep.32775 10.0.0.23

v1000 fa:ce:b0:01:00:34 D vtep.32775 10.0.0.23

v1000 fa:ce:b0:01:00:42 DR esi.1799 00:01:01:01:01:01:01:01:01:01



Meaning The switching table displays MAC addresses that were learned locally (xe-0/0/12 and

xe-0/0/13) and remotely (vtep.* and esi.*).

Spine: Verifying Interfaces

Purpose Verify that the fabric-facing and leaf-facing interfaces are up.

Action Display the fabric-facing and leaf-facing interfaces:

user@spine-1> show interfaces terse

Interface Admin Link Proto Local Remotegr-0/0/0 up uppfe-0/0/0 up uppfe-0/0/0.16383 up up inet inet6 pfh-0/0/0 up uppfh-0/0/0.16383 up up inet pfh-0/0/0.16384 up up inet et-0/0/18 up upet-0/0/19 up upet-0/0/20 up upxe-0/0/34:0 up downxe-0/0/34:1 up upxe-0/0/34:1.0 up up eth-switchxe-0/0/34:2 up downxe-0/0/34:3 up downet-0/0/58 up upet-0/0/58.0 up up inet 172.16.0.1/31 ## <<< To Fabric 1et-0/0/59 up upet-0/0/59.0 up up inet 172.16.0.9/31 ## <<< To Fabric 2et-0/0/60 up upet-0/0/60.0 up up inet 172.16.0.25/31 ## <<< To Fabric 4et-0/0/61 up upet-0/0/61.0 up up inet 172.16.0.17/31 ## <<< To Fabric 3et-0/0/66 up upet-0/0/66.0 up up inet 172.16.0.32/31 ## <<< To Leaf 1et-0/0/67 up upet-0/0/67.0 up up inet 172.16.0.34/31 ## <<< To Leaf 2

Meaning The fabric-facing and leaf-facing interfaces are connected and operating correctly.

Spine: Verifying IPv4 BGP Sessions

Purpose Verify the state of the underlay (EBGP) and overlay (IBGP) sessions that connect the

spine devices to the leaf devices, the fabric devices, and the other spine devices.



Action Verify that IPv4 EBGP and IBGP sessions are established with the other devices in the IP

fabric:

user@spine-1> show bgp summaryGroups: 3 Peers: 11 Down peers: 0Table Tot Paths Act Paths Suppressed History Damp State Pendingbgp.evpn.0 1232 967 0 0 0 0inet.0 104 44 0 0 0 0Peer AS InPkt OutPkt OutQ Flaps Last Up/Dwn State|#Active/Received/Accepted/Damped...10.0.0.12 65200 216957 214460 0 0 1w1d 11:06:20 Establ bgp.evpn.0: 104/104/104/0 default-switch.evpn.0: 100/100/100/0 __default_evpn__.evpn.0: 0/0/0/0 TYPE-5.evpn.0: 4/4/4/010.0.0.13 65200 167514 214452 0 0 1w1d 11:06:16 Establ bgp.evpn.0: 369/369/369/0 default-switch.evpn.0: 342/342/342/0 __default_evpn__.evpn.0: 0/0/0/0 TYPE-5.evpn.0: 4/4/4/010.0.0.14 65200 167663 214446 0 0 1w1d 11:06:12 Establ bgp.evpn.0: 104/369/369/0 default-switch.evpn.0: 91/342/342/0 __default_evpn__.evpn.0: 0/0/0/0 TYPE-5.evpn.0: 4/4/4/010.0.0.21 65200 139039 246623 0 0 1w1d 11:06:19 Establ bgp.evpn.0: 240/240/240/0 default-switch.evpn.0: 240/240/240/0 __default_evpn__.evpn.0: 240/240/240/010.0.0.22 65200 96675 289414 0 0 1w1d 11:06:14 Establ bgp.evpn.0: 150/150/150/0 default-switch.evpn.0: 150/150/150/0 __default_evpn__.evpn.0: 150/150/150/0172.16.0.0 65001 255 253 0 87 1:49:31 Establ inet.0: 8/26/23/0172.16.0.8 65002 26787 26824 0 0 1w1d 11:06:08 Establ inet.0: 8/26/23/0172.16.0.16 65003 26849 26870 0 0 1w1d 11:06:28 Establ inet.0: 13/26/23/0172.16.0.24 65004 26928 26881 0 0 1w1d 11:06:24 Establ inet.0: 13/24/21/0172.16.0.33 65021 26805 26957 0 0 1w1d 11:06:20 Establ inet.0: 1/1/1/0172.16.0.35 65022 26798 26956 0 0 1w1d 11:06:16 Establ inet.0: 1/1/1/0

Meaning Because there are peer connections to AS 65001, AS 65002, AS 65003, and AS 65004



(the four fabric devices), AS 65021 and AS 65022 (the underlay for Leaf 1 and Leaf 2),

and AS 65200 (overlay to Spine 2, Spine 3, Spine 4, Leaf 1, and Leaf 2), all EBGP and

IBGP sessions are established and functioning correctly.

Spine: Verifying BFD

Purpose Verify that Bidirectional Forwarding Detection (BFD) is operating correctly between the

leaf, spine, and fabric devices.

Action Verify that BFD is operating between the devices in the IP fabric:

user@spine-1> show bfd session

Detect TransmitAddress State Interface Time Interval Multiplier10.0.0.12 Up 1.050 0.350 3 10.0.0.13 Up 1.050 0.350 3 10.0.0.21 Up 1.050 0.350 3 10.0.0.22 Up 1.050 0.350 3 10.0.0.14 Up 1.050 0.350 3 172.16.0.0 Up et-0/0/58.0 1.050 0.350 3 172.16.0.8 Up et-0/0/59.0 1.050 0.350 3 172.16.0.16 Up et-0/0/61.0 1.050 0.350 3 172.16.0.24 Up et-0/0/60.0 1.050 0.350 3 172.16.0.33 Up et-0/0/66.0 1.050 0.350 3 172.16.0.35 Up et-0/0/67.0 1.050 0.350 3


Meaning BFD isoperating correctly between the leaf, spine, and fabric devices for both theunderlay

and the overlay.

Spine: Verifying the IRB Interfaces

Purpose Verify that the IRB interfaces are up.



Action Display the summary status for the IRB interfaces:

user@spine-1> show interfaces irb terseInterface Admin Link Proto Local Remoteirb up upirb.100 up up inet 10.1.100.211/24 inet6 2001:db8:10:1:100::211/112 fe80::231:4600:647b:e118/64irb.101 up up inet 10.1.101.211/24 inet6 2001:db8:10:1:101::211/112 fe80::231:4600:657b:e118/64irb.102 up up inet 10.1.102.211/24 inet6 2001:db8:10:1:102::211/112 fe80::231:4600:667b:e118/64irb.103 up up inet 10.1.103.211/24 inet6 2001:db8:10:1:103::211/112 fe80::231:4600:677b:e118/64irb.104 up up inet 10.1.104.211/24 inet6 2001:db8:10:1:104::211/112 fe80::231:4600:687b:e118/64irb.105 up up inet 10.1.105.211/24 inet6 2001:db8:10:1:105::211/112 fe80::231:4600:697b:e118/64irb.106 up up inet 10.1.106.211/24 inet6 2001:db8:10:1:106::211/112 fe80::231:4600:6a7b:e118/64irb.107 up up inet 10.1.107.211/24 inet6 2001:db8:10:1:107::211/112 fe80::231:4600:6b7b:e118/64irb.108 up up inet 10.1.108.211/24 inet6 2001:db8:10:1:108::211/112 fe80::231:4600:6c7b:e118/64irb.999 up up inet 10.255.99.211/24 inet6 2001:db8:10:255:99::211/112 fe80::231:4603:e77b:e118/64

Meaning The IRB interfaces are established and functioning correctly.

Spine: Verifying VTEP Interfaces

Purpose Verify the overall status of the VTEP interfaces.

Action Display the summary status of the VTEP interfaces:

user@spine-1> show interfaces vtep terseInterface Admin Link Proto Local Remotevtep up upvtep.32768 up up vtep.32769 up up eth-switchvtep.32770 up up eth-switchvtep.32771 up up eth-switchvtep.32772 up up eth-switchvtep.32773 up up eth-switchvtep.32774 up up eth-switchvtep.32775 up up eth-switch

Meaning Because all eight VTEP interfaces are up, the VTEP interfaces are functioning normally.



Spine: Verifying VTEP Destination Addresses

Purpose Verify the full status of the VTEP interfaces.



Action Display the full status of the VTEP interfaces:

user@spine-1> show interfaces vtep*Physical interface: vtep , Enabled, Physical link is Up Interface index: 641, SNMP ifIndex: 503 Type: Software-Pseudo, Link-level type: VxLAN-Tunnel-Endpoint, MTU: 1600, Speed: Unlimited Device flags : Present Running Link type : Full-Duplex Link flags : None Last flapped : Never Input packets : 0 Output packets: 0

Logical interface vtep.32768 (Index 816) (SNMP ifIndex 656) Flags: Up SNMP-Traps Encapsulation: ENET2 VXLAN Endpoint Type: Source, VXLAN Endpoint Address: 10.0.0.11, L2 Routing Instance: default-switch, L3 Routing Instance: default Input packets : 0 Output packets: 0

Logical interface vtep.32769 (Index 827) (SNMP ifIndex 640) Flags: Up SNMP-Traps Encapsulation: ENET2 VXLAN Endpoint Type: Remote, VXLAN Endpoint Address: 10.0.0.21, L2 Routing Instance: default-switch, L3 Routing Instance: default Input packets : 264939940 Output packets: 181948295 Protocol eth-switch, MTU: 1600 Flags: Trunk-Mode




Logical interface vtep.32773 (Index 831) (SNMP ifIndex 670) Flags: Up SNMP-Traps Encapsulation: ENET2 VXLAN Endpoint Type: Remote, VXLAN Endpoint Address: 10.0.0.14, L2 Routing



Instance: default-switch, L3 Routing Instance: default Input packets : 1154027 Output packets: 35820 Protocol eth-switch, MTU: 1600 Flags: Trunk-Mode



Meaning Because each VTEP interface terminates remotely at one of the leaf and spine devices,

the VTEP interfaces are functioning correctly.

Spine: Verifying Inter-Spine ECMP

Purpose Verify that ECMP is working between the spine devices.

Action Verify the preferred paths between selected spine devices.

a. Display the preferred paths between Spine 1 and Spine 2:

user@spine-1> show route 10.0.0.12

inet.0: 54 destinations, 132 routes (51 active, 0 holddown, 12 hidden)+ = Active Route, - = Last Active, * = Both

10.0.0.12/32 *[BGP/170] 1w1d 11:21:19, localpref 100 AS path: 65003 65012 I, validation-state: unverified to 172.16.0.0 via et-0/0/58.0 to 172.16.0.8 via et-0/0/59.0 to 172.16.0.24 via et-0/0/60.0 > to 172.16.0.16 via et-0/0/61.0 [BGP/170] 00:05:20, localpref 100 AS path: 65001 65012 I, validation-state: unverified > to 172.16.0.0 via et-0/0/58.0 [BGP/170] 1w1d 11:20:59, localpref 100 AS path: 65002 65012 I, validation-state: unverified > to 172.16.0.8 via et-0/0/59.0 [BGP/170] 00:25:22, localpref 100 AS path: 65004 65012 I, validation-state: unverified > to 172.16.0.24 via et-0/0/60.0

:vxlan.inet.0: 22 destinations, 22 routes (22 active, 0 holddown, 0 hidden)+ = Active Route, - = Last Active, * = Both



10.0.0.12/32 *[Static/1] 1w1d 11:21:18, metric2 0 to 172.16.0.0 via et-0/0/58.0 to 172.16.0.8 via et-0/0/59.0 > to 172.16.0.24 via et-0/0/60.0 to 172.16.0.16 via et-0/0/61.0

b. Display the preferred paths between Spine 1 and Spine 3:

user@spine-1> show route 10.0.0.13

inet.0: 54 destinations, 132 routes (51 active, 0 holddown, 12 hidden)+ = Active Route, - = Last Active, * = Both

10.0.0.13/32 *[BGP/170] 1w1d 11:22:04, localpref 100 AS path: 65003 65013 I, validation-state: unverified to 172.16.0.0 via et-0/0/58.0 to 172.16.0.8 via et-0/0/59.0 to 172.16.0.24 via et-0/0/60.0 > to 172.16.0.16 via et-0/0/61.0 [BGP/170] 00:01:36, localpref 100 AS path: 65001 65013 I, validation-state: unverified > to 172.16.0.0 via et-0/0/58.0 [BGP/170] 1w1d 11:21:44, localpref 100 AS path: 65002 65013 I, validation-state: unverified > to 172.16.0.8 via et-0/0/59.0 [BGP/170] 1w1d 11:22:00, localpref 100 AS path: 65004 65013 I, validation-state: unverified > to 172.16.0.24 via et-0/0/60.0

:vxlan.inet.0: 22 destinations, 22 routes (22 active, 0 holddown, 0 hidden)+ = Active Route, - = Last Active, * = Both

10.0.0.13/32 *[Static/1] 1w1d 11:22:03, metric2 0 to 172.16.0.0 via et-0/0/58.0 to 172.16.0.8 via et-0/0/59.0 > to 172.16.0.24 via et-0/0/60.0 to 172.16.0.16 via et-0/0/61.0

Meaning Because there are four equal-cost paths to reach the other spine devices, inter-spine

ECMP is functioning correctly.

Spine: Verifying the Routing Instances

Purpose Verify the routing tables for the customer routing instances Tenant 10 and Tenant 20,

and the EVPN Type 5 routing instance.

Action Verify the IPv4 routing table for Tenant 10:a.

user@spine-1> show route table VRF_TENANT_10.inet.0

VRF_TENANT_10.inet.0: 11 destinations, 11 routes (11 active, 0 holddown, 0 hidden)+ = Active Route, - = Last Active, * = Both

10.1.100.0/24 *[Direct/0] 1w1d 11:22:40 > via irb.10010.1.100.211/32 *[Local/0] 1w1d 11:22:40 Local via irb.100



10.1.101.0/24 *[Direct/0] 1w1d 11:22:40 > via irb.10110.1.101.211/32 *[Local/0] 1w1d 11:22:40 Local via irb.10110.1.102.0/24 *[Direct/0] 1w1d 11:22:40 > via irb.10210.1.102.211/32 *[Local/0] 1w1d 11:22:40 Local via irb.10210.1.103.0/24 *[Direct/0] 1w1d 11:22:40 > via irb.10310.1.103.211/32 *[Local/0] 1w1d 11:22:40 Local via irb.10310.1.104.0/24 *[Direct/0] 1w1d 11:22:40 > via irb.10410.1.104.211/32 *[Local/0] 1w1d 11:22:40 Local via irb.10410.10.0.11/32 *[Direct/0] 1w1d 11:22:40 > via lo0.10

b. Verify the IPv4 routing table for Tenant 20:

user@spine-1> show route table VRF_TENANT_20.inet.0

VRF_TENANT_20.inet.0: 9 destinations, 9 routes (9 active, 0 holddown, 0 hidden)+ = Active Route, - = Last Active, * = Both

10.1.105.0/24 *[Direct/0] 1w1d 11:22:40 > via irb.10510.1.105.211/32 *[Local/0] 1w1d 11:22:40 Local via irb.10510.1.106.0/24 *[Direct/0] 1w1d 11:22:40 > via irb.10610.1.106.211/32 *[Local/0] 1w1d 11:22:40 Local via irb.10610.1.107.0/24 *[Direct/0] 1w1d 11:22:40 > via irb.10710.1.107.211/32 *[Local/0] 1w1d 11:22:40 Local via irb.10710.1.108.0/24 *[Direct/0] 1w1d 11:22:40 > via irb.10810.1.108.211/32 *[Local/0] 1w1d 11:22:40 Local via irb.10810.20.0.11/32 *[Direct/0] 1w1d 11:22:40 > via lo0.20

c. Verify the IPv4 routing table for the EVPN Type 5 instance:

user@spine-1> show route table TYPE-5.inet.0

TYPE-5.inet.0: 7 destinations, 8 routes (7 active, 0 holddown, 0 hidden)+ = Active Route, - = Last Active, * = Both

10.255.99.0/24 *[Direct/0] 1w1d 09:23:52 > via irb.999 [EVPN/170] 1w1d 09:23:52 to 172.16.0.0 via et-0/0/58.0 > to 172.16.0.8 via et-0/0/59.0 to 172.16.0.24 via et-0/0/60.0 to 172.16.0.16 via et-0/0/61.010.255.99.211/32 *[Local/0] 1w1d 09:23:52 Local via irb.99910.255.99.212/32 *[EVPN/170] 1w1d 09:23:52



> to 172.16.0.0 via et-0/0/58.0 to 172.16.0.8 via et-0/0/59.0 to 172.16.0.24 via et-0/0/60.0 to 172.16.0.16 via et-0/0/61.010.255.100.0/24 *[EVPN/170] 1w1d 09:23:52 > to 172.16.0.0 via et-0/0/58.0 to 172.16.0.8 via et-0/0/59.0 to 172.16.0.24 via et-0/0/60.0 to 172.16.0.16 via et-0/0/61.0 > to 172.16.0.0 via et-0/0/58.0 to 172.16.0.8 via et-0/0/59.0 to 172.16.0.24 via et-0/0/60.0 to 172.16.0.16 via et-0/0/61.010.255.100.2/32 *[EVPN/170] 1w1d 09:23:52 > to 172.16.0.0 via et-0/0/58.0 to 172.16.0.8 via et-0/0/59.0 to 172.16.0.24 via et-0/0/60.0 to 172.16.0.16 via et-0/0/61.010.255.100.212/32 *[EVPN/170] 1w1d 09:23:52 to 172.16.0.0 via et-0/0/58.0 > to 172.16.0.8 via et-0/0/59.0 to 172.16.0.24 via et-0/0/60.0 to 172.16.0.16 via et-0/0/61.010.9.9.9/32 *[Direct/0] 1w1d 09:23:52 > via lo0.999

d. Verify the IPv6 routing table for Tenant 10:

user@spine-1> show route table VRF_TENANT_10.inet6.0

VRF_TENANT_10.inet6.0: 15 destinations, 15 routes (15 active, 0 holddown, 0 hidden)+ = Active Route, - = Last Active, * = Both

2001:db8:10:1:100::/112 *[Direct/0] 1w1d 11:22:30 > via irb.1002001:db8:10:1:100::211/128 *[Local/0] 1w1d 11:22:40 Local via irb.1002001:db8:10:1:101::/112 *[Direct/0] 1w1d 11:22:30 > via irb.1012001:db8:10:1:101::211/128 *[Local/0] 1w1d 11:22:40 Local via irb.1012001:db8:10:1:102::/112 *[Direct/0] 1w1d 11:22:30 > via irb.1022001:db8:10:1:102::211/128 *[Local/0] 1w1d 11:22:40 Local via irb.1022001:db8:10:1:103::/112 *[Direct/0] 1w1d 11:22:31 > via irb.1032001:db8:10:1:103::211/128 *[Local/0] 1w1d 11:22:40 Local via irb.1032001:db8:10:1:104::/112 *[Direct/0] 1w1d 11:22:31 > via irb.104



2001:db8:10:1:104::211/128 *[Local/0] 1w1d 11:22:40 Local via irb.104fe80::231:4600:647b:e118/128 *[Local/0] 1w1d 11:22:40 Local via irb.100 fe80::231:4600:657b:e118/128 *[Local/0] 1w1d 11:22:40 Local via irb.101fe80::231:4600:667b:e118/128 *[Local/0] 1w1d 11:22:40 Local via irb.102fe80::231:4600:677b:e118/128 *[Local/0] 1w1d 11:22:40 Local via irb.103fe80::231:4600:687b:e118/128 *[Local/0] 1w1d 11:22:40 Local via irb.104

e. Verify the IPv6 routing table for Tenant 20:

user@spine-1> show route table VRF_TENANT_20.inet6.0

VRF_TENANT_20.inet6.0: 12 destinations, 12 routes (12 active, 0 holddown, 0 hidden)+ = Active Route, - = Last Active, * = Both

2001:db8:10:1:105::/112 *[Direct/0] 1w1d 11:22:30 > via irb.1052001:db8:10:1:105::211/128 *[Local/0] 1w1d 11:22:40 Local via irb.1052001:db8:10:1:106::/112 *[Direct/0] 1w1d 11:22:30 > via irb.1062001:db8:10:1:106::211/128 *[Local/0] 1w1d 11:22:40 Local via irb.1062001:db8:10:1:107::/112 *[Direct/0] 1w1d 11:22:30 > via irb.1072001:db8:10:1:107::211/128 *[Local/0] 1w1d 11:22:40 Local via irb.1072001:db8:10:1:108::/112 *[Direct/0] 1w1d 11:22:30 > via irb.1082001:db8:10:1:108::211/128 *[Local/0] 1w1d 11:22:40 Local via irb.108fe80::231:4600:697b:e118/128 *[Local/0] 1w1d 11:22:40 Local via irb.105fe80::231:4600:6a7b:e118/128 *[Local/0] 1w1d 11:22:40 Local via irb.106fe80::231:4600:6b7b:e118/128 *[Local/0] 1w1d 11:22:40 Local via irb.107fe80::231:4600:6c7b:e118/128



*[Local/0] 1w1d 11:22:40 Local via irb.108

f. Verify the IPv6 routing table for the EVPN Type 5 instance:

user@spine-1> show route table TYPE-5.inet6.0TYPE-5.inet6.0: 7 destinations, 8 routes (7 active, 0 holddown, 0 hidden)+ = Active Route, - = Last Active, * = Both

2001:db8:10:255:99::/112 *[Direct/0] 1w1d 09:23:41 > via irb.999 [EVPN/170] 1w1d 09:23:52 to 172.16.0.0 via et-0/0/58.0 to 172.16.0.8 via et-0/0/59.0 > to 172.16.0.24 via et-0/0/60.0 to 172.16.0.16 via et-0/0/61.02001:db8:10:255:99::211/128 *[Local/0] 1w1d 09:23:52 Local via irb.9992001:db8:10:255:99::212/128 *[EVPN/170] 1w1d 09:23:52 to 172.16.0.0 via et-0/0/58.0 to 172.16.0.8 via et-0/0/59.0 > to 172.16.0.24 via et-0/0/60.0 to 172.16.0.16 via et-0/0/61.02001:db8:10:255:100::/112 *[EVPN/170] 1w1d 09:23:52 > to 172.16.0.0 via et-0/0/58.0 to 172.16.0.8 via et-0/0/59.0 to 172.16.0.24 via et-0/0/60.0 to 172.16.0.16 via et-0/0/61.0 > to 172.16.0.0 via et-0/0/58.0 to 172.16.0.8 via et-0/0/59.0 to 172.16.0.24 via et-0/0/60.0 to 172.16.0.16 via et-0/0/61.02001:db8:10:255:100::211/128 *[EVPN/170] 1w1d 09:23:52 > to 172.16.0.0 via et-0/0/58.0 to 172.16.0.8 via et-0/0/59.0 to 172.16.0.24 via et-0/0/60.0 to 172.16.0.16 via et-0/0/61.02001:db8:10:255:100::212/128 *[EVPN/170] 1w1d 09:23:52 > to 172.16.0.0 via et-0/0/58.0 to 172.16.0.8 via et-0/0/59.0 to 172.16.0.24 via et-0/0/60.0 to 172.16.0.16 via et-0/0/61.0fe80::231:4603:e77b:e118/128 *[Local/0] 1w1d 09:23:52 Local via irb.999

Meaning The two customer routing instances and the EVPN Type 5 routing instance for both IPv4

and IPv6 are functioning correctly.

Spine: Verifying the Layer 3 Gateway

Purpose Verify that each tenant host resolves the gateway MAC address by using the Layer 3

gateway IRB interface on the spine devices.



Action Display the ARP table to verify that the hosts use the IRB interface as a Layer 3 gateway:

user@spine-1> show arp no-resolveMAC Address Address Interface Flags00:00:5e:00:01:01 10.1.100.1 irb.100 permanent published gatewayde:ad:be:e1:00:10 10.1.100.10 irb.100 [vtep.32769] nonede:ad:be:e1:00:11 10.1.100.11 irb.100 [vtep.32769] nonede:ad:be:e1:00:12 10.1.100.12 irb.100 [vtep.32769] nonede:ad:be:e1:00:13 10.1.100.13 irb.100 [vtep.32769] nonede:ad:be:e1:00:14 10.1.100.14 irb.100 [vtep.32769] nonede:ad:be:e1:00:20 10.1.100.20 irb.100 [vtep.32770] nonede:ad:be:e1:00:21 10.1.100.21 irb.100 [vtep.32770] nonede:ad:be:e1:00:23 10.1.100.23 irb.100 [vtep.32770] nonede:ad:be:e1:00:24 10.1.100.24 irb.100 [vtep.32770] nonede:ad:be:e1:00:30 10.1.100.30 irb.100 [vtep.32774] nonede:ad:be:e1:00:31 10.1.100.31 irb.100 [vtep.32774] nonede:ad:be:e1:00:32 10.1.100.32 irb.100 [vtep.32774] nonede:ad:be:e1:00:33 10.1.100.33 irb.100 [vtep.32774] nonede:ad:be:e1:00:34 10.1.100.34 irb.100 [vtep.32774] nonede:ad:be:e1:00:40 10.1.100.40 irb.100 [vtep.32772] nonede:ad:be:e1:00:41 10.1.100.41 irb.100 [vtep.32772] nonede:ad:be:e1:00:43 10.1.100.43 irb.100 [vtep.32772] nonede:ad:be:e1:00:44 10.1.100.44 irb.100 [vtep.32772] nonefa:ce:b0:01:00:10 10.1.100.50 irb.100 [vtep.32769] nonefa:ce:b0:01:00:11 10.1.100.51 irb.100 [vtep.32769] nonefa:ce:b0:01:00:13 10.1.100.53 irb.100 [vtep.32769] nonefa:ce:b0:01:00:14 10.1.100.54 irb.100 [vtep.32769] nonefa:ce:b0:01:00:20 10.1.100.60 irb.100 [vtep.32770] nonefa:ce:b0:01:00:21 10.1.100.61 irb.100 [vtep.32770] nonefa:ce:b0:01:00:23 10.1.100.63 irb.100 [vtep.32770] nonefa:ce:b0:01:00:24 10.1.100.64 irb.100 [vtep.32770] nonefa:ce:b0:01:00:30 10.1.100.70 irb.100 [vtep.32774] nonefa:ce:b0:01:00:31 10.1.100.71 irb.100 [vtep.32774] nonefa:ce:b0:01:00:32 10.1.100.72 irb.100 [vtep.32774] nonefa:ce:b0:01:00:33 10.1.100.73 irb.100 [vtep.32774] nonefa:ce:b0:01:00:34 10.1.100.74 irb.100 [vtep.32774] nonede:ad:fa:c1:00:12 10.1.100.112 irb.100 [vtep.32769] nonede:ad:fa:c1:00:13 10.1.100.113 irb.100 [vtep.32769] nonede:ad:fa:c1:00:14 10.1.100.114 irb.100 [vtep.32769] nonede:ad:fa:c1:00:22 10.1.100.122 irb.100 [vtep.32769] nonede:ad:fa:c1:00:23 10.1.100.123 irb.100 [vtep.32769] nonede:ad:fa:c1:00:24 10.1.100.124 irb.100 [vtep.32769] none00:00:5e:00:01:01 10.1.101.1 irb.101 permanent published gatewayde:ad:be:e1:01:10 10.1.101.10 irb.101 [vtep.32769] nonede:ad:be:e1:01:11 10.1.101.11 irb.101 [vtep.32769] nonede:ad:be:e1:01:12 10.1.101.12 irb.101 [vtep.32769] nonede:ad:be:e1:01:13 10.1.101.13 irb.101 [vtep.32769] nonede:ad:be:e1:01:14 10.1.101.14 irb.101 [vtep.32769] nonede:ad:be:e1:01:20 10.1.101.20 irb.101 [vtep.32770] nonede:ad:be:e1:01:21 10.1.101.21 irb.101 [vtep.32770] nonede:ad:be:e1:01:23 10.1.101.23 irb.101 [vtep.32770] nonede:ad:be:e1:01:24 10.1.101.24 irb.101 [vtep.32770] nonede:ad:be:e1:01:30 10.1.101.30 irb.101 [vtep.32774] nonede:ad:be:e1:01:31 10.1.101.31 irb.101 [vtep.32774] nonede:ad:be:e1:01:32 10.1.101.32 irb.101 [vtep.32774] nonede:ad:be:e1:01:33 10.1.101.33 irb.101 [vtep.32774] nonede:ad:be:e1:01:34 10.1.101.34 irb.101 [vtep.32774] nonede:ad:be:e1:01:40 10.1.101.40 irb.101 [vtep.32772] none



de:ad:be:e1:01:41 10.1.101.41 irb.101 [vtep.32772] nonede:ad:be:e1:01:42 10.1.101.42 irb.101 [vtep.32772] nonede:ad:be:e1:01:43 10.1.101.43 irb.101 [vtep.32772] nonede:ad:be:e1:01:44 10.1.101.44 irb.101 [vtep.32772] nonefa:ce:b0:01:01:10 10.1.101.50 irb.101 [vtep.32769] nonefa:ce:b0:01:01:11 10.1.101.51 irb.101 [vtep.32769] nonefa:ce:b0:01:01:13 10.1.101.53 irb.101 [vtep.32769] nonefa:ce:b0:01:01:14 10.1.101.54 irb.101 [vtep.32769] nonefa:ce:b0:01:01:20 10.1.101.60 irb.101 [vtep.32770] nonefa:ce:b0:01:01:21 10.1.101.61 irb.101 [vtep.32770] nonefa:ce:b0:01:01:22 10.1.101.62 irb.101 [vtep.32770] nonefa:ce:b0:01:01:23 10.1.101.63 irb.101 [vtep.32770] nonefa:ce:b0:01:01:24 10.1.101.64 irb.101 [vtep.32770] nonefa:ce:b0:01:01:30 10.1.101.70 irb.101 [vtep.32774] nonefa:ce:b0:01:01:31 10.1.101.71 irb.101 [vtep.32774] nonefa:ce:b0:01:01:32 10.1.101.72 irb.101 [vtep.32774] nonefa:ce:b0:01:01:33 10.1.101.73 irb.101 [vtep.32774] nonefa:ce:b0:01:01:34 10.1.101.74 irb.101 [vtep.32774] nonede:ad:fa:c1:01:12 10.1.101.112 irb.101 [vtep.32769] nonede:ad:fa:c1:01:13 10.1.101.113 irb.101 [vtep.32769] nonede:ad:fa:c1:01:14 10.1.101.114 irb.101 [vtep.32769] nonede:ad:fa:c1:01:22 10.1.101.122 irb.101 [vtep.32769] nonede:ad:fa:c1:01:23 10.1.101.123 irb.101 [vtep.32769] nonede:ad:fa:c1:01:24 10.1.101.124 irb.101 [vtep.32769] none00:00:5e:00:01:01 10.1.102.1 irb.102 permanent published gatewayde:ad:be:e1:02:10 10.1.102.10 irb.102 [vtep.32769] nonede:ad:be:e1:02:11 10.1.102.11 irb.102 [vtep.32769] nonede:ad:be:e1:02:12 10.1.102.12 irb.102 [vtep.32769] nonede:ad:be:e1:02:13 10.1.102.13 irb.102 [vtep.32769] nonede:ad:be:e1:02:14 10.1.102.14 irb.102 [vtep.32769] nonede:ad:be:e1:02:20 10.1.102.20 irb.102 [vtep.32770] nonede:ad:be:e1:02:21 10.1.102.21 irb.102 [vtep.32770] nonede:ad:be:e1:02:22 10.1.102.22 irb.102 [vtep.32770] nonede:ad:be:e1:02:23 10.1.102.23 irb.102 [vtep.32770] nonede:ad:be:e1:02:24 10.1.102.24 irb.102 [vtep.32770] nonede:ad:be:e1:02:30 10.1.102.30 irb.102 [vtep.32774] nonede:ad:be:e1:02:31 10.1.102.31 irb.102 [vtep.32774] nonede:ad:be:e1:02:32 10.1.102.32 irb.102 [vtep.32774] nonede:ad:be:e1:02:33 10.1.102.33 irb.102 [vtep.32774] nonede:ad:be:e1:02:34 10.1.102.34 irb.102 [vtep.32774] nonede:ad:be:e1:02:40 10.1.102.40 irb.102 [vtep.32772] nonede:ad:be:e1:02:41 10.1.102.41 irb.102 [vtep.32772] nonede:ad:be:e1:02:42 10.1.102.42 irb.102 [vtep.32772] nonede:ad:be:e1:02:43 10.1.102.43 irb.102 [vtep.32772] nonede:ad:be:e1:02:44 10.1.102.44 irb.102 [vtep.32772] nonefa:ce:b0:01:02:10 10.1.102.50 irb.102 [vtep.32769] nonefa:ce:b0:01:02:11 10.1.102.51 irb.102 [vtep.32769] nonefa:ce:b0:01:02:12 10.1.102.52 irb.102 [vtep.32769] nonefa:ce:b0:01:02:13 10.1.102.53 irb.102 [vtep.32769] nonefa:ce:b0:01:02:14 10.1.102.54 irb.102 [vtep.32769] nonefa:ce:b0:01:02:20 10.1.102.60 irb.102 [vtep.32770] nonefa:ce:b0:01:02:21 10.1.102.61 irb.102 [vtep.32770] nonefa:ce:b0:01:02:22 10.1.102.62 irb.102 [vtep.32770] nonefa:ce:b0:01:02:23 10.1.102.63 irb.102 [vtep.32770] nonefa:ce:b0:01:02:24 10.1.102.64 irb.102 [vtep.32770] nonefa:ce:b0:01:02:30 10.1.102.70 irb.102 [vtep.32774] nonefa:ce:b0:01:02:31 10.1.102.71 irb.102 [vtep.32774] nonefa:ce:b0:01:02:32 10.1.102.72 irb.102 [vtep.32774] nonefa:ce:b0:01:02:33 10.1.102.73 irb.102 [vtep.32774] nonefa:ce:b0:01:02:34 10.1.102.74 irb.102 [vtep.32774] none



de:ad:fa:c1:02:12 10.1.102.112 irb.102 [vtep.32769] nonede:ad:fa:c1:02:13 10.1.102.113 irb.102 [vtep.32769] nonede:ad:fa:c1:02:14 10.1.102.114 irb.102 [vtep.32769] nonede:ad:fa:c1:02:22 10.1.102.122 irb.102 [vtep.32769] nonede:ad:fa:c1:02:23 10.1.102.123 irb.102 [vtep.32769] nonede:ad:fa:c1:02:24 10.1.102.124 irb.102 [vtep.32769] none00:00:5e:00:01:01 10.1.103.1 irb.103 permanent published gatewayde:ad:be:e1:03:10 10.1.103.10 irb.103 [vtep.32769] nonede:ad:be:e1:03:11 10.1.103.11 irb.103 [vtep.32769] nonede:ad:be:e1:03:12 10.1.103.12 irb.103 [vtep.32769] nonede:ad:be:e1:03:13 10.1.103.13 irb.103 [vtep.32769] nonede:ad:be:e1:03:14 10.1.103.14 irb.103 [vtep.32769] nonede:ad:be:e1:03:20 10.1.103.20 irb.103 [vtep.32770] nonede:ad:be:e1:03:21 10.1.103.21 irb.103 [vtep.32770] nonede:ad:be:e1:03:22 10.1.103.22 irb.103 [vtep.32770] nonede:ad:be:e1:03:23 10.1.103.23 irb.103 [vtep.32770] nonede:ad:be:e1:03:24 10.1.103.24 irb.103 [vtep.32770] nonede:ad:be:e1:03:30 10.1.103.30 irb.103 [vtep.32774] nonede:ad:be:e1:03:31 10.1.103.31 irb.103 [vtep.32774] nonede:ad:be:e1:03:32 10.1.103.32 irb.103 [vtep.32774] nonede:ad:be:e1:03:33 10.1.103.33 irb.103 [vtep.32774] nonede:ad:be:e1:03:34 10.1.103.34 irb.103 [vtep.32774] nonede:ad:be:e1:03:40 10.1.103.40 irb.103 [vtep.32772] nonede:ad:be:e1:03:41 10.1.103.41 irb.103 [vtep.32772] nonede:ad:be:e1:03:42 10.1.103.42 irb.103 [vtep.32772] nonede:ad:be:e1:03:43 10.1.103.43 irb.103 [vtep.32772] nonede:ad:be:e1:03:44 10.1.103.44 irb.103 [vtep.32772] nonefa:ce:b0:01:03:10 10.1.103.50 irb.103 [vtep.32769] nonefa:ce:b0:01:03:11 10.1.103.51 irb.103 [vtep.32769] nonefa:ce:b0:01:03:12 10.1.103.52 irb.103 [vtep.32769] nonefa:ce:b0:01:03:13 10.1.103.53 irb.103 [vtep.32769] nonefa:ce:b0:01:03:14 10.1.103.54 irb.103 [vtep.32769] nonefa:ce:b0:01:03:20 10.1.103.60 irb.103 [vtep.32770] nonefa:ce:b0:01:03:21 10.1.103.61 irb.103 [vtep.32770] nonefa:ce:b0:01:03:22 10.1.103.62 irb.103 [vtep.32770] nonefa:ce:b0:01:03:23 10.1.103.63 irb.103 [vtep.32770] nonefa:ce:b0:01:03:24 10.1.103.64 irb.103 [vtep.32770] nonefa:ce:b0:01:03:30 10.1.103.70 irb.103 [vtep.32774] nonefa:ce:b0:01:03:31 10.1.103.71 irb.103 [vtep.32774] nonefa:ce:b0:01:03:32 10.1.103.72 irb.103 [vtep.32774] nonefa:ce:b0:01:03:33 10.1.103.73 irb.103 [vtep.32774] nonefa:ce:b0:01:03:34 10.1.103.74 irb.103 [vtep.32774] nonede:ad:fa:c1:03:12 10.1.103.112 irb.103 [vtep.32769] nonede:ad:fa:c1:03:13 10.1.103.113 irb.103 [vtep.32769] nonede:ad:fa:c1:03:14 10.1.103.114 irb.103 [vtep.32769] nonede:ad:fa:c1:03:22 10.1.103.122 irb.103 [vtep.32769] nonede:ad:fa:c1:03:23 10.1.103.123 irb.103 [vtep.32769] nonede:ad:fa:c1:03:24 10.1.103.124 irb.103 [vtep.32769] none00:00:5e:00:01:01 10.1.104.1 irb.104 permanent published gatewayde:ad:be:e1:04:10 10.1.104.10 irb.104 [vtep.32769] nonede:ad:be:e1:04:11 10.1.104.11 irb.104 [vtep.32769] nonede:ad:be:e1:04:12 10.1.104.12 irb.104 [vtep.32769] nonede:ad:be:e1:04:13 10.1.104.13 irb.104 [vtep.32769] nonede:ad:be:e1:04:14 10.1.104.14 irb.104 [vtep.32769] nonede:ad:be:e1:04:20 10.1.104.20 irb.104 [vtep.32770] nonede:ad:be:e1:04:21 10.1.104.21 irb.104 [vtep.32770] nonede:ad:be:e1:04:22 10.1.104.22 irb.104 [vtep.32770] nonede:ad:be:e1:04:23 10.1.104.23 irb.104 [vtep.32770] nonede:ad:be:e1:04:24 10.1.104.24 irb.104 [vtep.32770] none



de:ad:be:e1:04:30 10.1.104.30 irb.104 [vtep.32774] nonede:ad:be:e1:04:31 10.1.104.31 irb.104 [vtep.32774] nonede:ad:be:e1:04:32 10.1.104.32 irb.104 [vtep.32774] nonede:ad:be:e1:04:33 10.1.104.33 irb.104 [vtep.32774] nonede:ad:be:e1:04:34 10.1.104.34 irb.104 [vtep.32774] nonede:ad:be:e1:04:40 10.1.104.40 irb.104 [vtep.32772] nonede:ad:be:e1:04:41 10.1.104.41 irb.104 [vtep.32772] nonede:ad:be:e1:04:42 10.1.104.42 irb.104 [vtep.32772] nonede:ad:be:e1:04:43 10.1.104.43 irb.104 [vtep.32772] nonede:ad:be:e1:04:44 10.1.104.44 irb.104 [vtep.32772] nonefa:ce:b0:01:04:10 10.1.104.50 irb.104 [vtep.32769] nonefa:ce:b0:01:04:11 10.1.104.51 irb.104 [vtep.32769] nonefa:ce:b0:01:04:12 10.1.104.52 irb.104 [vtep.32769] nonefa:ce:b0:01:04:13 10.1.104.53 irb.104 [vtep.32769] nonefa:ce:b0:01:04:14 10.1.104.54 irb.104 [vtep.32769] nonefa:ce:b0:01:04:20 10.1.104.60 irb.104 [vtep.32770] nonefa:ce:b0:01:04:21 10.1.104.61 irb.104 [vtep.32770] nonefa:ce:b0:01:04:22 10.1.104.62 irb.104 [vtep.32770] nonefa:ce:b0:01:04:23 10.1.104.63 irb.104 [vtep.32770] nonefa:ce:b0:01:04:24 10.1.104.64 irb.104 [vtep.32770] nonefa:ce:b0:01:04:30 10.1.104.70 irb.104 [vtep.32774] nonefa:ce:b0:01:04:32 10.1.104.72 irb.104 [vtep.32774] nonefa:ce:b0:01:04:33 10.1.104.73 irb.104 [vtep.32774] nonefa:ce:b0:01:04:34 10.1.104.74 irb.104 [vtep.32774] nonede:ad:fa:c1:04:12 10.1.104.112 irb.104 [vtep.32769] nonede:ad:fa:c1:04:13 10.1.104.113 irb.104 [vtep.32769] nonede:ad:fa:c1:04:14 10.1.104.114 irb.104 [vtep.32769] nonede:ad:fa:c1:04:22 10.1.104.122 irb.104 [vtep.32769] nonede:ad:fa:c1:04:23 10.1.104.123 irb.104 [vtep.32769] nonede:ad:fa:c1:04:24 10.1.104.124 irb.104 [vtep.32769] none00:00:5e:00:01:01 10.1.105.1 irb.105 permanent published gatewayde:ad:be:e1:05:10 10.1.105.10 irb.105 [vtep.32769] nonede:ad:be:e1:05:11 10.1.105.11 irb.105 [vtep.32769] nonede:ad:be:e1:05:12 10.1.105.12 irb.105 [vtep.32769] nonede:ad:be:e1:05:13 10.1.105.13 irb.105 [vtep.32769] nonede:ad:be:e1:05:14 10.1.105.14 irb.105 [vtep.32769] nonede:ad:be:e1:05:20 10.1.105.20 irb.105 [vtep.32770] nonede:ad:be:e1:05:21 10.1.105.21 irb.105 [vtep.32770] nonede:ad:be:e1:05:22 10.1.105.22 irb.105 [vtep.32770] nonede:ad:be:e1:05:23 10.1.105.23 irb.105 [vtep.32770] nonede:ad:be:e1:05:24 10.1.105.24 irb.105 [vtep.32770] nonede:ad:be:e1:05:30 10.1.105.30 irb.105 [vtep.32774] nonede:ad:be:e1:05:31 10.1.105.31 irb.105 [vtep.32774] nonede:ad:be:e1:05:32 10.1.105.32 irb.105 [vtep.32774] nonede:ad:be:e1:05:33 10.1.105.33 irb.105 [vtep.32774] nonede:ad:be:e1:05:34 10.1.105.34 irb.105 [vtep.32774] nonede:ad:be:e1:05:40 10.1.105.40 irb.105 [vtep.32772] nonede:ad:be:e1:05:41 10.1.105.41 irb.105 [vtep.32772] nonede:ad:be:e1:05:42 10.1.105.42 irb.105 [vtep.32772] nonede:ad:be:e1:05:43 10.1.105.43 irb.105 [vtep.32772] nonede:ad:be:e1:05:44 10.1.105.44 irb.105 [vtep.32772] nonefa:ce:b0:01:05:10 10.1.105.50 irb.105 [vtep.32769] nonefa:ce:b0:01:05:11 10.1.105.51 irb.105 [vtep.32769] nonefa:ce:b0:01:05:12 10.1.105.52 irb.105 [vtep.32769] nonefa:ce:b0:01:05:13 10.1.105.53 irb.105 [vtep.32769] nonefa:ce:b0:01:05:14 10.1.105.54 irb.105 [vtep.32769] nonefa:ce:b0:01:05:20 10.1.105.60 irb.105 [vtep.32770] nonefa:ce:b0:01:05:21 10.1.105.61 irb.105 [vtep.32770] nonefa:ce:b0:01:05:22 10.1.105.62 irb.105 [vtep.32770] nonefa:ce:b0:01:05:23 10.1.105.63 irb.105 [vtep.32770] none



fa:ce:b0:01:05:24 10.1.105.64 irb.105 [vtep.32770] nonefa:ce:b0:01:05:30 10.1.105.70 irb.105 [vtep.32774] nonefa:ce:b0:01:05:32 10.1.105.72 irb.105 [vtep.32774] nonefa:ce:b0:01:05:33 10.1.105.73 irb.105 [vtep.32774] nonefa:ce:b0:01:05:34 10.1.105.74 irb.105 [vtep.32774] nonede:ad:fa:c1:05:12 10.1.105.112 irb.105 [vtep.32769] nonede:ad:fa:c1:05:13 10.1.105.113 irb.105 [vtep.32769] nonede:ad:fa:c1:05:14 10.1.105.114 irb.105 [vtep.32769] nonede:ad:fa:c1:05:22 10.1.105.122 irb.105 [vtep.32769] nonede:ad:fa:c1:05:23 10.1.105.123 irb.105 [vtep.32769] nonede:ad:fa:c1:05:24 10.1.105.124 irb.105 [vtep.32769] none00:00:5e:00:01:01 10.1.106.1 irb.106 permanent published gatewayde:ad:be:e1:06:10 10.1.106.10 irb.106 [vtep.32769] nonede:ad:be:e1:06:11 10.1.106.11 irb.106 [vtep.32769] nonede:ad:be:e1:06:12 10.1.106.12 irb.106 [vtep.32769] nonede:ad:be:e1:06:13 10.1.106.13 irb.106 [vtep.32769] nonede:ad:be:e1:06:14 10.1.106.14 irb.106 [vtep.32769] nonede:ad:be:e1:06:20 10.1.106.20 irb.106 [vtep.32770] nonede:ad:be:e1:06:21 10.1.106.21 irb.106 [vtep.32770] nonede:ad:be:e1:06:22 10.1.106.22 irb.106 [vtep.32770] nonede:ad:be:e1:06:23 10.1.106.23 irb.106 [vtep.32770] nonede:ad:be:e1:06:24 10.1.106.24 irb.106 [vtep.32770] nonede:ad:be:e1:06:30 10.1.106.30 irb.106 [vtep.32774] nonede:ad:be:e1:06:32 10.1.106.32 irb.106 [vtep.32774] nonede:ad:be:e1:06:33 10.1.106.33 irb.106 [vtep.32774] nonede:ad:be:e1:06:34 10.1.106.34 irb.106 [vtep.32774] nonede:ad:be:e1:06:40 10.1.106.40 irb.106 [vtep.32772] nonede:ad:be:e1:06:41 10.1.106.41 irb.106 [vtep.32772] nonede:ad:be:e1:06:42 10.1.106.42 irb.106 [vtep.32772] nonede:ad:be:e1:06:43 10.1.106.43 irb.106 [vtep.32772] nonede:ad:be:e1:06:44 10.1.106.44 irb.106 [vtep.32772] nonefa:ce:b0:01:06:10 10.1.106.50 irb.106 [vtep.32769] nonefa:ce:b0:01:06:11 10.1.106.51 irb.106 [vtep.32769] nonefa:ce:b0:01:06:12 10.1.106.52 irb.106 [vtep.32769] nonefa:ce:b0:01:06:13 10.1.106.53 irb.106 [vtep.32769] nonefa:ce:b0:01:06:14 10.1.106.54 irb.106 [vtep.32769] nonefa:ce:b0:01:06:20 10.1.106.60 irb.106 [vtep.32770] nonefa:ce:b0:01:06:21 10.1.106.61 irb.106 [vtep.32770] nonefa:ce:b0:01:06:22 10.1.106.62 irb.106 [vtep.32770] nonefa:ce:b0:01:06:23 10.1.106.63 irb.106 [vtep.32770] nonefa:ce:b0:01:06:24 10.1.106.64 irb.106 [vtep.32770] nonefa:ce:b0:01:06:30 10.1.106.70 irb.106 [vtep.32774] nonefa:ce:b0:01:06:32 10.1.106.72 irb.106 [vtep.32774] nonefa:ce:b0:01:06:33 10.1.106.73 irb.106 [vtep.32774] nonefa:ce:b0:01:06:34 10.1.106.74 irb.106 [vtep.32774] nonede:ad:fa:c1:06:12 10.1.106.112 irb.106 [vtep.32769] nonede:ad:fa:c1:06:13 10.1.106.113 irb.106 [vtep.32769] nonede:ad:fa:c1:06:14 10.1.106.114 irb.106 [vtep.32769] nonede:ad:fa:c1:06:22 10.1.106.122 irb.106 [vtep.32769] nonede:ad:fa:c1:06:23 10.1.106.123 irb.106 [vtep.32769] nonede:ad:fa:c1:06:24 10.1.106.124 irb.106 [vtep.32769] none00:00:5e:00:01:01 10.1.107.1 irb.107 permanent published gatewayde:ad:be:e1:07:10 10.1.107.10 irb.107 [vtep.32769] nonede:ad:be:e1:07:11 10.1.107.11 irb.107 [vtep.32769] nonede:ad:be:e1:07:12 10.1.107.12 irb.107 [vtep.32769] nonede:ad:be:e1:07:13 10.1.107.13 irb.107 [vtep.32769] nonede:ad:be:e1:07:14 10.1.107.14 irb.107 [vtep.32769] nonede:ad:be:e1:07:20 10.1.107.20 irb.107 [vtep.32770] nonede:ad:be:e1:07:21 10.1.107.21 irb.107 [vtep.32770] none



de:ad:be:e1:07:22 10.1.107.22 irb.107 [vtep.32770] nonede:ad:be:e1:07:23 10.1.107.23 irb.107 [vtep.32770] nonede:ad:be:e1:07:24 10.1.107.24 irb.107 [vtep.32770] nonede:ad:be:e1:07:30 10.1.107.30 irb.107 [vtep.32774] nonede:ad:be:e1:07:32 10.1.107.32 irb.107 [vtep.32774] nonede:ad:be:e1:07:33 10.1.107.33 irb.107 [vtep.32774] nonede:ad:be:e1:07:34 10.1.107.34 irb.107 [vtep.32774] nonede:ad:be:e1:07:40 10.1.107.40 irb.107 [vtep.32772] nonede:ad:be:e1:07:41 10.1.107.41 irb.107 [vtep.32772] nonede:ad:be:e1:07:42 10.1.107.42 irb.107 [vtep.32772] nonede:ad:be:e1:07:43 10.1.107.43 irb.107 [vtep.32772] nonede:ad:be:e1:07:44 10.1.107.44 irb.107 [vtep.32772] nonefa:ce:b0:01:07:10 10.1.107.50 irb.107 [vtep.32769] nonefa:ce:b0:01:07:11 10.1.107.51 irb.107 [vtep.32769] nonefa:ce:b0:01:07:12 10.1.107.52 irb.107 [vtep.32769] nonefa:ce:b0:01:07:13 10.1.107.53 irb.107 [vtep.32769] nonefa:ce:b0:01:07:14 10.1.107.54 irb.107 [vtep.32769] nonefa:ce:b0:01:07:20 10.1.107.60 irb.107 [vtep.32770] nonefa:ce:b0:01:07:21 10.1.107.61 irb.107 [vtep.32770] nonefa:ce:b0:01:07:22 10.1.107.62 irb.107 [vtep.32770] nonefa:ce:b0:01:07:23 10.1.107.63 irb.107 [vtep.32770] nonefa:ce:b0:01:07:24 10.1.107.64 irb.107 [vtep.32770] nonefa:ce:b0:01:07:30 10.1.107.70 irb.107 [vtep.32774] nonefa:ce:b0:01:07:32 10.1.107.72 irb.107 [vtep.32774] nonefa:ce:b0:01:07:33 10.1.107.73 irb.107 [vtep.32774] nonefa:ce:b0:01:07:34 10.1.107.74 irb.107 [vtep.32774] nonede:ad:fa:c1:07:12 10.1.107.112 irb.107 [vtep.32769] nonede:ad:fa:c1:07:13 10.1.107.113 irb.107 [vtep.32769] nonede:ad:fa:c1:07:14 10.1.107.114 irb.107 [vtep.32769] nonede:ad:fa:c1:07:21 10.1.107.121 irb.107 [vtep.32769] nonede:ad:fa:c1:07:22 10.1.107.122 irb.107 [vtep.32769] nonede:ad:fa:c1:07:23 10.1.107.123 irb.107 [vtep.32769] nonede:ad:fa:c1:07:24 10.1.107.124 irb.107 [vtep.32769] none00:00:5e:00:01:01 10.1.108.1 irb.108 permanent published gatewayde:ad:be:e1:08:10 10.1.108.10 irb.108 [vtep.32769] nonede:ad:be:e1:08:11 10.1.108.11 irb.108 [vtep.32769] nonede:ad:be:e1:08:12 10.1.108.12 irb.108 [vtep.32769] nonede:ad:be:e1:08:13 10.1.108.13 irb.108 [vtep.32769] nonede:ad:be:e1:08:14 10.1.108.14 irb.108 [vtep.32769] nonede:ad:be:e1:08:20 10.1.108.20 irb.108 [vtep.32770] nonede:ad:be:e1:08:21 10.1.108.21 irb.108 [vtep.32770] nonede:ad:be:e1:08:22 10.1.108.22 irb.108 [vtep.32770] nonede:ad:be:e1:08:23 10.1.108.23 irb.108 [vtep.32770] nonede:ad:be:e1:08:24 10.1.108.24 irb.108 [vtep.32770] nonede:ad:be:e1:08:30 10.1.108.30 irb.108 [vtep.32774] nonede:ad:be:e1:08:31 10.1.108.31 irb.108 [vtep.32774] nonede:ad:be:e1:08:32 10.1.108.32 irb.108 [vtep.32774] nonede:ad:be:e1:08:33 10.1.108.33 irb.108 [vtep.32774] nonede:ad:be:e1:08:34 10.1.108.34 irb.108 [vtep.32774] nonede:ad:be:e1:08:40 10.1.108.40 irb.108 [vtep.32772] nonede:ad:be:e1:08:41 10.1.108.41 irb.108 [vtep.32772] nonede:ad:be:e1:08:42 10.1.108.42 irb.108 [vtep.32772] nonede:ad:be:e1:08:43 10.1.108.43 irb.108 [vtep.32772] nonede:ad:be:e1:08:44 10.1.108.44 irb.108 [vtep.32772] nonefa:ce:b0:01:08:10 10.1.108.50 irb.108 [vtep.32769] nonefa:ce:b0:01:08:11 10.1.108.51 irb.108 [vtep.32769] nonefa:ce:b0:01:08:12 10.1.108.52 irb.108 [vtep.32769] nonefa:ce:b0:01:08:13 10.1.108.53 irb.108 [vtep.32769] nonefa:ce:b0:01:08:14 10.1.108.54 irb.108 [vtep.32769] nonefa:ce:b0:01:08:20 10.1.108.60 irb.108 [vtep.32770] none



fa:ce:b0:01:08:22 10.1.108.62 irb.108 [vtep.32770] nonefa:ce:b0:01:08:23 10.1.108.63 irb.108 [vtep.32770] nonefa:ce:b0:01:08:24 10.1.108.64 irb.108 [vtep.32770] nonefa:ce:b0:01:08:30 10.1.108.70 irb.108 [vtep.32774] nonefa:ce:b0:01:08:32 10.1.108.72 irb.108 [vtep.32774] nonefa:ce:b0:01:08:33 10.1.108.73 irb.108 [vtep.32774] nonefa:ce:b0:01:08:34 10.1.108.74 irb.108 [vtep.32774] nonede:ad:fa:c1:08:11 10.1.108.111 irb.108 [vtep.32769] nonede:ad:fa:c1:08:12 10.1.108.112 irb.108 [vtep.32769] nonede:ad:fa:c1:08:13 10.1.108.113 irb.108 [vtep.32769] nonede:ad:fa:c1:08:14 10.1.108.114 irb.108 [vtep.32769] nonede:ad:fa:c1:08:21 10.1.108.121 irb.108 [vtep.32769] nonede:ad:fa:c1:08:22 10.1.108.122 irb.108 [vtep.32769] nonede:ad:fa:c1:08:23 10.1.108.123 irb.108 [vtep.32769] nonede:ad:fa:c1:08:24 10.1.108.124 irb.108 [vtep.32769] none5c:5e:ab:79:42:81 10.94.191.254 em0.0 none00:00:5e:00:01:01 10.255.99.1 irb.999 permanent published gatewayde:ad:be:e9:99:10 10.255.99.10 irb.999 [vtep.32769] nonede:ad:be:e9:99:11 10.255.99.11 irb.999 [vtep.32769] nonede:ad:be:e9:99:12 10.255.99.12 irb.999 [vtep.32769] nonede:ad:be:e9:99:13 10.255.99.13 irb.999 [vtep.32769] nonede:ad:be:e9:99:14 10.255.99.14 irb.999 [vtep.32769] nonede:ad:be:e9:99:20 10.255.99.20 irb.999 [vtep.32770] nonede:ad:be:e9:99:21 10.255.99.21 irb.999 [vtep.32770] nonede:ad:be:e9:99:22 10.255.99.22 irb.999 [vtep.32770] nonede:ad:be:e9:99:23 10.255.99.23 irb.999 [vtep.32770] nonede:ad:be:e9:99:24 10.255.99.24 irb.999 [vtep.32770] nonefe:54:00:1c:95:51 128.0.0.16 bme0.0 none88:a2:5e:cc:89:bf 172.16.0.0 et-0/0/58.0 none88:a2:5e:cc:98:bf 172.16.0.8 et-0/0/59.0 none88:a2:5e:cd:4c:b7 172.16.0.16 et-0/0/61.0 none88:a2:5e:cd:60:b7 172.16.0.24 et-0/0/60.0 none88:a2:5e:cc:d9:bb 172.16.0.33 et-0/0/66.0 none88:a2:5e:cc:48:bb 172.16.0.35 et-0/0/67.0 none00:31:46:7b:e1:1b 192.168.1.1 em2.32768 noneTotal entries: 387

Meaning The host MAC addresses aremapped to the corresponding IRB interfaces that are being

used as a Layer 3 gateway.

Spine: Verifying the Switching Table

Purpose Verify that VNI, VTEP, and ESI information appears in the switching table for a

corresponding VLAN.



Action Display switching table information for VLAN 100:

user@spine-1> show ethernet-switching table vlan-id 100

MAC flags (S - static MAC, D - dynamic MAC, L - locally learned, P - Persistent static SE - statistics enabled, NM - non configured MAC, R - remote PE MAC, O - ovsdb MAC)

Ethernet switching table : 71 entries, 71 learnedRouting instance : default-switch Vlan MAC MAC Logical Active

name address flags interface source

v1000 00:00:5e:00:01:01 DR,SD esi.1937 v1000 00:00:5e:00:02:01 DR,SD esi.1937 v1000 00:00:5e:00:53:00 D vtep.32769 v1000 00:00:5e:00:53:01 D vtep.32769 v1000 00:00:5e:00:53:02 D vtep.32769 v1000 00:00:5e:00:53:03 D vtep.32769 v1000 00:00:5e:00:53:04 D vtep.32769 v1000 00:00:5e:00:54:10 D vtep.32770 v1000 00:00:5e:00:54:11 D vtep.32770 v1000 00:00:5e:00:54:12 D vtep.32770 v1000 00:00:5e:00:54:13 D vtep.32770 v1000 00:00:5e:00:54:14 D vtep.32770 v1000 00:00:5e:00:55:10 D vtep.32774 v1000 00:00:5e:00:55:11 D vtep.32774 v1000 00:00:5e:00:55:12 D vtep.32774 v1000 00:00:5e:00:55:13 D vtep.32774 v1000 00:00:5e:00:55:14 D vtep.32774 v1000 00:00:5e:00:56:10 D vtep.32772 v1000 00:00:5e:00:56:11 D vtep.32772 v1000 00:00:5e:00:56:12 D vtep.32772 v1000 00:00:5e:00:56:13 D vtep.32772 v1000 00:00:5e:00:56:14 D vtep.32772 v1000 00:31:46:79:e4:9a D vtep.32775 v1000 00:31:46:7a:04:9a D vtep.32773 v1000 de:ad:be:e1:00:10 D vtep.32769 v1000 de:ad:be:e1:00:11 D vtep.32769 v1000 de:ad:be:e1:00:12 D vtep.32769 v1000 de:ad:be:e1:00:13 D vtep.32769 v1000 de:ad:be:e1:00:14 D vtep.32769 v1000 de:ad:be:e1:00:20 D vtep.32770 v1000 de:ad:be:e1:00:21 D vtep.32770 v1000 de:ad:be:e1:00:22 D vtep.32770 v1000 de:ad:be:e1:00:23 D vtep.32770 v1000 de:ad:be:e1:00:24 D vtep.32770 v1000 de:ad:be:e1:00:30 D vtep.32774 v1000 de:ad:be:e1:00:31 D vtep.32774 v1000 de:ad:be:e1:00:32 D vtep.32774 v1000 de:ad:be:e1:00:33 D vtep.32774 v1000 de:ad:be:e1:00:34 D vtep.32774 v1000 de:ad:be:e1:00:40 D vtep.32772 v1000 de:ad:be:e1:00:41 D vtep.32772 v1000 de:ad:be:e1:00:42 D vtep.32772 v1000 de:ad:be:e1:00:43 D vtep.32772 v1000 de:ad:be:e1:00:44 D vtep.32772



v1000 de:ad:fa:c1:00:10 D vtep.32769 v1000 de:ad:fa:c1:00:11 D vtep.32769 v1000 de:ad:fa:c1:00:12 D vtep.32769 v1000 de:ad:fa:c1:00:13 D vtep.32769 v1000 de:ad:fa:c1:00:14 D vtep.32769 v1000 de:ad:fa:c1:00:20 D vtep.32769 v1000 de:ad:fa:c1:00:21 D vtep.32769 v1000 de:ad:fa:c1:00:22 D vtep.32769 v1000 de:ad:fa:c1:00:23 D vtep.32769 v1000 de:ad:fa:c1:00:24 D vtep.32769 v1000 ec:3e:f7:89:15:1a D vtep.32771 v1000 fa:ce:b0:01:00:10 D vtep.32769 v1000 fa:ce:b0:01:00:11 D vtep.32769 v1000 fa:ce:b0:01:00:12 D vtep.32769 v1000 fa:ce:b0:01:00:13 D vtep.32769 v1000 fa:ce:b0:01:00:14 D vtep.32769 v1000 fa:ce:b0:01:00:20 D vtep.32770 v1000 fa:ce:b0:01:00:21 D vtep.32770 v1000 fa:ce:b0:01:00:22 D vtep.32770 v1000 fa:ce:b0:01:00:23 D vtep.32770 v1000 fa:ce:b0:01:00:24 D vtep.32770 v1000 fa:ce:b0:01:00:30 D vtep.32774 v1000 fa:ce:b0:01:00:31 D vtep.32774 v1000 fa:ce:b0:01:00:32 D vtep.32774 v1000 fa:ce:b0:01:00:33 D vtep.32774 v1000 fa:ce:b0:01:00:34 D vtep.32774 v1000 fa:ce:b0:01:00:42 DR,SD esi.1939

Meaning Because VNI, VTEP, and ESI information appears in the switching table, the mapping of

this information to the VLANs is functioning correctly.

Spine: Verifying the Source of the VXLAN Tunnel

Purpose Verify source information for the VXLAN tunnel to confirm the correct VLAN-to-VNI

mappings and local VTEP configuration.



Action Display source information for the VXLAN tunnel:

user@spine-1> show ethernet-switching vxlan-tunnel-end-point sourceLogical System Name Id SVTEP-IP IFL L3-Idx<default> 0 10.0.0.11 lo0.0 0 L2-RTT Bridge Domain VNID MC-Group-IP default-switch TYPE-5+999 1999 0.0.0.0

default-switch v1000+100 1000 0.0.0.0

default-switch v1001+101 1001 0.0.0.0

default-switch v1002+102 1002 0.0.0.0

default-switch v1003+103 1003 0.0.0.0

default-switch v1004+104 1004 0.0.0.0

default-switch v1005+105 1005 0.0.0.0

default-switch v1006+106 1006 0.0.0.0

default-switch v1007+107 1007 0.0.0.0

default-switch v1008+108 1008 0.0.0.0

Meaning Because theVLAN-to-VNImappingsand localVTEPconfigurationarecorrect, theVXLAN

tunnel source is functioning correctly.

Spine: Verifying VNI-to-VXLAN Tunnel Mapping

Purpose Verify that each VNI maps properly to each VXLAN tunnel, the spine device is properly

connected to the remote VTEPs, and has the correct reachability..



Action Verify the mapping of the VNIs to the VXLAN tunnels by displaying the remote VTEP

information:

user@spine-1> show ethernet-switching vxlan-tunnel-end-point remoteLogical System Name Id SVTEP-IP IFL L3-Idx<default> 0 10.0.0.11 lo0.0 0 RVTEP-IP IFL-Idx NH-Id 10.0.0.12 829 1928 VNID MC-Group-IP 1999 0.0.0.0 1008 0.0.0.0 1007 0.0.0.0 1006 0.0.0.0 1005 0.0.0.0 1004 0.0.0.0 1003 0.0.0.0 1002 0.0.0.0 1001 0.0.0.0 1000 0.0.0.0 RVTEP-IP IFL-Idx NH-Id 10.0.0.13 833 1942 VNID MC-Group-IP 1999 0.0.0.0 1008 0.0.0.0 1007 0.0.0.0 1006 0.0.0.0 1005 0.0.0.0 1004 0.0.0.0 1003 0.0.0.0 1002 0.0.0.0 1001 0.0.0.0 1000 0.0.0.0 RVTEP-IP IFL-Idx NH-Id 10.0.0.14 831 1940 VNID MC-Group-IP 1999 0.0.0.0 1008 0.0.0.0 1007 0.0.0.0 1006 0.0.0.0 1005 0.0.0.0 1004 0.0.0.0 1003 0.0.0.0 1002 0.0.0.0 1001 0.0.0.0 1000 0.0.0.0 RVTEP-IP IFL-Idx NH-Id 10.0.0.21 827 1906 VNID MC-Group-IP 1999 0.0.0.0 1008 0.0.0.0 1007 0.0.0.0 1006 0.0.0.0 1005 0.0.0.0 1004 0.0.0.0 1003 0.0.0.0 1002 0.0.0.0 1001 0.0.0.0 1000 0.0.0.0 RVTEP-IP IFL-Idx NH-Id 10.0.0.22 828 1927



VNID MC-Group-IP 1999 0.0.0.0 1008 0.0.0.0 1007 0.0.0.0 1006 0.0.0.0 1005 0.0.0.0 1004 0.0.0.0 1003 0.0.0.0 1002 0.0.0.0 1001 0.0.0.0 1000 0.0.0.0 RVTEP-IP IFL-Idx NH-Id 10.0.0.23 806 1796 VNID MC-Group-IP 1008 0.0.0.0 1007 0.0.0.0 1006 0.0.0.0 1005 0.0.0.0 1004 0.0.0.0 1003 0.0.0.0 1002 0.0.0.0 1001 0.0.0.0 1000 0.0.0.0 1999 0.0.0.0 RVTEP-IP IFL-Idx NH-Id 10.0.0.24 807 1797 VNID MC-Group-IP 1008 0.0.0.0 1007 0.0.0.0 1006 0.0.0.0 1005 0.0.0.0 1004 0.0.0.0 1003 0.0.0.0 1002 0.0.0.0 1001 0.0.0.0 1000 0.0.0.0 1999 0.0.0.0

Meaning TheVNIsaremapped to thecorrectVXLANtunnels, thespinedevice isproperly connected

to the remote VTEPs, and has the correct reachability..

Spine: VerifyingMACAddress Learning

Purpose Verify that the MAC addresses are learned through the VXLAN tunnels.



Action Display the MAC addresses that are learned through the VXLAN tunnels:

user@spine-1> show ethernet-switching vxlan-tunnel-end-point remotemac-table


Logical system : defaultRouting instance : default-switch Bridging domain : TYPE-5+999, VLAN : 999, VNID : 1999 MAC MAC Logical Remote VTEP address flags interface IP address 00:00:5e:00:01:01 DR,SD esi.1693 10.0.0.12 00:00:5e:00:02:01 DR,SD esi.1693 10.0.0.12 de:ad:be:e9:99:10 D vtep.32769 10.0.0.21 de:ad:be:e9:99:11 D vtep.32769 10.0.0.21 de:ad:be:e9:99:12 D vtep.32769 10.0.0.21 de:ad:be:e9:99:13 D vtep.32769 10.0.0.21 de:ad:be:e9:99:14 D vtep.32769 10.0.0.21 de:ad:be:e9:99:20 D vtep.32770 10.0.0.22 de:ad:be:e9:99:21 D vtep.32770 10.0.0.22 de:ad:be:e9:99:22 D vtep.32770 10.0.0.22 de:ad:be:e9:99:23 D vtep.32770 10.0.0.22 de:ad:be:e9:99:24 D vtep.32770 10.0.0.22 ec:3e:f7:89:15:1a D vtep.32771 10.0.0.12


Bridging domain : v1000+100, VLAN : 100, VNID : 1000 MAC MAC Logical Remote VTEP address flags interface IP address 00:00:5e:00:01:01 DR,SD esi.1937 10.0.0.12 00:00:5e:00:02:01 DR,SD esi.1937 10.0.0.12 fa:ce:b0:01:00:42 DR,SD esi.1939 10.0.0.23 00:00:5e:00:53:00 D vtep.32769 10.0.0.21 00:00:5e:00:53:01 D vtep.32769 10.0.0.21 00:00:5e:00:53:02 D vtep.32769 10.0.0.21 00:00:5e:00:53:03 D vtep.32769 10.0.0.21 00:00:5e:00:53:04 D vtep.32769 10.0.0.21 de:ad:be:e1:00:10 D vtep.32769 10.0.0.21 de:ad:be:e1:00:11 D vtep.32769 10.0.0.21 de:ad:be:e1:00:12 D vtep.32769 10.0.0.21 de:ad:be:e1:00:13 D vtep.32769 10.0.0.21 de:ad:be:e1:00:14 D vtep.32769 10.0.0.21 de:ad:fa:c1:00:10 D vtep.32769 10.0.0.21 de:ad:fa:c1:00:11 D vtep.32769 10.0.0.21 de:ad:fa:c1:00:12 D vtep.32769 10.0.0.21 de:ad:fa:c1:00:13 D vtep.32769 10.0.0.21 de:ad:fa:c1:00:14 D vtep.32769 10.0.0.21 de:ad:fa:c1:00:20 D vtep.32769 10.0.0.21 de:ad:fa:c1:00:21 D vtep.32769 10.0.0.21 de:ad:fa:c1:00:22 D vtep.32769 10.0.0.21 de:ad:fa:c1:00:23 D vtep.32769 10.0.0.21 de:ad:fa:c1:00:24 D vtep.32769 10.0.0.21 fa:ce:b0:01:00:10 D vtep.32769 10.0.0.21 fa:ce:b0:01:00:11 D vtep.32769 10.0.0.21 fa:ce:b0:01:00:12 D vtep.32769 10.0.0.21 fa:ce:b0:01:00:13 D vtep.32769 10.0.0.21 fa:ce:b0:01:00:14 D vtep.32769 10.0.0.21



00:00:5e:00:54:10 D vtep.32770 10.0.0.22 00:00:5e:00:54:11 D vtep.32770 10.0.0.22 00:00:5e:00:54:12 D vtep.32770 10.0.0.22 00:00:5e:00:54:13 D vtep.32770 10.0.0.22 00:00:5e:00:54:14 D vtep.32770 10.0.0.22 de:ad:be:e1:00:20 D vtep.32770 10.0.0.22 de:ad:be:e1:00:21 D vtep.32770 10.0.0.22 de:ad:be:e1:00:22 D vtep.32770 10.0.0.22 de:ad:be:e1:00:23 D vtep.32770 10.0.0.22 de:ad:be:e1:00:24 D vtep.32770 10.0.0.22 fa:ce:b0:01:00:20 D vtep.32770 10.0.0.22 fa:ce:b0:01:00:21 D vtep.32770 10.0.0.22 fa:ce:b0:01:00:22 D vtep.32770 10.0.0.22 fa:ce:b0:01:00:23 D vtep.32770 10.0.0.22 fa:ce:b0:01:00:24 D vtep.32770 10.0.0.22 ec:3e:f7:89:15:1a D vtep.32771 10.0.0.12 00:00:5e:00:56:10 D vtep.32772 10.0.0.24 00:00:5e:00:56:11 D vtep.32772 10.0.0.24 00:00:5e:00:56:12 D vtep.32772 10.0.0.24 00:00:5e:00:56:13 D vtep.32772 10.0.0.24 00:00:5e:00:56:14 D vtep.32772 10.0.0.24 de:ad:be:e1:00:40 D vtep.32772 10.0.0.24 de:ad:be:e1:00:41 D vtep.32772 10.0.0.24 de:ad:be:e1:00:42 D vtep.32772 10.0.0.24 de:ad:be:e1:00:43 D vtep.32772 10.0.0.24 de:ad:be:e1:00:44 D vtep.32772 10.0.0.24 00:31:46:7a:04:9a D vtep.32773 10.0.0.14 00:00:5e:00:55:10 D vtep.32774 10.0.0.23 00:00:5e:00:55:11 D vtep.32774 10.0.0.23 00:00:5e:00:55:12 D vtep.32774 10.0.0.23 00:00:5e:00:55:13 D vtep.32774 10.0.0.23 00:00:5e:00:55:14 D vtep.32774 10.0.0.23 de:ad:be:e1:00:30 D vtep.32774 10.0.0.23 de:ad:be:e1:00:31 D vtep.32774 10.0.0.23 de:ad:be:e1:00:32 D vtep.32774 10.0.0.23 de:ad:be:e1:00:33 D vtep.32774 10.0.0.23 de:ad:be:e1:00:34 D vtep.32774 10.0.0.23 fa:ce:b0:01:00:30 D vtep.32774 10.0.0.23 fa:ce:b0:01:00:31 D vtep.32774 10.0.0.23 fa:ce:b0:01:00:32 D vtep.32774 10.0.0.23 fa:ce:b0:01:00:33 D vtep.32774 10.0.0.23 fa:ce:b0:01:00:34 D vtep.32774 10.0.0.23 00:31:46:79:e4:9a D vtep.32775 10.0.0.13

Meaning MAC addresses are being shared across the VXLAN tunnels correctly.

Fabric: Verifying Interfaces

Purpose Verify the state of the spine-facing interfaces.



Action Verify that the spine-facing interfaces are up:

user@fabric-1> show interfaces terseInterface Admin Link Proto Local Remoteet-0/0/12 up upet-0/0/12.0 up up inet 172.16.0.6/31 ## <<< To Spine 4 et-0/0/13 up upet-0/0/13.0 up up inet 172.16.0.4/31 ## <<< To Spine 3 et-0/0/14 up upet-0/0/14.0 up up inet 172.16.0.2/31 ## <<< To Spine 2 et-0/0/15 up up et-0/0/15.0 up up inet 172.16.0.0/31 ## <<< To Spine 1

Meaning The spine-facing interfaces for the fabric devices are operating correctly.

Fabric: Verifying IPv4 BGP Sessions

Purpose Verify the state of spine-facing IPv4 BGP sessions.

Action Verify that the IPv4 BGP sessions are established:

user@fabric-1> show bgp summaryGroups: 1 Peers: 4 Down peers: 0Table Tot Paths Act Paths Suppressed History Damp State Pendinginet.0 10720 2082 0 0 0 0Peer AS InPkt OutPkt OutQ Flaps Last Up/Dwn State|#Active/Received/Accepted/Damped...172.16.0.1 65011 53 59 0 0 17:52 16/22/20/0 0/0/0/0172.16.0.3 65012 53 61 0 0 17:52 16/22/20/0 0/0/0/0172.16.0.5 65013 43 47 0 0 13:23 16/22/20/0 0/0/0/0172.16.0.7 65014 51 54 0 0 17:52 16/22/20/0 0/0/0/0

Meaning The BGP sessions are established and functioning correctly.

Fabric: Verifying BFD

Purpose Verify that Bidirectional Forwarding Detection (BFD) is operating correctly between the

fabric and spine devices.



Action Verify that BFD is operating between the fabric and spine devices:

user@fabric-1> show bfd session Detect TransmitAddress State Interface Time Interval Multiplier172.16.0.1 Up et-0/0/15.0 1.050 0.350 3 172.16.0.3 Up et-0/0/14.0 1.050 0.350 3 172.16.0.5 Up et-0/0/13.0 1.050 0.350 3 172.16.0.7 Up et-0/0/12.0 1.050 0.350 3


Meaning BFD is operating correctly between the fabric and spine devices.

All Devices: Verifying Port Mirroring

Purpose Verify that port mirroring is operating correctly.

Action Display the port-mirroring firewall filters:a.

user@spine-1> show firewall

Filter: PM-et-0/0/58.0-i Counters:Name Bytes PacketsDEF-et-0/0/58.0-i 812944 14343PM_T1-et-0/0/58.0-i 3042 39





Filter: PM-et-0/0/67.0-i Counters:Name Bytes Packets



DEF-et-0/0/67.0-i 1507781 28060PM_T1-et-0/0/67.0-i 3370166920 36631985

b. Display the port-mirroring statistics:

user@spine-1> show forwarding-options port-mirroringInstance Name: &global_instance Instance Id: 1 Input parameters: Rate : 1 Run-length : 0 Maximum-packet-length : 0 Output parameters: Family State Destination Next-hop ethernet-switching up xe-0/0/34:1.0

Instance Name: VXLAN_PM_Instance Instance Id: 2 Input parameters: Rate : 1 Run-length : 0 Maximum-packet-length : 0 Output parameters: Family State Destination Next-hop ethernet-switching up xe-0/0/34:1.0

Meaning Port mirroring is operating correctly.

RelatedDocumentation

• Understanding the IaaS: EVPN and VXLAN Solution on page 5



solution guide infrastructure as a service: evpn and vxlan · fabric1 10.0.0.1 65001 fabric2...

Documents