expressroute: connecting private and public clouds through

ExpressRoute: Connecting Private and Public Clouds through Network Service ProvidersVenkat GattamneniMicrosoft Azure

DCIM-B423

Doug SinkAT&TGene BakerMcKesson

Jon OrmondMicrosoft IT

ExpressRoute Overview

AT&T NetBond and ExpressRoute

Customer case studies – Microsoft IT, McKesson

Agenda

Cloud on your WANAvoids risks from exposure to InternetAvoids complexity and added costsProvides lower latency, higher bandwidth and greater availability

Public Cloud

WAN

Customer DC

Customer site 1

Customer site 2

Public internet

Customers want Cloud on their networks

IPsec VPN over InternetGreater networking costs and higher latencyData traverses the Internet to reach public cloudLimited bandwidth

Public Cloud

WAN

Customer DC

Customer site 1

Customer site 2

Public internet

Security

Lower cost

Predictable performance

High throughput

What is ExpressRoute?

Connect your private network with Azure via secure, high-throughput, low latency connections bypassing the Internet

Azure

Private Networ

k

Customer DC

Customer site 1

Customer site 2

ExpressRoute

Enterprise workloads Dev/test lab BI/big data

Media Productivity apps

Storage, backup, and recovery

Hybrid apps

ExpressRoute Flavors and PartnersConnecting at an Exchange provider

Connecting via a Network service provider

ExpressRoutepartner location

Publicinternet

Customer site

Microsoft Azure

Customer site 1

Customer site 2

Customer site 3

WANPublic

internet

Microsoft Azure

ExpressRoute Bandwidth tiersNetwork Service Provider ScenarioMonthly dual-port fee.Unlimited data transfer (in and out) included

10 Mbps 50 Mbps

100 Mbps 500 Mbps

99.9% SLA

DedicatedCircuit uptime

50 Mbps

Available Today• Washington D.C. • Silicon Valley, CA• London, UK

Coming Soon...• Additional sites in Europe,

Asia, and North America

Locations:

Global datacenters

ExpressRoute locations today

ExpressRoute locations

Doug Sink, AT&T Enterprise Solution Consulting

ExpressRoute and AT&T

AT&T MPLS OverviewAT&T NetBondIntegration with ExpressRoute and AzureNetBond Configuration and Orchestration

Agenda

© 2014 AT&T Intellectual Property. All rights reserved. AT&T and the AT&T logo are trademarks of AT&T Intellectual Property.12

MPLS Primer, Terminology, Topology

MPLS: Multi Protocol Label SwitchingRFC 4364

CER: Customer Edge RouterIP routing not MPLS

PER: Provider Edge RouterIP routing to CERLabel switching to the core

Control Plane VRF: Virtual Route Forwarding,

separate routing table per customer on the PER

RD: Route DistinguisherRT: Route TargetMP-BGP: Multi-Protocol BGPForwarding PlaneLDP: Label Distribution Protocol, assign

and distribute forwarding labelsLSP: Label Switched Paths

Customer separation, security• MPLS VPN attributes (VRF, RD, RT) to separate IP routing within the PE and PE-to-PE• MPLS label switches traffic, not IP forwarded, separating customers. • IP routes not known in the core (route free core)

MPLS Product Offers • AVPN, PNT, IPFR, EVPN,

Hybrids

10.1.1.0/24 NH=CER

AS:65000IP Traffic

Customer 1

MPiBGP

OSPFLDP

10.1.1.0 : RDNH (next hop=PE)RT, inner Label

Label Switched Path

MPLS LSR

eBGP

CER

RR

PER PER

Customer 1

CER

VRF1 VRF1eBGP

Outerlabel

No IP route lookups except at first PER

OSPFLDP


Basic MPLS

US VPNAS 13979

CER

CER

CER

Customer Routing

Route Reflector

PER

PER

Customer Routing

CER

CERPER

PERCustomer Routing

Customer Routing

Core IGP

Core IGP

Core IGP

Core IGP

LSR LSRLSR

LSR LSRLSR

LSR

MP-iBGP


BenefitsPerformance, COSReliabilityScalability SecurityAny-to-any or Hub-spoke connectivity Reporting Service Level AgreementsFast Failover & Disaster Recovery

AT&T VPN Service Overview

Customer Access

Firewall

AT&T MPLSNetwork

The Internet

AT&TPER

PPP, POSFR* Ethernet

DSL*

ATM*

Smartphone

*AT&T VPN Frame, ATM & DSL Ports are on Sales Hold and Unavailable to New Customers


AVPN Connectivity


VPNAccess

Value-AddedSecurity

AT&TVPN Appliance

3G & 4G

PPP

Ethernet

DSL

FR

ATM

AT&TPOP AT&T MPLS VPNPrivate

InternetPublic

Backbone

Remote Access (ANIRA)

NB-FW

MobilityPrivate & Public

AT&T Client

Internet

Cloud ServicesU

C Business VoIP Telepresenc

e

• Remote access & Managed Tunneling• Managed Firewalls• Intrusion Detection

AT&T Connect

• Network Based Firewall• ANIRA remote access• Mobility• AT&T Connect• Business VoIP• NetBond to Cloud

ServicesCCS

Performance: Class of Service End-to-EndPrioritizing your traffic

None of this matters when there is no congestion

MPLS Core

Traffic Flow

Class 1VOIP

Ingress policing:• RT excess dropped• Set EXP for core

PER Queuing at “egress”

Backbone COSLAN queuing LAN queuing

PER PER CERCER

CER Marks & Queues: • DSCP• CBWFQ• Police RT

Class 2vVideo

Class 2SAP, OracleApplication

sClass 3

Web Traffic

Class 4Email, FTP

Class 5Scavenger


AT&T NetBond


Enabling the virtual private cloudAT&T allows enterprises to connect to the ‘cloud’ as an extension of their VPN, leveraging their existing IT assets and operations

StorageCompute

UsersInternal IT

MPLS VPN access – Today: fixed connectionsFuture: on demand, self service, consumptionbased connections

Private Cloud

MPLS VPN

MPLS VPN

Base or persistent loads

IT resources – on demand, self service, consumption based, dynamically scalable, logically isolated

Enterprise A

Enterprise B


Traditional Model versus NetBond

Users

Benefits Avoids exposure to Internet risks (DDOS) Greater Performance and Availability than

alternative solutions Scales Dynamically with cloud usage Elasticity creates added pricing value Provisions in hours vs. weeks Avoids complexity and added costs of

managing multiple networking solution components

IPSec or Direct Connect models add cost and complexity to build and manage, and do not provide flexibility for the cloud

CorporateData Center

IPSEC Tunnelor Private Line

Cloud Service

MPLS VPNUsers

AT&T VPN

NetBond

Fixed MobileCloud Service


http://ongadgetnow.com/wp-content/uploads/2011/01/Samsung-Tablets-PCs-3-D-TVs.jpg

AT&T NetBond Pay As You Go Model

Infrastructure Capacity

Change pricing BW anytime—applies to whole current

month

Price Model like cloud service(& similar to

High Cap Flex)

95% tile of 5 Min Averages

Highest of In & Out

Charges based on actual network

consumption

Quick turn-upwithout lengthycommitments

Customer 1

Customer n

Customer 2

AT&T VPN NetworkAVPN, EVPN, IPFR, PNT

NetBond

Customer Traffic Separated

PhysicalConnection

Cloud Service Provider

IBM/Microsoft/AT&T/Partners


AT&T NetBond Reference Architecture


AVPN/PNT/IPeFR/EVPN

AT&T NetBond

AT&T Common Backbone

Cloud Vendor Edge

AT&T IPE

AT&T NetBond Physical Infrastructure

Cloud Product

Cloud Provider infrastructure

AT&T/Provider PeeringMethod Varies

AT&T Common

Cloud Infrastructur

e

AT&T MPLS Offerings

Provider Data CenterCage

NNI

nx10G Data Path (Label Switched)

Routing Updates

VLAN per Customer

Customer Location on MPLS VPN

PER


Integration to ExpressRoute and Azure


AT&T NetBond with Azure


AVPN/PNT/IPeFR/EVPN

AT&T Cloud Services


Microsoft ExpressRout

e Routers

AT&T IPEMicrosoft

Windows Azure

Platform

nx10GVLAN per Customer


PER


/30

/30

AT&T NetBond Physical Infrastructure• Routing

• COS

BGP RoutingCustomer

Provided /29

Data Path(Label

Switched)Routing Updates

QOS/COSDSCP set by Cloud Service

All Cloud Traffic Transmittedin EXP3 Queue in CBB

COS Egress PER based on

DSCP

No QOS Egress IPE

Transmitted in appropriate Queue in CBB

COS Ingress PER based on

DSCP

MS Azure VNC Drilldown

Microsoft ExpressRoute

Routers

Microsoft Azure Cloud

10G EthernetVLAN 100VLAN 20010.50.1.2/30 10.50.1.1/30

192.168.0.1/30

BFD & BGP Peering per /30

VRF AVRF B

VRF AVRF B

AVPN

VPN AVPN B10G Ethernet

VLAN 100VLAN 200

192.168.0.6/3010.50.1.6/30 10.50.1.5/30

192.168.0.5/30

BFD & BGP Peering per /30

VRF AVRF B

VRF AVRF B

Cust A

Cust B

192.168.0.2/30

AT&T IPE

NetBond to Vendor Demarc

RT Stitching

Tunnels

192.168.0.0/29 192.168.0.0/30

& 192.168.0.4/30

• Customer provides a /29 subnet to the Synaptic Portal when enabling the VNC. The /29 is broken into 2 /30 subnets which are applied to the redundant 10G connections and used for MS to AT&T BGP peering

• Routing is dynamic between MS and AT&T• Failover of redundant links is accomplished using a Primary/Secondary design utilizing

prepends© 2014 AT&T Intellectual Property. All rights reserved. AT&T and the AT&T logo are trademarks of AT&T Intellectual Property.25

NetBond Configuration and Orchestration


VNC: Virtual Network Connection. Container that ties together a chosen AVPN VPN, Cloud VPN, and VLAN(s)

VLAN: Per VNC logical link(s) to cloud vendor from the AT&T iPE. The only customer assignable attribute is the IP address space.

AT&T Synaptic Portal Terminology

AVPN VPNCloud Services


Cloud Vendor Edge

AT&TIPE

Cloud Product

Customer Location on MPLS

VPN

Customer VPNCustomer

Cloud VPN

Redundant 10G

VLAN

VNC

PER


VNC creation in AT&T Synaptic Portal

Choose CSPChoose CSP LocationName the VNCChoose AVPN VPNChoose BW


VLAN creation in AT&T Synaptic Portal

Provide /29 SubnetName VLANProvide Service Key


ExpressRoute and Microsoft ITJon OrmondDirector MSIT NetworkingMicrosoft IT

100k+ Users on Office 365 Exchange110+

Modern Apps Delivered

300k System Center managed devices

180k+Users

40kMSIT Servers in On-Prem Data Centers

The Microsoft IT Environment

513Site locations (113 countries)

<10%LOB apps run in PaaS today

1m+Devices hit the Microsoft network

210kSharePoint Sites in the Cloud

1,300+LOB apps managed by IT

4.5m Remote connections/ month

75kMSFT employees on Yammer

80%LOB apps in Azure in 5 years

153kManaged Windows 8.1 Systems

200k+Unique devices connect to wireless/day

Enterprise First & Best Program

95%In the Cloud, WAP and Azure, in 5 years

21,470Wireless Access Points

2,302Routers

22gbSustained Internet Traffic

7,241Ethernet Switches

12,055Total Managed Network Devices

939SAN Switches

288Firewalls

131Load Balancers

>24M ft2Wireless Coverage

519Wireless Controllers

49,152Strands of Dark Fiber (Puget Sound Campus)

600Managed Circuits

9IT Datacenters

Network Infrastructure Services

Gene BakerEnterprise Architect – Office of the CTOMcKesson

McKesson and ExpressRoute

McKesson at a Glance

Company Founded:1833

Fortune 500:Ranked 14th

Revenue:$122.5 billion

America’s oldest and largest healthcare services company Headquarters:

San Francisco

Employees:43,500

Segments: Distribution Solutionsand Technology Solutions

Together with our customers and partners, we are creating a sustainable future for healthcare. Together we are charting a

course to better health.

Leadership Positions in Both Segments

TechnologySolutions

DistributionSolutions

#1 pharmaceutical distributor in U.S. and Canada

#1 generics distributor

#1 in medical-surgical distribution to alternate care sites

leader in clinical, revenue-cycle and resource-management solutions

leading RelayHealth claims-processing and connectivity business

#1 in medical-management software and services to payers

Our Azure Journey

• Evaluation of top 5 IaaS and PaaS Cloud Providers – Microsoft was chosen

• Put in place an Enterprise Agreement inclusive of a BAA with Microsoft

• Built and Evaluated Point to Site, Site to Site and ExpressRoute POC communication paths – ExpressRoute was chosen

• Evaluation of HDInsight underway, and System Center POC beginning this month

We did not just go all –in with Microsoft and AT&T, we had a long journey with many evaluation points along the way:

Reasons for ExpressRoute

• Infrastructure and Administrative burden for adding new Accounts or Business Units was very taxing, and opportunity for mistakes or non-standard deployment

• Requirements for high level SLA’s that cannot be guaranteed by Public Internet links

• Need for hybrid deployments with some components housed in McKesson Data Centers

We chose to become an early adopter of ExpressRoute for many reasons, but the key reasons can be summarized quickly:

Solution Features Performance Security Administration Workloads

ExpressRoute

Committed bandwidthCan commit to SLA for performance and up

time

Non-Public MPLS HealthCare Framework

Trusted business partnerEnterprise Class Firewall end point

Single installation with no changes for moves or

adds

•SMB & Enterprsie•Moving VHD / Images•DR/Archive •SLA driven apps

Site to Site VPN

Subject to Internet performance

No performance commitment

Encrypted tunnel over open internet

Enterprise Class Firewall end point

Requires IP block, routing updates, VPN

configRequires firewall & perimeter changes

•Development•IaaS and PaaS•SMB

Point to Site VPN

Subject to Internet performance

No performance commitment

Encrypted tunnel over open internet

Requires IP block, routing updates, VPN

configRequires firewall & perimeter changes

•Development•POC•Small non-critical

Solution Advanceme

nt

Proof of Concept Successes

• Hybrid Application (Database at McKesson – App in Azure) – easy and smooth, with no latency issues

• Application Disaster Recovery Testing – successfully failed over a hosted application to Azure, but did not test end user experience

• Federation/Domain Joining – Easily Domain Joined Azure to McKesson

• High Speed File Transfer – Very fast FTP and SMB file transfers• Public Peering - Easily leveraged Azure storage and other services

Our team went through an extensive checklist of requirements with specific success criteria, but there were some key take aways:

In Summary

• Commitment to GA ExpressRoute affords us the opportunity to build our cloud architecture from the ground up – not just an extension of our current compute capabilities

• Pricing will play a key factor in this space

We are very happy with the outcome of the evaluation, and are in the planning phase for a live roll out. Some departing thoughts:

DocumentationExpressRoute

Azure ExpressRoute overviewAzure ExpressRoute technical overviewAzure ExpressRoute FAQsAzure ExpressRoute API reference for customersAzure PowerShell cmdlet reference for customers

AT&T AT&T Netbond

Resources

http://azure.microsoft.com/en-us/documentation/services/expressroute/

http://azure.microsoft.com/en-us/documentation/services/expressroute/

http://msdn.microsoft.com/library/azure/dn606309.aspx

http://msdn.microsoft.com/en-us/library/azure/dn606292.aspx






https://www.synaptic.att.com/clouduser/html/productdetail/ATT_NetBond.htm

https://www.synaptic.att.com/clouduser/html/productdetail/ATT_NetBond.htm

DEV-B312 What’s new in Windows Azure IaaSDEV-B346 What’s new in Windows Azure NetworkingDEV-B311 Building highly available and scalable applications in Windows AzureDEV-B360 Extending your premises to Windows Azure with Virtual Networks and ExpressRouteDEV-B415 ExpressRoute: Connecting private and public clouds through Exchange ProvidersDEV-B422 ExpressRoute: Connecting private and public clouds through WAN providersDEV-B324 Security and Windows Azure IaaSDEV-B328 Running your Dev/Test in Windows AzureDEV-B375 Public Cloud Security: Surviving in a Hostile Multitenant Environment DEV-B334 Disaster Recovery and Windows Azure IaaSDEV-B338 IaaS: Hosting a Microsoft SharePoint 2013 Farm on Windows AzureDEV-B361 Oracle in Windows Azure

Related Sessions

Come Visit Us in the Microsoft Solutions Experience!Look for Datacenter and Infrastructure Management

TechExpo Level 1 Hall CD

For More InformationWindows Server 2012 R2http://technet.microsoft.com/en-US/evalcenter/dn205286

Windows Server

Microsoft Azure

Microsoft Azurehttp://azure.microsoft.com/en-us/

System Center

System Center 2012 R2http://technet.microsoft.com/en-US/evalcenter/dn205295

Azure Pack Azure Packhttp://www.microsoft.com/en-us/server-cloud/products/windows-azure-pack

http://technet.microsoft.com/en-US/evalcenter/dn205286


http://azure.microsoft.com/en-us/



http://www.microsoft.com/en-us/server-cloud/products/windows-azure-pack

http://www.microsoft.com/en-us/server-cloud/products/windows-azure-pack

ResourcesLearning

Microsoft Certification & Training Resourceswww.microsoft.com/learning

msdnResources for Developers

http://microsoft.com/msdn

TechNetResources for IT Professionals

http://microsoft.com/technet

Sessions on Demandhttp://channel9.msdn.com/Events/TechEd

http://www.microsoft.com/learning

http://microsoft.com/msdn

http://microsoft.com/technet

http://channel9.msdn.com/Events/TechEd

http://channel9.msdn.com/Events/TechEd

Complete an evaluation and enter to win!

Evaluate this session

Scan this QR code to evaluate this session.

© 2014 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

expressroute: connecting private and public clouds through

Documents