4th sdn interest group seminar-session 2-3(130313)

2013 OpenFlow Korea All Rights Reserved

The Great Transformation

Feb, 2013

OpenFlow Korea

(www.OPENFLOW.or.kr)

발표자 : 서영석 기술매니저 [email protected]


OFA

Data Center Site 3

HW Tables

OFC RCS

Quagga

Site Broker

Google’s Software Defined WAN

SDN Gateway Global

Broker

iBGP / ISIS

Non-TE(ISIS) Path

Tunneling App

OFA

Data Center Site 2

HW Tables

OFC RCS

Quagga

Site Broker

Tunneling App

PaxOS PaxOS

Cluster Border Router


Flow Manager

Path Allocation Algorithm

Path Selection

Topology Manager

OFA

HW Tables

TE Server

Site Level TE Path

OFC Tunneling

App

PaxOS

Quagga


Site Broker

RCS

iBGP / ISIS

EBGP EBGP

Data Center Site 1 EBGP

Demand Matrix {src, dst -> utility curve}

Abstract Path Assignment {src, dst -> paths and weights}

Interface up/down status

Per Site Path Manipulation Commands

Site level edges with RTT and Capacity

4가지 주요 기술

- Global Broker : 각 사이트의 필요 대역폭 산정

- TE Server : 필요 대역폭을 위한 경로 설정

- SDN GateWay : TE와 OFC 간 연동

- Quagga : 기존 네트워크와 연동 iBGP / ISIS

iBGP / ISIS



Google은 전세계에 13개의 Data Center를 가지고 있다.

ATLAS 2010 Traffic Report, Arbor Networks

• Data Growth – Web expands/changes : billions of new / modified

pages every month – Every few hours we crawl / refresh more than

entire Library of Congress – YouTube gains 4+ billion views every day

7%


Google’s Data Center

• Two backbones

– I-Scale : Internet facing (user traffic)

– G-Scale : Datacenter traffic (Data Center간 traffic)

• Cloud Computing Requires Massive Bandwidth

– Low latency access

• Vast majority of data migrating to cloud

• Data must be replicated at multiple sites

• Computation & Storage migration to Data Centers

– Data storage : personal files, logs, company data

– Application execution : word processing, email, calendar

– Content retrieval : photos, music, video

– Web services : search, social, e-commerce

– Large-scale data processing : MapReduce, Hadoop


Google’s Data Center

• 기존 네트워크 Architecture의 치명적 한계에 직면

– Networks don’t function exactly as we would like

• Not deterministic

• Suboptimal behavior and failure handling

• Hard to configure and operate at scale

• Expensive, manual, error prone systems

– Networking is hard

• Performance

• Scale

• Failure Handling

• 혁신적인 네트워크 Architecture의 필요성 증가

– WAN을 하나의 system으로 구현하는 Architecture 필요

• 높은 네트워크 활용

• Routing 최적화 / Traffic Engineering

• non-equal cost /shortest path

• Global knowledge, simplicity, Ops friendly technology

• Multi-chassis architecture


WAN Fabrics Architecture

Gateway

Bandwidth Broker

Flow Manager


Path Selection

Topology Manager

TE Server


Abstract Path Assignment

Demand Matrix



Data Center Site 1

…

Data Center Site 2 Data Center Site 3 Data Center Site N


WAN Fabrics Network 구성 절차(1/5)

• 기존의 Data Center 연결 구조



• Data Center간 WAN Fabrics Network 구현을 위해 SDN 도입



• 단계적 SDN 구성



• SDN으로 WAN Fabrics 전체 연결



• Traffic Engineering을 통한 WAN Fabrics 구현


WAN Fabrics Network

• 평균 95% 이상의 회선 사용률


WAN Fabrics Network Hardware

• 수 백 개의 non-blocking 10GE port로 구성

• OpenFlow 지원

• BGP, ISIS를 위한 Open source routing stack을

지원

• 선별적 기능 구현

• Site별 Multiple chassis로 구성

– Fault tolerance

– Scale to multiple Tbps


OFA

Data Center Site 3

HW Tables

OFC RCS

Quagga

Site Broker


SDN Gateway Global

Broker

iBGP / ISIS

Non-TE(ISIS) Path

Tunneling App

OFA

Data Center Site 2

HW Tables

OFC RCS

Quagga

Site Broker

Tunneling App

PaxOS PaxOS



Flow Manager


Path Selection

Topology Manager

OFA

HW Tables

TE Server

Site Level TE Path

OFC Tunneling

App

PaxOS

Quagga


Site Broker

RCS

iBGP / ISIS

EBGP EBGP

Data Center Site 1 EBGP

Demand Matrix {src, dst -> utility curve}

Abstract Path Assignment {src, dst -> paths and weights}




4가지 주요 기술

- Global Broker : 각 사이트의 필요 대역폭 산정

- TE Server : 필요 대역폭을 위한 경로 설정

- SDN Gateway : TE와 OFC 간 연동

- Quagga : 기존 네트워크와 연동 iBGP / ISIS

iBGP / ISIS


Trust but Verify : Consistency Checks


High Level Architecture



B/W Broker

TE Server



• TE Server는 SDN Gateway에 Traffic Engineering Service를 적용

• SDN Gateway는 모든 Site에 TE Service를 적용하기 위해 각 Site의 Tunneling App과 연동

• Tunneling App을 통해 각 Site의 OFC는 TE Service가 적용된 flow table을 생성하여 OFA에게 전달

• 각 Site의 flow는 TE path를 통해 전달

• 일부 flow는 non-TE path를 통해 전달



• OFC는 두 가지 flow 정보를 이용하여

flow table을 생성

– Quagga를 통해 받은 Legacy Routing을 위한 Flow 정보

– Tunneling App을 통해 받은 TE Service가 적용된 Flow 정보

• OFC는 DataCenter Switch의 OFA에게

flow table을 전달

• OFA는 flow table을 이용하여 HW Table을 생성


Traffic Engineering Example

• West -> East demand

– 100Gb/s low latency

– 200Gb/s bulk transfer


TE Path Allocation

Flow group Allocation Paths and splits

CHS to MRN BE1 320 Gbps out of 320 Gbps CHS-MRN : 75% CHS-ATL-MRN : 25%

• Path Selection – Find Static k shortest passible paths between src and dst

• Path Ordering and Grouping

– Group similar latency paths into path preference groups

– Sort paths preference group by latency

• Compute Flow Group Allocation :

– For each flow group, input:

• Sorted paths preference groups

• Demand with priority (utility function) from broker

– Exhaustive waterfill algorithm

• Fill preferred paths first


Convergence under Failures

• Without TE : Failure detection and convergence is slower :

– Delay ‘inside’ TE << timers for detecting and communicating failures (in ISIS)

– Fast failover may be milliseconds, but not guaranteed to be either accurate or “good”


Centralized TE의 이점

• Better efficiency with global visibility

• Converges faster to target optimum on failure

• Higher Efficiency

– allows for explicit definition of cost functions

– allows for in-house development of optimization algorithms

• Deterministic behavior

– Simplifies planning vs. over-provisioning for worst case variability

– Can directly mirror production event streams for testing

• Supports innovation and more robust SW development

• Controller uses modern server hardware

– Significantly higher performance


Conclusions

• Dramatic growth in WAN bandwidth requirements

– Every 10x, something breaks

– Existing software/hardware architectures make it impractical to deliver cheap bandwidth globally

• Software Defined Networking enables

– Separation of hardware from software

– Efficient logically centralized control/management

– Innovation and flexibility

• Deployment experience with Google’s global SDN production WAN

– It’s real and it works

– This is just the beginning…


OpenFlow Korea

(www.OPENFLOW.or.kr)

4th sdn interest group seminar-session 2-3(130313)

Technology

te ofc

te server

global broker

weights interface

2013 openflow korea

sdn gateway

scale

paths