cellsdn: taking control of cellular core networks - laurent vanbever

CellSDN: Taking control of cellular core networks

IBM Watson Research Center

Tue 9 April 2013

Laurent Vanbever

Princeton University

Joint work with Xin Jin, Li Erran Li, and Jennifer Rexford

Cellular core networks are not flexible

access core

Packet Data Network Gateway

Serving Gateway

Internet

Serving Gateway

Base Station (BS)

User Equipment (UE)


GPRS Tunnel (GTP)



Application optimization

Stateful firewall, DPI, lawfull intercept

Paging

Monitoring and Billing

Most functionalities are implemented at the PGW:

z

z

z

z

Application optimization

Stateful firewall, DPI, lawfull intercept

Paging

Monitoring and Billing

Most functionalities are implemented at the PGW:

z

z

z

z


Hard to mix-and-match

Slow innovation

z

z



Serving Gateway

Serving Gateway

Millions of $ per box

Cellular core networks are not flexible and costly!

CellSDN enables flexible and cost-effective networks



Key idea: separate Software from Hardware


Software

Hardware



Software

Virtualized


Hardware


easy deployment

up-/down-scale Software

Virtualized


Hardware

Hardware

Commodity

easy deployment

up-/down-scale


Software

Virtualized



easy deployment

up-/down-scale

cheap Hardware

Commodity

Software

Virtualized


To enable CellSDN flexibility, the challenge is scalability

Cellular policies are fine-grained, based on

customer-attributes z smartphone model, OS, billing plan, …

applications z video, web, voice, …

a combination of both z video from iPhone 5 gold subscribers, …


can lead to million of different flows

customer-attributes z smartphone model, OS, billing plan, …

applications z video, web, voice, …

a combination of both z video from iPhone 5 gold subscribers, …

Cellular policies are fine-grained, based on


Data-plane

Control-plane

Forwarding table size

Overhead

Slow reactivity

Challenges


Data-plane

Control-plane


Overhead

Slow reactivity

Hierarchical tagging

Challenges Solutions


Data-plane

Control-plane


Overhead

Slow reactivity

Hierarchical tagging

Hierarchical controller

Challenges Solutions


Scaling the data-plane

multi-dimensional tagging

Scaling the control-plane

tasks delegation

Architecture

software-defined network

1





tasks delegation

Architecture


S7

S6

S9

S8

S4

S5

Serving Gateway


Serving Gateway

CellSDN relies on commodity, SDN-enabled switches

S7

S6

S9

S8

S4

S5

Serving Gateway


Serving Gateway

CellSDN relies on commodity, SDN-enabled switches

OpenFlow switches

S7

S6

S9

S8

S4

S5

Serving Gateway


Serving Gateway

CellSDN also relies on commodity SDN switches for specialized appliances

S7

S6

S9

S8

S4

S5

S4

S5 S3 S2

S1

S0

CellSDN also relies on commodity SDN switches for specialized appliances

S7

S6

S9

S8

S4

S5

CellSDN distributes network processing

Firewall

Monitoring

DPI

S4

S5 S3 S2

S1

S0

S7

S6

S9

S8

S4

S5 S3 S2

S1

S0

DPI

DPI

Firewall

Monitoring

DPI

S7

S6

S9

S8

S4

S5 S3 S2

S1

S0

Monitoring DPI

DPI Monitoring

Firewall

Monitoring

S7

S6

S9

S8

S4

S5 S3 S2

S1

S0

Firewall Monitoring DPI

DPI Monitoring Firewall

Firewall

S7

S6

S9

S8

S4

S5 S3 S2

S1

S0

Firewall Monitoring DPI

DPI Monitoring Firewall

CellSDN route traffic based on high-level service policies

A service policy is composed of a

predicate A boolean expression, e.g. on subscriber a:ributes, applica;ons a:ributes, cell proper;es …

An integer. To disambiguate overlapping policies priority

A list of middleboxes, access-‐control specifica;ons (i.e., allow/deny) …

ac;on

provider is B

provider is not A

traffic is video and plan is Silver

*

1.

2.

3.

4.

Firewall

Drop

Firewall Transcoder

Firewall

priority predicate ac;on

Example of service policy

S7

S6

S9

S8

S4

S5 S3 S2

S1

S0

Transcoder Firewall DPI

DPI Transcoder Firewall

video & provider A

Internet

S7

S6

S9

S8

S4

S5 S3 S2

S1

S0



1

video & provider A

Internet

S7

S6

S9

S8

S4

S5 S3 S2

S1

S0



2

Internet

S7

S6

S9

S8

S4

S5 S3 S2

S1

S0



3

Internet

S7

S6

S9

S8

S4

S5 S3 S2

S1

S0

Controller



CellSDN uses a centralized controller to populate forwarding tables

Internet





tasks delegation

Architecture


2

CellSDN relies on tagging to implement forwarding policies

SW

SW

SW

SW

SW

SW

SW

SW

SW

SW

MB

MB

CellSDN installs stable policy path in the core

SW

SW

SW

SW

SW

SW

SW

SW

SW

SW

core

MB

MB

tag1

tag2

CellSDN classifies and tags traffic at the user-facing edge

SW

SW

SW

SW

SW

user-facing edge

SW

SW

SW

SW

SW

core

MB

MB

tag1

tag2

tag1

CellSDN caches tags at the Internet-facing edge for the return traffic

SW

SW

SW

SW

SW

user-facing edge

SW

SW

SW

SW

SW

core

MB

MB

tag1

tag2

internet-facing edge

IP header Tag

Cache

If each path gets one tag, millions of tag are needed

1000 1000 1 million *

# base station # policy paths

base station

=

# paths

If each path gets one tag, millions of tag are needed

1000 1000 1 million *

# base station # policy paths

base station

=

# paths

At best, switches support a few tens of thousands entries

[Stephen, Conext12]

CellSDN relies on multi-dimensional tag

Location

Policy

User Equipment Identifier

CellSDN tags are composed of three parts:

z

z

z

switches can selectively match on any part

Location

Policy



z

z

z

S5

S4

S3 S2 S1 S0

DPI Firewall Transcoder

Video Traffic Gold Membership + = Firewall || Transcoder

S5

S4

S3 S2 S1 S0



tag1

S5

S4

S3 S2 S1 S0

DPI Firewall

Match Action

tag1 Forward to Transcoder

Transcoder

Match Action

tag1 Forward to Firewall

Firewall || Transcoder

tag1

Video Traffic Gold Membership + =

S5

S4

S3 S2 S1 S0



tag1 tag1

video

video

Match Action


Match Action


S5

S4

S3 S2 S1 S0

DPI Firewall

Match Action


Transcoder

Match Action


1 vid.

Firewall || Transcoder

tag1

Video Traffic Gold Membership + =

S5

S4

S3 S2 S1 S0


Web Traffic Feature Phone + = Firewall || DPI

S5

S4

S3 S2 S1 S0


tag2

Web Traffic Feature Phone + = Firewall DPI

S5

S4

S3 S2 S1 S0

DPI Firewall

Match Action

tag2 Forward to DPI

Transcoder

Match Action


tag2

Web Traffic Feature Phone + = Firewall DPI

2 @

@

Location

Policy



z

z

z

S7

S6

S9

S8

S4

S5

S4

S5

In CellSDN, each base station is associated with an IP prefix

Access switches

S3 S2

S1

S0

S7

S6

S9

S8

In CellSDN, each base station is associated with an IP prefix

Access switches

10.0.0.0/16

10.0.1.0/16

10.0.2.0/16

10.0.3.0/16

S7

S6

S9

S8

User Equipment receive a unique IP address

Access switches

10.0.0.0/16

10.0.1.0/16

10.0.2.0/16

10.0.3.0/16

172.16.0.1

S7

S6

S9

S8

Access switches rewrite the UE IP into a location-dependent address

Access switches

10.0.0.0/16

172.16.0.1

10.0.0.10

UE ID

LocIP

S5

S4

S3 S2 S1 S0


CellSDN can route based on Base Station prefixes

S5

S4

S3 S2 S1 S0

DPI Firewall Transcoder 10.0.0.0/16

10.1.0.0/16


S5

S4

S3 S2 S1 S0

DPI Firewall

Match Action

10.0.0.0/16 Forward to S4


Transcoder 10.0.0.0/16

10.1.0.0/16


S5

S4

S3 S2 S1 S0

DPI Firewall

Match Action



Transcoder

Match Action


10.0.0.0/16

10.1.0.0/16

CellSDN automatically aggregates adjacent prefixes

S7

S6

S9

S8

S4

S5 S3

S1

Transcoder #1

10.0.0.0/16

10.0.1.0/16

10.0.2.0/16

10.0.3.0/16

CellSDN can selectively match on tag and BS ID for load-balancing

Match Action

tag1 & 10.0.0.0/15

Forward to Transcoder #1


Transcoder #2

Location

Policy



z

z

z

CellSDN UE tag enables mobility

S7

S6 S4 S2

S1

10.0.0.0/16

10.1.0.0/16

S5 S3

Transcoder

Transcoder

old flow

10.0.0.7 Old LocIP

S7

S6 S4 S2

S1

10.0.0.0/16

10.1.0.0/16

S5 S3

Transcoder

Transcoder

old flow

10.0.0.7 Old LocIP

When an user moves, she receives a new LocIP

S7

S6 S4 S2

S1

10.0.0.0/16

10.1.0.0/16

S5 S3

Transcoder

Transcoder

old flow

10.1.0.7 New LocIP

10.0.0.7 Old LocIP

When an user moves, she receives a new LocIP

S4 S2

S1

10.0.0.0/16

10.1.0.0/16

S3

Transcoder

Transcoder

old flow S6

encapsulate to S7

S7 S5

10.1.0.7 New LocIP

10.0.0.7 Old LocIP

Old flows reach the previous base station, ensuring policy consistency

To avoid triangle routing, CellSDN can install shortcut path after the last MB

S4 S2

S1

S3

Transcoder

Transcoder

old flow S6

S7 S5

10.1.0.7 New LocIP

10.0.0.7 Old LocIP

Match Action

10.0.0.7 Forward to S5



New flows automatically flow along new policy paths

S7

S6 S4 S2

S1

10.0.0.0/16

10.1.0.0/16

S5 S3

Transcoder

Transcoder

new flow

10.1.0.7 New LocIP

10.0.0.7 Old LocIP

Location

Policy



z

z

z

Location

Policy


z

z

z

How does CellSDN compute these tags?

Given a service policy-path,

CellSDN minimizes the forwarding tables size by reusing tags

Compute the actual path P used in the network 1.

For each candidate tag t used on the path P,

Compute the # of new rules needed if t is used

2.

Select the candidate tag minimizing the # of new rules, Create new tag if none available 3.

Install the forwarding entries in the network 4.

S5

S4

S3 S2 S1 S0


10.0.0.0/16

10.1.0.0/16

S7

S6 10.2.0.0/16

10.3.0.0/16

S5

S4

S3 S2 S1 S0


10.0.0.0/16

10.1.0.0/16

S7

S6 10.2.0.0/16

10.3.0.0/16

S4 DPI Firewall policy path

S5

S4

S3 S2 S1 S0


10.0.0.0/16

10.1.0.0/16

Match Action

tag1 Forward to DPI

Match Action

tag1 Forward to FW

S7

S6 10.2.0.0/16

10.3.0.0/16


S5

S4

S3 S2 S1 S0


10.0.0.0/16

10.1.0.0/16

Match Action

tag1 Forward to DPI

Match Action

tag1 Forward to FW

S7

S6 10.2.0.0/16

10.3.0.0/16

S6 Firewall policy path

S5

S4

S3 S2 S1 S0


10.0.0.0/16

10.1.0.0/16

Match Action

tag1 Forward to DPI

tag1, src: 10.2.0.0/16

Forward to S1

Match Action

tag1 Forward to FW

S7

S6 10.2.0.0/16

10.3.0.0/16


S5

S4

S3 S2 S1 S0


10.0.0.0/16

10.1.0.0/16

Match Action

tag1 Forward to DPI

tag1, src: 10.2.0.0/16

Forward to S1

tag1,src: 10.3.0.0/16

Forward to S1

Match Action

tag1 Forward to FW

S7

S6 10.2.0.0/16

10.3.0.0/16


S5

S4

S3 S2 S1 S0


10.0.0.0/16

10.1.0.0/16

Match Action

tag1 Forward to DPI

tag1, src: 10.2.0.0/15

Forward to S1

Match Action

tag1 Forward to FW

S7

S6 10.2.0.0/16

10.3.0.0/16


•  FatTree topology

•  128 switches && 1280 base stations

•  8 types of middleboxes

•  5 middleboxes long service policy

CellSDN dataplane can support a large number of service policies

Simulation

z

z

z

Evaluation table size in function of the # of policies

z

1000 2000 3000 4000 5000 6000 7000 80000

2500

5000

7500

10000

12500

15000

Number of service policy clauses

Sw

itch table

siz

e (

num

ber

of ru

les)

MaximumMedian

Switches table size increase linearly wrt to the # of policy paths

1000 2000 3000 4000 5000 6000 7000 80000

2500

5000

7500

10000

12500

15000


Sw

itch table

siz

e (

num

ber

of ru

les)

MaximumMedian


Only 5k entries are required to support 4k policy paths

1000 2000 3000 4000 5000 6000 7000 80000

2500

5000

7500

10000

12500

15000


Sw

itch table

siz

e (

num

ber

of ru

les)

MaximumMedian


In the worst case, 8k policy paths require 13.6k entries

1000 2000 3000 4000 5000 6000 7000 80000

2500

5000

7500

10000

12500

15000


Sw

itch table

siz

e (

num

ber

of ru

les)

MaximumMedian


In the worst case, 8k policy paths require 13.6k entries

Today, operators require a few hundreds policies

1000 2000 3000 4000 5000 6000 7000 80000

2500

5000

7500

10000

12500

15000


Sw

itch table

siz

e (

num

ber

of ru

les)

MaximumMedian


CellSDN works on commodity switches





tasks delegation

Architecture


3

Traffic Management Layer

Mechanism Layer

Service Policy

Traffic Management Policy

“Transcoder || Firewall”

“Balance load across all MB instances”

OpenFlow messages

CellSDN controller separates traffic management from rules installation


Mechanism Layer

Path Optimizer

Service Policy




OpenFlow messages


Mechanism Layer

Service Policy


Packet classification

Path implementation



OpenFlow messages

S7

S6

S9

S8

S4

S5 S3 S2

S1

S0

Controller

To scale, CellSDN uses a hierarchical controller

S7

S6

S9

S8

S4

S5 S3 S2

S1

S0

Controller

Most of the tasks are delegated to local-agents

Local-Agent

Local-Agent

Local-Agent

Local-Agent

Local agents handle locally most frequent events

-  cache a list of packet classifiers

-  contact central controller upon cache miss

-  tag flows

z

z z

Local-agents act as cache, reducing the load on the main controller

-  UE arrival, handoff

-  topology changes

-  dynamic policies

Central controller globally handle less frequent events

-  cache a list of packet classifiers

-  contact central controller upon cache miss

-  tag flows

z

z z

z z

z

The central controller deals with less frequent, but more complex events

Local agents handle locally most frequent events

•  1 week of traces from a large LTE network

•  1500 base stations

•  1 million users

CellSDN control-plane can handle the load of large cellular networks

Dataset

z

z

z

Evaluation •  # of events per second

•  # of active users per second z

z

0 50 100 150 200 250 3000

0.2

0.4

0.6

0.8

1

Number of events per second in the whole network

CD

F

UE ArrivalHandoff

The number of events going to the main controller is small

0 50 100 150 200 250 3000

0.2

0.4

0.6

0.8

1


CD

F

UE ArrivalHandoff

214/s 99th percentile 280/s Total << 1000


0 50 100 150 200 250 3000

0.2

0.4

0.6

0.8

1


CD

F

UE ArrivalHandoff

214/s 99th percentile 280/s

Today’s controllers can process tens of thousands events/s

Total << 1000


100

101

102

1030

0.2

0.4

0.6

0.8

1

Number of active UEs per base station

CD

F

514/s 99th percentile ~ 5k flows

The number of flows handled by the local-agent is small

100

101

102

1030

0.2

0.4

0.6

0.8

1

Number of active UEs per base station

CD

F

514/s 99th percentile ~ 5k flows

Today’s controllers can process tens of thousands events/s

The number of flows handled by the local-agent is small





tasks delegation

Architecture


CellSDN supports flexible fine-grained policies

CellSDN enables flexible and cost-effective cellular networks

CellSDN achieves scalability with

Multi-dimensional aggregation z

Asymmetric edge design z

Hierarchical controller z


IBM Watson Research Center

Tue 9 April 2013

Laurent Vanbever

www.vanbever.eu

cellsdn: taking control of cellular core networks - laurent vanbever

Documents