scalable and crash-tolerant load balancing based on switch migration for multiple openflow...

　Scalable and Crash-Tolerant Load Balancing based on Switch Migration for Multiple OpenFlow Controllers

Chu LIANG ＊ Ryota KAWASHIMA ＊ Hiroshi MATSUO ＊

＊ Nagoya Institute of Technology, Japan1/22

The spread of Software-Defined NetworkingEasier management and faster innovationA centralized controller and programmable network devices

The complication of controller is increasingLoad balancing, QoS controlling, Security…

The size of networks continues to increaseThe growing number of OpenFlow-enabled devices

A centralized controller has a potential bottleneck

Research Background

Distributed control plane has been proposed

2/22

Distributed OpenFlow Controllers

OpenFlow controller cluster

OpenFlow switches

Group A Group B

OFC 1 OFC 2

OpenFlow : one of the mostly representative protocol for SDN Packet-in : not match any forwarding rule forward to controller

Packet-in

physically distributed controllers achieve better scalability

OpenFlow Controller

3/22

Problems in Distributed Controllers Load imbalance results in suboptimal performance

Topology change Variety user traffic

• Elastic resources, VMs migration, Variety utilizations

OpenFlow controller cluster

OpenFlow switches

Group A Group B

OFC 1 OFC 2

Static mapping configuration

High loaded

Low loaded

Dynamic load balance among the controllers is required4/22

Switch OFC 1 OFC 2

S1

S2

S3

Problems in OpenFlow Controllers Multiple Controllers in OpenFlow

Roles : Master/ Slave/ Equal

The coordination of “role changing” is not provided

？

Master-R

equest

Role-R

eply

OFC2

S2S1 S3

OFC1

Master Slave

Master Slave

Each controller only has one role

Master Slave

Master SlaveR

ole-

Rep

lyM

aste

r-Req

uest

Master-RequestRole-Reply

5/22

Scalable OpenFlow Controller Redundancy Tackling Local and Global Recoveries 　

• Keisuke Kuroki, Nobutaka Matsumoto, Michiaki Hayashi, The Fifth International Conference on Advances in Future Internet. 2013.

Towards an Elastic Distributed SDN Controller 　• Advait Dixit, Fang Hao, Sarit Mukherjee, T.V. Lakshman, Ramana Kompella, ACM SIGCOMM

HotSDN, 2013

Proposed a switch migration protocol according to controller load

Proposed a crash-tolerant method based on with multiple controllers of OpenFlow1.3

Related Work

Do not support load balancing among the controllers

Be complex and do not support the crash-tolerant for master controller

Role-Management server can be single point of failure

6/22

Proposed MethodDynamically shift the load across the multiple controllers

different controllers can be set master for individual switchswitch migration

Support the crash-tolerant for controllers distributed architecture automatic failover in the event of a failureJGroups based communication

Simplification of the switch management by groupingeach controller only manage switches in the same groupswitch migration is performed in group

7/22

Proposed Architecture

Group A

OpenFlow switch

Group B

Group C

Global controller cluster

Global DB

Local DB

Local controller cluster

8/22

t



Global DB

Based on the global JGroups channelShare global data

tenant information, user data etc.

Provide global view of network to upper controllercan be considered as a logically centralized controller plane

9/22

Local controller cluster

Local controller clusterreduce network delayreduce communicate traffic

Synchronize network statusswitch-controller mapping, link, port

Perform controller load schedulingCoordinate switch migration

set master/slave role for switches

Dynamically shift the load across the multiple controllers

Local DB

10/22

C : Switch Migration Module

A : Load Monitoring Module A : Load Monitoring Module

B : Load Scheduling Module B : Load Scheduling Module

Implementation

OpenDaylight Core

OpenFlow Driver

LinkDiscovery

SwitchManger

HostManger

Eve

nt N

otif

icat

ion

(JG

roup

s)OpenDaylight　 APIs

Dis

trib

uted

Key

-Val

ue S

tore

(In

fini

span

)

• Collect and calculate controllers load

Application Application

Controller structure (OpenDaylight based)

• Selected master controller

• Perform switch migration

11/22

A. Load Calculation

coordinator

OFC1 OFC2

OFC4 OFC3

Coordinator : collecting and computing load informationController Load : switch metric and server metric

the number of active switches, packets requests rate (switch metric )usage of cpu, memory, network bandwidth (server metric)

Local Controller Cluster

12/22

B. Load Scheduling

When and Which controller should be elected as masterThe lightest-load controller

OFC1 OFC2 OFC3 Perform Switch Migration

Controller failover

Add new switches

Which switches should be selected to migrate Dynamic round-trip time feedback based switch selection

？ 13/22

C. Switch Migration

Initial heaviest-loadController A

Initial lightest-load Controller B

OpenFlow Switch T

Switch T migration Request

Role_request to Master

Role_reply for Master

Switch T migration Reply

Slave for TMaster for T

Master for TSlave for T

14/22

C. Switch Migration



OpenFlow Switch T






Role_reply to Master

Slave for T

Master for T

Master for T

Slave for T

Master for T

Failover time

15/22

Preliminary evaluation (1/2)

The switch migration processThe migration process takes about 2ms



OpenFlow Switch T





Slave for TMaster for T


2ms

16/22

Preliminary evaluation (2/2)

The controller failover process The failover process takes about an average of 20ms mostly affected by the failure detection provided by JGroups.



OpenFlow Switch T


Role_reply to Master


Failover time

20ms

17/18

Failure detection

17/22

Evaluation environment

(Traffic Generator)

(OFC A)

Host Host Host HostHost Host Host Host

(Mininet Network)

SW 1

(OFC B)

SW 2 SW 3 SW 4 SW 5 SW 6 SW 8SW 7

VM1

VM2

Iperf client

Iperf server

Host 1 Host 2

Host 3

Host 4 18/22

Evaluation

Controller Node Traffic Generator Evaluation Node

OS Ubuntu-server 12.04 Centos 6.5 64bit

CPU Core i5(4 core) Core i7(1 core) Core i5(4 core)

Memory 16GB 8GB

Network 100Mbps Ethernet

OpenFlow Switch

- Open vSwtich-1.10.0

Machine specifications

Switch switch 1 ∼ 2 switch 3 ∼ 4 switch 5 ∼ 6 switch 7 ∼ 8

Workload A 1000 pps 2000 pps 2000 pps 4000 pps

Workload B 1000 pps 2000 pps 4000 pps 6000 pps

Workload C 1000 pps 2000 pps 6000 pps 8000 pps

Three kind of workloads

Load

19/22

0 20 40 60 80 100 120 140 160 18010

15

20

25

30

35

40

static

proposal

0 5 10 15 20 25 30 35 40

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Workload A Workload B Workload C

Results : Throughput static : existing ( static switch-controller mapping) proposal : dynamically switch migration

Thr

ough

put (

Mbi

t/se

c)

Run Time (in sec)

OFC A:5 switchesOFC B:3 switches



0 5 10 15 20 25 30 35 40

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Loa

d D

iffe

renc

e


20/22

0 5 10 15 20 25 30 350

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

workload C (static)workload C (proposal)workload B (proposal)workload B (static)workload A (static)workload A (proposal)

Results : Response Time

cum

ulat

ive

dist

ribu

tion

func

tion

(CD

F)

Response Time: VM——OFC B (Ping)

workload A (static)workload A (proposal)workload B (static)workload B (proposal)workload C (static)workload C (proposal)

Packets loss

Response Time (in msec)21/22

Conclusion & Future Work

ConclusionProposed a scalable and crash-tolerant load balancing based

on switch migration for multiple OpenFlow controllersEnable the controllers coordinate actions to dynamically

shift the load across the multiple controllers Improve the throughput and response time of control plane

Future WorkOptimize of the load scheduling modulesImplement a topology aware switch migration algorithms to

improve the scalability in the real large scale networkEvaluate the performance in vary applications and topologies

with more practical traffics22/22

scalable and crash-tolerant load balancing based on switch migration for multiple openflow...

Documents

controller load

centralized controller

complication of controller

controller packet

openflow roles

group switch migration

switch management

crashtolerant load balancing