analytics driven sdn and commodity switches

Analytics driven SDN and commodity switches

Peter Phaal Founder and President, InMon Corp.Silicon Valley SDN Group, May, 2014

Copyright © 2014 InMon Corporation

Controller

Analyze

Plan

Act

Network

MeasurementProtocol Control

Protocol

Feedback control

“You can’t control what you can’t measure” Tom DeMarco


Separation of edge and core

Fabric: A Retrospective on Evolving SDN by Martin Casado, Teemu Koponen, Scott Shenker, and Amin Tootoonchian

Network Elements Controller Functions

Edge flexible software virtual switches network virtualization, tenant isolation, security, NFV… e.g. NSX, Nuage …

Fabric simple, low cost, vendor neutral, hardware switches

traffic analytics and control to increase efficiency

between practicality (support matching on standard headers)and generality (match on all headers). However, this requiresswitch hardware to support lookups over hundreds of bits;in contrast, core forwarding with MPLS need only matchover some tens of bits. Thus, with respect to the forwardinghardware alone, an OpenFlow switch is clearly far from thesimplest design achievable.

• Second, it does not provide sufficient flexibility. We expecthost requirements to continue to evolve, leading to increasinggenerality in the Host-Network interface, which in turn meansincreasing the generality in the matching allowed and theactions supported. In the current OpenFlow design paradigm,this additional generality must be present on every switch. Itis inevitable that, in OpenFlow’s attempt to find a sweet spotin the practicality vs generality tradeoff, needing functionalityto be present on every switch will bias the decision towards amore limited feature set, reducing OpenFlow’s generality.

• Third, it unnecessarily couples the host requirements to thenetwork core behavior. This point is similar to but moregeneral than the point above. If there is a change in theexternal network protocols (e.g., switching from IPv4 to IPv6)which necessitates a change in the matching behavior (becausethe matching must be done over different fields), this requiresa change in the packet matching even in the network core.

Thus, our goal is to extend the SDN model in a way that avoidsthese limitations yet still retains SDN’s great control plane flexibility.To this end, it must retain its programmatic control plane interface(so that it provides a general Operator-Network interface), whilecleanly distinguishing between the Host-Network and Packet-Switchinterfaces (as is done in MPLS). We now describe such a design.

3 Extending SDN3.1 OverviewIn this section we explore how the SDN architectural frameworkmight be extended to better meet the goals listed in the introduction.Our proposal is centered on the introduction of a new conceptualcomponent which we call the “network fabric”. While a commonterm, for our purposes we limit the definition to refer to a collectionof forwarding elements whose primary purpose is packet transport.Under this definition, a network fabric does not provide morecomplex network services such as filtering or isolation.

The network then has three kinds of components (see Figure1): hosts, which act as sources and destinations of packets; edgeswitches, which serve as both ingress and egress elements; and thecore fabric. The fabric and the edge are controlled by (logically)separate controllers, with the edge responsible for complex networkservices while the fabric only provides basic packet transport. Theedge controller handles the Operator-Network interface; the ingressedge switch, along with its controller, handle the Host-Networkinterface; and the switches in the fabric are where the Packet-Switchinterface is exercised.

The idea of designing a network around a fabric is well understoodwithin the community. In particular, there are many examples oflimiting the intelligence to the network edge and keeping the coresimple.4 Thus, our goal is not to claim that a network fabric is4This is commonly done for example in datacenters whereconnectivity is provided by a CLOS topology running an IGP andECMP. It is also reflected in WANs where interdomain policies areimplemented at the provider edge feeding packets into a simplerMPLS core providing connectivity across the operator network.

Fabric Elements

Fabric Controller

SrcHost

DstHost

Edge Controller

Ingress Edge Switch

Egress Edge Switch

Figure 1: The source host sends a packet to an edge switch, whichafter providing network services, sends it across the fabric for theegress switch to deliver it to the destination host. Neither host seesany internals of the fabric. The control planes of the edge and fabricare similarly decoupled.

a new concept but rather we believe it should be included as anarchitectural building block within SDN. We now identify the keyproperties for these fabrics.

Separation of Forwarding. In order for a fabric to remain decou-pled from the edge it should provide a minimal set of forwardingprimitives without exposing any internal forwarding mechanismsthat would be visible from the end system if the fabric werereplaced. We describe this in more detail below but we believeit is particularly important that external addresses are not used inforwarding decisions within the fabric both to simplify the fabricforwarding elements, but also to allow for independent evolution offabric and edge.

Separation of Control. While there are multiple reasons to keepthe fabric and the edge’s control planes separate, the one we wouldlike to focus on is that they are solving two different problems. Thefabric is responsible for packet transport across the network, whilethe edge is responsible for providing more semantically rich servicessuch as network security, isolation, and mobility. Separating thecontrol planes allows them each to evolve separately, focusing onthe specifics of the problem. Indeed, a good fabric should be ableto support any number of intelligent edges (even concurrently) andvice versa.

Note that fabrics offer some of the same benefits as SDN.In particular, if the fabric interfaces are clearly defined andstandardized, then fabrics offer vendor independence, and (as wedescribe in more detail later) limiting the function of the fabric toforwarding enables simpler switch implementations.

3.2 Fabric Service Model

Under our proposed model, a fabric is a system component whichroughly represents raw forwarding capacity. In theory, a fabricshould be able to support any number of edge designs includingdifferent addressing schemes and policy models. The reverse shouldalso be true; that is, a given edge design should be able to takeadvantage of any fabric regardless of how it was implementedinternally.

The design of a modern router/switch chassis is a reasonablygood analogy for an SDN architecture that includes a fabric. Ina chassis, the line cards contain most of the intelligence and theyare interconnected by a relatively dumb, but very high bandwidth,backplane. Likewise, in an SDN architecture with a fabric, the edgewill implement the network policy and manage end-host addressing,while the fabric will effectively interconnect the edge as fast andcheaply as possible.

The chassis backplane therefore provides a reasonable starting

Simple, low cost, vendor neutral → merchant silicon


Rise of merchant silicon

20132011

Por

ts

Opportunity to leverage merchant silicon traffic analytics and apply targeted controls to increase fabric efficiency


Large “Elephant” flows

http://research.microsoft.com/en-us/UM/people/srikanth/data/imc09_dcTraffic.pdf

Elephant flows are the small number of long lived large flows responsible

for majority of bytes on network

http://blog.sflow.com/2013/02/sdn-and-large-flows.html

http://research.microsoft.com/en-us/UM/people/srikanth/data/imc09_dcTraffic.pdf

http://blog.sflow.com/2013/02/sdn-and-large-flows.html


packets

decode hash sendflow cache flushsample

Flow Records

flow cache embedded on switchcustom ASIC based switch

NetFlow IPFIX …

decode hash sendflow cache flush

Flow Records

packets

send

polli/f counters

sample

multiple switches export sFlow

packets

send

polli/f counters

sample

...

external software flow cache

merchant silicon based switch (Broadcom, Intel/Fulcrum, and Marvell)

JSON/RESTNetFlow IPFIX …

• Reduce ASIC cost / complexity • Fast response (data not sitting on switch) • Centralized, network-wide visibility • Increase flexibility → software defined analytics

Move flow cache from ASIC to external software

Scale-out alternative to SNMP polling

Traffic analytics with sFlow

Centralized real-time analytics identifies large flows, paths, hot spots etc. → plan corrective actionsHow can controls be efficiently deployed?


• Simple, no change to normal forwarding behavior - BGP, OSPF, SPB, TRILL, LAG/MLAG etc. used to control L2 / L3 forwarding tables

• Efficient, merchant silicon hardware multipath forwarding efficiently handles most flows. OpenFlow used to control ACL table and selectively override forwarding of specific flows (block, mark, steer, rate-limit), maximizing effectiveness of limited general match capacity.Note: very few ACLs needed in fabric since policy has shifted to edge - mainly required to protect control plane

• Scaleable, flows handled by existing control plane, OpenFlow only used when controller wants to make an exception. Note: An OpenFlow controller could pro-actively configure L2/L3 tables to define “NORMAL” forwarding and still support hybrid control of ACL table

• Robust, if controller fails, network keeps forwarding

Traffic control with hybrid OpenFlow

Hybrid Programmable Forwarding Plane, David Ward, ONF Summit, 2011


sFlow-RT feedback controller

Large flow steering

DDoS Mitigation

REST API

Open “Southbound” APIs

Data Plane

Real-time analytics and control

Hosts

Open “Northbound” APIs

User defined policy

sFlow-RT controller

real-time analytics hybrid OpenFlow controller

Open JavaScript/ECMAScript API optimized for SDN traffic engineering applications

Large flow marking

…

Web portal OpenStack

etc.


Brocade/InMon: DDoS mitigation

http://www.opennetsummit.org/pdf/2014/sdn-idol/Brocade-SDN-Idol-Proposal.pdf

“Real-Time SDN Analytics for DDoS Mitigation” winner of ONS SDN Idol 2014

http://www.opennetsummit.org/pdf/2014/sdn-idol/Brocade-SDN-Idol-Proposal.pdf


ALU/InMon: Large flow marking

http://enterprise.alcatel-lucent.com/docs/?id=23847

http://enterprise.alcatel-lucent.com/docs/?id=23847


Extend control beyond network

Analyze

Plan

Act

Network, Storage, Compute

sFlow-RT Feedback Controller

Maximize data center efficiency through coordinated workload placement and resource allocation of network, storage, and

compute based on measured loads and communication patternse.g. reduce network congestion by instructing OpenStack to move virtual machine


• InMon.com

• blog.sFlow.com

• sFlow.org

• Host-sFlow.SourceForge.net

• Velocity 2012http://blog.sflow.com/2013/04/velocity-conference-talk.html

• Bay Area Network Virtualization Meetuphttp://blog.sflow.com/2013/06/bay-area-network-virtualization-talk.html

• Mininet testbedhttp://blog.sflow.com/2014/04/mininet-integrated-hybrid-openflow.html

Explore further

http://InMon.com

http://blog.sflow.com

http://sFlow.org

http://Host-sFlow.SourceForge.net

http://blog.sflow.com/2013/04/velocity-conference-talk.html

http://blog.sflow.com/2013/06/bay-area-network-virtualization-talk.html

http://blog.sflow.com/2014/04/mininet-integrated-hybrid-openflow.html