document: cnet-ict-619543-netide/d 2.7 date: … single module without composition ... nbi...

Document: CNET-ICT-619543-NetIDE/D 2.7Date: Security: PublicStatus: Final Version: 1.0

Document Properties

Document Number: D 2.7

Document Title: NetIDE Manual

Document Responsible: CREATE-NET (CN)

Document Editor: Roberto Doriguzzi Corin (CN)

Authors:

Roberto Doriguzzi Corin (CN), Pedro A. Aranda (TID),Elisa Rojas (TELCA), Arne Schwabe (UPB),Christian Stritzke (IEM), Kévin Phemius (THALES),Alec Leckey (INTEL), Carmen Guerrero (IMDEA),Sergio Tamurejo (IMDEA), Antonio Marsico (CN)

Target Dissemination Level: PU

Status of the Document: Final

Version: 1.0

Production Properties:

Reviewers: Holger Karl (UPB), Pedro A. Aranda (TID)

Document History:

Version Date Issued by

1.0 21-October-2016 Roberto DoriguzziCorin

Disclaimer:This document has been produced in the context of the NetIDE Project. The research leading to these results hasreceived funding from the European Community's Seventh Framework Programme (FP7/2010�2013) under grantagreement n◦ 619543.All information in this document is provided �as is� and no guarantee or warranty is given that the informationis �t for any particular purpose. The user thereof uses the information at its sole risk and liability.For the avoidance of all doubts, the European Commission has no liability in respect of this document, which ismerely representing the authors view.

NetIDEPublic

Ref. Ares(2017)4238357 - 30/08/2017

Abstract:The NetIDE Framework delivers an integrated environment for SDN development that uni�esdi�erent controller technologies and allows developers to write, test and deploy modular networkapplications that are independent from the underlying SDN technology.In this deliverable we illustrate the most relevant architectural and implementation conceptsdeveloped during the project, as well as we provide manuals, explaining how end users (softwaredevelopers, network administrators, network operators) can install, con�gure and run the NetIDEFramework.

Keywords:

Software De�ned Networking (SDN), Manual, NetIDE Framework, Integrated Development Environment,SDN Application Re-Usability, SDN Tools

NetIDEPublic

Contents

List of Figures v

List of Acronyms viii

Executive Summary 1

1 Introduction 2

2 The NetIDE Framework 42.1 The Network Engine . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

2.1.1 The Core . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62.1.2 Message format . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112.1.3 NetIDE Intermediate Protocol . . . . . . . . . . . . . . . . . . . . . . . . . . 122.1.4 Shim . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132.1.5 Backend . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

2.2 The NetIDE Integrated Development Environment . . . . . . . . . . . . . . . . . . . 19

3 Network Engine manuals 223.1 General requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 233.2 The Core . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24

3.2.1 Installing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 243.2.2 Running . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

3.3 ONOS Shim . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 263.3.1 Installing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 263.3.2 Running . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26

3.4 OpenDaylight Shim . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 273.4.1 Installing and running . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27

3.5 Ryu Shim . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 283.5.1 Installing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 283.5.2 Running . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28

3.6 Testing a Shim . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 293.7 Ryu Backend . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30

3.8 Floodlight Backend . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 313.8.1 Installing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 313.8.2 Running . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31

3.9 Testing Backends . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 323.9.1 Testing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32

4 IDE manual 334.1 Installation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33

4.1.1 Prerequisites . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 334.1.2 Installation via the Eclipse Marketplace . . . . . . . . . . . . . . . . . . . . . 33

NetIDEPublic

4.1.3 Installation using the update site . . . . . . . . . . . . . . . . . . . . . . . . . 334.1.4 Installation by building from source . . . . . . . . . . . . . . . . . . . . . . . 33

4.2 Usage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 344.2.1 Application format management . . . . . . . . . . . . . . . . . . . . . . . . . 344.2.2 Specifying and using parameter schemas . . . . . . . . . . . . . . . . . . . . . 344.2.3 Con�guring Network Topologies . . . . . . . . . . . . . . . . . . . . . . . . . . 364.2.4 Testing network applications . . . . . . . . . . . . . . . . . . . . . . . . . . . 374.2.5 Monitoring the network application . . . . . . . . . . . . . . . . . . . . . . . . 384.2.6 Global Con�guration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 394.2.7 NetIDE Application Loader . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40

5 Tools manuals 425.1 Logger . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42

5.2 Debugger . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 435.2.1 Installing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 445.2.2 Running . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45

5.3 Pro�ler . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 465.3.1 Installing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 475.3.2 Running . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47

5.4 Tra�c Emulator . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 475.4.1 Installation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 485.4.2 Running . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48

5.5 Memory Management System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 515.5.1 Installation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 515.5.2 Running . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51

6 NetIDE framework at work 536.1 Demo scenario . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54

7 Advanced architectural concepts 577.1 De�nitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 577.2 Types of composition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58

7.2.1 Single module without composition . . . . . . . . . . . . . . . . . . . . . . . . 587.2.2 Multiple modules without composition . . . . . . . . . . . . . . . . . . . . . . 587.2.3 Multiple modules with harmonizing . . . . . . . . . . . . . . . . . . . . . . . . 587.2.4 Parallel composition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 597.2.5 Serial composition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 597.2.6 Approximate serial composition . . . . . . . . . . . . . . . . . . . . . . . . . . 607.2.7 Composition and order of middle boxes . . . . . . . . . . . . . . . . . . . . . 61

7.3 Composition with OpenFlow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 617.3.1 De�nitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 627.3.2 Multiple modules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 627.3.3 Achieving consistent run-to-completion . . . . . . . . . . . . . . . . . . . . . . 637.3.4 Parallel composition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 637.3.5 Serial composition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63

8 Beyond the end of NetIDE 658.1 Eclipse Marketplace . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65

NetIDEPublic

8.2 OpenDaylight . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 658.3 The Network Modelling (NEMO) language . . . . . . . . . . . . . . . . . . . . . . . . 65

8.3.1 Evolution during the lifetime of NetIDE (NetIDE) . . . . . . . . . . . . . . . 658.3.2 Beyond the lifetime of NetIDE . . . . . . . . . . . . . . . . . . . . . . . . . . 66

8.4 ONF Englewood . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 668.4.1 Beyond the lifetime of NetIDE . . . . . . . . . . . . . . . . . . . . . . . . . . 68

9 Conclusions 69

A NetIDE Protocol Speci�cation 70A.1 The NetIDE protocol header . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70A.2 Module announcement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71A.3 Heartbeat . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72A.4 Handshake . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72A.5 The fence mechanism . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73A.6 The OpenFlow protocol . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74

A.6.1 Properly handling reply messages . . . . . . . . . . . . . . . . . . . . . . . . . 75A.7 Other SBI protocols . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76

B XML Schema for Composition speci�cation 77

Bibliography 84

NetIDEPublic

List of Figures

1.1 The NetIDE Framework. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

2.1 Network Engine architecture. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52.2 Core Bundle Overview. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62.3 Location of the LogPub within the NetIDE Core. . . . . . . . . . . . . . . . . . . . . 102.4 The inner working of the LogPub in PUB mode. . . . . . . . . . . . . . . . . . . . . 112.5 The inner working of the LogPub in SUB mode. . . . . . . . . . . . . . . . . . . . . . 112.6 Architecture of a Shim. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132.7 Architecture of ONOS Shim. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142.8 Architecture of ODL Shim within Beryllium release. . . . . . . . . . . . . . . . . . . 152.9 Architecture of the Ryu Shim. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 162.10 Architecture of a Backend. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 172.11 The Backend within the Ryu architecture. . . . . . . . . . . . . . . . . . . . . . . . . 172.12 The Floodlight Client Controller Backend. . . . . . . . . . . . . . . . . . . . . . . . . 182.13 NetIDE Developer Toolkit Architecture. . . . . . . . . . . . . . . . . . . . . . . . . . 21

3.1 NetIDE's Network Engine repository at GitHub [1]. . . . . . . . . . . . . . . . . . . . 223.2 Engine Demo v0.3 (2016) at YouTube [2]. . . . . . . . . . . . . . . . . . . . . . . . . 23

4.1 The application format project tree. . . . . . . . . . . . . . . . . . . . . . . . . . . . 354.2 Parameter con�gurator. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 364.3 NetIDE Topology Editor. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 374.4 SDN network application testing dashboard. . . . . . . . . . . . . . . . . . . . . . . . 384.5 Logpub interface and view. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 394.6 Global con�guration view. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39

5.1 Logger trace. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 435.2 Debugger's output with OpenFlow 1.0. . . . . . . . . . . . . . . . . . . . . . . . . . . 465.3 Debugger's output with OpenFlow 1.3. . . . . . . . . . . . . . . . . . . . . . . . . . . 465.4 NetIDE Wireshark dissector. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 465.5 Application Pro�ler GUI. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48

6.1 Demonstration work�ow. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 536.2 Demonstration scenario. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54

7.1 Using a virtual overlay network for composition of module A and B. . . . . . . . . . 617.2 Typical order of IDS and Firewall and NAT load balancer middle boxes in a tradi-

tional network. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 617.3 Parallel composition of IDS, FW and NAT load-balancer modules . . . . . . . . . . . 62

8.1 Simpli�ed diagram of the Englewood architecture. . . . . . . . . . . . . . . . . . . . 678.2 Integration of Englewood with the Network Engine. . . . . . . . . . . . . . . . . . . . 67

A.1 Fence mechanism work�ow. Both nxid and module_id refer to the NetIDE header�elds. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74

NetIDEPublic

A.2 Request/reply message handling. xid refers to the OpenFlow header �eld. . . . . . . 75

NetIDEPublic

List of Acronyms

AMQP Advanced Message Queueing Protocols

API Application Programming Interface

CLI Command Line Interface

FIB Forwarding Information Base

FOSS Free and Open-Source Software

GUI Graphical User Interface

IDE Integrated Development Environment

IETF Internet Engineering Task Force

JAXB Java Architecture for XML Binding

LAN Local Area Network

LLDP Link Layer Discovery Protocol

MSM Most Speci�c Match

MMS Memory Management System

NBI Northbound Interface

NEMO Network Modelling

NETCONF Network Con�guration Protocol

NetIDE NetIDE

NFV Network Functions Virtualisation

ODL OpenDaylight

OF OpenFlow

ONF Open Networking Foundation

ONOS Open Network Operating System

OO object-oriented

OSGi Open Source Gateway initiative

OVSDB Open vSwitch Database Management Protocol

REST Representational State Transfer

NetIDEPublic

RFC Request For Comment

SBI Southbound Interface

SDN Software De�ned Networking

SoC Separation of Concerns

T-API Transport API

TID Telefónica, I+D

UA Union of Actions

UC Use Case

VNF Virtual Network Function

VNFD Virtual Network Function (VNF) Descriptor

XML eXtensible Markup Language [3]

NetIDEPublic

Executive Summary

This �nal WP2 deliverable comprises the results achieved during the whole project, in two words: theNetIDE framework. The NetIDE Engine (and its accompanying tools) and the NetIDE IntegratedDevelopment Environment (IDE) are the main components of the Framework. Its architecture andimplementation have been developed in work packages 2, 3 and 4. As a �manual�, this documentincludes implementation details as well as administrator and end-user documentation to guide in-stallation and usage of the NetIDE Framework. It also provides guidelines for developers willing toenhance the framework with new components for SDN controller platforms or new control protocolsnot considered during the project.

NetIDEPublic

1 Introduction

The Software De�ned Networking (SDN) paradigm allows networking hardware to be made �mal-leable� and remotely manageable by the so-called SDN controllers. However, the current SDNlandscape is extremely fragmented. Di�erent open and closed-source controller frameworks � suchas ONOS [4], OpenDaylight [5], Ryu [6], or Floodlight [7] � exist. Porting SDN applications fromone such platform to another is practically impossible and SDN users like network operators henceface a situation where they are either con�ned to applications working for the platform of theirchoice or are forced to re-implement their solutions every time they encounter a new platform.The goal of the NetIDE Framework is to deliver an integrated environment for SDN development

that uni�es di�erent controller technologies and allows developers to write, test and deploy modularnetwork applications that are independent from the underlying SDN technology. In this regard, itre-de�nes the concept of Network Application as a set of SDN software modules, possibly writtenfor di�erent SDN controller platforms, that are con�gured to cooperate on controlling the network.A Network Application hence allows to run such a set of modules as a single entity.As shown in Figure 1.1, the NetIDE Framework is an integrated environment that comprises

three main components: an Integrated Development Environment (IDE), a multi-controller SDNplatform called Network Engine, and a set of Tools for SDN developers.

Figure 1.1: The NetIDE Framework.

The IDE is built as a set of plugins for the Eclipse framework [8]. It provides editors that supportnetwork programming languages, a graphical editor to specify network topologies and the interfacesfor tools to debug network applications and monitor the network. Such an IDE includes a graphicalinterface to create complex network applications as composition of modules that are independentfrom the SDN platform.The composite network application is deployed onto the so-called Network Engine, the second

key element of the NetIDE Framework. The Network Engine is a multi-layer SDN platform whichorchestrates the execution of multiple SDN modules to control the network cooperatively as a singlenetwork application. The Network Engine comprises (1) a client controller layer that executes the

NetIDEPublic

SDN modules constituting the network application and (2) a server SDN controller layer that drivesthe underlying infrastructure. Between them, we have introduced an intermediate layer called Core:it hosts all logic and data structures that are independent from the particular controller frameworksand communicates with client and server controllers using the NetIDE Intermediate Protocol.Both Network Engine and IDE can be used to test the network applications before the actual

deployment onto an operational network. To this purpose, the IDE implements an automaticprocedure that generates a virtual machine where network applications can be executed over a pre-con�gured Network Engine and a network emulator. For the testing phase, many tools are providedto validate the network applications, such as the Logger, the Debugger, the Veri�cator, the Pro�lerand the Tra�c Emulator. Other tools, like the Memory Management System are meant to improvethe performance and the resiliency of the network especially in environments where multiple SDNmodules cooperate on controlling the same network.The remainder of this document starts at Chapter 2 with architectural and implementation as-

pects of the two main building blocks of the NetIDE Framework: the Network Engine and theIDE. We defer a comprehensive description of the tools to Deliverable D4.3 of the NetIDE project(NetIDE App Engine �nal release and evaluation).Chapters 3, 4 and 5 provide installation and running manuals for developers and end-users for all

the components of the framework, tools included. We conclude the actual manual of the NetIDEFramework in Chapter 6 by presenting a concrete use case where the NetIDE framework is not onlyused to design, implement, and test network applications, but also to support the network operatorin providing new functions to operational SDN networks.Chapter 7 discusses the most challenging aspects of the design of the Network Engine: trying to

de�ne a proper mechanism to compose SDN modules written for di�erent platforms into a singlenetwork application. In Chapter 8, we explore the technical concepts that we have developed inNetIDE and how they will be maintained beyond the lifetime of the project. For the sake ofcompleteness, in Annex A we provide the latest speci�cation of the NetIDE protocol. The �nalversion of the Network Engine uses it to communicate between Core, Backends and Shim.

NetIDEPublic

2 The NetIDE Framework

NetIDE has developed a framework that fully covers the design and execution cycle for NetworkApplications and introduces Software Development paradigms into the SDN world. The frameworkincludes a multi-platform SDN application execution environment (the Network Engine) and anIntegrated Development Environment to guide Network Application developers through the wholedevelopment cycle. In addition, we initially envisaged the possibility of implementing a NetworkApplication repository. During the lifetime of NetIDE, we have interacted with di�erent researchprojects that were developing application repositories with the intent of integrating support forNetIDE Applications where suitable. At the end, we decided to concentrate on the Network Engineand the IDE to provide a viable environment and downgraded the activities around the NetworkApplication repository. In the course of NetIDE, several other projects, including T-NOVA [9]in the FP7 Framework programme developed concepts for repositories that can host our NetworkApplications and thus nicely complement the NetIDE ecosystem.In this chapter, we provide a full description of the �nal Network Engine architecture and the

NetIDE Integrated Development Environment as well as how they related to one another.

2.1 The Network Engine

Controlling an SDN network with multiple, independent SDN applications is challenging for twomain reasons: (i) the Northbound Interface (NBI) and East-West Interface (EWI) of SDN controllersare still not standardized, therefore each SDN platform implements its own interfaces making ithard for SDN applications written for di�erent controllers to coexist in the same network, and (ii)SDN controllers only provide basic mechanisms to coordinate independent SDN modules, basicallyindividual SDN modules can transfer network events to other modules, creating simple chains ofindependent modules that handle an event.The Network Engine (Figure 2.1) has been designed and implemented to tackle these challenges.

The Network Engine is an SDN component that provides to SDN applications (called Modules inthe �gure) the runtime environment they expect and makes them cooperating with each other,e�ectively building a single Network Application. The Network Engine generalises the notion ofchaining provided by some controllers by (a) supporting explicit and �exible composition semantics,which (b) the Network Engine can execute even across multiple controllers.Our Network Engine follows the layered SDN controller approach proposed by the Open Net-

working Foundation in their SDN Architecture Technical Reference TR-502 [10]. It comprises aclient controller layer that executes the modules that form the Network Application and a serverSDN controller layer that drives the underlying infrastructure.The challenge is to integrate client and server controllers. A �rst idea is to connect the Southbound

Interface (SBI) for the (possibly multiple) client controllers to the Northbound Interface (NBI) ofthe (single) server controller. But as these interfaces do normally not match, adaptation is necessary.This adaptation has to cater for the idiosyncrasies of the di�erent controller frameworks we intendto support and has to be implemented for each of them. Conceivably, we could translate eachsupported controller's SBI into the NBI of each any other support controller, but that quicklygets out of hand and also voids the opportunity to coordinate execution between multiple clientcontrollers. We hence impose a protocol between server controller's NBI and client controller's

NetIDEPublic

Figure 2.1: Network Engine architecture.

SBI, the NetIDE Intermediate Protocol (cf. Annex A). To maximize reuse of adaptation code fromthe speci�c controller to the Intermediate Protocol, we use separate adaptors for the SBI, calledBackend, and for the NBI, called Shim.In our �rst implementations [11], we have proved that the Shim/Backend structure connected by

an intermediate protocol is feasible and sensible. However, they still left the fundamental componentin these modules: the composition logic. This implied that it needed to be reimplemented foreach controller framework we wanted to support. To overcome this shortcoming, we introducean intermediate layer, the Core (cf. Section 2.1.1): it hosts all logic and data structures thatare independent of the particular controller frameworks and communicates with both Shim andBackend using the same NetIDE Intermediate Protocol. The Core makes both Shim and Backendlight-weight and easier to implement for new controllers.We implemented the NetIDE Network Engine based on SDN controller platforms available as Free

and Open-Source Software (FOSS). Since we provide the functional speci�cations and platform-independent libraries [12], it should be easy to implement any Network Engine component fora closed-source commercial SDN environment as well. However, due to potential restrictions onpublishing results, we deemed them as impractical to work with within a research project andconcentrated on FOSS implementations.In consequence, we adopted OpenDaylight (ODL) and ONOS as server controllers, Floodlight as

client controller, and Ryu as either one. ODL and ONOS are carrier-grade controllers targeted toservice providers, enterprises and mainstream deployments. Being able to scale to a large numberof network elements, we recognized both ODL and ONOS as the most suitable SDN platforms forplaying the server controller's role. On the other hand, Ryu and Floodlight are aimed at facilitatingthe rapid prototyping of SDN applications. We were able to isolate their SBI drivers to add ourBackend alongside and to hook into their event dispatching mechanism to drive the execution logic(e.g. see Section 2.1.5.1).In the rest of this section, we provide architectural details of the components of our Network

Engine. With respect to previous deliverables, we also include some useful hints to help developersto extend the Network Engine to other platforms. We also describe the composition mechanismimplemented in the Core and how to extend it to support other control/management protocols.

NetIDEPublic

Moreover, we present the NetIDE Intermediate Protocol as a means for the Core to drive theexecution �ow of the whole Engine.

2.1.1 The Core

The Core contains several components that connect to the Shim and Backends, provide interfacesfor the NetIDE tools, and handle composition and con�ict resolution. In this section, we focuson the architecture of the Core; we defer the discussion on composition and con�ict resolution toChapter 7.The NetIDE Core is implemented as a modular Java-based application on top of the Apache

Karaf platform. Apache Karaf is an Open Source Gateway initiative (OSGi) runtime distributionfor highly dynamic applications and supports features such as runtime recon�guration, servicediscovery and modularization [13, 14]. Since it uses the same platform, the NetIDE Core can beintegrated into the OpenDaylight or ONOS platforms, opening up synergies.

NetIDE Core

core.api bundle

Core Interfaces

Shared Classes

core.connectivity bundle

Shim Connection

BackendConnection

core.caos bundle

SpecificationParser

Composition

Conflict Resolution

core.management bundle

ManagementConnection

ConfigurationChange Handling

Karaf CLICommandhandlers

core.routing bundle

OpenFlow RequestMessage Routing

eu.netip.library bundle

Message Parser

Message Builder

Abstraction Classes

Dependencies

Figure 2.2: Core Bundle Overview.

As an OSGi-based application that adheres to the patterns of modular applications and service-oriented architectures, the Core is built out of several bundles, each implementing a particularfunctionality of the Core. An overview of the Core bundles can be found in Figure 2.2. The bundleswere chosen to get a high reusability as the bundle are almost self-contained. For example putting thecomposition engine itself in its own bundle allows to reuse the composition engine in other contexts,e.g. in a SDN controller. Or that a bundle can be simply exchanged with another functionallysimilar bundle. Similarly, the communication that is handled by ZeroMQ [15] is implemented inthe core.connectivity bundle. If, for example, the communication to Backends and Shim should bedone via HTTP (not a good idea, just as an example), a core.httpconnectivity could replace it. Theindividual bundles are explained in detail in the following sections.The individual bundles provide services to and consume services of other bundles. The service

references are managed by the Apache Aries Blueprint Dependency Injection module [16]. It is inte-grated in Karaf and was therefore selected as the dependency injection framework for the prototype.Service exports and imports are described in a blueprint.xml �le that is part of each bundle.

2.1.1.1 The core.api bundle

The core.api bundle contains shared classes and interfaces for the Core bundles. It does notcontain any logic itself and does not provide or consume services. It is referenced as a dependencyby the other bundles and provided at runtime in Karaf as part of the Core.

2.1.1.2 The lib.netip bundle (dependency)

With the NetIDE Intermediate Protocol also came the need for libraries that make handling theprotocol's messages easy. This bundle contains a Java implementation of the NetIDE Intermediate

NetIDEPublic

Protocol, i.e., it contains classes for parsing and creating NetIDE messages. It is not directly part ofthe Core project but is implemented separately as a Java library so that other Java-based projectsof NetIDE (e.g., our OpenDaylight Shim, ONOS Shim and the Floodlight Backend) can utilize it.Therefore, it is added as a dependency and deployed in Karaf as part of the Core.

2.1.1.3 The core.connectivity bundle

This bundle contains all classes necessary for the connection to the Shim and the Backends. Asmentioned earlier, these classes utilize the ZeroMQ library for connections. In particular, it usesthe JeroMQ library [17], an open-source pure-Java implementation of ZeroMQ, which is referencedas a compiled dependency.The connections are handled by so-called managers that run tasks like message parsing and

calling events on registered listeners, following common object-oriented (OO) and Java patterns.There are dedicated managers for Shim and Backend connections. The managers do not handlenetwork connections directly; instead, they use a connector that is responsible for receiving andsending data. This is valuable since it allowed a fast migration from TCP sockets to ZeroMQ-basedcommunication, having to rewrite only the connector and none of the managers, and also followsthe Separation of Concerns (SoC) principle.

2.1.1.4 The core.caos bundle

The core.caos (Composition and Orchestration) bundle contains the most important logic of theCore, namely the composition and con�ict resolution mechanisms.Since it is responsible for handling the composition as speci�ed in the composition speci�cation,

the bundle also includes abstraction classes that enable the automatic deserialization of a providedeXtensible Markup Language (XML)-speci�cation (Section 2.1.1.8). The deserialization mechanismuses the Java Architecture for XML Binding (JAXB) reference implementation [18], guided by an-notations in the abstraction classes. JAXB is also used to generate the XML schema for compositionspeci�cations out of these annotated classes.

2.1.1.5 The core.routing bundle

This bundle manages the mapping of requests and answers in order to give only the right Backend/-module the response to a request and not to broadcast the response to all Backends, risking that aBackend gets confused by a response that it did not request. See Annex A.6.1 for more details.

2.1.1.6 The core.management bundle

This bundle provides a management interface for the Core using ZeroMQ and NetIDE Managementmessages. This interface allows external tools to change the con�guration of the Core at runtime(i.e. by installing a new composition speci�cation).Runtime re-con�guration is achieved by using the Apache Aries Blueprint Con�guration service

[16], which is managed by Karaf's Con�gAdmin service. The bundle de�nes con�guration propertiesin their blueprint.xml con�guration �le whose values are resolved at runtime and injected into theclass instance. The Blueprint con�guration system also supports dynamic con�guration changeseither by reloading the a�ected bundle or by calling the speci�c method for the changed property.When a con�guration change request is received, the change is re�ected in Blueprint con�gurationspace, which takes care of reloading the a�ected bundles. The Core bundles make use of Blueprintcon�guration for several con�guration properties such as connection ports. An overview of theavailable con�guration properties can be found in Table 2.1; guidance on how to use them can befound in the o�cial Karaf documentation.

NetIDEPublic

Table 2.1: Core con�guration properties

Bundle Key Value Description

core.connectivity port 1-65535 The port to listen for incoming Shim andBackend connections.

core.caos maxWaitTime Integer (>= 0) The maximum time in seconds to wait for re-quired modules to connect before aborting. 0means �wait inde�nitely�.

bypassUnsupported-Messages

Boolean If set to �true�, unsupported messages that ar-rive from the Shim will be relayed to the tar-geted Backend, if possible. If set to �false�,such messages will be discarded.

composition-Speci�cation

XML compositionspeci�cation string

The composition speci�cation that should beused. If the string is not a valid compositionspeci�cation, the change will be rejected andthe former speci�cation will be used.

core.management port 1-65535 The port to listen for management connec-tions.

2.1.1.7 Extending the Core to non-OpenFlow protocols

Almost all modules of the Core are written in a protocol-agnostic way. The only modules thatknow about and handle the content of NetIDE messages with OpenFlow payload are core.caos

and core.routing. core.routing serves a very special purpose of mapping/remapping OpenFlowXIDs and is therefore OpenFlow speci�c.Most of the core.caos module is also protocol-agnostic. It will do sequential or parallel compo-

sition by instancing the ConflictResolvers class, which in turn calls the speci�c con�ict resolverdepending on the message content. At the moment this is only implemented for OpenFlow in theform of the DefaultOFConflictResolver class. Extending caos for other protocols simply needsimplementing resolver classes for the other protocols.

2.1.1.8 Composition speci�cation

The speci�cation language has to be able to express all the semantics that specify how to composeSDN application modules running in the SDN controllers in the Network Engine to a NetworkApplication.We used an XML-based composition speci�cation since that allows us to o�oad checking the

validity of the �le to an XML validation tool and enables us to use well established XML toolsand libraries. The application is composed in the Modules element (line 2). This informationis used to start modules and client controllers (if necessary) when the new application is loaded.Additionally, �lters can be applied to modules or invocations to only apply them to selected events.The Composition element (line 11) de�nes the execution �ow on network events. An example forsuch a speci�cation is given in Listing 2. It uses an XML-based notation. For the underlying XMLschema, refer to Annex B.

NetIDEPublic

Listing 2.1: Composition Speci�cation Example.

1 <CompositionSpecification xmlns="http://netide.eu/schemas/compositionspecification/v1">

2 <Modules>

3 <Module id="fw" loaderIdentification="ryu-fw.py"/>

4 <Module id="appA" loaderIdentification="appA.py"/>

5 <Module id="appB" loaderIdentification="appB.py"/>

6 <Module id="lb" loaderIdentification="loadbalancer.jar">

7 <CallCondition events="packetIn" datapaths="0 42 43 45"/>

8 </Module>

9 <Module id="log" loaderIdentification="logger.py"/>

10 </Modules>

11 <Composition>

12 <ModuleCall module="fw" />

13 <ParallelCall resolutionPolicy="priority">

14 <ModuleCall module="appA" priority="1"/>

15 <ModuleCall module="appB" priority="2"/>

16 </ParallelCall>

17 <Branch>

18 <BranchCondition events="flowMod"/>

19 <If>

20 <ModuleCall module="log"/>

21 </If>

22 <Else>

23 <ModuleCall module="lb"/>

24 </Else>

25 </Branch>

26 </Composition>

27 </CompositionSpecification>

In the following section we will describe the most important elements of the composition XMLformat in greater detail.

2.1.1.9 Con�ict resolution policies

Currently, the following con�ict resolution policies are de�ned:

Ignore A baseline policy, simply ignores all potential con�icts and returns the union of the com-mands.

Pass If two commands con�ict, discard them both. Also a baseline policy; currently not imple-mented (but trivial to do).

Priority Assuming that priorities have been assigned to the applications in the composition spec-i�cation (even if only implicitly by the order in which they appear in the speci�cation �le),then pick the command resulting from the application with the higher priority (and ignorethe other commands).

Auto The auto policy tries to determine a smart resolution. It applies only to commands withmatch con�ict but no action con�icts. First, it generates the Most Speci�c Match (MSM) outof the two:

• for each match �eld that has wildcards in both commands, keep the wildcard;

• if there is a wildcard in one command and a speci�c value in the other, keep the speci�cvalue;

• if there is the same speci�c value in both commands, keep that value;

• otherwise, the auto policy fails.

NetIDEPublic

As candidate set of commands for this MSM, we consider the Union of Actions (UA) of C1

and C2. By assumption these commands have no action con�ict, hence can be combined.

The actions for this match are de�ned as follows:

• If the MSM equals the matches of both commands, replace C1 and C2 by the new rulewith the MSM and the UA.

• If the MSM equals the match of C1 and UA equals the actions of C1, then keep both C1

and C2 and ignore the MSM (the new rules would not add anything, it is equivalent toC1 but does not contradict C2). Similarly for C2.

• Otherwise, add a new command with the MSM and the UA.

If this policy fails for two con�icting C1 and C2, fall back to a default policy.

Other merge policies are easy to add to the Core if needed, for example automatic rewriting ofrules by priorities.

2.1.1.10 The LogPub module

The LogPub is one of the modules of the NetIDE Core. It acts as a unique interface between thetools and the other modules of the Core. With this architecture the LogPub is seen by the restof the Core as another Backend and allows an abstraction of the tools. The LogPub tasks can besummarized as:

1. converting messages from NetIDE to ZeroMQ format (and the other way around);

2. transmitting messages originated by the Shim/Backend(s) to the tools;

3. receiving messages from the tools and sends them to the Shim/Backend(s).

The LogPub uses a PUB/SUB paradigm to facilitate the handling of messages in both directionsand thus exposes two queues to the tools. The PUB queue is where the LogPub publishes messagesand the tools are subscribed to it; the SUB queue is where the tools push messages and the LogPubis subscribed to it.

Figure 2.3: Location of the LogPub within the NetIDE Core.

The LogPub module receives the messages from the Shim and Backend(s) using the Java interfacecore_connector that is used by the core.routing module to send messages to the LogPub. Themain receiving loop of the LogPub runs in a Thread (to allow concurrent access between the di�erent

NetIDEPublic

queues); hence, there is an initial process to deposit the message in the ZeroMQ in_process queue(as depicted in Fig. 2.4).1 This prevents the Thread from hanging as ZeroMQ handles all commu-nication from/to the process. This functionality is integral for ZeroMQ to handle concurrency. Themessages are then classi�ed by the Identifier; its role is to assign them a topic; when the messagesare dispatched in the PUB queue, the tools will only receive the messages they are subscribed to(see Fig. 2.4). For example the Logger is supposed to get all messages and will subscribe to alltopics (represented by the * mnemonic), while the IDE will subscribe to the topology update topic.

Figure 2.4: The inner working of the LogPub in PUB mode.

In SUB mode (see Fig. 2.5), the LogPub has an open queue which accepts every message sent fromthe tools and transmits messages either to the core.routing module which, as its name suggests,handles the routing of synchronous OpenFlow requests messages between the LogPub, the Shimand the Backends (e.g. STATISTICS_REQUESTS) or directly to the Shim/Backend for asynchronousmessages (e.g. FLOW_MOD).

Figure 2.5: The inner working of the LogPub in SUB mode.

2.1.2 Message format

Messages exchanged between the LogPub and the tools must use the format inside a ZeroMQ multi-part message shown in Listing 2.2. The location �eld contains a string identifying the destinationor source of the message (Shim or Backend) and the message �eld contains a netide_message:

• In the Tool−→ LogPub direction, the location �eld indicates whether the message is intendedfor the Shim or a Backend (eg. {shim,message}).

1We introduced the LogPub architecture in Deliverable D2.5 [19], while in Deliverable D2.3 [20] described the twoAdvanced Message Queueing Protocols (AMQP) options we evaluated to implement it: RabbitMQ and ZeroMQ.

NetIDEPublic

• In the LogPub −→ Tool direction, the location �eld indicates whether the message originatesfrom the Shim or a Backend. As mentioned earlier, the message will also have a topic (eg.{topic,shim,message}).

Listing 2.2: ∅MQ multi-part message header

struct logpub_message{

struct zeromq_message location;

struct zeromq_message message;

2.1.2.1 Interface with the IDE

The NetIDE Core connects with the IDE via two separate channels. The testing dashboard usesSSH to start and stop the Core as well as load a composition speci�cation. SSH is also used toautomatically install the Core, the Shim and the Backends. The output and prompt of the NetIDECore can also be displayed in the IDE via an SSH connection.The connection to the LogPub interface (Section 2.1.1.10) is established via a ∅MQ connection.

The IDE listens to the LogPub interface, deserializes the messages and uses them for visualizationand monitoring.A detailed explanation of the connections from the IDE to a NetIDE engine setup can be found

in Section 2.2.

2.1.3 NetIDE Intermediate Protocol

The NetIDE Intermediate protocol implements the following functions: (i) it carries managementmessages between the Network Engine's layers (Core, Shim and Backends), e.g., to exchange infor-mation on the supported SBI protocols, to provide unique identi�ers for application modules, or toimplement the fence mechanism (Section A.5 in Annex A or Section 3.1.3 in Deliverable D2.5 [19]).(ii) It carries event and action messages between Shim, Core, and Backend, properly demultiplexingsuch messages to the right module based on identi�ers. (iii) It encapsulates messages speci�c toa particular SBI protocol version (e.g., OpenFlow 1.X, NetConf, etc.) with proper information torecognize these messages as such.The NetIDE protocol covers the communication between the di�erent components of our architec-ture: (i) Module advertisement using the NETIDE_MODULE_ANN and NETIDE_MODULE_ACK messages byclient controllers (ii) Transaction control with the NETIDE_FENCEmessage, and (iii) NETIDE_OPENFLOW,NETIDE_NETCONF or others, to exchange SBI speci�c messages like OpenFlow packets, etc.The NetIDE protocol provides a 64-bit datapath_id �eld to uniquely identify network elements

when SBI messages are carried. It also provides a 32-bit module_id to uniquely identify applicationmodules running on top of each client controller. The composition mechanism in the Core leverageson this �eld to implement the execution �ow of these modules. Finally, the nxid �eld is used in thecontext of the fence mechanism (Section Section 2.1.1). The full protocol speci�cation is providedin Annex A.

NetIDEPublic

2.1.4 Shim

The Shim is a platform-speci�c component that translates the NBI of the server controller to theNetIDE Application Programming Interface (API), thereby exposing it to the other components ofthe Network Engine.

Figure 2.6: Architecture of a Shim.

Figure 2.6 shows the internal architecture of the Shim integrated with a server controller platform.It includes hooks placed inside SBIs (small grey boxes in the �gure) to route all messages fromthe network to the Shim itself, overriding the server controller's processing logic. This preventsother application modules running on top of the server controller from interacting directly withthe network (i.e. by-passing the NetIDE Core's logic). Its implementation depends on the servercontroller platform and it does not necessarily necessitate the modi�cation of the Southboundinterfaces, as represented in Figure 2.6, but it can involve a platform's core components instead(such as the application manager in Ryu, cf. Section 2.1.4.3).The Core Connection component handles the communication with the Core through the NetIDE

Intermediate Protocol. The API Translator converts between the NetIDE protocol and platform-speci�c NBI messages. Please note that, in this case, NBI means a bi-directional and event-drivenserver controller's northbound interface, like the Java interface available in ONOS and OpenDaylightor the Python interface in Ryu. In fact, the Core, as well as the client controllers on top of it, needs tomonitor and handle the events that are asynchronously generated by the network, such as new �ows,port status changes, etc. This operation would be possible but not e�cient with Request/Responseinterfaces like Representational State Transfer (REST), also available in many SDN controllers.The Topology module allows the IDE, via a REST interface (see Section Section 2.2), to retrieve

the topology underneath the Network Engine. Topology discovery is achieved by interacting withthe network through the server controller's SBI or by retrieving the topology information collectedby speci�c core components available in advanced controller platforms (e.g. ONOS and OpenDay-light). Topology details include datapath_ids, number of supported �ow tables, (administrativeand operational) port information and other properties of the network elements.We implemented the Shim for ODL and ONOS as OSGi bundles, leveraging their respective Java

NBIs to interact with the network elements. Both implementations support OpenFlow versions1.0 and 1.3 and make use of serialization methods to convert messages from the controller's NBIformat into OpenFlow byte arrays before encapsulating them with the NetIDE header. Vice-versa,deserializers are used to convert byte arrays contained in the payload of NetIDE messages (OpenFlowcommands) into NBI Java classes. While for ONOS we could leverage the implementation ofserializer/deserializer methods provided by the OpenFlowJ-Loxi library [21], in ODL (which usesits own library Open�owJava [22]) we had to implement the serializer from scratch and to extendthe existing deserializer in order to cover all OpenFlow message types.We also implemented the Shim for Ryu, based on Ryu's Python NBI. It includes serializer and

deserializer methods based on Python's struct module [23].

NetIDEPublic

2.1.4.1 ONOS Shim

The ONOS Shim is implemented as an OSGi bundle and it is based on the OpenFlowControllerAPI [24] (Figure 2.7). This API provides the complete management of the OpenFlow protocol anduses the OpenFlowJ-Loxi library to serialize and de-serialize OpenFlow messages. This strategyallows the Shim to handle OpenFlow messages without any translation into/from the abstractedONOS NBI. This avoids an additional computational burden due to the translation and any lossof information this translation may cause. For example, when a PACKET_IN arrives at the ONOSOpenFlow southbound, the ONOS OpenFlow provider converts the packet to a high-level represen-tation, called the PacketContext [25]. This interface does not maintain all the information carriedby a PACKET_IN, such as the buffer_id, that is often relevant for SDN forwarding applications.In short, the ONOS Shim simply encapsulates the OpenFlow message received by the OpenFlow-

Controller API in the NetIDE Intermediate Protocol and then it relays the encapsulated messageto the Core.

Figure 2.7: Architecture of ONOS Shim.

Although the ONOS Shim may appear bound to the OpenFlow protocol (speci�cally, it currentlysupports OpenFlow v1.0 and v1.3), actually it can potentially handle several SBIs at the sametime, such as OpenFlow, NETCONF and OVSDB. The ONOS Shim is implemented to importany new SBI through the available Java APIs, without requiring any modi�cation to its architec-ture. For example, one can add NETCONF support to the ONOS Shim by simply importing theNetConfController interface [26] (Listing 2.3).

Listing 2.3: Example of Java import for NETCONF in the ONOS Shim.

import org.onosproject.netconf.NetconfController

After that, the ONOS Shim has to be registered as a message listener to the NETCONF south-bound in order to receive messages from the NETCONF devices.

2.1.4.2 OpenDaylight Shim

The ODL Shim has been implemented as an OSGI bundle. OpenDaylight uses Apache Karaf (anOSGI container) to manage the di�erent bundles that make up the project, with each o�ering theirown service interface as a Karaf Feature. The NetIDE feature provides the required functionality tomanage the OpenFlow protocol with each connected switch. Unlike the ONOS Shim and the Corewhich use the NetIP library to generate NetIDE Intermediate Protocol messages (Section 2.1.1.2),the ODL Shim had to use a modi�ed version of this library. The NetIP library uses OpenFlowJ-Loxi to serialize Open�ow messages, also used by Floodlight. However, OpenDaylight uses its own

NetIDEPublic

internal project Open�owJava to serialize Open�ow messages. As such, we needed to create analternative NetIP library that uses Open�owJava for serialization. This ODL-NetIP library residesin the OpenDaylight GIT repository [27] and is built as part of the nightly build process.Unfortunately, the Open�owJava project only provides serialization functions in one direction,

e.g. serializing a Packet_Out java class to a byte array for sending to a switch. As the OpenFlowprotocol dictates a Packet_Out will never arrive from a switch, there is no corresponding functionto deserialize a byte array back to a Packet_Out java class. This is true for all messages for a givendirection, eg, Packet_In arrive from switches so they do not require serialization to byte arrays.However, as the Shim passes these serialized objects to the Core for routing to the Backends,the project required these additional serialization functions. These functions are speci�c to justOpenFlow and were coded and submitted as patches to the Open�owJava serialization library ofOpenDaylight. These patches have been accepted ensuring these functions are now maintainedgoing forward.The ODL Shim uses the ZeroMQ library (a message queue wrapper library) to place messages

received from the switches onto the Message Queue that is managed by the NetIDE Core. Inaddition, the Shim subscribes to this same message queue so that it can receive the messagescoming from connected Backends. These outgoing messages are then sent to the relevant switchbased on the datapath ID that is part of the NetIDE Intermediate Protocol.

Figure 2.8: Architecture of ODL Shim within Beryllium release.

2.1.4.3 Ryu Shim

The Ryu Shim is implemented as an application for the Ryu controller [6] based the Python Open-Flow libraries included in Ryu's source code [28]. Fig. 2.9 shows the Shim in the context of Ryu'sarchitecture when Ryu is deployed as server controller. In this �gure, modules besides the Shim areSDN applications that run on Ryu and that are part of the composite network application alongwith the modules running on client controllers.The Shim Layer is a single-threaded entity that interacts with Ryu's internal OpenFlow controller

to communicate with the underlying OpenFlow devices. The �nal version of the Ryu Shim workswith any version of the OpenFlow protocol (1.0-1.5.1 at the time of writing) and is compatible withthe latest version of Ryu (v4.6 at the time of writing).The Ryu Shim converts messages from the controller's NBI format into OpenFlow byte arrays

before encapsulating them with the NetIDE header. Vice-versa, it converts byte arrays contained inthe payload of NetIDE messages (OpenFlow commands) into NBI Python structures. Such format

NetIDEPublic

Figure 2.9: Architecture of the Ryu Shim.

translations are based on Python's struct module [23].The latest version of the Ryu Shim implements the Shim architecture version 2.0, as presented

in Deliverable 2.6 and depicted in Figure 2.6. To route all the network events to the Shim, wemodi�ed send_event_to_observers in app_manager.py, which is the method that distributes theevents to the registered application modules (called observers in the Ryu controller framework).As a consequence, only the observer with name RyuShim gets the events from the OpenFlow SBI(Listing 2.4). The others, i.e. the SDNs modules running on Ryu, only receive the events via aBackend SBI, which is driven by the Core with NetIDE intermediate protocol.

Listing 2.4: Augmented send_event_to_observers for Ryu.

def send_event_to_observers(self, ev, state=None):

for observer in self.get_observers(ev, state):

if observer is "RyuShim":

self.send_event(observer, ev, state)

2.1.5 Backend

The Backend is designed as an additional southbound interface for the client controller that inter-acts with the underlying layers of the Network Engine. Figure 2.10 shows the internal architectureof a Backend. At boot time, the Backend starts the discovery of the application modules runningon top of the client controller (Module Discovery) and registers them to the Core (AnnouncementHandler). The Core, in turn, assigns a speci�c identi�er (module_id) for each registered module.As part of its initialization process, the Backend also queries the Shim for the physical topology.Backends store the topology information as instances of the network elements (BackendDPs in Fig-ure 2.10), which are exposed to the application modules. Further details on Backend initializationare provided as part of the NetIDE protocol speci�cation in Annex A, Sections A.2 and A.4.At runtime, Backends use the module_id in the NetIDE protocol header to tell the Core which

module is sending the message. On the other side, the Core uses the module_id based on pre-de�ned policies to indicate which module handles the event. The Event Handler steers the eventdistribution inside the Backend, ensuring events are sent to the correct module.

2.1.5.1 Ryu Backend

The Ryu Backend represents an SBI for the Ryu controller [6] and it is implemented as an inde-pendent module (its implementation does not impact Ryu's core modules) that can be loaded at

NetIDEPublic

Figure 2.10: Architecture of a Backend.

boot time along with all the user's applications. The Backend is a modi�ed version of Ryu's inter-nal OpenFlow protocol SBI which (i) uses the NetIDE protocol to communicate with the NetIDECore, (ii) implements the fence mechanism to synchronize with the Core (cf. Section A.5) and (iii)maintains internal representations of the network elements that are exposed to the applications.The �nal version of the Ryu Backend has been tested with OpenFlow v1.0 and v1.3, although it isimplemented to work with any version of the protocol (1.0-1.5.1 at the time of writing), and it iscompatible with the latest release of Ryu (v4.6 at the time of writing).

Figure 2.11: The Backend within the Ryu architecture.

Like the Shim, the Ryu Backend converts messages from the controller's NBI format into Open-Flow byte arrays before encapsulating them with the NetIDE header. Vice-versa, byte arrays con-tained in the payload of NetIDE messages (OpenFlow commands) are converted into NBI Pythonstructures which are relayed to the running SDN modules.In Figure 2.11, the general Backend architecture is integrated with the rest of the Ryu architecture.

The Event handler implements the fence mechanism which executes the event handlers implementedin the SDN modules and it takes care of intercepting the exit points of such handlers that triggerthe NETIDE_FENCE message (cf. Section A.5).

Listing 2.5: Implementation of the fence mechanism in the Ryu Backend.

NetIDEPublic

1 if value == module_id and key in event_observers:

2 module_brick = ryu.base.app_manager.lookup_service_brick(key)

3 module_brick_handlers = module_brick.get_handlers(ev)

4 for handler in module_brick_handlers:

5 handler(ev)

6 break

78 msg = NetIDEOps.netIDE_encode('NETIDE_FENCE',self.netide_xid,module_id,0,"")

9 self.channel.socket.send(msg)

Listing 2.5 provides an excerpt of Ryu Backend's code. In line 1, the Backend checks whetherthe SDN module with the module_id (speci�ed by the Core in the NetIDE header) implementsthe handler for the event. In line 5, the Backend passes the event to the module by executing itshandler. Please note that, since the SDN module receives the events from the Backend, it usesthe send_msg method implemented in the Backend to respond. In such a method, the Backendintercepts the message (e.g. a FLOW_MOD or a PACKET_OUT) and encapsulates it with the NetIDEheader before sending it to the Core.In lines 8�9, a fence message is sent to the Core indicating that the event has been processed, no

matter if the module has produced zero, one, or many control messages to handle the event.

2.1.5.2 Floodlight Backend

The Floodlight Backend was developed as a Floodlight Module that is loaded when the Floodlightcontroller starts. By accessing the native Floodlight application interface (API), this gives themodule access to additional events and functions that are not available from the RESTful interface.While Floodlight REST applications can start up and connect over HTTP whenever they want,Floodlight Modules are required to register with the controller and specify where in the order ofother Modules they start.

Figure 2.12: The Floodlight Client Controller Backend.

By default, Floodlight listens on port 6633 for all connecting OpenFlow switches. By changing

NetIDEPublic

this to a di�erent port 6634, no physical/virtual switch will now connect to Floodlight. For eachSwitch connected event that is received by the NetIDE module from the Core, a new Dummy Switchclass is created in the backend that connects to Floodlight on this separate port, 6634, representingthe actual switch. The Dummy Switch replays and responds to OpenFlow protocol messages (hello, echo_reply, config_reply, stats_reply, etc). Floodlight will now add this switch to itslist of Managed Switches and trigger the addedSwitch event to other IOFSwitchListener listeners.This is where SDN applications developers traditionally push flow_mod messages proactively to theswitch.As SDN Applications and other Floodlight modules send OpenFlow messages to switches, each

Dummy Switch will now receive this message. After inspecting the message, it then creates therelevant NetIDE Intermediate Protocol message and sends it to the NetIDE Core. When the responseis received from the Core, the equivalent OpenFlow Reply message is created and the relevantproperties are set. The message is then sent back to Floodlight through the correct Dummy Switch.Similarly, all switch-initiated messages (eg, packet_in) are sent by the Shim Layer (through thecore) to the NetIDE Module of Floodlight, which in turn creates the relevant OpenFlow responsemessage and sends it out through the correct Dummy Switch. The latest version of Floodlightnow supports the NetIDE proposed fence mechanism to notify completion of message processing.As each Backend completes the processing of an inbound message, the associated fence message isnow sent. This ensures that the Backend supports the latest version of the NetIDE IntermediateProtocol.To manage connected switches, Floodlight maintains a separate TCP channel for each connected

switch. It is therefore important that each response from the Core is relayed through the correctDummy Switch's TCP connection. This is tracked through the Datapath Identi�er (DPID) andprovides uniqueness. It is possible to view the list of switches that Floodlight is managing byusing the Floodlight Management Web interface available at http://localhost:8080/. Whatis novel here is that none of these physical switches are actually connected to Floodlight, butare in fact managed by the Server Controller (Shim component) and Core. But Floodlight willcontinue to operate and generate topology maps based on information provided to it by the Backendcomponent.

2.2 The NetIDE Integrated Development Environment

This chapter provides a short overview of the architecture of the NetIDE developer toolkit and itsinteractions with the NetIDE Engine. Please refer to WP3 Deliverables [29] for further details.We describe the architecture of the IDE in the context of running controller machines and the enginearchitecture. Figure 2.13 shows a component structure of the IDE and their connections to otherelements of the project.

Test UI The test UI in the IDE is used to deploy and run the Network Engine for testing purposes.This requires a connection to the test machine via SSH as well as a connection to the virtualmachine manager Vagrant [30] in case a test VM has to be created on the developers machine.

Copying SDN modules and topology con�gurations from the IDE to a remote machine is donevia the Remote Machine Manager which uses SCP. The App Manager starts and stops SDNmodules on remote machines and VMs using SSH.

All these management GUIs build upon the GUI framework SWT.

Monitoring UI The Monitoring UI processes information from the LogPub interface and makes ithuman-readable via graphical visualization.

NetIDEPublic

The necessary data is retrieved by a ZeroMQ listener which operates inside the IDE and listensto LogPub messages. These messages are parsed and can be passed on to other components.

A Visualizer creates graphical representations of the message log and displays it to the user.It is implemented upon the Eclipse-internal data visualization framework BIRT.

App Format Tools The App Format Tools mainly consist of editors and con�guration assistantswhich do not interact with the Network Engine or remote machines. These editors are used toedit the parameter schema and the system requirement speci�cation �les, and test topologies(Topology Editor). The �rst two editors use the Xtext framework and the latter uses thegraphical editor framework Sirius.

The Packager creates a gzipped package from a NetIDE project. This package can be used bythe NetIDE package Loader which provides a headless way to start SDN apps on the NetIDEEngine.

The Topology Importer is an application built on top of the OpenDaylight (ODL) SDN Con-troller [31]. It produces noti�cations when changes on the topology controller by ODL occur.An update would be expected in the following cases: (i) a switch is connected to ODL; (ii) aswitch is disconnected from ODL; (iii) a new host is connected to a switch controlled byODL; (iv) a host is disconnected from a switch controlled by ODL; (v) a connection is createdbetween two switches.

NetIDEPublic

Figure 2.13: NetIDE Developer Toolkit Architecture.

NetIDEPublic

3 Network Engine manuals

In this chapter we describe how to install and run the di�erent components of the Network Engine.As described in Section 2.1, the Network Engine is de�ned by three types of components: the Core(unique piece of code in the Network Engine), the Shim (only one instance can run at a time,although more than one Shim is implemented) and the Backends (one or more instances can run ata time). All the code of the Network Engine can be found at https://github.com/fp7-netide/Engine, where the master branch is the most updated and stable version of it.

Figure 3.1: NetIDE's Network Engine repository at GitHub [1].

Following Fig. 3.1, the Core component is contained in the core folder, three implementationsof the Shim for ODL, Open Network Operating System (ONOS) and Ryu are contained in the

NetIDEPublic

odl-shim, onos-shim and ryu-shim folders respectively, and two implementation of the Backendfor Floodlight and Ryu are in floodlight-backend and ryu-backend. We describe how to installall of these components in the next sections.Additionally, the Network Engine repository contains �ve more folders. The libraries/netip

folder has the libraries implementing the NetIDE Protocol for both Java and Python, which arecommon for the Core, Shims and Backends (because all of these components of the Network Engineare either developed in Java or Python). The loader folder is an application loader for the NetworkEngine, directly related with the IDE and the deployment process. The odl/topology-update

implements a topology loader for the ODL Shim, used in runtime by the IDE. The tests foldercontains several examples of composition to be executed in the Network Engine, including themodule applications and the composition speci�cations �les to be loaded. Finally the utils folderhas a mix of di�erent utils, including a Vagrant �le and scripts to generate a working instance ofthe Network Engine automatically, instead of installing components and dependencies one by one.Finally, the NetIDE YouTube channel [2] contains several demos of the Network Engine (as

illustrated in Fig. 3.2), in which any user can follow how to run the Network with an initial schemathat shows the deployment and step-by-step commands later on.

Figure 3.2: Engine Demo v0.3 (2016) at YouTube [2].

3.1 General requirements

The common requirements for all components of the Network Engine are:

• Git version control system. No particular version required, but we recommend to use version1.9.0 and beyond.

• Java 8 JDK, or Python 2.7/3.5 for Python-based platforms.

• Maven [32]. As of writing, the approved maven version is v3.3.9.

• A Linux-based operating system. Even though most components are developed and tested ondi�erent platforms as well, the setup on Linux has been tested most thoroughly.

NetIDEPublic

3.2 The Core

3.2.1 Installing

1. Clone the central Engine source repository

git clone https://github.com/fp7-netide/Engine.git netide-engine

git checkout master

2. The following commands will build the bundles needed by the Core and install them to yourlocal Maven repository:

cd $HOME/netide-engine/core

mvn clean install -Dgpg.skip=true

Gpg is skipped since normally parts of the builds are signed with gpg to able to publish them.

3. Download Apache Karaf from https://karaf.apache.org/index/community/download.html

in version 3.0.5 and extract it to $HOME/netide/apache-karaf-3.0.5

4. Install the branding osgi packet to the lib directory of karaf

cp core.branding/target/core.branding-1.1.0-SNAPSHOT.jar $HOME/netide/apache-

karaf-3.0.5/lib/

5. Start Karaf by running $HOME/netide/apache-karaf-3.0.5/bin/karaf

6. Install the netide-core feature:

feature:repo-add mvn:eu.netide.core/core.features/1.1.0-SNAPSHOT/xml/features

feature:install core

7. Load a composition speci�cation with netide:loadcomposition, for example the MinimalSpeci�cation:

netide:loadcomposition /Users/arne/netide/netide-engine/core/specification/

OneSimpleSwitch.xml

NetIDEPublic

Warning: the karaf shell does not expand variables and placeholder characters like ∼ or$HOME, so the path needs to be an absolute path like in the example.

8. Run now Mininet, the Shim and �nally the Backend (Note: If you are using the examplecomposition �le, you should use the Ryu simple_switch.py application)

3.2.2 Running

The Core provides several commands to perform debugging and simple status checks. Karafs com-mands all start with log:, e.g. to have a tail -f style log viewing, use:

log:tail

Additionally NetIDE speci�c status commands are available:

Show connected Backends/modules

karaf@root()> netide:listmodules

Id Name Backend Last message (s ago)

797 SimpleSwitch backend-ryu-14172 -

238 backend-ryu-14172 backend-ryu-14172 4,66

- shim - 0,32

The commands show all known Backends, the applications running on the Backend and the Shimand also the last time the Core received a messages from the Backends and applications.

Show OpenFlow requests from Backends

netide-core>netide:listroutingrequests

Shim xid Backend B xid #Resp Type Last message (s ago)

100002 backend-ryu-80695 107 0 STATS_REQUEST 75,31s

This command lists all symmetric requests between SDN modules and the network that are handledby the Core. Refer to Annex A.6.1 for more detail about this mechanism.

Show installed composed �ow mods

As another debugging feature, the Core allows to list all Flows (OpenFlow 1.3 only) that have beeninstalled by the composition mechanism:

netide-core>netide:listflowmods

DefaultFlowEntry{rule=DefaultFlowEntry{id=0, deviceId=of:0000000000000002,

priority=10, selector=[IN_PORT:2, ETH_DST:72:A3:E9:28:94:71], treatment=

DefaultTrafficTreatment{immediate=[OUTPUT:1], deferred=[], transition=None,

cleared=false, metadata=null}, tableId=0, created=1474979205923, payLoad=null},

state=ADDED}

NetIDEPublic

DefaultFlowEntry{rule=DefaultFlowEntry{id=0, deviceId=of:0000000000000002,

priority=10, selector=[IN_PORT:1, ETH_DST:5A:4F:00:AF:E8:56], treatment=

DefaultTrafficTreatment{immediate=[OUTPUT:2], deferred=[], transition=None,

cleared=false, metadata=null}, tableId=0, created=1474979205931, payLoad=null},

state=ADDED}

3.3 ONOS Shim

3.3.1 Installing

The ONOS Shim is developed for ONOS version 1.5.1. ONOS and Shim have been tested on Ubuntu14.04 LTS, with the supporting software as detailed in Section 3.1.To perform the ONOS and Shim installations, the following steps are required:

• Download ONOS:

git clone https://gerrit.onosproject.org/onos

git checkout v1.5.1

• Set-up the environment and build the ONOS source code. Please refer to this guide: https://wiki.onosproject.org/display/ONOS/Installing+and+Running+ONOS

• To simplify the execution of ONOS, the best way is to create a ONOS cell. This guide explainsthe creation and usage of cells: https://wiki.onosproject.org/display/ONOS/ONOS+from+Scratch#ONOSfromScratch-4.Createacustomcelldefinition

• Download and build the ONOS Shim application:

git clone https://github.com/fp7-netide/Engine.git

cd onos-shim && mvn clean install -DskipTests

3.3.2 Running

Once ONOS and Shim are compiled, they are ready to be executed. First, start ONOS by usingthis command:

ok clean

and the system prompt should look like this:

Now ONOS is running but the con�guration of FlowRuleManager must be modi�ed to allow the per-sistence of �ow rules that are not installed directly by ONOS. To this end, the following commandsmust be issued in the Karaf shell of ONOS:

onos:cfg set org.onosproject.net.flow.impl.FlowRuleManager allowExtraneousRules true

Open a new BASH shell and install the ONOS Shim:

onos-app $OC1 install target/onos-app-shim-1.0.0-SNAPSHOT.oar

The ONOS Shim is now installed but not running. The next step is the con�guration of theconnection with the Core. This step is done via the ONOS shell by entering the following commands:

NetIDEPublic

config:edit eu.netide.shim.ShimLayer

property-set coreAddress {CORE_IP}

property-set corePort {CORE PORT}

config:update

Where CORE_IP is the IP address of the machine where the Core is running (by default localhost)and CORE_PORT is the TCP port being listened by the Core (by default 5555).Now the ONOS Shim can be activated by using the following command in the ONOS shell:

app activate eu.netide.shim

3.4 OpenDaylight Shim

3.4.1 Installing and running

OpenDaylight can be downloaded from the OpenDaylight Software Downloads Page: http://www.opendaylight.org/software/downloads The Karaf distribution has no features enabled by de-fault. However, all of the features are available to be installed. The only system requirement is theJava 8 runtime:Extract OpenDaylight from the archive.

1. Unzip the zip �le

2. Navigate to the directory where you decompressed the ODL Karaf distribution

3. run ./bin/karaf

4. install the netide feature

[~/distribution-karaf]$ ./bin/karaf

________ ________ .__ .__ .__ __

\_____ \ ______ ____ ____ \______ \ _____ ___.__.| | |__| ____ | |___/ |_

/ | \\____ \_/ __ \ / \ | | \\__ \< | || | | |/ ___\| | \ __\

/ | \ |_> > ___/| | \| ` \/ __ \\___ || |_| / /_/ > Y \ |

\_______ / __/ \___ >___| /_______ (____ / ____||____/__\___ /|___| /__|

\/|__| \/ \/ \/ \/\/ /_____/ \/

Hit '<tab>' for a list of available commands

and '[cmd] --help' for help on a specific command.

Hit '<ctrl-d>' or type 'system:shutdown' or 'logout' to shutdown OpenDaylight.

opendaylight-user@root>

Install NetIDE using the karaf console.

opendaylight-user@root>feature:install odl-netide-rest

Con�guration of the ODL-shim component is through an xml �le located in the ./configurationdirectory. Default attributes are pre-set, but these can be overridden if required

<transport-protocol>TCP</transport-protocol>

NetIDEPublic

<switch-idle-timeout>15000</switch-idle-timeout>

<core-address>127.0.0.1</core-address>

<core-port>5555</core-port>

3.5 Ryu Shim

The Ryu Shim is implemented as application for the Ryu controller [6] by using the Python Open-Flow libraries included in the Ryu's source code [28, 33]. The current implementation supports theNetIDE Intermediate protocol v1.4.1 (cf. Annex A) and works with the latest version of Ryu (v4.6at the time of writing).

3.5.1 Installing

First, clone the Ryu code from [28] on a Linux-based local machine (let say, in your home folder$HOME). Copy the python folder fromhttps://github.com/fp7-netide/Engine/tree/master/libraries/netip

into $HOME/ryu/ryu and rename it as netide. After that, install Ryu by running the commandfrom $HOME/ryu:

cd $HOME/ryu

python ./setup.py install

Then, add installation path for Ryu to your PYTHONPATH variable in your pro�le (i.e. either$HOME/.profile or $HOME/.bashrc) by adding the following line:

export PYTHONPATH=/usr/local/lib/python2.7/dist-packages/ryu

The path may depend on the Linux distribution and on the version of Python used for the in-stallation. Additional Python packages may be required to successfully complete the installationprocedure. On a Ubuntu 14.04 Linux OS the following Debian and Python packages must beinstalled (using sudo to gain root privileges for installation):

apt-get install python-pip python-dev python-repoze.lru

apt-get install libxml2-dev libxslt1-dev zlib1g-dev python-zmq

pip install ecdsa stevedore greenlet oslo.config eventlet WebOb Routes

3.5.2 Running

In general, the Shim is a client for the NetIDE Core. Therefore, always start the Core �rst (cf.Section 3.2), then run

ryu-manager ryu-shim.py

This command will start the Shim along with the rest of the Ryu platform. The Shim will try toconnect to a running NetIDE Core which must be already running and listening on TCP port 5555.

Note: The Ryu controller platform listens for connections from the switches on the port 6633.Therefore, when using Mininet for testing purposes, start Mininet by specifying the controllerinformation as follows:

sudo mn --topo linear,4 --controller=remote,ip=IP_ADDR,port=6633

The IP address information is not needed when Ryu and Mininet are running on the same machine.

NetIDEPublic

3.6 Testing a Shim

To test on of the Shims presented above, it is necessary to run the Core and at least one of theBackends described in the following sections. In this Section we provide a testing procedure basedon the Ryu Backend.First, start the Core by following the procedure described in the accompanying README or inSection Section 3.2 of this document and then, run either the ONOS, Opendaylight or the RyuShim.A network emulator such as Mininet can be used to test the software. In folder ryu-backend/testsof the Github repository, one can �nd script netide-topo.py that automatically con�gures Mininetwith a 4 switches and 4 hosts topology.

sudo mn --custom netide-topo.py --topo mytopo --controller=remote,ip=IP_ADDR,port=6633

Where IP_ADDR is the IP address of the machine where the Network Engine is running. It is notneeded when the Network Engine and Mininet are running on the same machine. To use OpenFlowv1.3 and test applications, add �switch ovs,protocols=OpenFlow13 to the options.This script con�gures the following topology:

alice alice-eth0:s22-eth1

bob bob-eth0:s22-eth2

charlie charlie-eth0:s11-eth1

www www-eth0:s23-eth1

s11 lo: s11-eth1:charlie-eth0 s11-eth2:s21-eth1

s21 lo: s21-eth1:s11-eth2 s21-eth2:s22-eth3 s21-eth3:s23-eth2

s22 lo: s22-eth1:alice-eth0 s22-eth2:bob-eth0 s22-eth3:s21-eth2

s23 lo: s23-eth1:www-eth0 s23-eth2:s21-eth3

Where alice, bob and www belong to a hypothetical LAN protected by a �rewall (switch s11), whilecharlie is outside the LAN.Once both Core and the Shim are running, the Ryu Backend and OpenFlow 1.0 network appli-

cations can be started with:

ryu-manager --ofp-tcp-listen-port 7733 ryu-backend.py \

tests/simple_switch.py tests/firewall.py

ryu-manager --ofp-tcp-listen-port 7733 ryu-backend.py \

tests/simple_switch_13.py tests/firewall_13.py

if you want to test the OpenFlow 1.3 applications.For instance, the composition con�guration for the Core may assign switches S21, S22 and S23

to the simple_switch application, while the S11 to the firewall application.As an alternative, one may want to test di�erent applications running on di�erent instances of

the client controller. In this case, just open two terminals and run the following commands (one foreach terminal):

ryu-manager --ofp-tcp-listen-port 7733 ryu-backend.py tests/simple_switch.py

ryu-manager --ofp-tcp-listen-port 7734 ryu-backend.py tests/firewall.py

Within the Mininet CLI, a pingall command demonstrates what tra�c is allowed and what isnot, depending on the rules installed by the �rewall application.

NetIDEPublic

3.7 Ryu Backend

The Ryu Backend is implemented as an additional southbound interface for the Ryu controller [6]by using the Python OpenFlow libraries that are part of the Ryu source code [28, 33]. It isan adaptation of the original controller.py which implements the OpenFlow SBI in Ryu. Thecurrent implementation supports the NetIDE Intermediate protocol v1.4.1 (cf. Annex A) and workswith the latest version of Ryu (v4.6 at the time of writing).

3.7.1 Installing

The Ryu Backend is provided as module for the Ryu controller. To load it, use the followingprocedure:

1. Clone the Ryu code from [28] on a Linux-based local machine.

2. Copy the python folder fromhttps://github.com/fp7-netide/Engine/tree/master/libraries/netip into $HOME/ryu/ryu

3. Rename it as netide.

4. Install Ryu by running the command python ./setup.py install from $HOME/ryu.

5. Then, add the installation path of Ryu to your PYTHONPATH variable in your $HOME/.profileor $HOME/.bashrc by adding the following line:

export PYTHONPATH=/usr/local/lib/python2.7/dist-packages/ryu

The path may depend on the Linux distribution and on the version of Python used for the in-stallation. Additional system and Python packages may be required to successfully complete theinstallation procedure.The following commands ensure that all required packages are installed on a Linux Ubuntu 14.04distribution:

apt install python-pip python-dev python-repoze.lru

apt install libxml2-dev libxslt1-dev zlib1g-dev python-zmq

pip install ecdsa stevedore greenlet oslo.config eventlet WebOb Routes

3.7.2 Running

Backends are clients for the NetIDE Core. Therefore, always start the Core �rst (cf. Section 3.2).The command below starts the Ryu Backend along with the rest of the Ryu platform. The Backendwill try to connect to a running NetIDE Core which must be already running and listening on TCPport 5555. Recall that, Ryu is not built using containers, such as Apache Karaf, that allows modulesand applications to be loaded and removed dynamically. Thus, all modules and SDN applicationsmust be included in the start up command. In the example below, we show how to run Ryu plus theBackend and two applications: a L2 forwarding (simple_switch.py) and a �rewall (firewall.py).Both applications are available in the Ryu Backend folder on github. Other applications can be usedas well. A few sample applications are available in the Ryu source tree in /home/foo/ryu/ryu/app.

ryu-manager --ofp-tcp-listen-port 7733 ryu-backend.py simple_switch.py firewall.py

The �ofp-tcp-listen-port 7733 is used to avoid possible con�icts that may happen when twodi�erent controller platforms are running on the same machine. In our case, we could have Ryuhosting the Backend and, e.g., ONOS running the Shim. By default, they both bind the TCP port6633 to accept OpenFlow connections from the network elements. Such a parameter forces Ryuto listen on a di�erent port (in this case 7733), which is anyhow never used, as the only activesouthbound interface on client controllers is the Backend.

NetIDEPublic

3.8 Floodlight Backend

3.8.1 Installing

When developing a module for Floodlight, you must download the Floodlight source code �rst andadd your module as a new namespace.In particular, follow these steps:

• Clone locally Floodlight (https://github.com/floodlight/floodlight.git) and then switchto v1.1 version:

git checkout v1.1

• Build �oodlight using make

• Copy the package net.floodlight.interceptor in the �oodlight source folder.(Copy floodlight-backend/v1.2/src/main/java/net/floodlightcontroller/interceptorfolder to floodlight/src/main/java/net/floodlightcontroller).

• Copy the jar �les under lib folder in �oodlight lib folder (except floodlight.jar used onlyduring development for compilation).

• Replace /floodlightv1.1/src/main/resources/floodlightdefault.properties with thegiven one that can be found at:/floodlight-backend/v1.2/src/main/resources/floodlightdefault.properties.

• Add to build.xml of floodlightV1.1 the following lines inside <patternset id="lib"> tag:

(or directly replace the �le with the one provided in manual/build.xml).

• Add to /floodlightv1.1/src/main/resources/META-INF/services/net.floodlightcontroller.core.module.IFloodlightModule the following line:net.floodlightcontroller.interceptor.NetIdeModule

(or manually replace the �le with the one provided in manual/IFloodlightModule).

• Compile the project with make command.

3.8.2 Running

Before starting the Floodlight Backend, there is an assumption that the Core is already runningand listening for connectionsUse the script provided in the �oodlight folder floodlight.sh to start the controller. If the

script returns an error related to java options used to start �oodlight, replace the 14 line of thescript with the following:

JVM_OPTS="$JVM_OPTS -XX:CompileThreshold=1500" #-XX:PreBlockSpin=8"

Similarly for Ryu, if Floodlight and another controller are running on the same machine, you mayreceive a �Manager address already in use� error when starting Floodlight. In this case, manuallychange the manager port used by Floodlight in Class net.floodlightcontroller.core.internal.Controller, line 136, from 6653 to 7753. (or manually replace the �le with the one provided inmanual/Controller.java and recompile the project).

NetIDEPublic

3.9 Testing Backends

3.9.1 Testing

To test a Backend it is necessary to run the Core and at least one of the Shims described in theprevious sections (either OpenDaylight, ONOS or Ryu), plus the Mininet network emulator. Forthe Ryu Backend just follows the example in Section 3.6. For the Floodlight Backend, use theinstructions in Section 3.8.2.

NetIDEPublic

4 IDE manual

4.1 Installation

This chapter explains the installation of the Eclipse-based IDE. It lists the prerequisites and after-wards shows di�erent ways to install the IDE.

4.1.1 Prerequisites

In order to use NetIDE properly, you need to have an installation of Vagrant (>v1.6), VirtualBox,and SSH on your machine. Windows users, make sure that the Vagrant and the SSH executableare referenced in your PATH variable. NetIDE works on Eclipse Luna (4.4), Mars (4.5), and Neon(4.6). We recommend using Eclipse Neon.

4.1.2 Installation via the Eclipse Marketplace

We provide an Eclipse Marketplace listing of the latest stable version to provide an easy andfast installation of the NetIDE developer toolkit. The marketplace entry can be found at http:

//marketplace.eclipse.org/content/netide. The necessary steps to install NetIDE from theMarketplace are as follows.

1. Open your Eclipse installation and navigate to Help->Eclipse Marketplace. . . in the Menubar.

2. Search for NetIDE, click Install and follow the instructions. After restarting Eclipse, NetIDEis ready to run.

4.1.3 Installation using the update site

The NetIDE plugins are also hosted at an update site which can be found at http://updatesite.netide.eu. The update site hosts the stable version as well as nightly builds. The necessary stepsto install NetIDE from an update site are as follows:

1. Open your Eclipse installation and navigate to Help->Install New Software. . . .

2. Add the repository http://updatesite.netide.eu/dependencies.

3. Enter http://updatesite.netide.eu/stable for a stable installation or http://updatesite.netide.eu/nightly for nightly builds.

4. Select NetIDE, click Next and follow the instructions. After restarting Eclipse, NetIDE isready to run.

4.1.4 Installation by building from source

NetIDE can also be compiled from source using the build automation system Maven[32]. Thecompilation and installation consists of the following steps:

1. Check out the master branch on https://github.com/fp7-netide/IDE.

NetIDEPublic

2. Navigate to the folder releng/eu.netide.dependencies and type mvn p2:site. This willbuild an Eclipse repository for external dependencies at releng/eu.netide/dependencies/target/repository.

3. Navigate to the folder releng/eu.netide.parent and type MAVEN_OPTS="-Xmx1024M" Mavenpackage. The build process can take some time.

4. Once it has �nished, you'll �nd the contents of an Eclipse update site as a zip �le inreleng/eu.netide.product/target.

5. Open your Eclipse installation.

6. Go to Help->Install New Software->Add->Local and specify the location of the depen-dency repository.

7. Go to Help->Install New Software->Add->Archive and specify the location of the zip �le.

8. Select NetIDE from the list and follow the instructions. After restarting Eclipse, NetIDE isready to run. Cool.

4.2 Usage

NetIDE consists of various tools to assist the development, composition, deployment and testing ofSDN network applications. Namely, there are the following features: (i) managing the applicationformat for a NetIDE package via various helper functions, (ii) network topologies can be createdand edited using the graphical topology editor, (iii) network applications can be annotated withcon�gurable parameters and �nally run and tested out of the IDE on remote and virtual machines.We are going to explain each of these features in the following subsections.

4.2.1 Application format management

NetIDE provides a set of tools to manage the SDN modules composing the NetIDE network appli-cation in the NetIDE application format (see Chapter 3 in Deliverable D2.4 [34]). The functionsnecessary for that are activated in the NetIDE perspective. The perspective can be opened viaWindow->Perspectives->Other->NetIDE.A new NetIDE Project providing a skeleton of the application format can be created using the

NetIDE new project wizard. Select File->New->NetIDE Project and follow the instruction ofthe wizard. This creates a new project in the Eclipse workspace with the initial �les to compose aNetIDE app.New SDN modules can be added via the app wizard. Open the app wizard by right-clicking the

project folder and selecting New->Other->NetIDE->NetIDE App. Enter a name and a controller forthe module. This creates a folder containing a system requirements speci�cation and a parameterschema (see Subsection 4.2.2).

4.2.2 Specifying and using parameter schemas

NetIDE app developers can specify the con�gurable parameters for SDN modules. Every appcreated with the app wizard contains an empty parameter schema �le. This �le can be edited in theparameter schema editor which is also a part of NetIDE. Listing 4.1 shows a simple parameter schema�le which de�nes the parameters of a �rewall SDN modules. Each parameter in the parameters

section is de�ned by a name, a type and a default value. The complex type IP is de�ned in thetypes section as a string matching a certain regular expression.

NetIDEPublic

Figure 4.1: The application format project tree.

Listing 4.1: Parameter Schema

parameters: {

stateful_idle_timeout: "Integer = 5",

port_web: "Integer = 80",

host_web: "IP = '10.0.0.10'",

port_dns: "Integer = 53"

types: {

type: "String",

regex: "\"\\b\\d{1,3}\\.\\d{1,3}\\.\\d{1,3}\\.\\d{1,3}\\b\""

This �le can now be used for easy, platform-independent app con�guration. The graphicalform for parameter con�guration can be invoked by right-clicking the project folder and select-ing NetIDE->Configure Parameters. This brings up a GUI in which values can be assigned toparameters. The con�gured values for all con�gured apps are stored in a parameters.json �le.Figure 4.2 shows a con�guration GUI generated from the parameters �le in Listing 4.1 and a shortparameter �le for a simple MAC learner. The form entries are already �lled with the default values.Mapping of the values in parameters.json to concrete con�guration �les can be achieved by

creating con�guration �le templates. The following steps are needed to create a template for acon�guration �le.

NetIDEPublic

Figure 4.2: Parameter con�gurator.

1. Select a �le form an app and select NetIDE->Copy to Templates. A copy of the templatewill appear in the templates folder.

2. Edit the �le in the templates folder and enrich it with template tags. The tags must match theparameter names in the parameter schema �les as they will be substituted by the parametervalues later on. The syntax of the tags has to comply to the handlebars speci�cation [35].

With the templates and the parameters.json �le at hand, NetIDE can generate platform-speci�ccon�gurations for the SDN module in the package. The �les can be rendered by right-clicking theproject folder and selecting NetIDE->Render Templates.

4.2.3 Con�guring Network Topologies

NetIDE provides tools to test composed network applications on virtual machines and physicalmachines optionally with a virtual network. This section explains the editors needed to con�guretest topologies and environments.Setting up virtual topologies is supported by the Topology Editor. This editor supports the

graphical creation of SDN-enabled network topologies. Figure 4.3 shows a screenshot of the editor.There is a canvas on the left-hand side and a palette on the right-hand side. Network models can becreated by clicking on an element in the palette and clicking on the canvas. The �rst step towardscreating a network model is to add a network (white rectangle). A network contains switches (lightblue rectangles) and hosts which can be connected via ports and connectors. Choosing a connectorfrom the palette and subsequently clicking on two switches or switch and a host creates a port oneach switch and connects them. In order to create a controller to manage the switches, a controllerneeds to be added to the canvas (not a network) and connected to the switches it is supposed tomanage.Each element has properties such as a name, an IP address, or a datapath ID. These properties

can be edited by right-clicking on an element and selecting Properties. This opens a view in whichall properties of an element can be edited. The tab Semantics shows the aforementioned semanticproperties.

NetIDEPublic

Figure 4.3: NetIDE Topology Editor.

The topology model is stored as an EMF [36] compliant model. In order to create a runnablecon�guration for the Mininet network emulator for testing, expand the topology model in the projecttree, right-click on the Network Environment element and select Generate Mininet Configuration.This stores a python-based Mininet con�guration in the folder gen/mininet. The virtual networkcan be set up by running the python script <projectname>_run.py.

4.2.4 Testing network applications

Network applications can be executed directly from the NetIDE developer toolkit on top of theNetIDE Engine using a dashboard (cf. Figure 4.4). The dashboard can be shown by opening the�le <projectname>.wb in the project tree. Alternatively, such a �le can also be created using therespective wizard under New->Other->NetIDE->NetIDE Configuration Manager.Figure 4.4 shows the user interface of the dashboard. The elements of this interface are described

below:

1. The upper section of the UI shows the settings and controls to manage a connection to amachine on which the apps are executed. Users can create establish an SSH connection to aremote machine or set up a Vagrant-powered virtual machine.

2. The Network Engine's Core can be started by pressing the On button in the respective section.The Off button switches it o�. Starting the Core opens an integrated terminal in the IDE. Ifthe terminal closes, it can be displayed again using the Reattach button.

3. The Core's behaviour can be speci�ed via a composition speci�cation �le(Section 2.1.1.8).This �le can be loaded inside of the IDE by selecting a composition speci�cation �le andloading it into the Core by pushing the Load button.

4. Di�erent Shims can be started up via the IDE. Pushing the button Import Topology fetchesthe underlying topology of the Shim if the speci�c Shim supports it. The imported topologyis stored as imported.topology in the current project root.

NetIDEPublic

Figure 4.4: SDN network application testing dashboard.

5. Mininet can be started, stopped and reattached to a terminal by pushing the On button inthe Mininet section.

6. The topology section can be used to specify the topology used by Mininet. The Generate

button generates a Mininet con�guration from the topology speci�cation.

7. The On button starts the debugger which logs the Core messages in a human-readable format.

8. The Apps section contains a list of SDN modules to run on the NetIDE engine. New modulescan be added to the composed netowkr application via the Add button. This opens a dialog inwhich the controller platform, the entry point, and the OpenFlow port can be con�gured, aswell as whether it should run on the NetIDE engine. The Start, Stop, Reload, and Reattach

buttons are used to start, stop, and restart an app, as well as reattach a terminal to a runningapp.

4.2.5 Monitoring the network application

The NetIDE developer toolkit provides a ZeroMQ [15] interface to the NetIDE Core which listensto messages sent by the LogPub module. A basic list-based visualization of the messages can

NetIDEPublic

be invoked by the ZMQ Listeners view. This view can be opened by selecting Window->Show

View->Other->NetIDE->ZMQ Listeners. This opens up a list of ZeroMQ listeners. Right-clickingon a listener in the list and selecting Toggle activates the listener and double-clicking a listeneropens up a log window containing the logpub messages.

Figure 4.5: Logpub interface and view.

4.2.6 Global Con�guration

There are several global options to con�gure the NetIDE developer toolkit. The following listexplains the available parameters.

Figure 4.6: Global con�guration view.

1. The executable paths for Vagrant, SSH, and SCP are speci�ed in the �rst section. Theseparameters must have a value since the testing feature heavily uses these three tools. Windows

NetIDEPublic

users have to specify the paths manually, OSX and Linux users have default values for theseparameters which can be changed if necessary.

2. This section enables users to de�ne a proxy server for their test VMs. This server con�gurationis written into the shell pro�le as well as into the package manager con�guration of the VMduring the VM provisioning phase.

3. Custom boxes can be used to skip the provisioning process by providing an already provi-sioned VM image. This way, an image can be created once and reused among App testers.A provisioned VM can be exported to an image by navigating into the folder containingthe Vagrant�le and typing vagrant package �name <name> where <name> denotes the givenname of the box. The box can be copied on another machine added via vagrant box add

<filename>. The given name of the box can then be entered into the line in this section ofthe con�guration GUI.

4. The section on the bottom of the GUI contains pre-con�gured ZeroMQ listeners. All listenersare de�ned with the format <name>,<address>.

4.2.7 NetIDE Application Loader

This tool validates and runs deployed NetIDE packages on client controllers. It starts all controllersand apps as well as the Core in a detachable tmux session.

4.2.7.1 Setup

Use the script loader/setup.sh in the Engine repository for bootstrapping. Cleanest way to dothe setup is to either put it inside a docker container or a virtual machine. The scripts assume arecent 64-Bit Ubuntu as the OS.

4.2.7.2 Install Package

Before starting an package, it needs to be installed �rst. To do so run the loader/startLoader.shscript with the install command which will install all the prerequisites to make the containednetwork application runnable. Example usage: ./startLoader.sh install /path/to/package.The package can either be a folder or an archive in the tar.gz format. If an archive is used it willbe extracted to /tmp/netide/.

4.2.7.3 Available Commands

Every command is used through the startLoader.sh. Currently there are the following commandsavailable:

• install /path/to/package/ used to setup the chosen package for running.

• run /path/to/package/ --server --param runs the given package. Server is optional, choosefrom ODL and ryu. Choose ryu for ryu, else ODL will be used. param is optional, a parameter�le can be speci�ed with a con�guration used for this run.

• extractionPath /path/to/extract de�nes a path to which packages should be extracted.

• extractArchive /path/to/Archive used to extract the given archive to the extraction path.Default is /tmp/netide

NetIDEPublic

• generate /path/to/package --param generates the param con�guration speci�ed in the �legiven with �param.

• list lists all running SDN controllers.

• stop stops all running SDN controllers.

• attach the packages are started in tmux sessions. If there are any detached sessions availablethis command attaches them.

NetIDEPublic

5 Tools manuals

In this chapter, we describe how to install and execute the di�erent tools that NetIDE project o�ers.The main purpose of the tools is to diagnose, debug and troubleshoot the data and control planesof the SDN network. NetIDE Engine is connected to these tools thanks to the LogPub, an interfacewhich is in charge of capturing the messages exchanged among the client controllers and the servercontroller, and pushing them into a queue. The tools will retrieve the enqueued messages and willcarry out their task. The LogPub provides the capability to connect new tools to the Engine andit o�ers a complete independence of the SDN technology used.Although, most of the tools perform very di�erent tasks (Pro�ler whose aim is to show relevant

information of the control and data planes, the Tra�c Emulator which simulates several tra�cpatterns or the Memory Management System that optimizes the memory consumption of the de-vices), there are also tools which are tightly related as they share part of the code: the Logger, theDebugger and the Veri�cator. The �le debugger.py is in charge of obtaining all the distributedmessages enqueued by the LogPub. Although called debugger.py, it serves the three tools at thesame time, let us explain how:

• It prints all the messages received on runtime and generates a plan text �le for the Logger:results.txt

• It creates a PCAP [37] �le for the Debugger: results.pcap

• It produces a postcard �le for the Veri�cator: results.card

Later on, such tools can be used to analyse the output �les.All the code of the tools can be found at https://github.com/fp7-netide/Tools, where the

master branch is the most updated and stable version of it.

5.1 Logger

The Logger is in charge of retrieving all the messages that are exchanged between the di�erentmodules of the Engine and display them into a terminal. This tool lets the user know what ishappening in the network at runtime. It also saves all logs in a plain text �le results.txt foro�ine checking.

5.1.1 Installing

For this purpose, it is necessary that the Logger has access to the queue where the messages arekept. The queue mechanism is provided by ZeroMQ [15], which therefore must be installed. Theo�cial site of ZeroMQ provides a guide which explains how to install it. Here we summarize howto install it in Ubuntu 14.04:

• Make sure that libtool, pkg-config, build-essential, autoconf, and automake are in-stalled.

• Check whether uuid-dev package, uuid/e2fsprogs RPM or equivalent on your system is in-stalled.

NetIDEPublic

• Download one of the version of ZeroMQ:

wget http://download.zeromq.org/zeromq-4.1.2.tar.gz

• Unpack the source archive:

tar -xvf zeromq-4.1.2.tar.gz

• Install ZeroMQ:

cd zeromq-4.1.2

./autogen.sh ./configure && make check

sudo make install

sudo ldconfig

5.1.2 Running

The whole logic of the tool is implemented in one python script, therefore executing such script isenough to activate the Logger. The following command is used to run the Logger and the Debugger,as both tools are implemented with the same script:

python debugger.py

The user will be able to visualize the messages exchanged between the network and the SDNmodule composing the NetIDE network application. The green messages belong to the data planeand their destination is the controller, while the blue ones are sent from the control plane to thedatapath elements.

Figure 5.1: Logger trace.

5.2 Debugger

The Debugger saves all captured logs from the LogPub, and thus tra�c exchanged in the Ne-tIDE Network Engine, in a �le called results.pcap. As mentioned before, it shares the runtimecode debugger.py with the Logger and the Veri�cator. The di�erence is the resulting �le for lateranalysis with Wireshark [38]: results.pcap.Additionally, we have developed a NetIDE Wireshark dissector, a plugin for Wireshark to analyse

the messages encapsulated with the NetIDE header (see Annex A), either at runtime or for o�ineanalysis of the debugger's output results.pcap.

NetIDEPublic

5.2.1 Installing

For the installation, we will distinguish between two components: the Debugger and the Ne-tIDE Wireshark dissector. For both cases, we need the source code of Wireshark, because weare going add new dissectors to it.Therefore, �rst we install Wireshark's sources and we check that everything works correctly.

Typical dependencies on a Ubuntu 14.04 Linux OS to be installed before the compilation are: git,autoconf, automake, libtool, bison, flex, qt-sdk, libgtk-3-dev, libpcap-dev. However,please refer to the Wireshark documentation [38] for more detailed information. Then do as follows:

• Go to the source directory and con�gure your source so it will build correctly for your versionof UNIX:

./autogen

./configure

• Build the sources:

• Install the software in its �nal destination:

make install

• Finally, if you have the following error when running Wireshark:

wireshark: error while loading shared libraries: libwiretap.so.4: cannot open

shared object file: No such file or directory

then run the following command:

sudo ldconfig

5.2.1.1 Debugger

As the Logger, it requires the installation of ZeroMQ (follow the guide in 5.1) to run debugger.py.After this, we need to copy some �les from https://github.com/fp7-netide/Tools/tree/master/

debugger/Core and follow the steps below:

• Copy packet-user_encap.c in the epan/dissectors/ folder of the Wireshark sources. Thenrecompile Wireshark with the following three commands (where the �rst one speci�es thelocation of the compiled Wireshark binary):

./configure --prefix=<location-of-your-wireshark-installation>/wireshark

make install

• Copy the �les netide.so and netide.la in the lib folder of your Wireshark installation.Normally the path of the plugin library for most Wireshark installations from source codeis /usr/local/lib/wireshark/plugins/x.y.z (where x.y.z is the speci�c version you in-stalled).

NetIDEPublic

Currently, the packet-user_encap.c automatically associates the DLT_USER0 link-layer headertype with the NetIDE protocol. If for some reason this does not work, the �rst time you runWireshark you must do it manually. Normally, Wireshark tells you to do so at start-up, but if not,you can follow the instructions in: https://wiki.wireshark.org/HowToDissectAnything to tellWireshark how to decode our user link-layer header type, which consists of going to:Edit→Preferences→Protocols>DLT_USER→Edit Encapsulations Table

and adding an entry for NetIDE , according to: https://www.wireshark.org/docs/wsug_html_

chunked/ChUserDLTsSection.html

5.2.1.2 NetIDE Wireshark dissector

In the case of the simple dissector, the installation is much easier. We need to copy two �les fromhttps://github.com/fp7-netide/Tools/tree/master/wireshark-NetIDEdissector that containthe dissector and do as follows:

• Copy the �les netide.so and netide.la in the lib folder of your Wireshark installation.Normally the path of the plugin library for most Wireshark installations from source codeis /usr/local/lib/wireshark/plugins/x.y.z (where x.y.z is the speci�c version you in-stalled).

5.2.2 Running

Although the installation of the Debugger and the Wireshark dissector contains some steps in com-mon (as they both share the use of Wireshark libraries), we use di�erent approaches for runningthem. The Debugger needs the LogPub to attach to the Network Engine, while the Wireshark dis-sector captures the tra�c on any network interface of the machine where it is running, included themessages exchanged between the NetIDE components but without requiring the LogPub interface.

5.2.2.1 Debugger

To run the Debugger, you need to execute the following command once the NetIDE Engine isrunning:

python debugger.py

Once we stop the execution, a results.pcap �le will be generated, which can be opened withthe Wireshark installation we prepared in the previous sections. Figures 5.2 and 5.3 show examplesof the results.pcap inspected with Wireshark and the NetIDE dissector.

5.2.2.2 NetIDE Wireshark dissector

Open Wireshark and capture any tra�c in the network. Usually the components of the NetworkEngine communicate via the localhost interface using the NetIDE intermediate protocol when allits components are running on the same machine. The capture will show something like Fig. 5.4,where 127.0.0.1 is the IP address of the localhost interface.The advantage of using the NetIDE Wireshark dissector is that the installation is much simpler

and it does not require to directly interface with the Network Engine. However, the drawback isthat it indirectly requires the ZeroMQ dissector, which produces errors when a ZeroMQ messagecontains more than one NetIDE message.

NetIDEPublic

Figure 5.2: Debugger's output with OpenFlow 1.0.

Figure 5.3: Debugger's output with OpenFlow 1.3.

Figure 5.4: NetIDE Wireshark dissector.

5.3 Pro�ler

In software engineering the term pro�ling is known as the performance analysis of a program. Itsobjective is to study the behavior of a software in order to detect failure points or areas wherea performance optimization is needed. In this project, the concept of pro�ling is applied to SDNnetworks, implementing an SDN Application Pro�ler. Such a tool is focused on collecting relevantstatistics from the control plane, as the execution time of the applications executed on top of thecontroller platforms.

NetIDEPublic

5.3.1 Installing

The Application Pro�ler, is based on the pro�ling concept of software engineering. The goal ofthis approach is to obtain the execution time of the methods executed in a controller and in theirapplications.Firstly, it is necessary to install the R software [39], a software environment for statistical com-

puting. In this section, we explain how to install the R software in Ubuntu 14.04 LTS, for otheroperating systems please refer to the project's web page. As a �rst step, add repository debhttp://ftp.cixug.es/CRAN/bin/linux/ubuntu trusty/ to the list of software channels in /etc/apt/-sources.list �le. Secondly, execute the next commands in a terminal:

sudo apt-get update

sudo apt-get install r-base

This will install the latest version of R (otherwise, some packages might be missing and the visual-ization of data of the pro�ler might not work correctly). Note: If you obtain a GPG error when tryingto update apt-get, you might want to check this link: http://askubuntu.com/questions/13065/how-do-i-fix-the-gpg-error-no-pubkey which basically solves it by installing y-ppa-managerand then executing Advanced->Try to import all missing GPG keys.Additionally, users who need to compile R packages from source [e.g. package maintainers, or

anyone installing packages with install.packages()] should also install the r-base-dev package:

sudo apt-get install r-base-dev

Installation and compilation of R or some of its packages may require Ubuntu packages from the�backports� repositories. Therefore, it is suggested to activate the backports repositories with anentry like: deb https://<my.favorite.ubuntu.mirror>/ trusty-backports main restricted universe.Finally, install RStudio (a GUI for R) [40]. This is optional, but eases for the visualization of the

results.

5.3.2 Running

Here we show how to pro�le the Ryu controller and its SDN modules when running in the contextof the Network Engine. Compared to Section 3.7, the command for executing the Ryu controller,Backend and SDN modules includes additional parameters and becomes as follows:

python -m cProfile -o statistics ryu/bin/ryu-manager --ofp-tcp-listen-port 7733

$NETIDE/Engine/ryu-backend/ryu-backend.py \

$NETIDE/Engine/ryu-backend/tests/simple_switch.py

The output of this command is a text �le called statistics.txt which contains the executiontime of all the functions under pro�ling. Statistics can be analysed with R by running commandprofiler_graphics.R or with the RStudio Graphical User Interface (GUI) by double clicking onit (Figure 5.5).

5.4 Tra�c Emulator

This tool is composed of applications which can be used to generate tra�c for the Emulator.The current applications available are:

• http: a client-server application which simulate HTTP tra�c from a host to a server. Thetra�c patterns are similar to what can be found in this work [41].

• pcap: an application which takes .pcap �les like those who can be found in .pcap reposito-ries [42] [43] or using an external tool like DCT2Gen [44], and create traces playable by a toollike tcpreplay [45].

NetIDEPublic

Figure 5.5: Application Pro�ler GUI.

• tftp: a client-server application which simulate TFTP tra�c.

• iperfp: a application which uses iperf to simulate common �ow patterns (VoIP, video, ...) interms of packet size and bandwidth.

5.4.1 Installation

To get the Tra�c Emulator application, clone the NetIDE Tools repository.

git clone https://github.com/fp7-netide/Tools.git

cd traffem

The following packages are used as system requirements for the di�erent applications:

http : wget, python

pcap : tcpreplay, tshark, wireshark

tftp : tftp, python

iperfp : iperfThey can all be installed with

sudo apt-get install [name_of_package]

5.4.2 Running

5.4.2.1 http

This application is composed of two scripts, one to run on server(s) and another one on client(s).On the server(s) run:

./HTTPserver.sh

NetIDEPublic

you can quit by either pressing Enter of Ctrl+c. The server will create multiple �les in a foldercalled HTTP_�les. This folder will be deleted upon quitting the application.On the client(s) run:

./HTTPclient.sh <serverIP>

you can quit by either pressing Enter of Ctrl+c. The client will connect to the provided IP and sim-ulate the tra�c from a user. The user will browse the internet and send/receive email (mandatorypatterns). It will also randomly use a media pattern between listening to a web radio, stream musicor watch a video. It will also randomly use a �le synchronization service. The patterns availableare:

Mandatory pattern(s):

Internet: Simulate the loading of a web page (size 2MB)

and wait for 5-60s to simulate reading

Email: Simulate sending and receiving an email (75KB)

continuously

Random pattern(s):

Radio: Simulate a web radio. This pattern will

download a file at a low rate (20KB/s) constantly.

Music: Simulate a streaming service. This patten will

download a "song" (3-5MB file) and wait before

loading another "song" (3-5mn)

Sync: Simulate the synchronization of data

Video: Simulate the browsing and streaming of a video

all files created on the host will be deleted upon closing the application.

5.4.2.2 pcap

You can use the �le test.pcap and run

./producePCAP -t

to produce three pcap �les. Otherwise to use the application run:

Usage ./producePCAP -p <.pcap> -i <source IP> -m <source MAC> -f <host(s) file>

<.pcap>: the .pcap file

<source IP>: the IP of the desired host from the .pcap

<source MAC>: the MAC of the desired host from the .pcap

<host(s) file>: a file with the output host in this format:

HOSTNAME HOST_IP HOST_MAC

Example:

./producePCAP -f hosts.txt -p test.pcap -i 172.16.11.12 -m f8:1e:df:e5:84:3a

NetIDEPublic

With host.txt containing:

H1 10.0.0.1 04:08:15:16:23:42

H2 172.19.3.15 00:01:46:56:62:96

H3 192.168.3.15 00:09:53:59:84:87

When the .pcap �les are ready, use tcpreplay on the hosts command line. For example, to replaythe .pcap once out of eth1:

tcpreplay -i eth1 -K test.pcap

To replay the .pcap 5000 times1 at line rate:

tcpreplay -i eth1 -t -K --loop 5000 test.pcap

To replay the .pcap 5000 times and limiting the rate at 50 Mbps:

tcpreplay -i eth1 --mbps=50 -K --loop 5000 test.pcap

See the tcpreplay help for more options.

5.4.2.3 tftp

This two part application is composed of two scripts, one to run on server(s) and another one onclient(s).On the server run:

./tftpServer.sh

On the client(s) run:

./tftpClient.sh <serverIP> [test|small|medium|big]

By default the test �le is downloaded. The other available �les are small (1MB), medium (10MB)and big (100MB).

5.4.2.4 iperfp

This two part application is composed of two scripts, one to run on server(s) and another one onclient(s).On the server run:

./iperfp.sh --server

On the client(s) run:

./iperfp.sh <serverIP> [--imix|--voip|--video]

Options:

--server: run in server mode

--imix: play the Internet Mix pattern.

7 packets of 40B 4 packets of 576B 1 packet of 1500B

--voip: play a VoIP pattern

(G.726 audio codec) - 130B packets @ 55KB/s

--video: play a video straming pattern

(H.264 video codec) - 1200B packets @ 471KB/s

1tcpreplay does not replay by duration so the total time would be the duration of the .pcap times the �loopargument.

NetIDEPublic

5.5 Memory Management System

The Memory Management System (MMS) is a tool for SDN environments that aims at improvingthe usage of the memory of the network devices and, as a consequence, the robustness and theperformance of the network.The MMS automatically removes the �ow entries installed by applications that are no longer running(we called this feature memory de-allocation) and automatically moving the less used wildcardentries to a slower memory when the devices' TCAM is full (memory swapping).The MMS has been implemented as a bundle for the ONOS platform. However, as explained

in [46], its architecture is generic enough to be implemented for other platforms such as OpenDay-light and Ryu. Thus, the MMS has been designed to provide memory management techniques tothe Network Engine as a tool executed in the server controller.

5.5.1 Installation

The MMS is implemented as a bundle for ONOS version 1.5.1. First, the following general require-ments are necessary to run ONOS and the MMS on Ubuntu 14.04 LTS:

• Java 8 JDK

• Apache Maven 3.3.9

• git

In order to perform the installation of ONOS and MMS, the following steps are required:

• Download ONOS:

git clone https://gerrit.onosproject.org/onos

git checkout v1.5.1

• In order to setup the environment and build the ONOS source code, please refer to this guide:https://wiki.onosproject.org/display/ONOS/Installing+and+Running+ONOS

• To simplify the execution of ONOS, the best way is to create a ONOS cell. This guide explainsthe creation and usage of cells: https://wiki.onosproject.org/display/ONOS/ONOS+from+Scratch#ONOSfromScratch-4.Createacustomcelldefinition

• Download and build the MMS:

git clone https://github.com/fp7-netide/Tools.git

cd memory-management-system/onos && mvn clean install

• Install the MMS:

onos-app $OC1 install target/onos-app-mms-1.0.0-SNAPSHOT.oar

5.5.2 Running

Once ONOS and the MMS are compiled, they are ready to be executed. In order to start ONOS,run the following command from a terminal:

ok clean

and the Command Line Interface (CLI) should look like this:

NetIDEPublic

Now ONOS is running, so the MMS can be installed and activated from another terminal:

onos-app $OC1 install! target/onos-app-mms-1.0.0-SNAPSHOT.oar

At this point, the MMS is running. The memory swapping is automatically performed whenthe SDN device noti�es a table full message. To get information about the number of �ow rulesswapped out from a SDN device, this command can be used in the ONOS CLI:

onos> mms:swapsize {device_URI}

The user can tune the con�guration of swapping threshold, which is the amount of �ow rulesdeleted by the swapping out operation. The following command can be used:

onos> cfg set eu.netide.mms.MMSManager flowDeletionThreshold {value}

For example, to swap the 20% of �ow rules from a SDN device, value is 0.2.The memory deallocation is not automatically activated at the MMS start-up. The network

administrator can activate it for particular SDN applications running on ONOS. This commandregisters an application to MMS deallocation:

onos> mms:addtomms {application_URI}

In order to unregister an application use this command:

onos> mms:removeapplication {application_URI}

NetIDEPublic

6 NetIDE framework at work

We have presented the NetIDE framework at work in di�erent conferences. In this chapter wedescribe the setup and considerations for the demonstration we did at the NetSoft'16 [47] conference.To provide a comprehensive example of how the NetIDE engine can be used, we expand it to coveradditional component(s) that are not treated in depth in the demonstration paper. We defer anin-depth discussion of how to implement the project use cases to the �nal WP5 deliverable.

With NetIDE, we have envisaged a network application development work�ow centred on (i)the IDE, a tool for developers to implement, deploy and test new SDN applications, and (ii) theNetwork Engine, where SDN applications can be tested on an emulated environment or executedon a real SDN network. Below, we describe the scenario where a developer leverages NetIDE toimplement, con�gure and test a network application in an emulated environment before the actualdeployment on a production SDN network. In this scenario, we also show how IDE and NetworkEngine support the dynamic recon�guration of the Network Application to provide new networkservices.

1. We �rst use the NetIDE Integrated Development Environment to design the network topologyfor the network emulator and to build a network application by means of the implementationand composition of two or more modules.

2. Once the NetIDE application is ready, we start the testing phase. Here, the Network Engineis con�gured to meet the network application requirements in terms of client controllers andcomposition speci�cation. In this phase, Mininet loads the con�guration �le produced withthe IDE which contains the topology details de�ned by the developer (Section 4.2.3). Thenetwork application can be tested with any topology con�guration to verify its behaviour indi�erent conditions.

3. We show how di�erent modules, written for di�erent controller frameworks, cooperate as asingle network application in controlling the network.

4. We modify the composition speci�cation by adding or removing modules from it, then wereload the composition speci�cation, and then see how the corresponding modules are start-ed/stopped and the network behaviour changes accordingly. The process, up to this point,

Topology Design Code Editor

Composition Specification

Engine Configuration Network Application

Network Engine

Composition Spec.

Mininet

Network Application

NetIDE Development Environment

Learning Switch

PY/JavaFloodlight

Firewall Ryu

Monitor Floodlight

Server Controller ONOS

FloodLight Ryu

Virtual Machine

Deployment

Runtime

Load Balancer Ryu

DOS Protection Ryu

Load Balancer

Monitor

NetIDE Core

DOS prot.

Figure 6.1: Demonstration work�ow.

was demonstrated at Netsoft'16 and its rationale is shown in Figure 6.1.

NetIDEPublic

6.1 Demo scenario

To demonstrate the lifecycle described above, we will use the setup we demostrated at Netsoft'16(Figure 6.2). The initial con�guration includes a Local Area Network (LAN) controlled by a Layer-2 forwarding module and protected against unauthorized access by a network element acting as aFirewall. Tra�c injected into the LAN is monitored by an SDN module which intercepts relevantcontrol messages and displays them on a terminal.

Figure 6.2: Demonstration scenario.

Design and Development: The aforementioned scenario is prepared from the NetIDE Develop-ment Environment (white elements on left side of Figure 6.1), where we design the topology, selec-t/modify a set of SDN application modules (Learning Switch, Firewall and Monitor in the �gure),and �nally we prepare the con�guration for the Network Engine. Application modules are writteneither in Java or in Python and implemented for di�erent controller platforms (in this demo we useFloodlight and Ryu as client controllers). Such modules will cooperatively control the network as asingle network application thanks to the Composition Speci�cation, a set of XML-encoded policiesthat determine how the NetIDE Core coordinates the modules (Section 2.1.1.8).The output of this phase consists of: (i) a con�guration �le for Mininet, (ii) a con�guration �le for

the Network Engine (the Composition Speci�cation in Listing 6.1), and (iii) the network applicationas a set of modules written for Ryu and Floodlight.

Deployment: The output of the previous phase is deployed onto a pre-con�gured Virtual Machinewhere all the required components are con�gured and started. As represented by white elementson the right side of Figure 6.1, Mininet loads the topology model produced by the IDE and theNetwork Engine is assembled with the required controller frameworks, i.e., ONOS as server controllerand Ryu and Floodlight as client controllers along with the required SDN modules. As soon asthe NetIDE Core loads the Composition Speci�cation, the Network Engine starts controlling thenetwork through the network application based on the policies de�ned in the speci�cation. TheComposition Speci�cation used for this speci�c use case is shown in Listing 6.1, where switcheswith datapath_ids 0x21, 0x22, 0x23 and 0x24 within LAN and DMZ are controlled by Floodlightvia the LearningSwitch module. Switch 0x11 acts as a �rewall and is controlled by Ryu by means

NetIDEPublic

of the Firewall module. Tra�c injected in switch 0x22 is also monitored via the Monitor modulefor Ryu which is composed in parallel with LearningSwitch.With this con�guration, hosts inside the LAN can communicate with each other, the LAN is

protected from undesired tra�c and it is also monitored to discover possible security breachescoming from trusted users.

Listing 6.1: Initial con�guration.

1 <?xml version="1.0" encoding="utf-8"?>

2 <CompositionSpecification xmlns="http://netide.eu/schemas/specification/v1">

3 <Modules>

4 <Module id="SimpleSwitch" loaderIdentification="LearningSwitch.jar">

5 <CallCondition events="packetIn" datapaths="0x21 0x22 0x23 0x24"/>

6 </Module>

7 <Module id="Monitor" loaderIdentification="monitor.py">

8 <CallCondition events="packetIn" datapaths="0x22"/>

9 </Module>

10 <Module id="Firewall" loaderIdentification="firewall.py">

12 </Module>

13 </Modules>

14 <Composition>

16 <ModuleCall module="SimpleSwitch" priority="1"/>

17 <ModuleCall module="Monitor" priority="2"/>

18 </ParallelCall>

19 </Composition>

Runtime: The NetIDE framework can be also used to activate new network services at runtimeto improve, for instance, performance and security of a production network. Referring to the redelements in Figures 6.1 and 6.2, the IDE can be used to re-con�gure the Network Engine whichis currently controlling the network to satisfy new requirements without the need of restarting theNetwork Engine. As an example, we add a Load Balancer module to spread the users' requests overdi�erent web servers, we install another module that cooperates with the Firewall on protecting thenetwork from Denial of Service (DoS) attacks, and we move the monitoring module from one switchto another.

Listing 6.2: Updated composition speci�cation.

1 <?xml version="1.0" encoding="utf-8"?>

2 <CompositionSpecification xmlns="http://netide.eu/schemas/specification/v1">

3 <Modules>

4 <Module id="SimpleSwitch" loaderIdentification="LearningSwitch.jar">

5 <CallCondition events="packetIn" datapaths="0x21 0x22 0x23"/>

6 </Module>

7 <Module id="SimpleSwitch" loaderIdentification="LoadBalancer.jar">

9 </Module>

10 <Module id="Monitor" loaderIdentification="monitor.py">

12 </Module>

13 <Module id="Firewall" loaderIdentification="firewall.py">

15 </Module>

16 <Module id="DosProtection" loaderIdentification="dos.py">

18 </Module>

NetIDEPublic

19 </Modules>

20 <Composition>

22 <ModuleCall module="SimpleSwitch" priority="1"/>

23 <ModuleCall module="Monitor" priority="2"/>

24 <ModuleCall module="DosProtection" priority="1"/>

25 <ModuleCall module="Firewall" priority="2"/>

26 </ParallelCall>

27 </Composition>

Listing 6.2 shows the �nal Composition Speci�cation. At runtime, it has been updated (either inone single step or with multiple incremental changes) by (i) assigning the control of switch 0x24 tothe LoadBalancer module written for FloodLight, (ii) moving the Monitor module to switch 0x23and composing in parallel DosProtection written for Ryu with Firewall on switch 0x11.

NetIDEPublic

7 Advanced architectural concepts

Composition of SDN applications has been one of the main topics of NetIDE . At the end of theproject, we step back for a moment and look at the problem from a more formal and generalisedperspective.

7.1 De�nitions

We can see a software-de�ned network as a collection of interconnected nodes (switches) forming agraph G = V (V,E), where E describes the connectivity between the switches. We de�ne the stateNv of a node as the state of its Forwarding Information Base (FIB). A FIB entry is de�ned as tuple(p,m, i) specifying a priority, the packets to match and a list of instructions i.1 The network stateN is then de�ned as the collection of all node states

N = {Nv | v ∈ V }

The network state can be changed by a command C. A command is a sequence of basic commandsC = [c1, c2, . . .] with cj ∈ {Finst, Fdel} that modify the FIB of a switch:

• Finst = (v, p,m, i): Install a FIB entry on node v with priority p, match m and list of instruc-tions i.

• Fdel = (v, p,m, i): Remove the FIB entry that was installed by Finst(v, p,m, i)

We de�ne the function a : N × C → N as the function that applies a network command to anetwork state and produces the new network state:

a(N, [c1, . . . , ck]) = a(a(N, c1), [c2, . . . , ck])

a(N, cj) =

{Add (p,m,i) at v if cj = Finst

Remove (p,m,i) at v if cj = Fdel

for a basic command cj = (p,m, i, v).In a network, these commands are generated by a control application or module Mj . We de�ne a

network module as a state-based function Mj that reacts to an event ev with a network command(and possibly modi�es its internal state):

Mj : ev → C

Examples for network events are the arrival of certain packet types (like ARP request) or thearrival of a new �ow currently not handled by the FIB.

1A typical FIB entry used in IP forwarding is (100,if {ingress_port==*, dst_ip∈{192.168.100.0/24}},set {src_mac 00:00:00:ab:cd:ef, dst_mac aa:bb:cc:00:11:22, egress_port 3}).

NetIDEPublic

7.2 Types of composition

The goal of composition is to run multiple modules on the same physical network and incorporateall their network commands into the network state. We distinguish di�erent ways of composingapplications.

7.2.1 Single module without composition

We start with the simplest case of a single module M1. All commands are simply forwarded andapplied to the network and the resulting network state N ′ is:

N ′ = a(N,M1(ev))

This is the case in most toy applications and demonstration scenarios of SDN controller frame-works nowadays.

7.2.2 Multiple modules without composition

A typical SDN controller runs multiple SDN modules. Commonly, all outputs of these modules areapplied to the network without any explicit form of composition. In this case, each command isapplied to the network as it happens, just like in the single module scenario. The resulting networkstate N (k) for multiple applications looks like this:

N ′ = a(N,M1(ev))

N ′′ = a(N ′,M2(ev))

N (k) = a(N (k−1),Mk(ev))

The simplicity of this approach is also its biggest problem. Since every network command is ap-plied when it happens, the resulting network state depends on the order of transactions applied:a(a(N,M1(ev)),M2(ev) is not necessarily the same as a(a(N,M2(ev),M1(ev)). The result can evenbe non-deterministic, i.e. for multiple executions in the same network, di�erences in execution speedof modules M1 and M2 can result in di�erent sequences and thus results even when applying thesame modules to the same network state and the same event.Another problem are transient states. In the time after the �rst module has answered but not

the second, the transient network state N ′ is active. This transient is problematic since it onlyre�ects the output of �rst module but not the others. These ill-speci�ed, non-deterministic tran-sient networks states are usually undesirable and constitute the main reason to explicitly de�ne acomposition logic that is able to deal with the transient state.

7.2.3 Multiple modules with harmonizing

The output of the multiple modules might contain con�icting or overlapping commands. For ex-ample, two modules might instruct one switch to deal with the same packet by either forwardingor dropping it, at the same priority. To deal with such con�icts, a stateful function

h : command→ command

NetIDEPublic

to modify the commands can be used; the goal is to harmonize the outputs. The network stateafter executing modules and harmonizing their outputs can be expressed as:

N ′ = a(N,h(M1(ev)))

N (k) = a(N (k−1), h(Mk(ev)))

When the harmonizing function h is the identity function, we get the same result as in the previoussubsection.This harmonizing function can deal with some problems, but intermediate, hard-to-predict net-

work states still exist. If one of the modules a (slowly reacting) �rewall its rule get applied, unwantedtra�c may pass in a transitional state. It is hence not a satisfying solution for all problems.

7.2.4 Parallel composition

To overcome the problem with transient state and varying order of applied results, parallel compo-sition collects all commands for all composed modules and then resolves all con�icts betweens thesecommands, composing the results into a single command to be applied to the network. This isdone by a special resolving function r that gets all command outputs and generates a con�ict-freeversion that can be applied to the network.

r : command× . . .× command→ command

The new network state N ′ can then be expressed as:

N ′ = a(N, r(M1(ev),M2(ev), . . . ,Mk(ev)))

This composition requires all command outputs of an event to be available; it must also be possibleto tie a command output of a module to a speci�c event (necessary when multiple events are passedto a module before commands have been produced, compare challenges of OpenFlow, Section 7.3).A big di�erence between the parallel composition and the harmonizing composition is that the

parallel composition is reactive, i.e. it depends on the fact that the network commands generatedby the modules are a response to a network event. The harmonizing composition works withoutthis assumption and can also be applied to network commands that are sent proactively without anevent (C =Mj(∅)).

7.2.5 Serial composition

For the serial composition, one module is fed the output of a previous module. The desired networkstate of the �rst module only exists as an input to the second module.To really support this behavior, we need to change the signature of the modules. They must

accept a command as an additional input:

M : event× command→ command

With that, we can de�ne a new function

M12 : ev 7→ (M1 ◦M2)(ev, ∅)

and use that in place of a normal module function (in the harmonizing composition or parallelcomposition). For example, if only the serially composed function is used, the new network state

NetIDEPublic

will be:

N ′ = a(N,M12(ev))

= a(N, (M1 ◦M2)(ev, ∅)= a(N,M2(ev,M1(ev, ∅)))

Chaining more than two network modules, e.g. M1 ◦M2 ◦M3, is de�ned by obvious induction.Network programming languages like Pyretic [48] also de�ne their function signature to have

symmetrical input and output: f : policy → policy. This is the key insight to give them a wellde�ned semantics!The main distinction of the serial composition to composition strategies discussed so far, is that

the last module in the composition chain can provide consistent network commands. It also allowsa module to incorporate the decisions of a previous module into its own decision. The downside ofthe serial composition is, however, that modules need to be explicitly designed and programmed tobe used in this way. We will take a look how useful serial composition is with existing modules inthe next sub section (7.2.6).

7.2.6 Approximate serial composition

As most network modules are not designed for serial composition (i.e., they do not accept a commandas an input), we de�ne an approximate way to do serial composition with existing network modules.In this scenario, we need to incorporate as much as possible from the network command into theinput event of the following module by a function

α : N × command→ event

where N is the approximated network state resulting from applying the output of the �rst function tothe current network state. This approximated state is a representation of the state in the controller;its manipulation does not involve manipulation of actual state in network devices.What can be incorporated into the new event is often very limited as we will see in Section 7.3.

The new network state using this function can be expressed as:

N ′ = a(N, (M1◦M2)(ev))

= a(N,M2(α(N ,M1(ev))))

Similarly, chaining three modules in an approximate serial composition works as well:

N ′ = a(N, (M1◦M2◦M3)(ev)

= a(N,M3(α(˜N,α(N ,M1(ev)))))

where˜N is the approximated network state resulting from applying M1(ev) to N .

Using overlays for approximate serial composition A conceivable variant to implement this ap-proximation of the network state and the function α is to use an overlay of virtual switches to areal network as shown in Figure 7.1: For each physical switch, a number of virtual switches cor-responding to the number of modules is emulated. Each module is assigned to one virtual switch.The approximated network states are the state of the virtual switches and the function α would�process� the packet that the module sends to its virtual switch output ports as event for the nextmodule.

NetIDEPublic

Figure 7.1: Using a virtual overlay network for composition of module A and B.

This semantic is not as useful as it seems. In the pure form of this approach the modules areseeing a fragmented network of switches in which every neighboring switch is controlled by a di�erentmodule. Normal routing and Link Layer Discovery Protocol (LLDP) neighbor discovery will failor produce unreasonable results. Adding special logical to alleviate these problem gives up theadvantage of a clear simple semantic.

7.2.7 Composition and order of middle boxes

A common misconception is that the placement of middleboxes (a networking device that transforms,inspects, �lters, or otherwise manipulates forwarded tra�c) always carries over to the compositionorder. If multiple middleboxes, for example a �rewall and a monitoring/IDS system should act onall the same tra�c, these boxes are set up in sequence to pass tra�c to box after another. Figure 7.2shows an example of a traditional middle box setup with an IDS, a �rewall and a load balancer.

Monitoring Server

IDS Firewall NAT

ExternalNetwork

Backend Servers

Figure 7.2: Typical order of IDS and Firewall and NAT load balancer middle boxes in a traditionalnetwork.

In stark contrast, for a composition the modules would be typically setup in a parallel compositionto allow all modules to base their decisions on the original input packets. The merging process ofall the outputs will then give an equivalent solution to the middle box solution. Figure 7.3 showsthe setup of Figure 7.2 implemented with parallel composition.

7.3 Composition with OpenFlow

OpenFlow is the most commonly used protocol used in real-world deployments and a lot of existingapplication logic is implemented using OpenFlow protocols. This makes OpenFlow desirable as

NetIDEPublic

newflow accept

copy tomonitor server

destinationbackend 7

raccept,

destinationbackend 7,

copy tomonitoring

Figure 7.3: Parallel composition of IDS, FW and NAT load-balancer modules

API protocol for composition and con�ict resolution. On the other hand, OpenFlow itself wasnever designed to be used in a composition context. The implicit assumption that there is only oneentity controlling an OpenFlow device2 makes its use problematic as many of the API results etc.are not in way that multiple entities can safely use them at the same time.This problem is aggravated by the fact that OpenFlow is not only used as a control protocol for

switches as the southbound interface. Instead, its semantic has also left its mark on the design ofnorthbound interfaces, which often more or less directly mirror the OpenFlow semantics. In thissection, we will analyze the problems of OpenFlow in composition and con�ict detection and detailhow and to what degree they can be avoided and solved.

7.3.1 De�nitions

We will brie�y show the de�nition of the important packet types in OpenFlow for composition:

Packet_In The PACKET_IN, abbreviated PKT_IN, is the main event in OpenFlow and usually sig-ni�es the arrival of a new �ow. Whenever a packet arrives at a switch that is not handledby one of the FIB entries (or a FIB entry explicitly states to generate a PKT_IN) a copy ofthe packet and the meta information of the packet (ingress port, etc.) are forwarded to thecontroller.

Flow_Mod The FLOW_MOD, abbreviated FM, is the OpenFlow command that is analogous to ourFIB entry install command Fi.

Packet_Out The PACKET_OUT, abbreviated PKT_OUT, allows an OpenFlow controller to craft andsend a packet to the network. A typical use case for this is to reply to an ARP Request. ThePKT_OUT consists of a packet and action list that is identical in function and syntax to the FMaction list.

7.3.2 Multiple modules

The �multiple modules� approach without harmonization (Section 7.2.2) is easy to support withOpenFlow. Adding a harmonizing function (Section 7.2.3) is possible but requires to intercept FMcommands before sending them to the network. Depending on the speci�c controller architecture,this is a more or less easy task. Correctly treating timeouts of FIB entries is also not a trivial task.Hence, even the �rst non-trivial composition approach is not entirely straightforward to support.

2OpenFlow does allow multiple connections per switch from multiple hosts for load sharing/backup of a single(distributed) entity; but this is not relevant for composition of applications in a single controller.

NetIDEPublic

7.3.3 Achieving consistent run-to-completion

In the previous sections we de�ned the parallel composition to combine all commands triggeredby the same network event. The de�nition of network events in OpenFlow is straightforward andconsists of a small list of unsolicited messages of which the most important one is the PKT_IN event.Unfortunately, in OpenFlow there is no relationship between a network event and the responses of

a controller and thus also no reliable way to tie the responses obtained from a module to the originalnetwork events. PKT_OUTs may reference the original PKT_IN as optimization to avoid copying thepacket but this captures only a fraction of the PKT_OUTs. Also, there is no way to tell if an OpenFlowmodule will respond to an event at all.Hence, the basic assumption of composition � i.e., actions can be tied to events across multiple

modules � is not guaranteed by OpenFlow.

We overcame this problem by fencing our transactions in the client controllers. In our imple-mentation, we de�ne a hook in the controller framework that noti�es the shim when an inputevent has been completely consumed. This is then noti�ed back to the NetIDE Core in the formof an (extra) NETIDE_FENCE packet that signals that the event has been completely consumedby the Controller Framework.

This fencing concept has been adopted by the Floodlight development team and is availablestarting from version 1.2 of the controller framework.

7.3.4 Parallel composition

If ignoring the (major) run-to-completion problem, implementing a resolve function works as sketchedin the Section 7.2.4.

7.3.5 Serial composition

With OpenFlow we can at best try to achieve approximate serial composition � actual serial com-position is impossible as an OpenFlow-oriented northbound interface cannot express both eventsand commands as input.The input event in OpenFlow is the PKT_IN. The goal is to create a PKT_IN that carries as much

information from the outputs (PKT_OUT and FLOW_OUT) of the previous module as possible.The generated packets and functions involved in an OpenFlow serial composition chain with two

modules looks like this:

PKT_IN0 →M1 → PKT_OUT1, FM1

→ α→ PKT_IN1

→M2 → PKT_OUT2, FM2

The meta-information part of the new PKT_IN (produced by the network emulation function α)is a match that only carries the input port and no other information. The input port is usually thesame as the input of the PKT_IN0 unless an overlay composition is used in which case the input portis the output port designated by the �rst module.For the packet part we have two options: (1) Modify the original packet of the original PKT_IN0

or (2) use the packet of the PKT_OUT0 if there is any and fall back to the input or stop the chain ifthere is none.When choosing the �rst option and using the packet of PKT_OUT1, we can assume that all actions

are either already applied or are in the action set of the packet out. As consequence, we will ignore

NetIDEPublic

FM1 in this case. If we decide to use the packet of the original packet of PKT_IN0, we can apply theactions of FM1 to it and thus ignore PKT_OUT1. No matter what option we choose, we always ignorea signi�cant part of M1 output.In both cases we have to apply the instructions of the PKT_OUT or FM to preserve as much in-

formation as possible. Only the subset of instructions that mutates the packet itself (like addinga vlan id) can be preserved. Everything that is not directly related to the content of the packetcannot be represented in the new packet, which includes instructions like setting the output queue,rate limits, goto table x, etc.The workaround to preserve the information contained in instructions is to remember them and

then merge/intersect all actions from all modules of a sequential composition in the last step. Butthis creates an unintuitive, di�cult to understand and predict hybrid between serial and parallelcomposition.In summary, all these problems with generating a new PKT_IN make sequential composition in an

OpenFlow only usable in very limited circumstances.

NetIDEPublic

8 Beyond the end of NetIDE

In this chapter, we explore the technical concepts that we have developed in NetIDE and how theywill be maintained beyond the lifetime of the project. We look (1) at the Open Source communitieswhere we have placed our code and how they can contribute to maintain it alive and (2) at EU-funded activities, like H2020 projects, where concepts can be further developed.

8.1 Eclipse Marketplace

The Eclipse marketplace provides a convenient way to download and install the NetIDE developertoolkit. After the project, NetIDE will still stay on the Eclipse Market Place for distribution. Thedeveloper toolkit will still be enhanced and extended by further modeling and analysis features aspart of a dissertation.

8.2 OpenDaylight

NetIDE is a key project of the OpenDaylight SDN Controller since the Beryllium release (Mar2016). It is also part of the Boron release (Sept 2016). The OpenDaylight community has beenvery supportive of the NetIDE usecase to allow SDN applications written for other SDN controllersto execute on ODL managed infrastructure. To ensure longterm support for the code produces byNetIDE after the project has completed, we have been pushing code patches into other projects.As mentioned in Chapter 3, all the serialization code required by the NetIDE Shim componentis now part of the Open�owJava serialization project and maintained by their developers goingforward. We are currently in discussions with the Open�owPlugin project to accept a patch thatwould migrate the majority of the NetIDE functionality for managing switches into their project.

8.3 The Network Modelling (NEMO) language

We started to explore the Network Modelling (NEMO) language as a way to describe applicationcomposition scenarios at a very early stage of the project. The proponents of NEMO are Huaweiand, after early contacts between TID and Huawei were established, a good collaboration with theirR&D lab in Beijing, including joint audio conferences between the Huawei and NetIDE teams hasallowed fruitful interaction. Thanks to it, we were able to re�ne Use Case 1, get insights of theinteraction between application modules inside the OpenDaylight controller framework and publiciseNetIDE and our results in di�erent venues, including the Internet Engineering Task Force (IETF).

8.3.1 Evolution during the lifetime of NetIDE

During the lifetime of the project, we have explored the possibility of using the NEMO language toexpress the composition of SDN modules. We have used our Use Case 1 to de�ne speci�c featuresin the NEMO language. The main requirement we derived from it is recursiveness: de�ne a Module

NetIDEPublic

that can be used as a building block in new Modules you de�ne as you describe scenarios withNEMO. This requirement has entered into the NEMO language.1

We have explored the possibility of using the IBNEMO project as a part of the Network Engineto implement Network Application deployment. The advantage of this approach would have beento reused external code to implement this feature. The disadvantages were dependence on anexternal project, which at this point in time is still not 100% usable, and imposing the use of theOpenDaylight controller in the NetIDE network engine, which would have limited its �exibility andcontradicted some of the explicit goals of NetIDE.

8.3.2 Beyond the lifetime of NetIDE

The NEMO language is currently being used as a basis for one of the Intent-based networkingapproaches in the OpenDaylight community. This ensures that the language will survive beyondthe lifetime of NetIDE in this broader community. We expect some aspects of the language tochange in time to adapt to the evolution of the intent concept. However, the new versions of thelanguage will still need to conform to our Use Case 1.In parallel, NEMO is also being evaluated as a candidate VNF Descriptor (VNFD) language in

the scope of the H2020 SUPERFLUIDITY project [49]. This line of activity was started in NetIDEbut will be continued in SUPERFLUIDITY.TID decided to follow this path because the NEMOlanguage provides a reasonable approach to providing recursiveness to VNF description languages.This line of action will allow to extend the applicability of NetIDE architectural concepts to theNetwork Functions Virtualisation (NFV) world.A long-term goal agreed with the NEMO team is to standardise the NEMO language in the

IETF. The current understanding is that NEMO will be included in the IETF normative corpusas an Request For Comment (RFC) using the individual submission process. This strategy hasbeen followed by the proponents of other signi�cant SDN protocols like the Open vSwitch DatabaseManagement Protocol (OVSDB) [50].

8.4 ONF Englewood

Englewood is an Open Networking Foundation (ONF) project whose goal is to develop a set ofsoftware modules that facilitates the deployment of standard ONF Transport API (T-API) in SDNenvironments where di�erent platforms (e.g. ONOS and OpenDaylight) are used. Englewood aimsto develop software modules that would support T-API processing and translation towards platform-speci�c APIs. Such modules encompass:

• platform-independent Platform Abstraction Layer (PAL);

• platform-dependent Handler.

Figure 8.1 shows the most relevant building blocks of the Englewood architecture. While theT-API Layout takes care of the communication with the application by exposing a standard T-APIinterface, the Adapter Layout provides a common Java interface for the handlers.NetIDE tackles two important use cases in Englewood: (i) the current Englewood architecture

does not support the scenario where multiple T-API applications running simultaneously and con-trolling the same network, (ii) only Java-based SDN platforms are supported since the AdapterLayout exposes a Java interface that cannot be used to attach platform-speci�c handlers for con-trollers written in a di�erent programming language (e.g. Python for Ryu).

1The interaction with Huawei has triggered re�nements in the NEMO language and the implementation of theIBNEMO module in ODL by Huawei that have not received any funding from NetIDE .

NetIDEPublic

Figure 8.1: Simpli�ed diagram of the Englewood architecture.

As shown in Figure 8.2, NetIDE addresses the aforementioned limitations thanks to the NetIDEprotocol which allows any server controller (not only the ones written in Java) to be connected tothe Englewood PAL via the NetIDE handler and by adapting the con�ict resolution mechanism toTransport API to avoid network mis-con�gurations due to multiple independent applications tryingto control the same tra�c.

Figure 8.2: Integration of Englewood with the Network Engine.

NetIDEPublic

8.4.1 Beyond the lifetime of NetIDE

The integration of the Network Engine with the Englewood PAL requires the enhancement of somecomponents. We need to: (i) implement the NetIDE handler shown in Figure 8.2 which correspondsto the NetIDE Backend, for the Englewood PAL, (ii) add support to con�ict resolution for T-APIin the Core, and (iii) implement the translation between T-API and server controller NBI in theShim.As architecture and interfaces of the Englewood PAL are still under de�nition, the implementation

work will start after the end of the NetIDE project. In this regard, we have de�ned a concreteshort-term plan for the implementation of the Java-based NetIDE handler. This handler will re-usemost of the code implemented for the Floodlight Backend for its southbound interface. For thenorthbound one, they will re-use the code implemented for the ONOS handler, as ONOS is onethe target platforms of the Englewood project. Moreover, this handler will also implement thetranslation between T-API and ONOS NBI for its southbound interface. The plan is to integratethis handler with the ONOS Shim to obtain an enhanced Shim that can translate T-API messagescarried by the NetIDE protocol into ONOS NBI.

NetIDEPublic

9 Conclusions

In this manual we have illustrated the major outcomes of the NetIDE project grouped under the label�NetIDE Framework�. Following the initial goal of the project, i.e. to �deliver a single integrateddevelopment environment to support the whole development lifecycle of network controller programsin a vendor-independent fashion� [51], we have provided a toolbox that enables DevOps principlesin SDN environments, reduces cost of developing Network Applications, and shortens the time-to-market for new services.In summary, the project has successfully produced and released to the Free and Open Source

Software (FOSS) community (i) an Eclipse-based IDE [52, 53], which supports the design anddevelopment of network applications, (ii) the Network Engine [1], which realizes the composition ofnetwork applications written for di�erent platforms into new applications and (iii) SDN Tools [54]to diagnose, debug and troubleshoot the data and control planes of the SDN network.The scienti�c value of our work has been demonstrated at several conferences, such as Net-

Soft'15 [11], SOSR'15 [55], NOMS'16 [56], NetSoft'16 [47] and EuCNC'16 [57], Moreover, the archi-tecture of the Network Engine will be presented at IEEE/IFIP CNSM Conference [58], a premierconference on network and service management (www.cnsm-conf.org/2016/).

NetIDEPublic

A NetIDE Protocol Speci�cation

The Intermediate protocol accomplishes the following functions: (i) to carry management messagesbetween the Network Engine's layers (Core, Shim and Backend); e.g., to exchange information onthe supported SBI protocols, to provide unique identi�ers for application modules, (ii) to carry eventand action messages between Shim, Core, and Backend, properly demultiplexing such messages tothe right module based on identi�ers, (iii) to encapsulate messages speci�c to a particular SBIprotocol version (e.g., OpenFlow 1.X, Network Con�guration Protocol (NETCONF), etc.) withproper information to recognize these messages as such.Messages of the NetIDE protocol contain two basic elements: the NetIDE header and the data (orpayload). The NetIDE header, described below in Section A.1, is placed before the payload andserves as the communication and control link between the di�erent components of the NetworkEngine. The payload carries management messages or the SBI messages issued by either clientcontrollers or network elements.

A.1 The NetIDE protocol header

The NetIDE header can be represented in a C-style coding format as follows:

struct netide_header{

uint8_t netide_ver;

uint8_t type;

uint16_t length;

uint32_t nxid

uint32_t module_id

uint64_t datapath_id

The netide_ver is the version of the NetIDE protocol (the current version v1.4 which is identi�edwith value 0x05), length is the total length of the payload in bytes and type contains a code thatindicates the type of the message according with the following values1:

enum type{

NETIDE_HELLO = 0x01,

NETIDE_ERROR = 0x02,

NETIDE_MGMT = 0x03,

NETIDE_MODULE_ANN = 0x04,

NETIDE_MODULE_ACK = 0x05,

NETIDE_HEARTBEAT = 0x06,

NETIDE_TOPOLOGY = 0x07,

NETIDE_FENCE = 0x08,

NETIDE_OPENFLOW = 0x11,

NETIDE_NETCONF = 0x12,

NETIDE_OPFLEX = 0x13,

NETIDE_OFCONFIG = 0x14,

NETIDE_OTHER = 0xFF

1 NETIDE_MGMT and NETIDE_TOPOLOGY message types are not documented in the current speci�cation. They havebeen introduced to allow future extensions of the Network Engine capabilities.

NetIDEPublic

datapath_id is a 64-bits �eld that uniquely identi�es the network elements. module_id is a 32-bits �eld that uniquely identi�es Backends and application modules running on top of each clientcontroller. The composition mechanism in the Core layer leverages on this �eld to implement thecorrect execution �ow of these modules. Finally, nxid is the transaction identi�er associated to theeach message. Replies must use the same value to facilitate the pairing.

A.2 Module announcement

The Core executes composition and con�ict resolution operations based on a con�guration �lewhich speci�es how the applications modules cooperate in controlling the network tra�c. In par-ticular, con�guration parameters determine the way the Core handles the messages received fromthe applications modules running on top of the client controllers. To this purpose, each message isencapsulated with the NetIDE header containing a module_id value that identi�es the module thathas issued the message.module_id values are assigned by the Core during the modules announcement/acknowledge processdescribed in this Section. As a result of this process, each Backend and application module can berecognized by the Core through an identi�er (the module_id) placed in the NetIDE header.

As a �rst step, Backends register themselves by sending a module announcement message (messagetype NETIDE_MODULE_ANN) to the Core containing a human-readable identi�er such as: backend-

<platform_name>-<pid>. Where platform_name is the name of the client controller platform (ryu,onos, odl and �oodlight can be used) and pid is the process ID of the instance of the client controllerwhich is performing the registration. The format of the message is the following:

struct NetIDE_message{

netide_ver = 0x05

type = NETIDE_MODULE_ANN

length = len("backend-<platform_name>-<pid>")

nxid = 0

module_id = 0

datapath_id = 0

data = "backend-<platform_name>-<pid>"

The answer generated by the Core (message type NETIDE_MODULE_ACK) includes a module_id valueand the Backend name in the payload (the same indicated in the NETIDE_MODULE_ANN message):

netide_ver = 0x05

type = NETIDE_MODULE_ACK

length = len("backend-<platform_name>-<pid>")

nxid = 0

module_id = BACKEND_ID

datapath_id = 0

data = "backend-<platform_name>-<pid>"

After this step, all the messages generated by the Backend (e.g., heartbeat and hello messagesdescribed in the following Sections) will contain the BACKEND_ID value in the module_id �eld ofthe NetIDE header. Furthermore, BACKEND_ID is used to register the application modules that arerunning on top of the client controller:

netide_ver = 0x05

type = NETIDE_MODULE_ANN

NetIDEPublic

length = len("module_name")

nxid = 0

datapath_id = 0

data = "module_name"

where module_name is the name of the module under registration. The module's name can beassigned by the Backend or retrieved from the module itself via API calls. The Core replies with:

netide_ver = 0x05

type = NETIDE_MODULE_ACK

length = len("module_name")

nxid = 0

module_id = MODULE_ID

datapath_id = 0

data = "module_name"

After this last step, the Backend allows the application modules to control the network. Inparticular, network commands sent towards the network (e.g. OpenFlow FLOW_MODs, PACKET_OUTs,FEATURES_REQUESTs) are intercepted by the Backend, which encapsulates them with the NetIDEheader containing the MODULE_ID value. Such a value is then used by the Core to recognize thesender of the message and to properly feed the composition and con�ict resolution operators.

A.3 Heartbeat

The heartbeat mechanism has been introduced after the adoption of the ZeroMQ messaging queuinglibrary [15] to transmit the NetIDE messages. Unfortunately, the ZeroMQ library does not o�erany mechanism to �nd out about disrupted connections (and also completely unresponsive peers).This limitation can be an issue for the Core's composition mechanism and for the tools connectedto the Network Engine, as they are not able to understand when an client controller disconnectsor crashes. As a countermeasure, Backends must periodically send (let's say every 5 seconds) a�heartbeat� message to the Core. If the Core does not receive at least one �heartbeat� message fromthe Backend within a certain timeframe, the Core considers it disconnected, removes all the relateddata from its memory structures and informs the relevant tools. In order to minimize the servicedisruption, the Core applies default policies as speci�ed in the composition speci�cation (e.g. a"drop all" action in case of disconnected �rewall module).The format of the message is the following:

netide_ver = 0x05

type = NETIDE_HEARTBEAT

length = 0

nxid = 0

datapath_id = 0

data = 0

A.4 Handshake

Upon completion of the connection with the Core (and of the module announcement/acknowledgeprocess for the Backends), Backends must immediately send a hello message with the list of the

NetIDEPublic

supported control and/or management protocols. The format of the message is the following:

netide_ver = 0x05

type = NETIDE_HELLO

length = 2*NR_PROTOCOLS

nxid = 0

datapath_id = 0

data = [list of supported protocols]

Where data contains one 2-byte word (in big endian order) for each protocol, with the �rst bytecontaining the code of the protocol according to the above enum, while the second byte indicatesthe version of the protocol (e.g. according to the ONF speci�cation, 0x01 for OpenFlow v1.0, 0x02for OpenFlow v1.1, etc.). NETCONF version is marked with 0x01 that refers to the speci�cationin the RFC6241 [59], while OpFlex version is marked with 0x00 since this protocol is still in work-in-progress stage [60].The Shim responds with another hello message containing the following:

netide_ver = 0x05

type = NETIDE_HELLO

nxid = 0

datapath_id = 0

if at least one of the protocols in the request is used between the server controller and the networkdevices. In particular, data contains the codes of the protocols that match the client's request(2-bytes words, big endian order). The backend_id value is used in the NetIDE header to allowthe Core to forward the reply to the Backend that started the handshake. If none of the requestedprotocols is supported, the header of the reply is as follows:

netide_ver = 0x05

type = NETIDE_ERROR

nxid = 0

datapath_id = 0

where the payload of the message data contains the codes of all the protocols supported by theserver controller (2-bytes words, big endian order).

A.5 The fence mechanism

An application module may respond to a given network event (e.g., an OpenFlow PACKET_IN) witha set of zero, one or multiple network commands (e.g., OpenFlow FLOW_MODs and PACKET_OUTs).The so-called fence mechanism is a means for the Core to correlate events and commands and toknow when the module has �nished processing the input event.This mechanism is implemented through the message type NETIDE_FENCE which is sent by theBackend to the Core once a module has �nished processing a network event. Within the same

NetIDEPublic

transaction, fence message, the network event and related network commands use all the samemodule_id and nxid values in the NetIDE header so that the Core can correlate them.

Module(ID=X)BackendCore

NetIDE msg(mod_id=X, nxid=M)

packet_in

packet_out

NetIDE_FENCE (mod_id=X, nxid=M)

flow_mod

extract the packet_insend it to Module X

NetIDE msg (mod_id=X, nxid=M)

packet_out Seria

lise &

transm

ait fo

flow_mod

Figure A.1: Fence mechanism work�ow. Both nxid and module_id refer to the NetIDE header�elds.

The process is represented in Fig. A.1, where a PACKET_IN event is encapsulated with the NetIDEheader by the Core with values module_id=X and nxid=M and �nally sent to the Backend. TheBackend removes the NetIDE header and forwards the PACKET_IN the the application module X. Themodule reacts with zero, one or multiple network commands (represented by OpenFlow messagesPACKET_OUT and FLOW_MOD in the �gure). Each network command is encapsulated by the Backendwith the NetIDE header re-using the same module_id and nxid values received from the Core withthe PACKET_IN. Therefore, the Core uses the nxid value to pair the network commands generatedby the module and the previous network event.Once the module's event handling function returns, the Backend issues a fence message to signalthe completion of the transaction to the Core.

A.6 The OpenFlow protocol

In this speci�cation, the support for all versions of OpenFlow is achieved with the following:

struct netide_message{

struct netide_header header;

uint8 data[0]

Where header contains the following values: netide_ver=0x05, type=NETIDE_OPENFLOW and lengthis the size of the original OpenFlow message which is contained in data.

NetIDEPublic

A.6.1 Properly handling reply messages

The NetIDE protocol helps the Network Engine in pairing OpenFlow reply messages with thecorresponding requests issued by the application modules running on top of it (e.g. statistics,feature requests, con�gurations, etc., thus the so-called �controller-to-switch� messages de�ned inthe OpenFlow speci�cations). In this context, the xid �eld in the OpenFlow header is not helpful,as may happen that di�erent modules use the same values.In the proposed approach, represented in Fig. A.2, the task of pairing replies with requests isperformed by the Core which ensures that only the module that sends the request will receive thecorresponding reply by replacing the xid of the OpenFlow requests with new unique values. TheCore stores the original xid along with the module_id it �nds in the NetIDE header. As the networkelements use the same xid values in the replies, the Core can easily pair requests and replies andcan use the saved module_id to send the reply to the right application module.The diagram in Fig. A.2 shows how the Network Engine handles the controller-to-switch OpenFlowmessages. The work�ow starts with an application module that issues an OpenFlow request withxid=N. The Backend relays the message to the Core by encapsulating it with the NetIDE header byusing module_id=X previously assigned to the application module (see Section A.2). Once the Corereceives this message, it computes a new OpenFlow xid value M (e.g. by using a hashing algorithm)and ensures that such a value is not being used in other existing transactions.

Module(ID=X)

Backend(ID=Y)

Core Shim Layer NetworkElement

request msg(xid = N)

compute new unique xide.g.: M = hash(N,X)

NetIDE msg(module_id=X)request msg

(xid = N)

NetIDE msg(module_id=None)

request msg(xid = M)

reply msg(xid = M)

restore the old xidxid=N module_id=X

NetIDE msg(module_id=None)

reply msg(xid = M)

NetIDE msg(module_id=X)

reply msg(xid = N)

Figure A.2: Request/reply message handling. xid refers to the OpenFlow header �eld.

Before sending the reply to the Backend, the Core restores the original xid in the OpenFlowreply (the application module expects to �nd in the reply the same xid value that was used for therequest) and inserts the module_id previously saved in the NetIDE header. The Backend will usethis information to forward the message to the right application module.Asynchronous OpenFlow messages generated by the network elements are ignored by the above

described tracking mechanism. They are simply relayed to the Backends that eventually forwardthem to the relevant application modules based on the composition and topology speci�cations.Currently de�ned OpenFlow asynchronous messages are the following:

NetIDEPublic

Message Type ID Description OF Version

OFPT_PACKET_IN 10 New packet received by a switch 1.0-1.5

OFPT_FLOW_REMOVED 11 Flow rule removed from the table 1.0-1.5

OFPT_PORT_STATUS 12 Port added, removed or modi�ed 1.0-1.5

OFPT_ROLE_STATUS 30 Controller role change event 1.4-1.5

OFPT_TABLE_STATUS 31 Changes of the table state 1.4-1.5

OFPT_REQUESTFORWARD 32 Request forwarding by the switch 1.4-1.5

OFPT_CONTROLLER_STATUS 35 Controller status change event 1.5

A.7 Other SBI protocols

The NetIDE intermediate protocol can easily support other SBI protocols, such as NETCONF [59],OF-Con�g [61] or OpFlex [60].While OF-Con�g con�gurations are only encoded in eXtensible Markup Language (XML), NetConfand OpFlex speci�cations are more �exible and support both XML and JavaScript Object Notation(JSON) encoding formats. For this reason, we need an additional �eld in the NetIDE headerto indicate the format of the message contained in data and to allow the recipients to correctlyhandle it. To this purpose, when transmitting NetConf or OpFlex messages, the sender must settype=NETIDE_OTHER in the NetIDE header to indicate the presence of an additional 16-bits �eld atthe end of the header. This �eld, named ext_type, speci�es the SBI protocol and the format of themessage carried by data:

struct netide_message{

struct netide_header header;

uint16_t ext_type;

uint8_t data[0];

Where header contains the following values: netide_ver=0x05, type=NETIDE_OTHER and length

is the size of the original SBI message carried by data. The value of ext_type indicates the SBIprotocol in the most signi�cant byte (as speci�ed in Section A.1) and the format of the message(either 0x00 for XML or 0x01 for JSON) in the least signi�cant byte.

NetIDEPublic

B XML Schema for Composition speci�cation

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>

<xs:schema version="1.0" targetNamespace="http://netide.eu/schemas/specification/v1"

xmlns:tns="http://netide.eu/schemas/specification/v1" xmlns:xs="http://www.w3.org/2001/XMLSchema">

<xs:element name="Branch">

<xs:complexType>

<xs:complexContent>

<xs:extension base="tns:ExecutionFlowNode">

<xs:sequence>

<xs:element name="BranchCondition" type="tns:Condition" form="qualified" minOccurs="0"/>

<xs:element ref="tns:If"/>

<xs:element ref="tns:Else"/>

</xs:sequence>

</xs:extension>

</xs:complexContent>

</xs:complexType>

</xs:element>

<xs:element name="CompositionSpecification">

<xs:complexType>

<xs:sequence>

<xs:element name="Modules" form="qualified">

<xs:complexType>

<xs:sequence>

<xs:element ref="tns:Module" maxOccurs="unbounded"/>

</xs:sequence>

</xs:complexType>

</xs:element>

<xs:element name="Composition" form="qualified" minOccurs="0">

<xs:complexType>

<xs:sequence>

<xs:choice minOccurs="0" maxOccurs="unbounded">

<xs:element ref="tns:ModuleCall"/>

<xs:element ref="tns:ParallelCall"/>

<xs:element ref="tns:Branch"/>

</xs:choice>

</xs:sequence>

</xs:complexType>

</xs:element>

</xs:sequence>

</xs:complexType>

</xs:element>

<xs:element name="Else">

<xs:complexType>

<xs:complexContent>

<xs:extension base="tns:ExecutionFlowNodeContainer">

<xs:sequence/>

</xs:extension>

</xs:complexType>

</xs:element>

NetIDEPublic

<xs:element name="If">

<xs:complexType>

<xs:complexContent>

<xs:extension base="tns:ExecutionFlowNodeContainer">

<xs:sequence/>

</xs:extension>

</xs:complexType>

</xs:element>

<xs:element name="Module">

<xs:complexType>

<xs:sequence>

<xs:element name="CallCondition" type="tns:Condition" form="qualified" minOccurs="0"/>

</xs:sequence>

<xs:attribute name="id" type="xs:ID" use="required"/>

<xs:attribute name="loaderIdentification" type="xs:string" use="required"/>

<xs:attribute name="noFenceSupport" type="xs:boolean" use="optional" />

</xs:complexType>

</xs:element>

<xs:element name="ModuleCall">

<xs:complexType>

<xs:complexContent>

<xs:sequence>

<xs:element name="CallCondition" type="tns:Condition" form="qualified" minOccurs="0"/>

</xs:sequence>

<xs:attribute name="allowVetoResult" type="xs:boolean"/>

<xs:attribute name="module" type="xs:IDREF" use="required"/>

<xs:attribute name="priority" type="xs:int" use="optional"/>

</xs:extension>

</xs:complexType>

</xs:element>

<xs:element name="ParallelCall">

<xs:complexType>

<xs:complexContent>

<xs:sequence>

<xs:element ref="tns:ModuleCall" maxOccurs="unbounded"/>

</xs:sequence>

<xs:attribute name="resolutionPolicy" type="tns:ResolutionPolicy"/>

</xs:extension>

</xs:complexType>

</xs:element>

<xs:complexType name="Condition">

<xs:sequence/>

<xs:attribute name="datapaths">

<xs:simpleType>

<xs:list itemType="xs:long"/>

</xs:simpleType>

</xs:attribute>

<xs:attribute name="ethDst" type="xs:string"/>

<xs:attribute name="ethSrc" type="xs:string"/>

<xs:attribute name="ethType" type="tns:ethType"/>

<xs:attribute name="events">

NetIDEPublic

<xs:simpleType>

<xs:list itemType="tns:Events"/>

</xs:simpleType>

</xs:attribute>

<xs:attribute name="inPort" type="xs:int"/>

<xs:attribute name="ipv4Dst" type="xs:string"/>

<xs:attribute name="ipv4Src" type="xs:string"/>

<xs:attribute name="ipv6Dst" type="xs:string"/>

<xs:attribute name="ipv6Src" type="xs:string"/>

<xs:attribute name="ipProto" type="tns:ipProtocol"/>

<xs:attribute name="tcpDst" type="xs:int"/>

<xs:attribute name="tcpSrc" type="xs:int"/>

<xs:attribute name="udpDst" type="xs:int"/>

<xs:attribute name="udpSrc" type="xs:int"/>

</xs:complexType>

<xs:complexType name="ExecutionFlowNode" abstract="true">

<xs:sequence/>

</xs:complexType>

<xs:complexType name="ExecutionFlowNodeContainer" abstract="true">

<xs:sequence>

<xs:choice minOccurs="0" maxOccurs="unbounded">

<xs:element ref="tns:ModuleCall"/>

<xs:element ref="tns:ParallelCall"/>

<xs:element ref="tns:Branch"/>

</xs:choice>

</xs:sequence>

</xs:complexType>

<xs:simpleType name="Events">

<xs:restriction base="xs:string">

<xs:enumeration value="flowRemoved"/>

<xs:enumeration value="packetIn"/>

<xs:enumeration value="barrierIn"/>

<xs:enumeration value="connectionDown"/>

<xs:enumeration value="connectionUp"/>

<xs:enumeration value="flowMod"/>

<xs:enumeration value="errorIn"/>

<xs:enumeration value="portStatus"/>

</xs:restriction>

</xs:simpleType>

<xs:simpleType name="ResolutionPolicy">

<xs:enumeration value="ignore"/>

<xs:enumeration value="priority"/>

<xs:enumeration value="auto"/>

<xs:enumeration value="pass"/>

</xs:restriction>

</xs:simpleType>

<xs:simpleType name="ethType">

<xs:enumeration value="ETH_FLOW"/>

<xs:enumeration value="COBRANET"/>

<xs:enumeration value="UNDEFINED"/>

<xs:enumeration value="IPv6"/>

<xs:enumeration value="EAP_OVER_LAN"/>

<xs:enumeration value="JUMBO_FRAMES"/>

NetIDEPublic

<xs:enumeration value="POWERLINK"/>

<xs:enumeration value="IPX_8137"/>

<xs:enumeration value="PPPoE_DISCOVERY"/>

<xs:enumeration value="TRILL"/>

<xs:enumeration value="BRIDGING"/>

<xs:enumeration value="SLOW_PROTOCOLS"/>

<xs:enumeration value="ARP"/>

<xs:enumeration value="HSR"/>

<xs:enumeration value="Q_IN_Q"/>

<xs:enumeration value="CONF_TEST"/>

<xs:enumeration value="RoCE"/>

<xs:enumeration value="APPLE_TALK_ARP"/>

<xs:enumeration value="REV_ARP"/>

<xs:enumeration value="FCoE"/>

<xs:enumeration value="WAKE_ON_LAN"/>

<xs:enumeration value="HOMEPLUG_AV"/>

<xs:enumeration value="APPLE_TALK"/>

<xs:enumeration value="PPPoE_SESSION"/>

<xs:enumeration value="HOMEPLUG_10"/>

<xs:enumeration value="IPX_8138"/>

<xs:enumeration value="MAC_SEC"/>

<xs:enumeration value="MPLS_MULTICAST"/>

<xs:enumeration value="CFM"/>

<xs:enumeration value="PTP"/>

<xs:enumeration value="PROFINET"/>

<xs:enumeration value="DECNET_IV"/>

<xs:enumeration value="ATA_OVER_ETH"/>

<xs:enumeration value="FCoE_INIT"/>

<xs:enumeration value="ETHERCAT"/>

<xs:enumeration value="QNX"/>

<xs:enumeration value="HYPERSCSI"/>

<xs:enumeration value="LLDP"/>

<xs:enumeration value="MRP"/>

<xs:enumeration value="VLAN_FRAME"/>

<xs:enumeration value="LLT"/>

<xs:enumeration value="SERCOS"/>

<xs:enumeration value="MPLS_UNICAST"/>

</xs:restriction>

</xs:simpleType>

<xs:simpleType name="ipProtocol">

<xs:enumeration value="EMCON"/>

<xs:enumeration value="SCPS"/>

<xs:enumeration value="IS_IS_OVER_IPv4"/>

<xs:enumeration value="RDP"/>

<xs:enumeration value="RSVP"/>

<xs:enumeration value="BNA"/>

<xs:enumeration value="MERIT_INP"/>

<xs:enumeration value="WB_EXPAK"/>

<xs:enumeration value="DCN_MEAS"/>

<xs:enumeration value="LOCAL_NET"/>

<xs:enumeration value="NSFNET_IGP"/>

<xs:enumeration value="VRRP"/>

<xs:enumeration value="LARP"/>

<xs:enumeration value="NVP_II"/>

<xs:enumeration value="MANET"/>

<xs:enumeration value="TLSP"/>

<xs:enumeration value="MFE_NSP"/>

NetIDEPublic

<xs:enumeration value="COMPAQ_PEER"/>

<xs:enumeration value="TCF"/>

<xs:enumeration value="PTP"/>

<xs:enumeration value="IP_COMP"/>

<xs:enumeration value="SDRP"/>

<xs:enumeration value="IPv6_ICMP"/>

<xs:enumeration value="QNX"/>

<xs:enumeration value="ARIS"/>

<xs:enumeration value="UDP_LITE"/>

<xs:enumeration value="IPCV"/>

<xs:enumeration value="TTP_IPTM"/>

<xs:enumeration value="SECURE_VMTP"/>

<xs:enumeration value="IPv6_NO_NXT"/>

<xs:enumeration value="FIRE"/>

<xs:enumeration value="DCCP"/>

<xs:enumeration value="UTI"/>

<xs:enumeration value="HMP"/>

<xs:enumeration value="SAT_EXPAK"/>

<xs:enumeration value="GMTP"/>

<xs:enumeration value="ESP"/>

<xs:enumeration value="IATP"/>

<xs:enumeration value="ETHERIP"/>

<xs:enumeration value="ARGUS"/>

<xs:enumeration value="CHAOS"/>

<xs:enumeration value="WB_MON"/>

<xs:enumeration value="BR_SAT_MON"/>

<xs:enumeration value="IPX_IN_IP"/>

<xs:enumeration value="A_N"/>

<xs:enumeration value="IPv6_ROUTE"/>

<xs:enumeration value="LEAF_2"/>

<xs:enumeration value="CBT"/>

<xs:enumeration value="MICP"/>

<xs:enumeration value="SRP"/>

<xs:enumeration value="MHRP"/>

<xs:enumeration value="MOBILE"/>

<xs:enumeration value="TRUNK_1"/>

<xs:enumeration value="MUX"/>

<xs:enumeration value="TRUNK_2"/>

<xs:enumeration value="GRE"/>

<xs:enumeration value="SKIP"/>

<xs:enumeration value="IPIP"/>

<xs:enumeration value="IRTP"/>

<xs:enumeration value="DDX"/>

<xs:enumeration value="MPLS_IN_IP"/>

<xs:enumeration value="XTP"/>

<xs:enumeration value="VISA"/>

<xs:enumeration value="STP"/>

<xs:enumeration value="PGM"/>

<xs:enumeration value="PIM"/>

<xs:enumeration value="SUN_ND"/>

<xs:enumeration value="IPLT"/>

<xs:enumeration value="PVP"/>

<xs:enumeration value="KRYPTOLAN"/>

<xs:enumeration value="CFTP"/>

<xs:enumeration value="MTP"/>

<xs:enumeration value="CPHB"/>

<xs:enumeration value="IPv6_FRAG"/>

<xs:enumeration value="HOST_INTERNAL"/>

<xs:enumeration value="SWIPE"/>

<xs:enumeration value="PNNI"/>

<xs:enumeration value="XNS_IDP"/>

NetIDEPublic

<xs:enumeration value="AX_25"/>

<xs:enumeration value="RVD"/>

<xs:enumeration value="I_NLSP"/>

<xs:enumeration value="PRIVATE_ENCRYPT"/>

<xs:enumeration value="GGP"/>

<xs:enumeration value="SM"/>

<xs:enumeration value="_3PC"/>

<xs:enumeration value="SHIM6"/>

<xs:enumeration value="IDPR_CMTP"/>

<xs:enumeration value="DIST_FS"/>

<xs:enumeration value="HOPOPT"/>

<xs:enumeration value="CRUDP"/>

<xs:enumeration value="IGP"/>

<xs:enumeration value="BBN_RCC_MON"/>

<xs:enumeration value="IDRP"/>

<xs:enumeration value="NETBLT"/>

<xs:enumeration value="VINES"/>

<xs:enumeration value="RSVP_E2E_IGNORE"/>

<xs:enumeration value="IPv6_OPTS"/>

<xs:enumeration value="IPPC"/>

<xs:enumeration value="Sprite_RPC"/>

<xs:enumeration value="L2TP"/>

<xs:enumeration value="ICMP"/>

<xs:enumeration value="SSCOPMCE"/>

<xs:enumeration value="FC"/>

<xs:enumeration value="WSN"/>

<xs:enumeration value="SPS"/>

<xs:enumeration value="XNET"/>

<xs:enumeration value="SAT_MON"/>

<xs:enumeration value="VMTP"/>

<xs:enumeration value="EGP"/>

<xs:enumeration value="TCP"/>

<xs:enumeration value="CPNX"/>

<xs:enumeration value="ST"/>

<xs:enumeration value="AH"/>

<xs:enumeration value="PUP"/>

<xs:enumeration value="SNP"/>

<xs:enumeration value="MOBILITY_HEADER"/>

<xs:enumeration value="TP_PP"/>

<xs:enumeration value="NARP"/>

<xs:enumeration value="SMP"/>

<xs:enumeration value="CRTP"/>

<xs:enumeration value="DDP"/>

<xs:enumeration value="SCC_SP"/>

<xs:enumeration value="SCTP"/>

<xs:enumeration value="HIP"/>

<xs:enumeration value="IGMP"/>

<xs:enumeration value="UDP"/>

<xs:enumeration value="ZERO_HOP"/>

<xs:enumeration value="EIGRP"/>

<xs:enumeration value="PRM"/>

<xs:enumeration value="DGP"/>

<xs:enumeration value="ISO_TP4"/>

<xs:enumeration value="ENCAP"/>

<xs:enumeration value="IFMP"/>

<xs:enumeration value="IL"/>

<xs:enumeration value="ISO_IP"/>

<xs:enumeration value="LEAF_1"/>

<xs:enumeration value="IDPR"/>

<xs:enumeration value="OSPF"/>

NetIDEPublic

<xs:enumeration value="PIPE"/>

</xs:restriction>

</xs:simpleType>

</xs:schema>

Listing B.1: Composition Speci�cation Example

NetIDEPublic

Bibliography

[1] NetIDE Engine Repository. https://github.com/fp7-netide/Engine.

[2] NetIDE YouTube Channel. https://www.youtube.com/channel/

UCX7fK91ZaGeihLAaEjovOww.

[3] Extensible Markup Language (XML) 1.0 (Fifth Edition). https://www.w3.org/TR/xml/, nov2008.

[4] ONOS - Open Network Operating System. http://onosproject.org/.

[5] The OpenDaylight Platform. https://www.opendaylight.org/.

[6] Ryu SDN Framework. https://osrg.github.io/ryu/.

[7] Floodlight OpenFlow Controller.

[8] Eclipse framework. https://eclipse.org.

[9] T-NOVA: Network Functions-as-a-service over virtualised infrastructures. http://www.

t-nova.eu.

[10] Open Networking Foundation. TR-502: SDN architecture. Technical reference, Open Network-ing Foundation, 2014.

[11] R. Doriguzzi-Corin, E. Salvadori, P. A. Aranda Gutiérrez, C. Stritzke, A. Leckey, K. Phemius,E. Rojas, and C. Guerrero. NetIDE: Removing vendor lock-in in SDN. In Network Softwariza-tion (NetSoft), 2015 1st IEEE Conference on, 2015.

[12] NetIDE GitHub Repository. https://github.com/fp7-netide.

[13] Apache Karaf. http://karaf.apache.org/.

[14] OSGi: The Dynamic Module System for Java. https://www.osgi.org/, 2015.

[15] ZeroMQ (∅MQ) Distributed Messaging. http://zeromq.org/.

[16] Apache ARIES Blueprint. https://aries.apache.org/modules/blueprint.html.

[17] JeroMQ Pure Java implementation of libzmq. https://github.com/zeromq/jeromq.

[18] Project JAXB. https://jaxb.java.net/.

[19] The NetIDE Consortium. NetIDE deliverable 2.5 - NetIDE Runtime Architecture Consolida-tion, March 2016.

[20] The NetIDE Consortium. NetIDE deliverable 2.3 - NetIDE IRF APIs Integrated Platform v1,Apr 2015.

[21] OpenFlowJ Loxi. https://github.com/floodlight/loxigen/wiki/\OpenFlowJ-Loxi.

NetIDEPublic

[22] OpenDaylight Open�owJava. https://github.com/opendayligfloodlightht/

\openflowjava.

[23] Python struct. https://docs.python.org/2/library/struct.html.

[24] ONOS Interface OpenFlowController.

[25] ONOS Interface PacketContext.

[26] ONOS Interface NetConfController.

[27] Mirror of the OpenDaylight netide gerrit project. https://github.com/opendaylight/

netide.

[28] Ryu source code. https://github.com/osrg/ryu.

[29] The NetIDE Consortium. D3.5 - Developer Toolkit v2. Technical report, The EuropeanCommission, 2016.

[30] Vagrant by HashiCorp. =https://www.vagrantup.com/.

[31] OpenDaylight Developer WIKI, main page. https://wiki.opendaylight.org/view/Main_

Page, Nov 2015.

[32] The Apache Software Foundation. Apache Maven. https://maven.apache.org.

[33] Ryu API reference. http://ryu.readthedocs.io/en/latest/index.html, Sep 2016.

[34] The NetIDE Consortium. NetIDE deliverable 2.4 - NetIDE Core concepts and Architecturev2, Sep 2015.

[35] Handlebars template language. http://handlebarsjs.com/.

[36] Eclipse modeling framework. https://eclipse.org/modeling/emf/.

[37] Manpage of PCAP. http://www.tcpdump.org/manpages/pcap.3pcap.html.

[38] Wireshark. Network protocol analizer for Unix and Windows. https://www.wireshark.org/.

[39] The r project for statistical computing. https://www.r-project.org/.

[40] Rstudio - open source and enterprise-ready professional software for r. https://www.rstudio.com/.

[41] On Tra�c Patterns of HTTP Applications. https://hal.archives-ouvertes.fr/

hal-00685658/document.

[42] Publicly available PCAP �les. http://www.netresec.com/?page=PcapFiles.

[43] UMassTraceRepository. http://traces.cs.umass.edu/index.php/Network/Network.

[44] DCT2Gen. https://www-old.cs.uni-paderborn.de/en/research-group/

research-group-computer-networks/people/dr-philip-wette/dct2gen.html.

[45] TCP Replay. http://tcpreplay.appneta.com/.

[46] Roberto Doriguzzi-Corin, Domenico Siracusa, Elio Salvadori, and Arne Schwabe. EmpoweringNetwork Operating Systems with Memory Management Techniques. In IEEE/IFIP NetworkOperations and Management Symposium, Istanbul, Turkey, 25-29 April 2016.

NetIDEPublic

[47] P. A. A. Gutiérrez, E. Rojas, A. Schwabe, C. Stritzke, R. Doriguzzi-Corin, A. Leckey, G. Pe-tralia, A. Marsico, K. Phemius, and S. Tamurejo. Netide: All-in-one framework for next gener-ation, composed sdn applications. In 2016 IEEE NetSoft Conference and Workshops (NetSoft),2016.

[48] Joshua Reich, Christopher Monsanto, Nate Foster, Jennifer Rexford, and David Walker. Mod-ular SDN Programming with Pyretic. USENIX ;login, 38(5):128�134, Oct. 2013.

[49] Super�uidity Project | the ability to instantiate internet services on-the-�y. http://

superfluidity.eu/, jul 2015.

[50] Ben Pfa� and Bruce Davie. The Open vSwitch Database Management Protocol. RFC 7047,December 2013.

[51] The NetIDE Consortium. NetIDE - An integrated development environment for portablenetwork applications. Annex I: Description of work 619543, The European Commission, Nov2015.

[52] NetIDE Plugin. http://marketplace.eclipse.org/content/netide, 2016.

[53] NetIDE IDE Repository. https://github.com/fp7-netide/IDE.

[54] NetIDE Tools Repository. https://github.com/fp7-netide/Tools.

[55] R. Doriguzzi-Corin, E. Salvadori, M. Gerola, and M. Santuari. An Approach to Exposing andSharing Network Services in Software-De�ned Networking. In ACM Sigcomm Symposium onSDN Research (SOSR), 2015.

[56] Antonio Marsico, Roberto Doriguzzi-Corin, Matteo Gerola, Domenico Siracusa, and ArneSchwabe. A Non-disruptive Automated Approach to Update SDN Applications at Runtime. InIEEE/IFIP Network Operations and Management Symposium, Istanbul, Turkey, 25-29 April2016.

[57] P. A. A. Gutiérrez, T. Rasheed, C. Stritzke, R. Doriguzzi-Corin, and S. Tamurejo. NetIDE- Controller Independence supporting composed SDN Applications. http://www.eucnc.eu/

2016/www.eucnc.eu/indexe637.html?q=node/134, 2016.

[58] R. Doriguzzi-Corin, P. A. Aranda Gutiérrez, E. Rojas, H. Karl, and E. Salvadori. Reusabilityof Software-De�ned Networking Applications: A Runtime, Multi-Controller Approach. In 12thInternational Conference on Network and Service Management (CNSM), 2016.

[59] R. Enns, M. Bjorklund, J. Schoenwaelder, and A. Bierman. Network Con�guration Protocol(NETCONF). Technical Report 6241, IETF Secretariat, June 2011.

[60] M. Smith et al. OpFlex Control Protocol. Technical report, IETF, November 2014.

[61] OpenFlow Management and Con�guration Protocol (OF-Con�g 1.1). https://www.

opennetworking.org/images/stories/downloads/sdn-resources/onf-specifications/

openflow-config/of-config-1.1.pdf, Jun 2012.

NetIDEPublic

document: cnet-ict-619543-netide/d 2.7 date: … single module without composition ... nbi...

Documents

cnet architecture in netfpga

company profile - cnet solutions

cnet devicenet

cool tools. getting up to date… cnet 100 cnet reviews

iy cnet davidenetdevicenet_and_a_b_plc

cnet csh-2400s usermanual

contenido - cnet technology systems

cnet 301 presentation

netide runtime architecture consolidation - …...

lte3301-plus - cnet content

summit 16: netide: integrating and orchestrating sdn...

ml - cnet content

user's guide - cnet content solutions - english...user's...

overview - cnet content

user manual - cnet content

c o n t a n e t - cnet technology systems - cnet s.a

epsonaculaserc3900n - cnet content

basics - cnet content

cnet content solutions_intelligent_cross_sel

lcd monitor - cnet content