Building Open Source with Open Source

Susan Wu Director of Technical Marketing

Midokura

About Me §  Took the scenic drive to Open Source from proprietary software like Oracle, Citrix, Sun §  Product marketing for container technologies like Solaris Zones, Docker §  Plugins/Connectors for enterprise manager § Open Source community experience: Ubuntu, Docker, OpenStack, CloudStack, MidoNet

About Midokura § Global startup with about 50+ employees §  Founders built distributed systems at Amazon and Google §  Engineering in Barcelona, Tokyo, Tel Aviv §  Sales and Marketing in San Francisco §  Network Virtualization for OpenStack and Docker §  Customers in web scale, enterprise, higher ed and service provider segments

Why Open Source §  It’s actually a two-fold question…for Midokura

-  Why build virtualization software for networking with open source technologies -  Why did the company open source the software

§  Today, we’re covering the first topic -  Why we build with open source -  And how we use open source technologies like Zookeeper and Cassandra

/* Copyright (c) 2015 Midokura SARL, All Rights Reserved. */

import Ember from 'ember';import isValidIP4 from 'midonet-manager/utils/is-valid-ip4';import {  copyOpt121Route,  createOpt121Route} from 'midonet-manager/utils/dhcp-subnets';import DHCPSubnetCommons from 'midonet-manager/mixins/dhcp-subnet; -commons’;

const {  get,  set,  setMeta,  Logger: {    debug  },  Component} = Ember;

export default Component.extend(DHCPSubnetsCommons, {  selectedDevice: null,

 actions: {

   update() {      let model;      let networkId;

     let {       dnsServerAddrs,       defaultGateway,       subnetPrefix,       subnetLength,       interfaceMTU,       opt121Routes      } = this.validateDHCPSubnet();

     model = get(this, 'model');      networkId = get(this, 'selectedDevice').get('id');      setMeta(model, '_networkId', networkId);      setMeta(model, '_uuidBeforeSave', model.get('id'));

         

Open Source Leads in Innovation §  Used to be imitation and commoditization to an expensive, proprietary solution §  Now open source leads in innovation §  Leveraging open source as building blocks speed up innovation § OpenStack - building an advanced Infrastructure as a service (IaaS) platform with open

source

§  Same with open source MidoNet

Cloud Networking

MidoNet  Topology

Intelligence at the Edge MidoNet leverages a distributed architecture where the SDN intelligence is pushed at the edge

1. VM 1 sends a packet through the virtual network 2. MN Agent fetches the virtual topology/state 3. It simulates the packet through the virtual network 4. It installs a flow rule in the kernel at the ingress host 5. Tunnel packets to egress host

Data type drives technology used

Data type drives technology used

3. Host State 4. Flow State

Host State Active Inactive

Flow State Flow IP Source IP Destination Port Source Port Destination Protocol

NAT Router ID Type of NAT (dst or src) NAT Binding

Connection Tracking + Device ID

Can only optimize 2 out of 3

Zookeeper provides consistency

§  Suited for Consistency §  Performance (in memory, r > w) -  In memory: limits dataset size (backed by disk)

§  Reliable delivery -  If a node sees an update, all will eventually

§  Total & Causal Order - Data is delivered in the same order it is sent - A message m is delivered only after all messages sent before m have been delivered

§ Not suited for high write volume

Consistency Availability Partition Tolerance Solution Virtual Network Topology X X Zookeeper Virtual Network State X X Zookeeper Host State X X Zookeeper

Three Zookeeper use cases

§ Watch for Data Changes §  Service Discovery §  Coordination

Zookeeper tracks the changes

Zookeeper notifies subscribers Updating changes to logical network configurations

Deploy in odd numbers for decisive vote Leader

Service Discovery

Leader

Cassandra used for backup

§ Well suited for high write volume §  Used in storing stateful connection tracking information like flow state and metrics §  Chosen for availability §  Although the agent can function in the case of Cassandra failure as flow state is stored in

memory, and so Cassandra is primarily used as backup

Consistency Availability Partition Tolerance Solution

Flow State X X Cassandra

Fault Tolerance

FAIL

w(x)ok

ok

ok

Fault Tolerance

x

x

x

Trading off Consistency

read_repair

x1

x

x

Trading off Consistency

?

Must Haves: Low Latency, High Throughput for Network Address Translation

§  Trivial Schema Changes -  vs. schema changes in shards

§  Flexible (good on growth mode)

Putting it all together §  Start with a packet § What’s the virtual

topology? § Go there (you and

all other similar packets)

Ready to Try MidoNet?

https://www.midonet.org/#quickstart