automating the network

Automating the Network

1

Matt Peterson Office of the CTO

Cumulus Networks PuppetCamp London – November 3rd, 2015

whois!

AS36472 !

Puppet’ing since 0.24 (2008) Added CL support in 2013

Rocket Turtle?

§ Cumulus Networks®

Why {Net}DevOps?

cumulusnetworks.com 4

greater operational insight

Networking is the last infrastructure silo

{Net}DevOps


Leverage common DevOps tenants within Networking

§ Configuration management (today’s focus) § Make repetitive tasks predictable § Enforce policy to desired state - naming schemes,

software versions, … great audit trail mechanism

§ Infrastructure as code § Describe physical & logical components in human &

machine readable formats

{Net}DevOps


§ Reactive to infrastructure as a whole §  ie: L3 Clos architecture, single spine switch fans fail,

automatically adjust IGP cost to route around

§ Consistency (sometimes viewed as transparency) § Source of truth (across entire org, if possible), Git is

popular in compute circles § Common monitoring & escalation systems

A brief history of DevOps

§ …well, almost.


Traditional network automation

§ Top-down efforts §  SNMP §  YANG/NETCONF §  OpenFlow

§ Ansible §  Screen-scraping

§ Difficult to integrate with the rest of the Ops stack


“Whitebox switch” or bare metal networking

§  Treat as a server, preforms like a switch (or router) •  Kernel networking state sync to ASIC (and vice versa) •  ifconfig, ethtool, isc-dhcp … apt-get pkgname!

§  Consistent tooling across compute & networking •  CLI is usually bash, no walled garden •  Automation friendly from day 1

§  Choice on HW & SW suppliers •  Same as bare metal computing is today •  Applies to costly optics & cabling too!

routing bridging ops

Cumulus Linux

hardware

Modern network automation


Modern network automation

§ Zero Touch Provisioning (ZTP) §  Normally triggered by DHCP

§ Runs a provisioning script §  Performs personalization §  Install & enable Puppet

§ Puppet takes over after Cumulus install


ZTP with DHCP example dhcp.conf

§  ddns-‐update-‐style none; §  default-‐lease-‐time 4320; §  max-‐lease-‐time 8640; §  authoritative;

§  option cumulus-‐provision-‐url code 239 = text;

§  subnet 192.168.0.0 netmask 255.255.255.0 { §  range 192.168.0.100 192.168.0.200; §  option routers 192.168.0.1; §  option domain-‐name-‐servers 192.168.0.1; §  option domain-‐name "lab.mycompany.com"; §  option cumulus-‐provision-‐url "http://192.168.0.2/provision.sh"; §  }

ZTP with DHCP example dhcp.conf


dhclient.conf option cumulus-‐provision-‐url code 239 = text;

Exit hook 1. #!/bin/bash 2. # Copyright 2015 Cumulus Networks, inc all rights reserved 3. if [ ! -‐z $new_cumulus_provision_url ] 4. then 5.  /usr/lib/cumulus/autoprovision -‐-‐log syslog -‐-‐url $new_cumulus_provision_url & 6. fi

ZTP example with Puppet

1 #!/bin/bash 2 3 function error() { 4 echo -‐e "\e[0;33mERROR: Provisioning failed running $BASH_COMMAND at line $BASH_LINENO of $(basename $0) \e[0m" >&2

5 exit 1 6 } 7 trap error ERR 8 # Allow Cumulus testing repo 9 sed -‐i /etc/apt/sources.list -‐e 's/^#\s*$deb.*testing.*$$/\1/g'

10 11 # Upgrade and install Puppet 12 apt-‐get update -‐y

13 apt-‐get upgrade -‐y 14 apt-‐get install puppet -‐y 15 16 echo "Configuring puppet" | wall -‐n 17 sed -‐i /etc/default/puppet -‐e 's/START=no/START=yes/'

18 19 service puppet restart 20 21 # CUMULUS-‐AUTOPROVISIONING 22 23 exit 0

§ Debian based, so we can do everything via. Puppet

§  Interface configuration (including Bonds & Bridges)

§  Routing software (Quagga – BGP/OSPF) configuration

§  Users, authentication, logging, packages, monitoring…

Puppet

Managing network interfaces


auto swp1 iface swp1 address 192.168.1.10 netmask 255.255.255.0 mtu 9000 auto swp2 iface swp2 address 172.16.2.21 mtu 9000 … auto swp48 iface swp48 address 10.1.1.10 netmask 255.255.254.0 mtu 9000



$swp1 = { 'name' => 'swp1', 'address' => '192.168.1.10', 'netmask' => '255.255.255.0' } $swp2 = { 'name' => 'swp2', 'address' => '172.16.2.21' } $swp48 = { 'name' => 'swp48', 'address' => '10.1.1.10', 'netmask' => '255.255.254.0' } $ifaces = [ $swp1, $swp2, $swp48 ] file { '/etc/network/interfaces': ensure => file, content => template('interfaces.erb') }

<% @ifaces.each do |iface| -‐%> auto <%= iface['name'] %> iface <%= iface['name'] %> address <%= iface['address'] %> <% if iface.has_key?('netmask') -‐%> netmask <%= iface['netmask'] %> <% end -‐%> mtu 9000 <% end %>



1.  auto lo 2.  iface lo inet loopback 3.  <% if @int_loopback -‐%> 4.  address <%= int_loopback %> 5.  netmask 255.255.255.255 6.  <% end -‐%>

7.  auto eth0 8.  iface eth0 inet dhcp

9.  <% if @int_unnumbered -‐%> 10. # unnumbered interfaces 11. <% @int_unnumbered.each do |val| -‐%> 12. auto <%= val %> 13. iface <%= val %> inet static 14.  address <%= int_loopback %> 15.  netmask 255.255.255.255

16. <% end -‐%> 17. <% else %> 18. # no unnumbered interfaces 19. <% end -‐%>

20. <% if @int_layer3 %> 21. # l3 interfaces 22. <% int_layer3.each_pair do |key, value_hash| %>

23. auto <%= key %> 24. iface <%= key %> inet static 25.  address <%= value_hash['address'] %> 26.  netmask <%= value_hash['netmask'] %>

27. <% end %> 28. <% else %> 29. # no l3 interfaces 30. <% end %>

31. <% if int_bridges %> 32. # bridges 33. <% int_bridges.each_pair do |key, value_hash| %> 34. auto <%= key %> 35. iface <%= key %> inet static 36.  address <%= value_hash['address'] %> 37.  netmask <%= value_hash['netmask'] %> 38.  bridge_ports <% value_hash['members'].each do |val| -‐

%><%= val%> <% end -‐%>

39. <% end %> 40. <% else %> 41. # no bridges 42. <% end %>


§ Everything in one file §  Pros

§  Simple to implement and understand §  Standard (no surprises)

§  Cons §  Not simple if you have complex configurations §  If one interface changes everything changes




etc └── network ├── interfaces └── interfaces.d ├── eth0 ├── lo ├── swp1 ├── swp2 ├── ... └── swp48

auto swp48 iface swp48 address 10.1.1.10 netmask 255.255.254.0 mtu 9000

source /etc/network/interfaces.d/*


§ Idempotentency §  notify => Service['networking']

§  Change one interface, everything is reloaded

§ ifupdown2 §  Part of Cumulus Linux, optional in Debian, Ubuntu §  ifreload –a




define iface { $id = $name["name"] $address = $name["address"] $netmask = $name["netmask"] file { "/etc/network/interfaces.d/$id": ensure => file, content => template("iface.erb"), require => File["/etc/network/interfaces.d"], notify => Exec["reload_$id"], } exec { "reload_$id": command => "ifdown $id;ifup $id", path => ["/sbin"], refreshonly => true, } } iface{ $ifaces: }

Cumulus Linux Puppet Modules


cumulus_interface {'swp1': ipv4 => '192.168.1.10/24', } cumulus_interface {'swp2': ipv4 => '172.16.2.21', } cumulus_interface {'swp48': ipv4 => '10.1.1.10/24', }

Bridges & Bonds


cumulus_bond {'uplink': slaves => ['swp10-‐11'] } cumulus_bridge { 'br0': ports => ['swp1-‐2'] ipv4 => ['10.1.1.1/24'] ipv6 => ['2001:db8:abcd::/48'] alias_name => 'LXC bridge' mtu => 9000 }

Managing Layer 3

§ Quagga §  OSPF, ECMP, BGP §  Focused on an interactive CLI

•  Makes automation a little awkward §  Graceful reload is experimental

§ Other Open Source §  Bird §  OpenBGPD §  ExaBGP


Firewalls & ACLs

§ Firewalls §  UFW rules are dead easy

•  UFW Puppet modules exist §  RedHat users are also covered

•  Lokkit modules exist

§ IPTables §  Loads of IPTables modules to choose from

•  If you need it!


Testing

§ Network Infrastructure as Code §  It’s code!

§ Acceptance testing §  Many vendors provide a virtual machine §  You can connect them virtually into topologies


Testing


Testing

§ Vagrant

§ Serverspec §  BATS, Cucumber, behave?

§ Beaker

§ GNS3


Summary

§  Expressing networks as human (ie: virtual diagram) and machine (YAML) = powerful (see PTM as an example)

§  HW appliances of all shapes are turning into pure SW plays (some begrudgingly)

§  {Net}DevOps transformation has begun in networking •  Please help these men and women share your journey!

© 2014 Cumulus Networks. CUMULUS, the Cumulus Logo, CUMULUS NETWORKS, and the Rocket Turtle Logo (the “Marks”) are trademarks and service marks of Cumulus Networks, Inc. in the U.S. and other countries. You are not permitted to use the Marks without the prior written consent of Cumulus Networks. The registered trademark Linux® is used pursuant to a sublicense from LMI, the exclusive licensee of Linus Torvalds, owner of the mark on a world-wide basis. All other marks are used under fair use or license from their respective owners.

§ Thank You!


Matt Peterson [email protected] @dorkmatt

automating the network

Technology