packet project calico keynote presentation
TRANSCRIPT
![Page 1: Packet Project Calico Keynote Presentation](https://reader031.vdocuments.net/reader031/viewer/2022020123/55a77d521a28abce668b4813/html5/thumbnails/1.jpg)
Let’s Talk Container Networking: Packet and Project Calico
Adam Rothschild <[email protected]>Co-Founder and SVP, Infrastructure
New York, NY - April 16, 2015
![Page 2: Packet Project Calico Keynote Presentation](https://reader031.vdocuments.net/reader031/viewer/2022020123/55a77d521a28abce668b4813/html5/thumbnails/2.jpg)
AGENDA
● Packet and Container Networking Use Cases
● Container Networking 101
● Leading Projects and Architectures
● To Overlay or Not to Overlay
● Review and Calico Intro
![Page 3: Packet Project Calico Keynote Presentation](https://reader031.vdocuments.net/reader031/viewer/2022020123/55a77d521a28abce668b4813/html5/thumbnails/3.jpg)
ABOUT PACKETAnd what’s with the robot...
![Page 4: Packet Project Calico Keynote Presentation](https://reader031.vdocuments.net/reader031/viewer/2022020123/55a77d521a28abce668b4813/html5/thumbnails/4.jpg)
PACKET’S MANIFESTO
● Provide high-performance bare metal compute nodes, available globally in <5 minutes.
● Ensure high-performance networking at all layers:
○ Bonded NICs at 2x1G or 2x10G○ Dense 10G/40G on the backbone (soon: 100G)○ Backbone network designed with copious capacity and eyeball delivery in mind○ High availability: no SPOF on the top of rack, or core network!
● Design for all of the above with “Internet scale” in mind:
○ Support many thousands of hosts inside a datacenter○ Support many thousands of disparate customers○ Support backend networking between datacenters○ Support IP mobility, global load balancing, between datacenters○ Legacy-free layer-3 datacenter network to meet these needs; zero VLANs and STP
throughout our entire production network (* some on our parallel management network, out of full disclosure :-)
![Page 5: Packet Project Calico Keynote Presentation](https://reader031.vdocuments.net/reader031/viewer/2022020123/55a77d521a28abce668b4813/html5/thumbnails/5.jpg)
OUR INTERNETS; L3 EVERYWHERE
to POP to POP
Optical (n*10G)
Copper (1G)
2*1G bonded 2*1G bonded
BBR
DSR DSR
ESR ESR ESR
Servar Servar
BBRPacket
Datacenter
Carrier Hotel POPCarrier Hotel POP
ESR
to NSP/peering
CSR CSRto NSP/peering
![Page 6: Packet Project Calico Keynote Presentation](https://reader031.vdocuments.net/reader031/viewer/2022020123/55a77d521a28abce668b4813/html5/thumbnails/6.jpg)
CUSTOMER NEEDS
Stripped down to the essentials, Packet’s ecosystem and “Internet scale” customers want:
● Simple node discovery, ability to move a service/container between hosts without provider coordination
● Secure and free/cheap back-end transfer● IP takeover (e.g. heartbeat) for HA/DR use cases● Security zoning between server tiers● VPN between diverse providers or cloud environments (e.g. mix and match
AWS, DO, SL compute)
![Page 7: Packet Project Calico Keynote Presentation](https://reader031.vdocuments.net/reader031/viewer/2022020123/55a77d521a28abce668b4813/html5/thumbnails/7.jpg)
CONTAINER NETWORKING 101
Defaults and Concepts
![Page 8: Packet Project Calico Keynote Presentation](https://reader031.vdocuments.net/reader031/viewer/2022020123/55a77d521a28abce668b4813/html5/thumbnails/8.jpg)
DOCKER NETWORKINGThe Problem:
![Page 9: Packet Project Calico Keynote Presentation](https://reader031.vdocuments.net/reader031/viewer/2022020123/55a77d521a28abce668b4813/html5/thumbnails/9.jpg)
DOCKER NETWORKING
● Remember, Containers != Small VM’s● Environments can be expected to have rapid creation / deletion of
containers● Containers are expected to move around alot● Finally something that is cross provider -- but what about the network?● Security? What security?
![Page 10: Packet Project Calico Keynote Presentation](https://reader031.vdocuments.net/reader031/viewer/2022020123/55a77d521a28abce668b4813/html5/thumbnails/10.jpg)
DOCKER NETWORKING
To recap, if you’re running containers today, your networking options include:
● Run docker separately on each box; exposing ports on public or private interfaces so containers can talk to each other.
● Run in-between / overlay solutions like Weave to fully abstract the networking.● Run "ready-to-go" multi-host platforms for docker like Deis, Flynn, or Rancher.● Build a shared bridge on a meshed network among your boxes and get your Docker services to
spawn containers there.
![Page 11: Packet Project Calico Keynote Presentation](https://reader031.vdocuments.net/reader031/viewer/2022020123/55a77d521a28abce668b4813/html5/thumbnails/11.jpg)
LEADING PROJECTSOpen Source Projects and Architectures
![Page 12: Packet Project Calico Keynote Presentation](https://reader031.vdocuments.net/reader031/viewer/2022020123/55a77d521a28abce668b4813/html5/thumbnails/12.jpg)
Route based
THE LANDSCAPE
FLANNEL
WEAVE
Overlay / tunnel based
![Page 13: Packet Project Calico Keynote Presentation](https://reader031.vdocuments.net/reader031/viewer/2022020123/55a77d521a28abce668b4813/html5/thumbnails/13.jpg)
ROCKET (PROPOSED)
![Page 14: Packet Project Calico Keynote Presentation](https://reader031.vdocuments.net/reader031/viewer/2022020123/55a77d521a28abce668b4813/html5/thumbnails/14.jpg)
WEAVE
![Page 15: Packet Project Calico Keynote Presentation](https://reader031.vdocuments.net/reader031/viewer/2022020123/55a77d521a28abce668b4813/html5/thumbnails/15.jpg)
RANCHER
1. Creates a full-mesh network between containers using IPSEC tunnels2. Rancher server orchestrates IPSEC setup and key management. Fully automatic. No manual steps required3. IPSEC tunnels are created on demand to reduce overhead4. Rancher server manage DHCP server and IP address allocation. Allocates 10.42/16 address globally by default.5. Rancher takes over container linking from Docker6. Works with Docker native API and command line. Rancher listens for Docker events and sets up networking in the background.7. Optionally disable encryption to improve performance
![Page 16: Packet Project Calico Keynote Presentation](https://reader031.vdocuments.net/reader031/viewer/2022020123/55a77d521a28abce668b4813/html5/thumbnails/16.jpg)
THE APPROACHES: OVS
![Page 17: Packet Project Calico Keynote Presentation](https://reader031.vdocuments.net/reader031/viewer/2022020123/55a77d521a28abce668b4813/html5/thumbnails/17.jpg)
OCTO TECHNOLOGY
...seriously?!
![Page 18: Packet Project Calico Keynote Presentation](https://reader031.vdocuments.net/reader031/viewer/2022020123/55a77d521a28abce668b4813/html5/thumbnails/18.jpg)
TO OVERLAY OR NOT TO OVERLAYThat’s the Question
![Page 19: Packet Project Calico Keynote Presentation](https://reader031.vdocuments.net/reader031/viewer/2022020123/55a77d521a28abce668b4813/html5/thumbnails/19.jpg)
OVERLAY CONCERNSI am an infrastructure operator, not a software engineer, by trade, and I see many operational challenges:
● Performance bottleneck as additional encapsulations (e.g. VXLAN, GRE, MPLS) and crypto (IPSec, SSL/TLS) are handled by the host○ Not a problem with VMs, as network is constrained before compute○ Very much a problem with powerful bare metal
● Overlay agents are required on the host/hypervisor layer● Topology is extremely complex, difficult to understand and troubleshoot; is
the connectivity problem with the overlay or underlay?● IP multicast required for some implementations; “BUM” flooding● Annoying defaults: [double] NAT, hardcoded subnets (which could overlap)● Scaling to large host count is still an issue with many implementations!● All of this complexity for questionable gain, as we are about to see...
![Page 20: Packet Project Calico Keynote Presentation](https://reader031.vdocuments.net/reader031/viewer/2022020123/55a77d521a28abce668b4813/html5/thumbnails/20.jpg)
NEEDS REVISITED
Do we actually need an overlay for these?
● Secure and free/cheap back-end transfer● Simple node discovery, ability to move a service/container between hosts
without provider coordination● IP takeover (e.g. heartbeat) for HA/DR use cases● Security zoning between server tiers● VPN between diverse providers or cloud environments (e.g. mix and match
AWS, DO, SL compute)
VPN is the only real use case today, and it’s becoming decreasingly relevant as the cloud interconnection market takes off. See: Equinix Cloud Exchange, SoftLayer and Amazon Direct Connect, et al.
![Page 21: Packet Project Calico Keynote Presentation](https://reader031.vdocuments.net/reader031/viewer/2022020123/55a77d521a28abce668b4813/html5/thumbnails/21.jpg)
GET INVOLVED!
● Rocket Networking Spec: https://github.com/coreos/rkt/issues/273
● Docker Networking Spec: https://github.com/docker/docker/issues/8951