cilium: seattle kubernetes meetup dec 2017

Application-Aware Security for Microservices via BPF

Cynthia Thomas, Technology Evangelist@_techcet_

Seattle Kubernetes MeetUpDecember 12th, 2017

Open Source Cloud Native Security

Application Architectures

Delivery Frequency

Operational Complexity

Single Server App

Yearly

Low

Evolution of Application Design & Delivery Frequency


Delivery Frequency


Single Server App

Yearly

Low

3-Tier App

Monthly

Moderate



Delivery Frequency


Single Server App

Yearly

Low

Distributed Microservices

10-100 x’s / day

Extreme

3-Tier App

Monthly

Moderate


Network Securityhas barely evolved

$ iptables -A INPUT -p tcp \-s 15.15.15.3 --dport 80 \-m conntrack --ctstate NEW \-j ACCEPT

The world still runs on iptablesmatching IPs and ports:

Your HTTP ports be like …

Network Securityfor Microservices

Gordon the intern has a brilliant idea…

Gordon wants to build a serviceto tweet out all job offerings.

We’re Hiring!

TweetService

GET /healthz

GET /jobs/{id}

GET /applicants/{job-id}

POST /jobs

APIGET /jobs/{id}

Jobs APIService

TweetService

The Jobs API service has all thedata Gordon needs.

GET /healthz

GET /jobs/{id}


POST /jobs

APIGET /jobs/331

GET /jobs/{id}

Jobs APIService

TweetService

Gordon uses the GET /jobs/ API call

GET /healthz

GET /jobs/{id}


POST /jobs

APIGET /jobs/331

GET /jobs/{id}

TLS Jobs APIService

TweetService

Developer etiquette.Super simple stuff.

Gordon uses mutual TLS AuthGood thinking Gordon

L3/L4

GET /healthz

GET /jobs/{id}


POST /jobs

APIGET /jobs/331

The security team has L3/L4 network security in place for all services

GET /jobs/{id}

Jobs APIService

TweetService

TLS

iptables -s 10.1.1.1-p tcp --dport 80-j ACCEPT

Gordon could POST /jobs or GET /applicants(mistakenly or haphazardly).

POTUS job available!

TweetService

Jobs APIService

L3/L4

GET /healthz

GET /jobs/{id}


POST /jobs

API

exposed

exposed

exposed

GET /jobs/331

Large parts of the API are still exposed unnecessarily

TweetService

GET /jobs/{id}

TLS

iptables -s 10.1.1.1-p tcp --dport 80-j ACCEPT

Not exactlyleast privilegeSecurity

GET /healthz

GET /jobs/{id}


POST /jobs

APIGET /jobs/331

Back to the drawing board…

GET /jobs/{id}

TLS Jobs APIService

TweetService

L3/L4

GET /healthz

GET /jobs/{id}


POST /jobs

APIGET /jobs/331

Least privilege security for microservices

GET /jobs/{id}

FROM“TurtleTweets”ALLOW“GET/jobs/”

TLS Jobs APIService

TweetService

We demanda demo

BPF - TheSuperpowersinside Linux

KubernetesIntegration


NetworkPolicy

StandardResources

L3,L4policy(ingressonlyink8s1.7)


NetworkPolicy

Services

StandardResources

L3,L4policy

ClusterIP,NodePort,LoadBalancer


NetworkPolicy

Services

StandardResources

L3,L4policy

Pods PodLabelstospecifypolicyon



NetworkPolicy

Services

StandardResources

L3,L4policy

Nodes



NodeIP toNodeCIDRmapping


NetworkPolicy

CiliumNetworkPolicy

Services

StandardResources

CustomResourceDefinitions(CRD)

L3,L4policy

L3(Labels/CIDR),L4,L7(ingress&egress)

Nodes



NodeIP toNodeCIDRmapping

ShouldIencapsulateornot?

Node1

Node2

Node3

ModeI:Overlay


Node1

Node2

Node3

ModeI:Overlay

Name NodeIP Node CIDRNode 1 192.168.10.1 10.0.1.0/24Node 2 192.168.10.8 10.0.2.0/24Node 3 192.168.10.9 10.0.3.0/24

KubernetesNoderesourcestable:

Installation

Run the kube-controller-manager with the --allocate-node-cidrsoption


ModeI:Overlay ModeII:NativeRoutingNode1

Node2

Node3

L3 Network

Usecase:• Runyourownroutingdaemon• Usethecloudprovider’srouter

Usecase:• Simple• “Justworks”onKubernetes

Node1

Node2

Node3

L3 Policy (Labels Based)

Metadata

Allow frompods

Pods the policyapplies to…

From Pod

To Pod

L3 Policy (CIDR)

Metadata

Allow toIP 8.8.8.8/32

Pods the policyapplies to…

To CIDR

From Pod

L4 Policy

Metadata

Policy appliesto pods …

Allow incomingon port 80

Pod

To Port

L4 Policy

Rule 2:Allow PUTIf header is set

Rule 1:Allow “GET /v/1”

L7 Policy – Only allow “GET /v1/”

Allowed API

Calls

How are these policies enforced?


• L3 & L4: BPF in the kernel


• L3 & L4: BPF in the kernel

• L7: Sidecar proxy or KProxy / BPF

Node 2Node 1

ServiceService HTTPRequest

What is a sidecar proxy?

Node 1

Service

SidecarProxy


Node 2

Service

SidecarProxy

Node 2Node 1

ServiceService

HTTPRequestSidecarProxy

SidecarProxy


Node 2Node 1

ServiceService

HTTPRequestSidecarProxy

SidecarProxy


Provides L7 functionality• Routing / Load balancing• Retries

• Circuit breaking• Metrics

More info? Google is your friend “sidecar” / “service mesh”

Node 2Node 1

Service

OperatingSystem

Service

Network

SidecarProxy

SidecarProxy

Socket

TCP/IP

Socket

TCP/IP

Socket

TCP/IP

Socket

TCP/IP

Socket

TCP/IP

Socket

TCP/IP

• 3x Socket memory requirement• 3x TCP/IP stack traversals• 3x Context switches• Complexity

Networking Path with a Sidecar

Network

Canweturnthesidecarintoaracecar?

Node 2Node 1

Task

OperatingSystem

Kernel Proxy

Task

Network

Socket

KProxywithBPF

TCP/IP

Socket

TCP/IP

KProxywithBPF

kTLS kTLSSidecarProxy

SidecarProxy

Network

Socket Redirect

Task

Socket Socket

Task

TCP/IP TCP/IP

Loopback

Socket Redirect – Performance?

More info: https://www.cilium.io/blog/istio

Node 2Node 1

Service

OperatingSystem

Service

Network

SidecarProxy

SidecarProxy

Socket

TCP/IP

Socket

TCP/IP

Socket

TCP/IP

Socket

TCP/IP

Socket

TCP/IP

Socket

TCP/IP

The Before and After

Network

Node 1 Node 2

Service

OperatingSystem

Service

Network

Socket

TCP/IP

The Before and After

KProxy

Socket

TCP/IP

KProxy

Network

Cilium Summary• Kubernetes, Mesos, Docker

• CNI / libnetwork

• Networking: Overlay or Native Routing

• Network Security (ingress/egress)

• L3 (Identity or CIDR), L4

• L7: HTTP (0.11), Kafka (0.12), gRPC (0.12)

• Load Balancing (XDP / BPF)

• Dependencies: kvstore (etcd / consul)

Application-Aware Security for Microservices via BPF

@ciliumprojectStar Us on GitHub! http://github.com/cilium/cilium

Thank You! Questions?Tutorial / Getting Started:http://cilium.io/try