Download - Leverage LXC/LXD with Kubernetes
![Page 1: Leverage LXC/LXD with Kubernetes](https://reader033.vdocuments.net/reader033/viewer/2022052514/58e519881a28ab624e8b4d67/html5/thumbnails/1.jpg)
Leverage LXC/LXD with Kubernetes Jason McGee, IBM Fellow and VP, IBM Lin Sun, Senior Software Engineer, IBM
![Page 2: Leverage LXC/LXD with Kubernetes](https://reader033.vdocuments.net/reader033/viewer/2022052514/58e519881a28ab624e8b4d67/html5/thumbnails/2.jpg)
Agenda• Background • Why are we looking at this? • Experiments • Demo • Summary
![Page 3: Leverage LXC/LXD with Kubernetes](https://reader033.vdocuments.net/reader033/viewer/2022052514/58e519881a28ab624e8b4d67/html5/thumbnails/3.jpg)
Beta available March 20th.
Combining Docker and Kubernetes to deliver powerful tools, an intuitive user experience, and built-in security and isolation to enable rapid delivery of applications - all while leveraging IBM Cloud Services including cognitive capabilities from Watson.
www.ibm.com/cloud-computing/bluemix/containers
IBM Bluemix Container Service
![Page 4: Leverage LXC/LXD with Kubernetes](https://reader033.vdocuments.net/reader033/viewer/2022052514/58e519881a28ab624e8b4d67/html5/thumbnails/4.jpg)
Intelligent Scheduling Automated rollouts and rollbacks Container Security & IsolationDesign Your Own Cluster
Self-healing Horizontal scaling Leverages IBM Cloud & Watson Integrated Operational Tools
S M L
Service discovery & load balancing Secret & configuration management Simplified Cluster Management Native Kubernetes Experience
IBM Bluemix Container Service
![Page 5: Leverage LXC/LXD with Kubernetes](https://reader033.vdocuments.net/reader033/viewer/2022052514/58e519881a28ab624e8b4d67/html5/thumbnails/5.jpg)
IBM Bluemix | IBM Confidential | ©2017 IBM Corporation
Architecture
• Free tier worker is deployed in our account
• One free tier worker per account
• Paid tier workers are deployed in customer’s account
• Carrier-Cruiser model • Hub-Spoke model
![Page 6: Leverage LXC/LXD with Kubernetes](https://reader033.vdocuments.net/reader033/viewer/2022052514/58e519881a28ab624e8b4d67/html5/thumbnails/6.jpg)
Free tier of IBM Bluemix Container Service
![Page 7: Leverage LXC/LXD with Kubernetes](https://reader033.vdocuments.net/reader033/viewer/2022052514/58e519881a28ab624e8b4d67/html5/thumbnails/7.jpg)
Requirements for free tier
• Each tenant has only 1 kubernetes worker (2 CPU, 4 GB memory) • Isolation between each tenant • Fast launch and destroy clusters • Minimum cost yet providing a lightweight native kubernetes
experience • Easy migration to paid tier
![Page 8: Leverage LXC/LXD with Kubernetes](https://reader033.vdocuments.net/reader033/viewer/2022052514/58e519881a28ab624e8b4d67/html5/thumbnails/8.jpg)
Why are we looking at this?
• Increase density for free tier • Reduce cost for free tier • Fast deployment for free tier worker • Quick tear down for free tier worker • Many free tier clusters are idle
![Page 9: Leverage LXC/LXD with Kubernetes](https://reader033.vdocuments.net/reader033/viewer/2022052514/58e519881a28ab624e8b4d67/html5/thumbnails/9.jpg)
Experiments we explored• Run kubernetes worker in docker containers • Run kubernetes in LXC container • Run kubernetes worker in LXC Container
![Page 10: Leverage LXC/LXD with Kubernetes](https://reader033.vdocuments.net/reader033/viewer/2022052514/58e519881a28ab624e8b4d67/html5/thumbnails/10.jpg)
Run Kubernetes worker in Docker containers
• We started with running kubernetes worker in Docker • It works but requires Docker container in privileged mode
![Page 11: Leverage LXC/LXD with Kubernetes](https://reader033.vdocuments.net/reader033/viewer/2022052514/58e519881a28ab624e8b4d67/html5/thumbnails/11.jpg)
Introduction of LXD• LXD is a container hypervisor and a new user experience for LXC • Not a rewrite of LXC, led by Canonical, Ltd • 2 Key components
• A system-wide deamon (lxd) • A command line client (lxc)
• Docker vs LXD • Docker specializes in deploying applications • LXD specializes in deploying (Linux) Virtual Machines
![Page 12: Leverage LXC/LXD with Kubernetes](https://reader033.vdocuments.net/reader033/viewer/2022052514/58e519881a28ab624e8b4d67/html5/thumbnails/12.jpg)
Run kubernetes in LXC containers• Kubernetes (master + worker) in non privileged LXC container
docker profile • Can’t run Docker privileged container
• Kubernetes processes directly run in LXC • A few kubernetes containers require privileged access
![Page 13: Leverage LXC/LXD with Kubernetes](https://reader033.vdocuments.net/reader033/viewer/2022052514/58e519881a28ab624e8b4d67/html5/thumbnails/13.jpg)
Run kubernetes worker in LXC containers• Kubernetes worker in
non privileged LXC container docker profile
• Kubernetes worker processes directly run in LXC
• Easy migration to paid tier
![Page 14: Leverage LXC/LXD with Kubernetes](https://reader033.vdocuments.net/reader033/viewer/2022052514/58e519881a28ab624e8b4d67/html5/thumbnails/14.jpg)
Run kubernetes worker in LXC containers
![Page 15: Leverage LXC/LXD with Kubernetes](https://reader033.vdocuments.net/reader033/viewer/2022052514/58e519881a28ab624e8b4d67/html5/thumbnails/15.jpg)
Run kubernetes worker in LXC containers
![Page 16: Leverage LXC/LXD with Kubernetes](https://reader033.vdocuments.net/reader033/viewer/2022052514/58e519881a28ab624e8b4d67/html5/thumbnails/16.jpg)
Run kubernetes worker in LXC containers• Demo!
![Page 17: Leverage LXC/LXD with Kubernetes](https://reader033.vdocuments.net/reader033/viewer/2022052514/58e519881a28ab624e8b4d67/html5/thumbnails/17.jpg)
Density with LXC & Kubernetes• Current Free tier: 2 Core, 4 GB memory • With our LXC Experiment
• 8 Core, 8GB memory LXD host • Each LXC with idle k8s worker running: 140MB peak, 100MB average • Each LXC with k8s worker and guestbook example: 1.5GB peak, 800MB average • LXC supports hard memory limit by default but allows for soft limit • Can run 10+ LXC Kubernetes workers, assume 20% workers are highly used while
rest are idle
![Page 18: Leverage LXC/LXD with Kubernetes](https://reader033.vdocuments.net/reader033/viewer/2022052514/58e519881a28ab624e8b4d67/html5/thumbnails/18.jpg)
List of Issues we opened• Privileged Docker containers in LXD: https://github.com/lxc/lxd/
issues/2825 • Skip OOM score adjust in unprivileged containers
• https://github.com/kubernetes/kubernetes/pull/43079 • https://github.com/opencontainers/runc/pull/1386
![Page 19: Leverage LXC/LXD with Kubernetes](https://reader033.vdocuments.net/reader033/viewer/2022052514/58e519881a28ab624e8b4d67/html5/thumbnails/19.jpg)
Summary of the experiment• LXC/LXD provides fast deployment, much higher density thus lower cost • Easy migration to paid tier • Wish lists:
• Explore cpu/mem limits options • Explore copy/snapshot features • Explore DNS • Explore Kubernetes keys and certs
![Page 20: Leverage LXC/LXD with Kubernetes](https://reader033.vdocuments.net/reader033/viewer/2022052514/58e519881a28ab624e8b4d67/html5/thumbnails/20.jpg)
Thank you!