what can-be-done-around-mesos
TRANSCRIPT
![Page 2: What can-be-done-around-mesos](https://reader034.vdocuments.net/reader034/viewer/2022052705/58ab04c61a28abd35e8b64bd/html5/thumbnails/2.jpg)
Overview
• Frameworks on Mesos
• PaaS Based on Mesos
• CI/CD Based on Mesos
• Big Data Based on Mesos
![Page 3: What can-be-done-around-mesos](https://reader034.vdocuments.net/reader034/viewer/2022052705/58ab04c61a28abd35e8b64bd/html5/thumbnails/3.jpg)
Frameworks on Mesoshttp://mesos.apache.org/documentation/latest/
frameworks/
![Page 4: What can-be-done-around-mesos](https://reader034.vdocuments.net/reader034/viewer/2022052705/58ab04c61a28abd35e8b64bd/html5/thumbnails/4.jpg)
ways of running on Mesos
• 3 ways, while every coin has two sides
✓ develop scheduler running on Mesos
✓ components containerized(options) + marathon
✓ batch restful api
![Page 5: What can-be-done-around-mesos](https://reader034.vdocuments.net/reader034/viewer/2022052705/58ab04c61a28abd35e8b64bd/html5/thumbnails/5.jpg)
![Page 6: What can-be-done-around-mesos](https://reader034.vdocuments.net/reader034/viewer/2022052705/58ab04c61a28abd35e8b64bd/html5/thumbnails/6.jpg)
develop scheduler running on Mesos
• Myriad: yarn-on-mesos
![Page 7: What can-be-done-around-mesos](https://reader034.vdocuments.net/reader034/viewer/2022052705/58ab04c61a28abd35e8b64bd/html5/thumbnails/7.jpg)
Components containerized +
Marathon
![Page 8: What can-be-done-around-mesos](https://reader034.vdocuments.net/reader034/viewer/2022052705/58ab04c61a28abd35e8b64bd/html5/thumbnails/8.jpg)
PaaS
• Platform as a Service
• You can focus on your application, not the infrastructure
• We can focus on making the platform resilient and self-healing
![Page 9: What can-be-done-around-mesos](https://reader034.vdocuments.net/reader034/viewer/2022052705/58ab04c61a28abd35e8b64bd/html5/thumbnails/9.jpg)
PaaS• marathon
• mesos
• private docker registry
• Jenkins
• gitlab/github
• metrics! metrics! metrics!
• log&debug
![Page 10: What can-be-done-around-mesos](https://reader034.vdocuments.net/reader034/viewer/2022052705/58ab04c61a28abd35e8b64bd/html5/thumbnails/10.jpg)
Marathon• PaaS on Mesos
• init.d for the cluster
• Docker support
• Scales at the click of a button
• roll-back
• Blue-Green Deployment
• RESTful API
• Manages edge routes - HAproxy
![Page 11: What can-be-done-around-mesos](https://reader034.vdocuments.net/reader034/viewer/2022052705/58ab04c61a28abd35e8b64bd/html5/thumbnails/11.jpg)
Marathon• health check, instead of supervisord
• constraints
• resources
• force pull image
• zookeeper is the bottleneck
• docker images warm-up or dfs
![Page 12: What can-be-done-around-mesos](https://reader034.vdocuments.net/reader034/viewer/2022052705/58ab04c61a28abd35e8b64bd/html5/thumbnails/12.jpg)
Private docker registry
• Storage Strategy
• data disk: size planning, inode
• storage driver: device mapper, aufs
• Auth
• authorized push, allowed pull
• Frontend
![Page 13: What can-be-done-around-mesos](https://reader034.vdocuments.net/reader034/viewer/2022052705/58ab04c61a28abd35e8b64bd/html5/thumbnails/13.jpg)
docker
• docker build image
• base image + wget executable program: run-time
• tag image by code release version
• exited containers GC
![Page 14: What can-be-done-around-mesos](https://reader034.vdocuments.net/reader034/viewer/2022052705/58ab04c61a28abd35e8b64bd/html5/thumbnails/14.jpg)
Service Discovery• Mesos-DNS(actively developing)
• one mesos-dns one slave
• stateless, replica
• RESTful API
• HAproxy/Bamboo
• 1 container 1 IP
• IPAM
• mesos container, instead docker container
• docker-Weave integration
• network isolation IS NOT Service Discovery
• host or bridge
• performance vs security
![Page 15: What can-be-done-around-mesos](https://reader034.vdocuments.net/reader034/viewer/2022052705/58ab04c61a28abd35e8b64bd/html5/thumbnails/15.jpg)
Load Balance• Mesos-DNS
• round robin
• dynamic load
• HAproxy/Bamboo
• ip/source hash
• round robin
• least conn
• dynamic load
![Page 16: What can-be-done-around-mesos](https://reader034.vdocuments.net/reader034/viewer/2022052705/58ab04c61a28abd35e8b64bd/html5/thumbnails/16.jpg)
Schedule• Docker-Swarm
• constraint: storage==ssd io
• affinity: container or image affinity
• volume-from, link, net: container
• strategy: bin-pack, spread, random
• Mesos
• resource: cpus:24;mem:24576;disk:409600;ports:[21000-24000,30000-34000];bugs(debug_role):{a,b,c}'
• attributes: 'rack:abc;zone:west;os:centos5;level:10;keys:[1000-1500]'
![Page 17: What can-be-done-around-mesos](https://reader034.vdocuments.net/reader034/viewer/2022052705/58ab04c61a28abd35e8b64bd/html5/thumbnails/17.jpg)
Metrics&Auto-Scaling
• Metrics
• host
• docker
• app
![Page 18: What can-be-done-around-mesos](https://reader034.vdocuments.net/reader034/viewer/2022052705/58ab04c61a28abd35e8b64bd/html5/thumbnails/18.jpg)
Metrics&Auto-Scaling• the trigger depends
• cpu/mem metrics
• app request rate
• app force/load
• how
• IaaS
• PaaS
• where
![Page 19: What can-be-done-around-mesos](https://reader034.vdocuments.net/reader034/viewer/2022052705/58ab04c61a28abd35e8b64bd/html5/thumbnails/19.jpg)
Log&Debug• client
• stdout/stderr VS. /var/log/app/*.log
• logspout/heka VS. logstash file input
• Server
• ES
• frontend
• Kibana
![Page 20: What can-be-done-around-mesos](https://reader034.vdocuments.net/reader034/viewer/2022052705/58ab04c61a28abd35e8b64bd/html5/thumbnails/20.jpg)
Multiple Roles
• DRF
• Static Reservation VS Dynamic Reservation
![Page 21: What can-be-done-around-mesos](https://reader034.vdocuments.net/reader034/viewer/2022052705/58ab04c61a28abd35e8b64bd/html5/thumbnails/21.jpg)
Q&A