michail koltsov, automate yourself out of running your applications with apache mesos
TRANSCRIPT
![Page 1: Michail koltsov, automate yourself out of running your applications with apache mesos](https://reader031.vdocuments.net/reader031/viewer/2022030313/58cec8021a28ab09548b5dc7/html5/thumbnails/1.jpg)
How I automated myself out of my job
with Apache Mesos
Michail Kolcow, 2017
![Page 2: Michail koltsov, automate yourself out of running your applications with apache mesos](https://reader031.vdocuments.net/reader031/viewer/2022030313/58cec8021a28ab09548b5dc7/html5/thumbnails/2.jpg)
About me• 12+ years in IT • Java, Scala, Lisp, Python, … • Tatar, Russian, English, Latvian, Polish • Spark, Hadoop, Couchbase • I love castles & craft beer • 2 lovely kids: Kira & Adam
![Page 3: Michail koltsov, automate yourself out of running your applications with apache mesos](https://reader031.vdocuments.net/reader031/viewer/2022030313/58cec8021a28ab09548b5dc7/html5/thumbnails/3.jpg)
The most British thing ever?
![Page 4: Michail koltsov, automate yourself out of running your applications with apache mesos](https://reader031.vdocuments.net/reader031/viewer/2022030313/58cec8021a28ab09548b5dc7/html5/thumbnails/4.jpg)
The most British thing ever
![Page 5: Michail koltsov, automate yourself out of running your applications with apache mesos](https://reader031.vdocuments.net/reader031/viewer/2022030313/58cec8021a28ab09548b5dc7/html5/thumbnails/5.jpg)
The most British thing ever
![Page 6: Michail koltsov, automate yourself out of running your applications with apache mesos](https://reader031.vdocuments.net/reader031/viewer/2022030313/58cec8021a28ab09548b5dc7/html5/thumbnails/6.jpg)
Kettle-o-strophe
![Page 7: Michail koltsov, automate yourself out of running your applications with apache mesos](https://reader031.vdocuments.net/reader031/viewer/2022030313/58cec8021a28ab09548b5dc7/html5/thumbnails/7.jpg)
Kettle-o-strophe is real
![Page 8: Michail koltsov, automate yourself out of running your applications with apache mesos](https://reader031.vdocuments.net/reader031/viewer/2022030313/58cec8021a28ab09548b5dc7/html5/thumbnails/8.jpg)
Autoscaling to prevent the future kettle-o-strophe
Dinorwig Power Station, Wales, UK
![Page 9: Michail koltsov, automate yourself out of running your applications with apache mesos](https://reader031.vdocuments.net/reader031/viewer/2022030313/58cec8021a28ab09548b5dc7/html5/thumbnails/9.jpg)
Why do we care about kettles?
![Page 10: Michail koltsov, automate yourself out of running your applications with apache mesos](https://reader031.vdocuments.net/reader031/viewer/2022030313/58cec8021a28ab09548b5dc7/html5/thumbnails/10.jpg)
My first kettle-o-strophe
![Page 11: Michail koltsov, automate yourself out of running your applications with apache mesos](https://reader031.vdocuments.net/reader031/viewer/2022030313/58cec8021a28ab09548b5dc7/html5/thumbnails/11.jpg)
My first kettle-o-strophe
• Start of the main product development: 2015 • Autoscaling groups, Cloudwatch & Elastic IP • Mandatory code review • ~100 microservices • Fully automated deployment w/Ansible &
Jenkins • Scala, Go, Java • Postgres, Couchbase, Influx, GraphDB • Non-distributed dev teams & DevOps
![Page 12: Michail koltsov, automate yourself out of running your applications with apache mesos](https://reader031.vdocuments.net/reader031/viewer/2022030313/58cec8021a28ab09548b5dc7/html5/thumbnails/12.jpg)
My first kettle-o-strophe
![Page 13: Michail koltsov, automate yourself out of running your applications with apache mesos](https://reader031.vdocuments.net/reader031/viewer/2022030313/58cec8021a28ab09548b5dc7/html5/thumbnails/13.jpg)
My first kettle-o-strophe
![Page 14: Michail koltsov, automate yourself out of running your applications with apache mesos](https://reader031.vdocuments.net/reader031/viewer/2022030313/58cec8021a28ab09548b5dc7/html5/thumbnails/14.jpg)
My first kettle-o-strophe
![Page 15: Michail koltsov, automate yourself out of running your applications with apache mesos](https://reader031.vdocuments.net/reader031/viewer/2022030313/58cec8021a28ab09548b5dc7/html5/thumbnails/15.jpg)
My second kettle-o-strophe
![Page 16: Michail koltsov, automate yourself out of running your applications with apache mesos](https://reader031.vdocuments.net/reader031/viewer/2022030313/58cec8021a28ab09548b5dc7/html5/thumbnails/16.jpg)
My second kettle-o-strophe
• Start of the main product development: 1948 • ~ 1000 of apps, code from different epochs • Distributed teams: EU, US, Australia • Highly motivated staff • Some apps got their own build & deployment
automation process • Scala, Java, Node, Python, Ruby, whatever… • Oracle, Sybase, Riak, Postgres, Couchbase,
Influx, GraphDB, choose your own DB
![Page 17: Michail koltsov, automate yourself out of running your applications with apache mesos](https://reader031.vdocuments.net/reader031/viewer/2022030313/58cec8021a28ab09548b5dc7/html5/thumbnails/17.jpg)
My second kettle-o-strophe
DATACENTRES ARE NOT IMMUTABLE!!!
![Page 18: Michail koltsov, automate yourself out of running your applications with apache mesos](https://reader031.vdocuments.net/reader031/viewer/2022030313/58cec8021a28ab09548b5dc7/html5/thumbnails/18.jpg)
To avoid the inevitable
![Page 19: Michail koltsov, automate yourself out of running your applications with apache mesos](https://reader031.vdocuments.net/reader031/viewer/2022030313/58cec8021a28ab09548b5dc7/html5/thumbnails/19.jpg)
To avoid the inevitable
![Page 20: Michail koltsov, automate yourself out of running your applications with apache mesos](https://reader031.vdocuments.net/reader031/viewer/2022030313/58cec8021a28ab09548b5dc7/html5/thumbnails/20.jpg)
To avoid the inevitable
![Page 21: Michail koltsov, automate yourself out of running your applications with apache mesos](https://reader031.vdocuments.net/reader031/viewer/2022030313/58cec8021a28ab09548b5dc7/html5/thumbnails/21.jpg)
Tech we use
![Page 22: Michail koltsov, automate yourself out of running your applications with apache mesos](https://reader031.vdocuments.net/reader031/viewer/2022030313/58cec8021a28ab09548b5dc7/html5/thumbnails/22.jpg)
Tech we build
![Page 23: Michail koltsov, automate yourself out of running your applications with apache mesos](https://reader031.vdocuments.net/reader031/viewer/2022030313/58cec8021a28ab09548b5dc7/html5/thumbnails/23.jpg)
Developers’ pipelines
![Page 24: Michail koltsov, automate yourself out of running your applications with apache mesos](https://reader031.vdocuments.net/reader031/viewer/2022030313/58cec8021a28ab09548b5dc7/html5/thumbnails/24.jpg)
Mesos as a DC abstraction
![Page 25: Michail koltsov, automate yourself out of running your applications with apache mesos](https://reader031.vdocuments.net/reader031/viewer/2022030313/58cec8021a28ab09548b5dc7/html5/thumbnails/25.jpg)
Mesos as a DC abstraction
![Page 26: Michail koltsov, automate yourself out of running your applications with apache mesos](https://reader031.vdocuments.net/reader031/viewer/2022030313/58cec8021a28ab09548b5dc7/html5/thumbnails/26.jpg)
Mesos is a distributed kernel
![Page 27: Michail koltsov, automate yourself out of running your applications with apache mesos](https://reader031.vdocuments.net/reader031/viewer/2022030313/58cec8021a28ab09548b5dc7/html5/thumbnails/27.jpg)
How it helps
![Page 28: Michail koltsov, automate yourself out of running your applications with apache mesos](https://reader031.vdocuments.net/reader031/viewer/2022030313/58cec8021a28ab09548b5dc7/html5/thumbnails/28.jpg)
How it helps
![Page 29: Michail koltsov, automate yourself out of running your applications with apache mesos](https://reader031.vdocuments.net/reader031/viewer/2022030313/58cec8021a28ab09548b5dc7/html5/thumbnails/29.jpg)
We need more abstractions
![Page 30: Michail koltsov, automate yourself out of running your applications with apache mesos](https://reader031.vdocuments.net/reader031/viewer/2022030313/58cec8021a28ab09548b5dc7/html5/thumbnails/30.jpg)
To track the tasks
![Page 31: Michail koltsov, automate yourself out of running your applications with apache mesos](https://reader031.vdocuments.net/reader031/viewer/2022030313/58cec8021a28ab09548b5dc7/html5/thumbnails/31.jpg)
To deploy the apps
![Page 32: Michail koltsov, automate yourself out of running your applications with apache mesos](https://reader031.vdocuments.net/reader031/viewer/2022030313/58cec8021a28ab09548b5dc7/html5/thumbnails/32.jpg)
It’s demo time
DEMO
![Page 33: Michail koltsov, automate yourself out of running your applications with apache mesos](https://reader031.vdocuments.net/reader031/viewer/2022030313/58cec8021a28ab09548b5dc7/html5/thumbnails/33.jpg)