Download - D-DAY 2015 Conteneurisé une startup
Comment nous avons conteneurisé une startup ?
#devopsday
Jonathan Martin @john_iw2
Stéphane Cottin @vixns
#architecture #hosting #opsdev #libre #r&d #techtrends #IoT #artprojects
#startup #newyork #marseille #r&d #frenchtech #ecommerce #bpi #shake #network #bigdata
#dockerlover #opensourcelover #hashtaglover
The first global cross-selling network between eMerchants
“ “
Yuzu By Example
At the beginning
months to do a MVP 4
No time to reinvent the wheel
Symfony - Java - MongoDB
PRODUCTION
Web BigData
- 2 physical servers - Multiple VMs - Chef & Chef server
- 2 physical servers - Stack Cloudera (Hadoop/Spark/...)
Six months later...
A lot of technos joined Yuzu during the MVP
Welcome to nodejs, scala, elasticsearch, hbase, redis, kafka, couchbase,...
They lived happily ever after
It doesn’t work in production… … that’s works fine in my local env
“ “
This Lib isn’t in the right version... It’s slow… why??
Where are my logs...
“ “
Time to look around...
Be kind, rewind Spring cleaning Microservices / 12 factors Resources isolation Improved security Continuous deployment Start-up compliant workflow Orchestration & supervision
AND WINNERS ARE ... THE
Docker + Mesos + Marathon + Consul
{"container":{"type":"DOCKER","docker":{"image":"redis:2.6.17","network":"BRIDGE","portMappings":[{"containerPort":6379,"hostPort":0,"protocol":"tcp"}]}},"id":"redis","instances":1,"cpus":0.5,"mem":1024,"healthChecks":[{"protocol":"TCP","portIndex":0}]}
MARATHON wants JSON
Dynamic Configuration Docker
Registrator
Consul
Consul-template
Logs, metrics, alerting...
Collect everything collectd / fluentd / rsyslog
Convert to async messages kafka
Filter and keep valuable data riemann / elasticsearch / influxdb
Generate realtime alerts riemann
Addictive dashboards kibana / grafana / riemann-dash
CHRONOS CRON for mesos
Our workflow
Git flow
Jenkins DinD
Private Registry Docker
Doploy
Marathon & Chronos
Boot2Docker + Parallels
Don’t / Do
use tag "latest" on production
use explicit version tags for your docker images
DON’T
use data containers for valuable data
storage layer may crash / change easy to forget when cleaning unused
containers / images
DON’T
use shared storage (glusterfs, nfs, ...) for your codebase, working dirs, config files or sessions
sloooooooooow generate scary lock errors and timeouts
fscache crash full systems easily
DON’T
tag your custom images with the VCS commit hash
● makes your workflow better (same version tag in VCS and images)
● prevent useless image rebuilds ( speedup deploys ) ● easy way to know exactly which code version is running,
even if the tag was changed / deleted in the VCS
DO
avoid shared filesystems
● put all versioned data in containers ● use external object storage for user files (S3, ceph, swift ,...) ● use database / memcached / couchbase for sessions ● use templates to generate local config files (consul-template ,
confd)
DO
Use docker independant storage for critical data
● lvm is your friend (unless you use and saturate thin volume metadata, use thin volumes with care)
● use storage plugins with docker >= 1.9.0 (convoy ?) ● redundant backups saves lives
DO
Mistaking is
learning
- we tried to put the chicken in the egg
- generate config files to shared storage
- dns for discovery
- use mongo replicaset without sharding
- ...
Chronos Collectd Consul Consul-template Couchbase Debian Docker Doploy ElasticSearch Fluentd Gearman Go Grafana Haproxy HBase InfluxDB Java Jenkins
Kafka Kibana Marathon Mesos MongoDB Nodejs OpenVpn PhalconPHP Redis Registrator Registry
Riemann Route53 S3 Scala Sentry Spark Statsd Symfony Zookeeper ...
Switch from registrator to mesos-consul
Remove the lasts SPOFs
Move user files to S3
Use the docker storage/network plugins
XDCC
Improve logs & metrics filtering and alerting
2015 Other mesos/docker stories
IPROFS A large scale php/drupal worldwide social app for “Institut français”
ARTE.TV We are migrating all their vod and svod services to mesos/docker (java apps)
VIXNS A mesos/docker cluster is collecting all logs and metrics from hundreds of servers
Conclusion
Questions?
@john_iw2 @vixns