configuration testing for better devops · configuration testing for better devops anatoly...
TRANSCRIPT
![Page 1: Configuration testing for better DevOps · Configuration testing for better DevOps Anatoly Vasilevskiy Stockholm, KTH, Chaos Engineering & DevOps Meetup . December 6, 2017](https://reader030.vdocuments.net/reader030/viewer/2022040612/5edd1d73ad6a402d666815d8/html5/thumbnails/1.jpg)
Configuration testing for better DevOps
Anatoly Vasilevskiy
Stockholm, KTH, Chaos Engineering & DevOps Meetup
December 6, 2017
![Page 2: Configuration testing for better DevOps · Configuration testing for better DevOps Anatoly Vasilevskiy Stockholm, KTH, Chaos Engineering & DevOps Meetup . December 6, 2017](https://reader030.vdocuments.net/reader030/viewer/2022040612/5edd1d73ad6a402d666815d8/html5/thumbnails/2.jpg)
Agenda
• Configuration testing
• Configuration amplification
• Automation of configuration testing
• Demo
2
![Page 3: Configuration testing for better DevOps · Configuration testing for better DevOps Anatoly Vasilevskiy Stockholm, KTH, Chaos Engineering & DevOps Meetup . December 6, 2017](https://reader030.vdocuments.net/reader030/viewer/2022040612/5edd1d73ad6a402d666815d8/html5/thumbnails/3.jpg)
Configuration testing in Nutshell
3
SUT & Tests
Config
Idea 1: Generate different configurations => Amplification of configurations
Idea 2: Integrate into continuous delivery pipeline => Automation of configuration testing
![Page 4: Configuration testing for better DevOps · Configuration testing for better DevOps Anatoly Vasilevskiy Stockholm, KTH, Chaos Engineering & DevOps Meetup . December 6, 2017](https://reader030.vdocuments.net/reader030/viewer/2022040612/5edd1d73ad6a402d666815d8/html5/thumbnails/4.jpg)
Challenges: Configuration testing
• Lacking formalization of configurations• Docker
• Lacking formalization of variation points• Features
• Lacking formalization of valid configuration• Constraints
• Lacking tooling to generate various configuration• Configuration amplification
• Lacking tooling to integrate into CI pipeline• Configuration Testing Framework
4
![Page 5: Configuration testing for better DevOps · Configuration testing for better DevOps Anatoly Vasilevskiy Stockholm, KTH, Chaos Engineering & DevOps Meetup . December 6, 2017](https://reader030.vdocuments.net/reader030/viewer/2022040612/5edd1d73ad6a402d666815d8/html5/thumbnails/5.jpg)
Use-case: XWiki
5
XWiki
AppServer(tomcat…)
Java (OpenJDK8…)
OS (Unbuntu…)
Test case
…
MySQL…
Postgres
…
?
Diverse environments
Diverse topology
![Page 6: Configuration testing for better DevOps · Configuration testing for better DevOps Anatoly Vasilevskiy Stockholm, KTH, Chaos Engineering & DevOps Meetup . December 6, 2017](https://reader030.vdocuments.net/reader030/viewer/2022040612/5edd1d73ad6a402d666815d8/html5/thumbnails/6.jpg)
What is configuration amplification
• Environment amplification• Monolithic architecture
• vary libraries, framework versions etc.
• Topology amplification• Distributed architectures (microservice, multitier)
• vary databases, application servers etc.
• Runtime amplification• Distributed architectures (microservice, multitier)
• scaling in/out, restarting services
6
![Page 7: Configuration testing for better DevOps · Configuration testing for better DevOps Anatoly Vasilevskiy Stockholm, KTH, Chaos Engineering & DevOps Meetup . December 6, 2017](https://reader030.vdocuments.net/reader030/viewer/2022040612/5edd1d73ad6a402d666815d8/html5/thumbnails/7.jpg)
How to amplify configurations
7
…
…
features and constraints
configuration model
constraint solver
sample config file
reusable pieces
generated config files
generated models
![Page 8: Configuration testing for better DevOps · Configuration testing for better DevOps Anatoly Vasilevskiy Stockholm, KTH, Chaos Engineering & DevOps Meetup . December 6, 2017](https://reader030.vdocuments.net/reader030/viewer/2022040612/5edd1d73ad6a402d666815d8/html5/thumbnails/8.jpg)
• Features• Linux (Alpine, Ubuntu…), Java (OpenJDK9, OpenJDK8…, IBMJDK9…), AppServer (Tomcat8,
Tomcat9…), Database (MySQL5, MySQL9…, Postgres9, Postgres10…)
• Reusable pieces • Docker files: Tomcat8 (requires Java, introduces Tomcat8)…• Standard Images: OpenJDK8, MySQL9
• Put them together in a meaningful way to cover different features• Constraint solving and optimization
• Output: • Docker images + Docker compose files – ready to run test with one command
8
Environment and topology amplification: XWiki
![Page 9: Configuration testing for better DevOps · Configuration testing for better DevOps Anatoly Vasilevskiy Stockholm, KTH, Chaos Engineering & DevOps Meetup . December 6, 2017](https://reader030.vdocuments.net/reader030/viewer/2022040612/5edd1d73ad6a402d666815d8/html5/thumbnails/9.jpg)
Runtime amplification
9
SUT
Scale service n up/down
Delay/lose network packets
Kill container n
Instrumentation
MonitoringLoad endpoints n with n req/s
…
RAM
CPU
Throughput
Latency
…
GloballyPer ServicePer Container
Assess resilience/robustness of the system andhelp defining auto-scaling, self-healing strategies
Very initial proof-of-concept:https://github.com/brice-morin/dracarys
![Page 10: Configuration testing for better DevOps · Configuration testing for better DevOps Anatoly Vasilevskiy Stockholm, KTH, Chaos Engineering & DevOps Meetup . December 6, 2017](https://reader030.vdocuments.net/reader030/viewer/2022040612/5edd1d73ad6a402d666815d8/html5/thumbnails/10.jpg)
Configuration Testing Framework (CTF)
10
• Pluggable build/test systems• maven• ant• tox• etc
• Pluggable reporting systems• mvn site• ant• etc
• Pluggable infrastructure management• sidecar
![Page 11: Configuration testing for better DevOps · Configuration testing for better DevOps Anatoly Vasilevskiy Stockholm, KTH, Chaos Engineering & DevOps Meetup . December 6, 2017](https://reader030.vdocuments.net/reader030/viewer/2022040612/5edd1d73ad6a402d666815d8/html5/thumbnails/11.jpg)
Prototype implementation of CTF
11
Configuration Test
Manager
(1) launches
stdout/stderr
path/../htmlreports/
(0) builds SUT Execution Manager
EnvironmentBuilder
EnvironmentExecutor
EnvironmentReport Builder
SUT Report Builder
(3) launchesstdout/stderr
reportreport
(2) launches
Jenkins Docker container
standardEnvcontainer
environmentcontainer 1
environmentcontainer N
SUT
CTF Maven Plugin
Configs
![Page 12: Configuration testing for better DevOps · Configuration testing for better DevOps Anatoly Vasilevskiy Stockholm, KTH, Chaos Engineering & DevOps Meetup . December 6, 2017](https://reader030.vdocuments.net/reader030/viewer/2022040612/5edd1d73ad6a402d666815d8/html5/thumbnails/12.jpg)
Demo
![Page 13: Configuration testing for better DevOps · Configuration testing for better DevOps Anatoly Vasilevskiy Stockholm, KTH, Chaos Engineering & DevOps Meetup . December 6, 2017](https://reader030.vdocuments.net/reader030/viewer/2022040612/5edd1d73ad6a402d666815d8/html5/thumbnails/13.jpg)
Links & Contacts
• STAMP Project at https://www.stamp-project.eu
• Configuration Testing Framework at https://github.com/SINTEF-9012/config-testing
• Environment and topology amplification at https://github.com/SINTEF-9012/ozepy
• Runtime amplification at https://github.com/brice-morin/dracarys
• Contacts• Anatoly Vasilevskiy ([email protected])• Brice Morin ([email protected])• Hui Song ([email protected])
13
![Page 14: Configuration testing for better DevOps · Configuration testing for better DevOps Anatoly Vasilevskiy Stockholm, KTH, Chaos Engineering & DevOps Meetup . December 6, 2017](https://reader030.vdocuments.net/reader030/viewer/2022040612/5edd1d73ad6a402d666815d8/html5/thumbnails/14.jpg)
14
Q&A