be fast or stay behind - building a continuous delivery platform
DESCRIPTION
Ten years of continuous growth leave many stretch marks on any website, like increasing maintenance overhead, lengthy and complex internal processes and lots of code and configuration that nobody knows about. A Windows-Linux migration in the past also does not help to achieve a clean platform. On the other hand, being a market leader does not leave any room for relaxation but requires us to stay ahead. We need to be faster than the competition and make sure that our own size and being part of a larger corporation does not slow us down. Continuous Delivery is not just a buzz word but the answer to many of our current problems. Today we envision our data center and the various IT departments as building blocks in a Continuous Delivery Platform (CDP) that strives to shorten the time it takes to convert an idea into productive code. This talk starts from a big picture of a typical web company and drills down into the technical and organizational challenges that stood in the way of creating a CDP. Our developers turning agile and doing everything through SCRUM was only the start of a series of profound changes that touched all IT departments and beyond. DevOps helped us a lot to explain to our management and colleagues what is going on and what we want. But only a brand-new deployment and configuration management brought the actual break-through to shared responsibility and teams developing operational thinking. An important learning was that engineers come together through solving common problems as a team. In our case we had to deal with two main concerns: Linux and Java. Linux being the choice operating environment we started to do things the Linux Way and package all our software, configuration files and even content into RPM packages, thus greatly simplifying the problem of deployment and system management. Since our developers build the RPM packages they also get much more involved in site operations and suddenly the whole DevOps idea actually works out for us. Java being the choice coding language (with a huge code base :-() and a lot of “Java thinking” lead us to write a bridge between Java and Linux: Our Nexus YUM plugin translates between a Java world that knows only Maven and a Linux world that likes to install packages via YUM. The automated build process in TeamCity creates RPM packages and puts them into the Nexus which serves the same RPM packages as a YUM repository to our servers. This simplifies the handover from development to operations and is a big performance boost for our delivery chain. These and other technical solutions come together with many organizational changes – e.g. giving developers more access in the data center – to create the foundation for our Continuous Delivery Platform and enable everyone to focus on working the software while relying on a reliable delivery tool chain below. Another important learning is the way how to ensure the management buy-in into what essentially startedTRANSCRIPT
![Page 1: Be Fast or Stay Behind - Building a Continuous Delivery Platform](https://reader033.vdocuments.net/reader033/viewer/2022060107/554baac5b4c905b3618b53c3/html5/thumbnails/1.jpg)
Be Fast Or Stay Behindwww.immobilienscout24.de
Schlomo Schapiro, Systems Architect & Open Source Evangelist
Ingmar Krusch, Team Lead in Operations
License: http://creativecommons.org/licenses/by-nc-nd/3.0/
Building a Continuous Delivery Platform
![Page 2: Be Fast or Stay Behind - Building a Continuous Delivery Platform](https://reader033.vdocuments.net/reader033/viewer/2022060107/554baac5b4c905b3618b53c3/html5/thumbnails/2.jpg)
2
www.ImmobilienScout24.de
2 billion PI per month
2 data center with ~800 VM
~30 crossfunctional IT teams
total of ~500 employees
14 years in business
part of Deutsche Telekom
Overview File Mgmt Systems Mgmt Lessons Learned
![Page 3: Be Fast or Stay Behind - Building a Continuous Delivery Platform](https://reader033.vdocuments.net/reader033/viewer/2022060107/554baac5b4c905b3618b53c3/html5/thumbnails/3.jpg)
3 Overview File Mgmt Systems Mgmt Lessons Learned
Close Collaboration
Package Deployment
![Page 4: Be Fast or Stay Behind - Building a Continuous Delivery Platform](https://reader033.vdocuments.net/reader033/viewer/2022060107/554baac5b4c905b3618b53c3/html5/thumbnails/4.jpg)
4
The Big Picture – Continuous Delivery Platform
BusinessDecision
To goLive
All Humans are on the Same Side
BuildConfigDeployTestSystems-Management
Automation
Scale OutData Centers
Overview File Mgmt Systems Mgmt Lessons Learned
![Page 5: Be Fast or Stay Behind - Building a Continuous Delivery Platform](https://reader033.vdocuments.net/reader033/viewer/2022060107/554baac5b4c905b3618b53c3/html5/thumbnails/5.jpg)
5
1st task: Reduce Systemic Complexity
File Management
Systems Management
Overview File Mgmt Systems Mgmt Lessons Learned
![Page 6: Be Fast or Stay Behind - Building a Continuous Delivery Platform](https://reader033.vdocuments.net/reader033/viewer/2022060107/554baac5b4c905b3618b53c3/html5/thumbnails/6.jpg)
6
File Management with RPM & YUM
Overview File Mgmt Systems Mgmt Lessons Learned
![Page 7: Be Fast or Stay Behind - Building a Continuous Delivery Platform](https://reader033.vdocuments.net/reader033/viewer/2022060107/554baac5b4c905b3618b53c3/html5/thumbnails/7.jpg)
7
File Management with RPM & YUM
Overview File Mgmt Systems Mgmt Lessons Learned
Libraries
Applications
Configuration
Operating System
100 %
![Page 8: Be Fast or Stay Behind - Building a Continuous Delivery Platform](https://reader033.vdocuments.net/reader033/viewer/2022060107/554baac5b4c905b3618b53c3/html5/thumbnails/8.jpg)
8
File Management with RPM & YUM
Overview File Mgmt Systems Mgmt Lessons Learned
PROD
DEV
InfrastructureBuild
ApplicationBuild
ConfigurationBuild
QA
YUMRepos
![Page 9: Be Fast or Stay Behind - Building a Continuous Delivery Platform](https://reader033.vdocuments.net/reader033/viewer/2022060107/554baac5b4c905b3618b53c3/html5/thumbnails/9.jpg)
9
File Management with RPM & YUM
Overview File Mgmt Systems Mgmt Lessons Learned
Interface
PROD
DEV
InfrastructureBuild
ApplicationBuild
ConfigurationBuild
QA
YUMRepos
![Page 10: Be Fast or Stay Behind - Building a Continuous Delivery Platform](https://reader033.vdocuments.net/reader033/viewer/2022060107/554baac5b4c905b3618b53c3/html5/thumbnails/10.jpg)
10
PROD
DEV
InfrastructureBuild
ApplicationBuild
QA
YUMRepos
Configuration Build – Pre-built Config Content
Overview File Mgmt Systems Mgmt Lessons Learned
ConfigurationBuild
![Page 11: Be Fast or Stay Behind - Building a Continuous Delivery Platform](https://reader033.vdocuments.net/reader033/viewer/2022060107/554baac5b4c905b3618b53c3/html5/thumbnails/11.jpg)
11
Configuration Build – Pre-built Config Content
Overview File Mgmt Systems Mgmt Lessons Learned
SVN
commitconfig file
buildconfig RPM
RPMswithstatic
content
YUMRepo
![Page 12: Be Fast or Stay Behind - Building a Continuous Delivery Platform](https://reader033.vdocuments.net/reader033/viewer/2022060107/554baac5b4c905b3618b53c3/html5/thumbnails/12.jpg)
12
Configuration Build – Pre-built Config Content
Overview File Mgmt Systems Mgmt Lessons Learned
SVN
RPMswithstatic
content
TemplatingVariables
Overlaying
YUMRepo
buildconfig RPM
commitconfig file
![Page 13: Be Fast or Stay Behind - Building a Continuous Delivery Platform](https://reader033.vdocuments.net/reader033/viewer/2022060107/554baac5b4c905b3618b53c3/html5/thumbnails/13.jpg)
13
Configuration Build – Pre-built Config Content
Overview File Mgmt Systems Mgmt Lessons Learned
SVN
RPMswithstatic
content
TemplatingVariables
Overlaying
we use1 RPM
per host
YUMRepo
commitconfig file
buildconfig RPM
![Page 14: Be Fast or Stay Behind - Building a Continuous Delivery Platform](https://reader033.vdocuments.net/reader033/viewer/2022060107/554baac5b4c905b3618b53c3/html5/thumbnails/14.jpg)
14
SVN
RPMswithstatic
content
TemplatingVariables
Overlaying
we use1 RPM
per host
Sharedresponsability
Ops, Dev, QA ...
Configuration Build – Pre-built Config Content
Overview File Mgmt Systems Mgmt Lessons Learned
YUMRepo
commitconfig file
buildconfig RPM
![Page 15: Be Fast or Stay Behind - Building a Continuous Delivery Platform](https://reader033.vdocuments.net/reader033/viewer/2022060107/554baac5b4c905b3618b53c3/html5/thumbnails/15.jpg)
15
Application Build – Continuous Release Delivery
Overview File Mgmt Systems Mgmt Lessons Learned
PROD
DEV
InfrastructureBuild
QA
YUMRepos
ConfigurationBuild
ApplicationBuild
![Page 16: Be Fast or Stay Behind - Building a Continuous Delivery Platform](https://reader033.vdocuments.net/reader033/viewer/2022060107/554baac5b4c905b3618b53c3/html5/thumbnails/16.jpg)
16
Application Build – Package Promotion Pipeline
YUMRepo
TestedRepo
YUMRepo
ReleasedRepo
YUMRepo
ComittedRepo
DEV Pre-PROD PROD
Overview File Mgmt Systems Mgmt Lessons Learned
![Page 17: Be Fast or Stay Behind - Building a Continuous Delivery Platform](https://reader033.vdocuments.net/reader033/viewer/2022060107/554baac5b4c905b3618b53c3/html5/thumbnails/17.jpg)
17
Application Build – Package Promotion Pipeline
CI YUMRepo
TestedRepo
YUMRepo
ReleasedRepo
YUMRepo
ComittedRepo
Business decisionto Go Live withtested versions
DEV Pre-PROD PROD
Auto-promote release if tests OK
Deploy&
Test
Deploy&
Test
Overview File Mgmt Systems Mgmt Lessons Learned
Deploy&
Test
![Page 18: Be Fast or Stay Behind - Building a Continuous Delivery Platform](https://reader033.vdocuments.net/reader033/viewer/2022060107/554baac5b4c905b3618b53c3/html5/thumbnails/18.jpg)
18
Application Build – Site-wide Releases
Overview File Mgmt Systems Mgmt Lessons Learned
PROD
DEV
InfrastructureBuild
QA
YUMRepos
ConfigurationBuild
ApplicationBuild
![Page 19: Be Fast or Stay Behind - Building a Continuous Delivery Platform](https://reader033.vdocuments.net/reader033/viewer/2022060107/554baac5b4c905b3618b53c3/html5/thumbnails/19.jpg)
19
Team City – CI
POM
Application Build – Site-wide Releases
Overview File Mgmt Systems Mgmt Lessons Learned
NEXUS„released“
Repo
mvn deploy
NEXUS Server
![Page 20: Be Fast or Stay Behind - Building a Continuous Delivery Platform](https://reader033.vdocuments.net/reader033/viewer/2022060107/554baac5b4c905b3618b53c3/html5/thumbnails/20.jpg)
20
Application Build – Site-wide Releases
NEXUS„released“
Repo
yum upgrade
Overview File Mgmt Systems Mgmt Lessons Learned
Team City – CI
POM
mvn deploy
NEXUS Server
NEXUS-YUMPlugin
YUM„released“
Repo
![Page 21: Be Fast or Stay Behind - Building a Continuous Delivery Platform](https://reader033.vdocuments.net/reader033/viewer/2022060107/554baac5b4c905b3618b53c3/html5/thumbnails/21.jpg)
21
File Management through many YUM repositories
Overview File Mgmt Systems Mgmt Lessons Learned
PROD
DEV
InfrastructureBuild
ApplicationBuild
ConfigurationBuild
QAYUMRepo
YUMRepoYUM
RepoYUMRepo
![Page 22: Be Fast or Stay Behind - Building a Continuous Delivery Platform](https://reader033.vdocuments.net/reader033/viewer/2022060107/554baac5b4c905b3618b53c3/html5/thumbnails/22.jpg)
22
Systems Management with YADT
Overview File Mgmt Systems Mgmt Lessons Learned
YADT – an Augmented Deployment Tool
![Page 23: Be Fast or Stay Behind - Building a Continuous Delivery Platform](https://reader033.vdocuments.net/reader033/viewer/2022060107/554baac5b4c905b3618b53c3/html5/thumbnails/23.jpg)
23
YADT – Unified Dependency Tree
Overview File Mgmt Systems Mgmt Lessons Learned
Web Server
HTTPD
Tomcat
App Server
Load Balancer
![Page 24: Be Fast or Stay Behind - Building a Continuous Delivery Platform](https://reader033.vdocuments.net/reader033/viewer/2022060107/554baac5b4c905b3618b53c3/html5/thumbnails/24.jpg)
24
YADT – Unified Dependency Tree
Web Server
HTTPD
Tomcat
App Server
Load Balancer
web-app
httpd
config
tomcat
Overview File Mgmt Systems Mgmt Lessons Learned
![Page 25: Be Fast or Stay Behind - Building a Continuous Delivery Platform](https://reader033.vdocuments.net/reader033/viewer/2022060107/554baac5b4c905b3618b53c3/html5/thumbnails/25.jpg)
25
YADT – Large Scale System Management
Web Server
HTTPD
Tomcatweb-app
httpd
config
tomcat
App Server
Java App
app
config
Overview File Mgmt Systems Mgmt Lessons Learned
![Page 26: Be Fast or Stay Behind - Building a Continuous Delivery Platform](https://reader033.vdocuments.net/reader033/viewer/2022060107/554baac5b4c905b3618b53c3/html5/thumbnails/26.jpg)
26
YADT – Large Scale System Management
LB: www.immobilienscout24.de
Web Server
HTTPD
Tomcatweb-app
httpd
config
tomcat
App Server
Java App
app
config
Web Server
HTTPD
Tomcatweb-app
httpd
config
tomcat
App Server
Java App
app
config
Overview File Mgmt Systems Mgmt Lessons Learned
![Page 27: Be Fast or Stay Behind - Building a Continuous Delivery Platform](https://reader033.vdocuments.net/reader033/viewer/2022060107/554baac5b4c905b3618b53c3/html5/thumbnails/27.jpg)
27
YADT – Large Scale System Management
LB: www.immobilienscout24.de
Web Server
HTTPD
Tomcatweb-app
httpd
config
tomcat
App Server
Java App
app
config
Web Server
HTTPD
Tomcatweb-app
httpd
config
tomcat
App Server
Java App
app
config
1st ch
unk
Overview File Mgmt Systems Mgmt Lessons Learned
![Page 28: Be Fast or Stay Behind - Building a Continuous Delivery Platform](https://reader033.vdocuments.net/reader033/viewer/2022060107/554baac5b4c905b3618b53c3/html5/thumbnails/28.jpg)
28
S: YADT – Large Scale System Management
LB: www.immobilienscout24.de
Web Server
HTTPD
Tomcatweb-app
httpd
config
tomcat
App Server
Java App
app
config
Web Server
HTTPD
Tomcatweb-app
httpd
config
tomcat
App Server
Java App
app
config
Overview File Mgmt Systems Mgmt Lessons Learned
1st ch
unk2nd chunk
![Page 29: Be Fast or Stay Behind - Building a Continuous Delivery Platform](https://reader033.vdocuments.net/reader033/viewer/2022060107/554baac5b4c905b3618b53c3/html5/thumbnails/29.jpg)
29
Systems Management with YADT
Reduce Systemic Complexity
Overview File Mgmt Systems Mgmt Lessons Learned
![Page 30: Be Fast or Stay Behind - Building a Continuous Delivery Platform](https://reader033.vdocuments.net/reader033/viewer/2022060107/554baac5b4c905b3618b53c3/html5/thumbnails/30.jpg)
30
Reduce Systemic Complexity
Decoupled Applications
Overview File Mgmt Systems Mgmt Lessons Learned
![Page 31: Be Fast or Stay Behind - Building a Continuous Delivery Platform](https://reader033.vdocuments.net/reader033/viewer/2022060107/554baac5b4c905b3618b53c3/html5/thumbnails/31.jpg)
31
Reduce Systemic Complexity
Developers Build Infrastructure
Overview File Mgmt Systems Mgmt Lessons Learned
![Page 32: Be Fast or Stay Behind - Building a Continuous Delivery Platform](https://reader033.vdocuments.net/reader033/viewer/2022060107/554baac5b4c905b3618b53c3/html5/thumbnails/32.jpg)
32
This Automation is our Continuous Delivery Platform
BusinessDecision
To goLive
All Humans are on the Same Side Scale Out
Data CentersAUTOMATION
Overview File Mgmt Systems Mgmt Lessons Learned
![Page 33: Be Fast or Stay Behind - Building a Continuous Delivery Platform](https://reader033.vdocuments.net/reader033/viewer/2022060107/554baac5b4c905b3618b53c3/html5/thumbnails/33.jpg)
33
DevOps
We build it together as one team.
We understand it completly.
And it is fully automated.
Overview File Mgmt Systems Mgmt Lessons Learned
![Page 34: Be Fast or Stay Behind - Building a Continuous Delivery Platform](https://reader033.vdocuments.net/reader033/viewer/2022060107/554baac5b4c905b3618b53c3/html5/thumbnails/34.jpg)
34
Open-Source from the start
https://code.google.com/p/yadt
https://code.google.com/p/nexus-yum-plugin
http://sourceforge.net/projects/yadtdownloads
Overview File Mgmt Systems Mgmt Lessons Learned
![Page 35: Be Fast or Stay Behind - Building a Continuous Delivery Platform](https://reader033.vdocuments.net/reader033/viewer/2022060107/554baac5b4c905b3618b53c3/html5/thumbnails/35.jpg)
35
What is the real competitive advantage?
https://code.google.com/p/yadt
https://code.google.com/p/nexus-yum-plugin
http://sourceforge.net/projects/yadtdownloads
Overview File Mgmt Systems Mgmt Lessons Learned
![Page 36: Be Fast or Stay Behind - Building a Continuous Delivery Platform](https://reader033.vdocuments.net/reader033/viewer/2022060107/554baac5b4c905b3618b53c3/html5/thumbnails/36.jpg)
36
Q&A
Overview File Mgmt Systems Mgmt Lessons Learned
Massively Reduce Systemic Complexity
Work As One Team
![Page 37: Be Fast or Stay Behind - Building a Continuous Delivery Platform](https://reader033.vdocuments.net/reader033/viewer/2022060107/554baac5b4c905b3618b53c3/html5/thumbnails/37.jpg)
37
Kontakt:Immobilien Scout GmbHAndreasstraße 1010243 Berlin
Email: [email protected]: [email protected]: www.immobilienscout24.de
Thank you very much!Please contact us for furtherquestions and discussions.