Subversion Day Berlin 2011 Configuration Management With Subversion And Rpm

Download Subversion Day Berlin 2011 Configuration Management With Subversion And Rpm

Post on 08-May-2015

2.250 views

Category:

Technology

1 download

Embed Size (px)

DESCRIPTION

ImmobilienScout24 makes use of SVN as a structured hierarchical configuration management backend that serves configuration files for development, testing and production platforms. Configuration files are automatically deployed to target systems via RPMs, which are automatically built from the information contained in a SVN repository. The components that comprise the solution (mostly a sophisticated post-commit hook) are open sourced under the GPL.See http://code.google.com/p/yadt for open source project homepage.

TRANSCRIPT

<ul><li>1.www.immobilienscout24.deConfiguration Managementwith Subversion and RPMThe Linux Way of Doing ThingsBerlin | 19.05.2011 | Schlomo SchapiroSystemarchitekt, Open Source Evangelist License: http://creativecommons.org/licenses/by-nc-nd/3.0/</li></ul> <p>2. http://code.google.com/p/yadt/Agendawww.immobilienscout24.de: Behind the scenesThe Linux Way of Doing ThingsRPM DeploymentConfiguration Management in SubversionYADT an Augmented Deployment ToolSlide 2 | Configuration Management with SVN and RPM | Schlomo Schapiro 3. http://code.google.com/p/yadt/www.immobilienscout24.de1.7 billion page impressions per monthhttp://www.immobilienscout24.de/de/anbieten/kennzahlen.jspLinux and standard open source toolchain RHEL LAMP server Tomcat, Spring, Hibernate MySQL and Oracle2 data centres with async WAN, &gt;600 VM, SAN, NAS 24/7 operationsAgile development, now going toward Continuous DeliveryAutomate, Automate, AutomateSlide 3 | Configuration Management with SVN and RPM | Schlomo Schapiro 4. http://code.google.com/p/yadt/Slide 4 | Configuration Management with SVN and RPM | Schlomo Schapiro 5. http://code.google.com/p/yadt/What problem do we solve?Standalone WebStandPHP Appalone Syslog SNMPContainerPHP/Perl Java VM Apache MySQLPostfix Linux OSPhysical/Virtual HWVirtualization LayerManage entire platform above (virtual) hardware layerSlide 5 | Configuration Management with SVN and RPM | Schlomo Schapiro 6. http://code.google.com/p/yadt/The UNIX Way of Thinking Web WebOn the Linux levelPHP/Perl Web TomcatApp ApacheJava VMIf we look very closely MySQLSyslog Net-SNMPStandaloneSSH PostfixAnd think UNIXLinux OSEverything is just a file! physical/virtual HWPrograms Environmental configurationLibrariesEven Startup dependenciesSoftware configuration And also what to run and whenOperational configurationReboot is the final test!Slide 6 | Configuration Management with SVN and RPM | Schlomo Schapiro 7. http://code.google.com/p/yadt/Linux tools we like to useSoftware Management: RPM Packages Versioning &amp; Transactions Validation and Rollback Deploy &amp; run actions: post-install, pre-remove, pre-install YUM Repositories Proven to work with millions of systems Release Upgrades, Repo Mirrors, Downgrades, Provisioning Bash &amp; Python scripts: /etc/init.d scripts for service management YADT for multi-server system management Automate, Automate, Automate :-)Slide 7 | Configuration Management with SVN and RPM | Schlomo Schapiro 8. http://code.google.com/p/yadt/Configuration = SoftwareSame tools usable for software and configurationOne tool to rule them all Software &amp; Configuration Dependencies between software and configurationRPM file conflicts enforce separation of concernsDoing this with a 100% coverage is probably a new idea!Slide 8 | Configuration Management with SVN and RPM | Schlomo Schapiro 9. http://code.google.com/p/yadt/Configuration Management in SVNConfig SVNpost-commit YUM reposSlide 9 | Configuration Management with SVN and RPM | Schlomo Schapiro 10. http://code.google.com/p/yadt/Big Picture Static Structureconfig hostPost-commit hook creates tstweb02is24-config-$hostname RPM svn export locsvn ci tstConfig patch VARIABLESfill in metadataSVN VARIABLES svn co RPM_REQUIRES RPM_PROVIDES DB_HOST DB_USER etc YUM IS24 software is24 web.properties Repository Automated RPMcreation db.properties typ webdb.host=@@@DB_HOST@@@db.user=@@@DB_USER@@@ is24-config-tstweb01-1.0-$rev.rpm: loctyp db.port=3306/etc/is24/system.properties tstweb etc /etc/is24/db.properties is24 /etc/is24/web.properties web.properties all VARIABLES SYSLOG_HOST etc is24loghost=log.domain.com[root@tstweb01 ~]# yum update system.properties Slide 10 | Configuration Management with SVN and RPM | Schlomo Schapiro 11. http://code.google.com/p/yadt/Static Structure Pros and Consconfig host Post-commit hook creates tstweb02 is24-config-$hostname RPM svn export locsvn ci tst Config patch VARIABLES fill in metadata SVN VARIABLESsvn co RPM_REQUIRES RPM_PROVIDES DB_HOST DB_USER etcYUM IS24 softwareVariables follow same is24Repository Automated RPM web.properties creation db.properties typstructure as config data web loctypdb.host=@@@DB_HOST@@@db.user=@@@DB_USER@@@ is24-config-tstweb01-1.0-$rev.rpm: tstwebdb.port=3306/etc/is24/system.propertiesOnly one tool (SVN) needed etc/etc/is24/db.properties is24 /etc/is24/web.properties web.properties allGUI talks only to SVN VARIABLES SYSLOG_HOST etc loghost=log.domain.com[root@tstweb01 ~]# yum update is24Versioning and change system.propertiestracking for variablesVariables and config files change together (atomic) easy rollbackIs this simple overlaying structure sufficient? KIS Simplify the world so that it is good enough! S!Slide 11 | Configuration Management with SVN and RPM | Schlomo Schapiro 12. http://code.google.com/p/yadt/ Big Picture Group Assignment meta hosts ttestserver-263 SPECIAL_KEY=value memberOf=apache tstweb02config memberOf=webapp all memberOf=tst etc groups is24 webapp system.properties member=tstweb02 groups memberOf=tomcat tst RPM_REQUIRES=is24-webapp-env, is24-jcr etc tst is24 member=tstweb02 web.properties DB_USER=tuv-db-user db.properties DB_HOST=dbserver-20 apache db.host=@@@DB_HOST@@@ etc db.user=@@@DB_USER@@@ db.port=3306 is24 svn ci httpd.conf webapp etc is24svn ci web.propertiesConfigYUM IS24 software SVN host Post-commitRepository Automated RPM hookcreation testserver-263 creates tstweb02is24-config-$hostname RPM Slide 12 | Configuration Management with SVN and RPM | Schlomo Schapiro 13. http://code.google.com/p/yadt/Group Assignment Pros and Consmeta hosts srv001 SPECIAL_KEY=value memberOf=apache testserver-263config memberOf=webappVariables follow same all memberOf=tuv etc groups is24 webapp member=testserver-263structure as config data system.properties groups memberOf=tomcat tuv RPM_REQUIRES=is24-webapp-env, is24-jcr etc tuv is24 member=testserver-263Only one tool (SVN) needed apache web.properties db.properties DB_USER=tuv-db-user DB_HOST=dbserver-20db.host=@@@DB_HOST@@@ etc db.user=@@@DB_USER@@@GUI talks only to SVNdb.port=3306 is24svn ci httpd.conf webapp etcVersioning and change host is24 web.propertiessvn ci ConfigSVN Post-commit YUM Repository IS24 softwareAutomated RPMtracking for variableshookcreation testserver-263creates tstweb02 is24-config-$hostnameRPMVariables and config files change together (atomic) easy rollbackFree assignment of hosts to groups, nested groupsGroups define function and pull in configuration and softwareFinal result is combination of all groups (in nesting order)Not any more so much KEEP IT SIMPLESlide 13 | Configuration Management with SVN and RPM | Schlomo Schapiro 14. http://code.google.com/p/yadt/System Personalisation: RPM Dependencies is24-config-tstweb02Requireshttpd is24-mod_jkis24-tomcatis24-base-envis24-appl-envis24-tomcat-data is24-jcris24-contentrulesis24-jcr-base is24-sitemapassembler is24-jtools is24-sitemapassembler-config jdk Slide 14 | Configuration Management with SVN and RPM | Schlomo Schapiro 15. http://code.google.com/p/yadt/YADT an Augmented Deployment ToolMeta Control LayerOpen Source GNU General Public License100% Python / BashLocal file management through RPMGlobal service managent through YADTModel datacenter dependencies: RPM packages Servers and services Between services on different serversStateless design current status is the state, define target statesAll actions restartable by following the dependency modelSVN RPM post-commit hook: Maintained as part of YADTSlide 15 | Configuration Management with SVN and RPM | Schlomo Schapiro 16. http://code.google.com/p/yadt/Immobilien Scout GmbHOperate Germanys Leading Real EstateListing Portal: www.immobilienscout24.deLinux, XEN, Apache, Tomcat, Java, LAMP,Oracle, MySQL, Nagios, DRBD, RPMOpen Source &amp; DevOps CompanyAlways seeking to hire new skill: Admins System Developershttp://bit.ly/is24greatplace System Engineers Architects Java &amp; PHP Developers Mobile (iOS &amp; Android) DevelopersSlide 16 | Configuration Management with SVN and RPM | Schlomo Schapiro 17. http://code.google.com/p/yadt/Thank you very much!Please contact me for furtherquestions and discussions.Kontakt:Immobilien Scout GmbH Fon: +49 30 243 01-1229Andreasstrae 10Email: schlomo.schapiro@immobilienscout24.de10243 BerlinURL: www.immobilienscout24.deSlide 17 | Configuration Management with SVN and RPM | Schlomo Schapiro </p>