an infrastructure for team development - gaylord aulke
TRANSCRIPT
An Infrastructure for Team Based PHP DevelopmentGaylord AulkeDirector Professional Services DACHZend Technologies
Agenda
• Why am i talking here? (About the Speaker)• Motivation / The early days• About Processes• About Infrastructure• About Tools
|
About me
• Gaylord Aulke, 38, geek and sometimes businessman
• Did some (serious) Assembler Coding in the past (40kb Machinecode)
• Commercial Projects in Pascal (yes i am THAT old)• 4GL Business Application Coding (i was young
and needed the money)• Started my Web-Life with Perl / CGI• Early Adoption of Java and J2EE (i love it)• Started using PHP in 1997 for very small projects
only
|
Motivation: The good old days
|
Developer
SAMBA Share
Change
Apache
Test
QM
Test
Server
Deploy
PM
Control/Start
Handcrafted approach
• Everyone developed on a common Network share• Apache had its Docroot there• PROs:
Very simple to set up FREQUENT integration of all changes Easy to learn, no errors to make
• CONs: Editing a central file may render the application untestable for
everyone Conflict resolution must be done manually Version Mgmt the hard way (.old, .old2, .veryold, .ancient) Copying code base for concurrent codelines
|
Motivation
• We have serveral software projects• Some are under active development• Some in production and need occasional fixes• We have a limited number of Developers• Everybody needs to work on different projects• Context switches are normal and need to be
efficient• We need to keep a high quality level• We want to scale the team and the customer base
|
About Processes
• Development• Integration• QS• Deployment• Maintenance
|
Processes: Development
• Every developer should be able to change and extend whatever he wants without restrictions
• She should be able to test her changes very frequently in a realistic context
• Changes of others must not influence the tests of her new code
• After successfully testing the changes for themselves they need to be integrated with the changes of everybody else
|
Processes: Integration
• Usually, code runs in a certain environment in production
• Developer needs a development Environment that is as close as possible to the production environment
• Integration needs to be done as early as possible in the process
• The „real“ integration is often only possible at the customer‘s site. A simple way to do this without risk is needed
|
Processes: QS
• „Quality“ is a process issue• Do reasonable tests already at development time
(test data, environment)
• Integrate all code on a continuous integration Server
• Test ALL functionality of the application frequently• Before a release do another test / approval
process• Define measures and threshold for acceptable
quality in all process steps
|
Processes: Deployment
• Automate deployment• Record all deployments made• If you have serveral customers: Manage the
Deployment information centrally
|
Processes: Maintenance
• Manage Defects, Feature Requests, open issues and Requirements
• Define a process for quick fixes/changes• Define a Release Process that coordinates Fixes
and new Releases
|
Local Development
|
Developers
Dev. DB.
Dev.Machinetest
change
SVN RepositoryUpdate/commit/revert
Checkout/delete
QS Machine(all projects)
Checkout/delete
QM Team
test
Packager
Deploy!
Project Config DB
Customer Servers
Scp/zip/rsync/ftp
Local development
• Every developer runs his own copy of the application
• A SCM (Source Code Mgmt) manages all the differerent changes and revisions of the Software Lorna will talk about SVN in a few minutes
• Changes are made to the local copy, local Apache has its docroot there
• Developer tests locally until his component works• Then „check in“ to the repository is done
|
About Databases
• A common database for all developers usually does the job well because: Changes are often backwards compatible (new fields, new
tables) Everybody can use shared test data If the DB is corrupted, it can be re-built from scripts
• Have a separate DB with „authentic“ test data for QS
• Generate temporary DBs during runtime of automated tests (we create and drop these for every single test)
• If a separate DB is needed, you can always copy…
|
Issues with local Development
• Everyone needs to build a local environment with all Extensions / PHP Versions etc.
• Some environment details might only be available at a central location in one instance (an SAP system for example)
• No Control about „the right environment“
|
How we cope with it
• Define a common environment for all applications Framework Set of extensions to use
• Define a common local apache and path setup• Put all environment information in config files• Manage local config files and Vhosts for this setup
in the SCM• Build the application for portability (Windows
Development Systems, Linux Servers)
|
More Sophisticated: Development Sandboxes
|
Developers
Priv. Workspace
Priv. VHostPriv. VHost
Priv. VHost
Priv. Workspace
Priv. Workspace
test
change
SVN Repository
Management ApplicationSelect project
Update/commit/revert
Checkout/delete
checkout
QS VHost
Shared Workspace
Checkout/delete
QM Team
test
Select project
PackagerDeploy!
Deploy!
Project Config DB
Customer Servers
Scp/zip/rsync/ftp
Development Sandboxes: File Structure
|
SymLink
BaseDir
Testserver
Sandboxes
Project#1
Project#2
Project#3
Dyn. Data
Core
Custom
Docroot
Björn
Eric
Thomas
Slot#1
Slot#2
Slot#3
Dyn. Data
Core
Custom
Docroot
Development Sandboxes
• One Development Server• All Environment set up there• Everyone has his own area there where he can
Checkout/Test• Every Developer has a number of „Project Slots“
that he can fill with any project (- revision)• Slots are re-used, can be freed after checkin
|
Development Sandboxes
• PROs: Single Point of administration Complete environment can be set up A common set of tools can be used
• CONs: Quite complex in multi-project environments Development / SCM checkout on network share Single Point of Failure
|
Middelway: VMWare
• Build VMWare Image for each project with all dependencies
• Use hgfs (host file system access) to access local code base (outside VM)
• Host OS can mount shares in the network for shared files and test data
• VM can use external or internal DB• Running a project is as easy as copy a VM, start it
|
Packaging and Branching
|
Change #1 Change #3 Change #5
Change #2 Change #4Live Branch
Development
Change #6
Merge Merge
Deploy new development Versionto Customer Server. Old Live Branch is deleted and then Created new fromdevelopment branch.
Create live branch when developmentVersion has been installed on the liveServer.
Each change in the liveBranch is immediatelyMerged to the dev. branch
Packaging
• Always have the current version that is installed on the customer‘s site in your SCM
• Always follow the same process steps: Change locally on your system Test changes Run automatic Deploy
• To do so, Manage configuration on customer‘s site in some tool Build an infrastructure for deployment to different sites with
different transport mechanisms
|
Tools
• IDE• SCM• Automated Testing / Metrics / Code Coverage• Code-Analyzing• Continuous Integration• Packaging / Deployment• Bug Tracking / Project management
|
IDE: Zend Studio
|
Zend Platform
|
SCM
• RCS works cool for config files• CVS can already manage development projects
with many users• SVN is „the better CVS“ (which is already a
question of believe for some)• GIT is extremely cool• Commercial tools are very different in focus and
functionality/quality• We bought 80 Perforce licenses during New
Economy times (600 $ per seat) and didn‘t regret it.
|
Tesing: PHPUnit
|
Selenium IDE
|
PHP Code Sniffer
|
Static Code Analyzer / Metrics
• „Project Mess Detection“ PMD scans source code and looks for potential problems like: Possible bugs Dead code - unused local variables, parameters and private methods Suboptimal code Overcomplicated expressions - unnecessary if statements, for loops etc. Duplicate code - copied/pasted code means copied/pasted bugs
• Coding Style Check• Static Security analysis (www.armorize.com)• Static Code Quality analysis (Zend Code Analyzer)• …
|
Continuous Integration: Cruise Control
|
Packaging/Deployment• Build-Tools:
ANT, PHING, RAKE, Shell Scripts
• They can generate Packages (ZIP or RPM)• Deployment to Staging Server• Then local deployment to Cluster
• Integrated Solution (Ruby): Capistrano (http://capify.org/) Deploy, Rollback Remote Execution via SSH
• Puppet (http://reductivelabs.com/trac/puppet/)
|
Packaging: What we do
• Intranet-App that has a „deploy“ button for each Project
• SCM Sync to Test-Server first (test-deploy-Button)• Run tests there• Then deploy to customer‘s site (deploy-life Button)• Sync to test server is done via SCM checkout• Sync to customer server is done with arbitrary
shell script ANT, PHING or RAKE for building packets FTP, SCP, RSYNC or HTTP Upload to tansfer Postsync Script on Target Machines to unpack, install
|
Bug Tracking, Project Mgmt
• Trac Integrates with SCM, many plugins (example: http://www.agile42.com/cms/pages/download/) Manages Changes, Tickets, Wiki
• ActiveCollab More PM/Collaboration features, no SCM integration http://www.activecollab.com/
• Phprojekt More Groupware oriented http://www.phprojekt.com/
|
phpUnderControl
• Cruise Control• PHPUnit• PHPCodeSniffer• PHPDocumentor• http://www.phpundercontrol.org
|
phpUnderControl
|
Buildix
http://buildix.thoughtworks.com/
Buildix includes:• Subversion for Source Control• Mingle for Agile Project Management• Cruise Control for Continuous Integration• Trac as a wiki and bug-tracker• …plus a little bit of our own ThoughtWorks magic,
to glue it all together
|
Learnings / Ideas
• For Local development: Set up a local DNS server that resolves all project names to 127.0.0.1
• Example: <projectname>.<customername>.local
• For SCM servers, File System can make a big difference. Choose the right one!
|
Reference to tools
• Zend Studio: http://www.zend.com/products/zend_studio• Cruise Control: http://cruisecontrol.sourceforge.net/• Xinc: http://code.google.com/p/xinc/• Apache ANT: http://ant.apache.org/• PHP Alternative to ANT: http://phing.info/• Subversion: http://subversion.tigris.org/• Tortoise SVN Explorer Extension: http://tortoisesvn.tigris.org/• Trac SVN+ Technical Project Management: http://trac.edgewall.org/• Nagios: http://www.nagios.org/• Zend Platform: http://www.zend.com/de/products/zend_platform• PHPUnit: http://www.phpunit.de/• Selenium: http://www.openqa.org/selenium/
|
Books I found useful
• „Projekt-Automatisierung“, Mike Clark, Carl Hanser Verlag, München, 2006
• (english: „Pragmatic Project Automation – How to build …“, The Pragmatic Programmers, LLC. 2004
• Subversion Book: http://svnbook.red-bean.com/
• “Effektive Softwarearchitekturen”, Gernot Starke, Hanser, 2005
• “Organisational Patterns for Agile Software Development”, Coplien, Harrison, Lucent Technologies, Prentice Hall, 2005
• “Test Driven Development. By Example”, Addison-Wesley, Kent Beck, 2002
|