technical debt - sotr14 - clarkie
DESCRIPTION
Technical Debt - Is your codebase approaching the fiscal cliff?TRANSCRIPT
Technical DebtIs your codebase approaching the fiscal cliff?
Andy Clarke [email protected] http://clarkie.io @clarkieclarkie
This is me
• Software Engineer for 8 years
• Never worked on a greenfield project (at work)
• Technical Architect @ Concrete
Outline
• What is Technical Debt?
• Debt Accumulation
• Debt Reduction
• Q + A
– Ward Cunningham, OOPSLA 1992
“Shipping first time code is like going into debt; !
a little debt speeds development so long as it is paid back promptly with a rewrite.”
– Intel, 2009
“Technical Debt is loosely defined as the volume of poorly written lines of code, poorly refactored, not following coding standards, not supported
with sufficient unit tests, and the amount of code duplication.”
– Michael (Doc) Norton, 2009
“Messy Code is not Technical Debt”
Technical DebtIs your code base project a ‘bag of spanners’?
Andy Clarke [email protected] http://clarkie.io @clarkieclarkie
Where does it come from?
• No vision
• No design
• Operational
• Experimentation
Where to keep your debt
• Code
• Persistent storage
• Architecture
• Infrastructure
Debt AccumulationCase Study: how Concrete got to where we are today
• A single platform to manage projects, plans, people, content, all from one place in the cloud.
• Enterprise collaboration
What is Concrete today?
• First client
• Need a picture library
In the beginning
• How do you create two new client sites?
• Easy
• Ctrl+C, Ctrl+V
In the beginning
• A client needs feature X
• A new client signs and requires feature X
Adding Features
• 2010 - 10 clients sites
• New client = new developer
• Cross selling features
• Bug fixes?
• 12 million lines of code!
Scaling
What we have today
• 30 production sites & 30 test sites
• Monthly releases
• Full LNSR HA stack
• Code deployed in sub 1 minute
• An understanding of our debt
What we have today
• 9 new applications
• A vision
• A product owner
• An architect
What we have today
0
10
20
30
40
50
60
70
80
90
2005
-04
2005
-07
2005
-10
2006
-01
2006
-04
2006
-07
2006
-10
2007
-01
2007
-04
2007
-07
2007
-10
2008
-01
2008
-04
2008
-07
2008
-10
2009
-01
2009
-04
2009
-07
2009
-10
2010
-01
2010
-04
2010
-07
2010
-10
2011
-01
2011
-04
2011
-07
2011
-10
2012
-01
2012
-04
2012
-07
2012
-10
2013
-01
2013
-04
2013
-07
2013
-10
2014
-01
2014
-04
Thou
san
ds
Total
Debt Reductionwhat we did to enable us to grow
How did we change
• Dev sites
• SubVersion
• Continuous Delivery
Continuous Delivery
Continuous Delivery
Continuous Delivery
• SVN post commit hooks
• Windows batch file & scheduled task
• Jenkins
Continuous Delivery
Consolidated Codebase
• 10 clients
• 6 applications
• 12,000,000 lines of code
Consolidated Codebase
• 30 clients
• 12 applications
• 72,000,000 million lines of code (estimate)
• 800,000 (actual)
Consolidated Codebase
• New framework
• Moved ‘core’ applications to a new repository
• ‘Core’ deployed independently
• Integration issues
• Shared database adds complexity (more debt)
Consolidated Codebase
• 3 new clients
• Usual copy and paste or…
• ConcretePlatform v2
Consolidated Codebase
• Merged ‘core’ back into a site
• Manually diff databases
• Merged everything on deployment
• Still deploying 60 copies of code base
• Deployments could take up to 1 hour
Consolidated Codebase
‘CP2’
New Clients
‘Coring’
Consolidated Codebase
• True single codebase (so close!)
• Deployed 10 times
• Each instance can run any client site
• Configuration driven
• Designed at SOTR 2013!
404 Handlers
• Tidy up static assets
• Enabled migration to Railo
404 Handlers
• <cfdirectory>
• QoQ
• Redirect or fail
Keeping dev sites up to date
• Code
• Database
• Configuration
Release Process• Monthly releases
• Run pre_core.sql
• Run pre_client.sql for each client db
• Deploy code (anything up to 30 minutes)
• Run post_core.sql
• Run post_client.sql for each client db
Database Rebuilds• Take back up from live
• Copy backup to dev server
• Drop the dev db
• Restore the backup from live
• ‘Massage’ the data
• Perform the release
• Every night
Database Rebuilds
Local Development
• Self installed + Configured
• Virtualbox Image shared
• Vagrant + Puppet
Monitoring and Metrics
• Statsd & Graphite
• Make it easy
• <cfset statsd.increment(“app.sessions.expired”) />
• Think about your metric structure
Monitoring and Metrics
• Logstash and Kibana
• Make it easy
Monitoring and Metricsfile { type => "statsD" path => "/var/www/*/WEB-INF/railo/logs/statsd.log" }
grok { type => “statsD" match => [ "message", ‘"%{DATA:level}" ,"%{DATA:thread}" ,"%{DATA:date}" ,"%{DATA:time}" ,”%{DATA:application}" ,"%{GREEDYDATA:app_message}"'] }
Monitoring and Metrics
So what’s next?
More to do• Decouple our backend services from the
databases
• Decouple the front end applications from the asset storage
• Consolidate our web servers
• Improve application usage metrics
• Unit testing - thanks Kev!
• Migrate to AWS
Codebase VCS Deployment Monitoring Logging
Crawl Individual _old FTP Nagios <cfmail
Walk ‘Core’ SubVersion Batch files Pingdom + PagerDuty <cflog
Run 1 Git Jenkins Cedexis + Statsd Kibana
Baby steps
Conclusion
• Technical debt isn’t bad
• Debt can give you a quicker time to market
• There isn’t a silver bullet
• You have to fight your corner
• It’s worth paying it off
