![Page 1: Pushing The Boundaries Of Continuous Integration](https://reader034.vdocuments.net/reader034/viewer/2022051015/5537f80c4a79597c778b4654/html5/thumbnails/1.jpg)
Pushing the Boundaries ofTesting and
Continuous Integration
Fabrizio Cannizzo Raghav RameshRobbie Clutton
![Page 2: Pushing The Boundaries Of Continuous Integration](https://reader034.vdocuments.net/reader034/viewer/2022051015/5537f80c4a79597c778b4654/html5/thumbnails/2.jpg)
http://flickr.com/photos/zoonabar/159442457/
Who are we?
http://flickr.com/photos/kerrybuckley/1356791754/in/pool-web21c
![Page 3: Pushing The Boundaries Of Continuous Integration](https://reader034.vdocuments.net/reader034/viewer/2022051015/5537f80c4a79597c778b4654/html5/thumbnails/3.jpg)
![Page 4: Pushing The Boundaries Of Continuous Integration](https://reader034.vdocuments.net/reader034/viewer/2022051015/5537f80c4a79597c778b4654/html5/thumbnails/4.jpg)
Agile to the Core!
XP/Scrum- TDD- Pair programming- Continuous Build/Integration
o Unit/Acceptance Testo Reporting
![Page 5: Pushing The Boundaries Of Continuous Integration](https://reader034.vdocuments.net/reader034/viewer/2022051015/5537f80c4a79597c778b4654/html5/thumbnails/5.jpg)
Feature completeness !=
completed product
http://flickr.com/photos/mattimattila/2188826561/
![Page 6: Pushing The Boundaries Of Continuous Integration](https://reader034.vdocuments.net/reader034/viewer/2022051015/5537f80c4a79597c778b4654/html5/thumbnails/6.jpg)
What's Missing?
Robustness Performance
Scalability
Failover
![Page 7: Pushing The Boundaries Of Continuous Integration](https://reader034.vdocuments.net/reader034/viewer/2022051015/5537f80c4a79597c778b4654/html5/thumbnails/7.jpg)
We needed to know the
thing would stand up
http://flickr.com/photos/alan-light/2435511982/
![Page 8: Pushing The Boundaries Of Continuous Integration](https://reader034.vdocuments.net/reader034/viewer/2022051015/5537f80c4a79597c778b4654/html5/thumbnails/8.jpg)
But we didn't want to wait for formal QA testing
![Page 9: Pushing The Boundaries Of Continuous Integration](https://reader034.vdocuments.net/reader034/viewer/2022051015/5537f80c4a79597c778b4654/html5/thumbnails/9.jpg)
We had to last three spooky nights
http://flickr.com/photos/luchilu/677786684/
![Page 10: Pushing The Boundaries Of Continuous Integration](https://reader034.vdocuments.net/reader034/viewer/2022051015/5537f80c4a79597c778b4654/html5/thumbnails/10.jpg)
After four weekends, we were done
http://flickr.com/photos/22243977@N06/2457787000/
BugsMemory LeaksPoor EnvironmentLog Extractors
![Page 11: Pushing The Boundaries Of Continuous Integration](https://reader034.vdocuments.net/reader034/viewer/2022051015/5537f80c4a79597c778b4654/html5/thumbnails/11.jpg)
What next?
Create scenarios Define expectations
Automate
Continuously test
![Page 12: Pushing The Boundaries Of Continuous Integration](https://reader034.vdocuments.net/reader034/viewer/2022051015/5537f80c4a79597c778b4654/html5/thumbnails/12.jpg)
So... how does it really work?
![Page 13: Pushing The Boundaries Of Continuous Integration](https://reader034.vdocuments.net/reader034/viewer/2022051015/5537f80c4a79597c778b4654/html5/thumbnails/13.jpg)
Technologies
Cruise Control
Java Spring Framework
Hand crafted Java code
JMX Console
UltraMonkey
SIPp
JainSIP
Mockphones
![Page 14: Pushing The Boundaries Of Continuous Integration](https://reader034.vdocuments.net/reader034/viewer/2022051015/5537f80c4a79597c778b4654/html5/thumbnails/14.jpg)
Cruise control is the engine that continuously runs typical usage scenarios.
http://flickr.com/photos/_fxr/2142296214/
Ensure the entire scenario played out as expected
http://flickr.com/photos/mike-burns/35534789/
connect connect
join
![Page 15: Pushing The Boundaries Of Continuous Integration](https://reader034.vdocuments.net/reader034/viewer/2022051015/5537f80c4a79597c778b4654/html5/thumbnails/15.jpg)
Why are scenarios relevant?
Robustness
Performance
Scalability
Failover
![Page 16: Pushing The Boundaries Of Continuous Integration](https://reader034.vdocuments.net/reader034/viewer/2022051015/5537f80c4a79597c778b4654/html5/thumbnails/16.jpg)
We automated phone behaviour using SIPp...
Different configurations are implemented using different spring application contexts and network setups
Scenario runners are java applications
Scenarios are POJOs loaded from spring application context
![Page 17: Pushing The Boundaries Of Continuous Integration](https://reader034.vdocuments.net/reader034/viewer/2022051015/5537f80c4a79597c778b4654/html5/thumbnails/17.jpg)
![Page 18: Pushing The Boundaries Of Continuous Integration](https://reader034.vdocuments.net/reader034/viewer/2022051015/5537f80c4a79597c778b4654/html5/thumbnails/18.jpg)
![Page 19: Pushing The Boundaries Of Continuous Integration](https://reader034.vdocuments.net/reader034/viewer/2022051015/5537f80c4a79597c778b4654/html5/thumbnails/19.jpg)
![Page 20: Pushing The Boundaries Of Continuous Integration](https://reader034.vdocuments.net/reader034/viewer/2022051015/5537f80c4a79597c778b4654/html5/thumbnails/20.jpg)
![Page 21: Pushing The Boundaries Of Continuous Integration](https://reader034.vdocuments.net/reader034/viewer/2022051015/5537f80c4a79597c778b4654/html5/thumbnails/21.jpg)
TestRR
• Generic Frameworko Robustnesso Performance
• http://code.google.com/p/testrr/
![Page 22: Pushing The Boundaries Of Continuous Integration](https://reader034.vdocuments.net/reader034/viewer/2022051015/5537f80c4a79597c778b4654/html5/thumbnails/22.jpg)
Who else is talking about this?
James Bull - Pragmatic Performance Testing - http://is.gd/SUv
Dr. Dobbs - Continuous Integration and Performance Testing - http://is.gd/12Ds
Manfred Lange - Introducing Automated Performance Testing - http://is.gd/SUj
Paul King & Dalton Cranston – Open Source Performance Testing Tools For The Web
![Page 23: Pushing The Boundaries Of Continuous Integration](https://reader034.vdocuments.net/reader034/viewer/2022051015/5537f80c4a79597c778b4654/html5/thumbnails/23.jpg)
What we have learned that you could take away...Automate early and often
unit and acceptance tests are an essential first step
maintaining these builds can have severe impact on velocity
the customer must buy in: formulate user stories to drive the creation of tests
![Page 24: Pushing The Boundaries Of Continuous Integration](https://reader034.vdocuments.net/reader034/viewer/2022051015/5537f80c4a79597c778b4654/html5/thumbnails/24.jpg)
... and also...
This drives creation of robust code earlier
Leverage existing frameworks for performance testing
Build up testing strategies incrementally
These tests are resource hungry, use an appropriate environment
![Page 25: Pushing The Boundaries Of Continuous Integration](https://reader034.vdocuments.net/reader034/viewer/2022051015/5537f80c4a79597c778b4654/html5/thumbnails/25.jpg)
Q & A Fabrizio Cannizzo:
- http://smartrics.blogspot.comRaghav Ramesh: - http://ragstorooks.wordpress.comRobbie Clutton: - http://blog.iclutton.com
Aloha SIP A/S: https://trac.osmosoft.com/AlohaWeb21C SDK: http://web21c.bt.com