Download - VodQA_Parallelizingcukes_AmanKing
![Page 1: VodQA_Parallelizingcukes_AmanKing](https://reader035.vdocuments.net/reader035/viewer/2022070301/5466d60eaf795965358b4ea7/html5/thumbnails/1.jpg)
Saturday, 26Saturday, 26thth March, 2011 March, 2011
![Page 2: VodQA_Parallelizingcukes_AmanKing](https://reader035.vdocuments.net/reader035/viewer/2022070301/5466d60eaf795965358b4ea7/html5/thumbnails/2.jpg)
Reducing build timewhen patience is not a virtue
Aman KingApplication Developer
ThoughtWorks
![Page 3: VodQA_Parallelizingcukes_AmanKing](https://reader035.vdocuments.net/reader035/viewer/2022070301/5466d60eaf795965358b4ea7/html5/thumbnails/3.jpg)
Recognize these?
![Page 4: VodQA_Parallelizingcukes_AmanKing](https://reader035.vdocuments.net/reader035/viewer/2022070301/5466d60eaf795965358b4ea7/html5/thumbnails/4.jpg)
Rank these
![Page 5: VodQA_Parallelizingcukes_AmanKing](https://reader035.vdocuments.net/reader035/viewer/2022070301/5466d60eaf795965358b4ea7/html5/thumbnails/5.jpg)
Did you choose this?
1 32
![Page 6: VodQA_Parallelizingcukes_AmanKing](https://reader035.vdocuments.net/reader035/viewer/2022070301/5466d60eaf795965358b4ea7/html5/thumbnails/6.jpg)
Or did you choose this?
1 23
![Page 7: VodQA_Parallelizingcukes_AmanKing](https://reader035.vdocuments.net/reader035/viewer/2022070301/5466d60eaf795965358b4ea7/html5/thumbnails/7.jpg)
Patience is not always a virtue!
![Page 8: VodQA_Parallelizingcukes_AmanKing](https://reader035.vdocuments.net/reader035/viewer/2022070301/5466d60eaf795965358b4ea7/html5/thumbnails/8.jpg)
Fail fast!quick feedback
![Page 9: VodQA_Parallelizingcukes_AmanKing](https://reader035.vdocuments.net/reader035/viewer/2022070301/5466d60eaf795965358b4ea7/html5/thumbnails/9.jpg)
Rank these
High functional coverage
Short build time
![Page 10: VodQA_Parallelizingcukes_AmanKing](https://reader035.vdocuments.net/reader035/viewer/2022070301/5466d60eaf795965358b4ea7/html5/thumbnails/10.jpg)
Did you choose this?
①High functional coverage
②Short build time
![Page 11: VodQA_Parallelizingcukes_AmanKing](https://reader035.vdocuments.net/reader035/viewer/2022070301/5466d60eaf795965358b4ea7/html5/thumbnails/11.jpg)
Or did you choose this?
①Short build time
②High functional coverage
![Page 12: VodQA_Parallelizingcukes_AmanKing](https://reader035.vdocuments.net/reader035/viewer/2022070301/5466d60eaf795965358b4ea7/html5/thumbnails/12.jpg)
You can choose both!
①Short build time
①High functional coverage
![Page 13: VodQA_Parallelizingcukes_AmanKing](https://reader035.vdocuments.net/reader035/viewer/2022070301/5466d60eaf795965358b4ea7/html5/thumbnails/13.jpg)
But before we see how…
![Page 14: VodQA_Parallelizingcukes_AmanKing](https://reader035.vdocuments.net/reader035/viewer/2022070301/5466d60eaf795965358b4ea7/html5/thumbnails/14.jpg)
Project background
![Page 15: VodQA_Parallelizingcukes_AmanKing](https://reader035.vdocuments.net/reader035/viewer/2022070301/5466d60eaf795965358b4ea7/html5/thumbnails/15.jpg)
Content driven, community oriented
website
![Page 16: VodQA_Parallelizingcukes_AmanKing](https://reader035.vdocuments.net/reader035/viewer/2022070301/5466d60eaf795965358b4ea7/html5/thumbnails/16.jpg)
Ruby on Rails
Cucumber+
Capybara+
Selenium
![Page 17: VodQA_Parallelizingcukes_AmanKing](https://reader035.vdocuments.net/reader035/viewer/2022070301/5466d60eaf795965358b4ea7/html5/thumbnails/17.jpg)
4 : 1 ratio of Dev : QA
Everyone writes functional tests in a common automation
suite
Acceptance Test Driven Development (ATDD)
![Page 18: VodQA_Parallelizingcukes_AmanKing](https://reader035.vdocuments.net/reader035/viewer/2022070301/5466d60eaf795965358b4ea7/html5/thumbnails/18.jpg)
Our problems
![Page 19: VodQA_Parallelizingcukes_AmanKing](https://reader035.vdocuments.net/reader035/viewer/2022070301/5466d60eaf795965358b4ea7/html5/thumbnails/19.jpg)
Long build time~ 55 minutes
![Page 20: VodQA_Parallelizingcukes_AmanKing](https://reader035.vdocuments.net/reader035/viewer/2022070301/5466d60eaf795965358b4ea7/html5/thumbnails/20.jpg)
Non-deterministic failures
![Page 21: VodQA_Parallelizingcukes_AmanKing](https://reader035.vdocuments.net/reader035/viewer/2022070301/5466d60eaf795965358b4ea7/html5/thumbnails/21.jpg)
Manual reruns needed
![Page 22: VodQA_Parallelizingcukes_AmanKing](https://reader035.vdocuments.net/reader035/viewer/2022070301/5466d60eaf795965358b4ea7/html5/thumbnails/22.jpg)
Our solution:Parallelization
![Page 23: VodQA_Parallelizingcukes_AmanKing](https://reader035.vdocuments.net/reader035/viewer/2022070301/5466d60eaf795965358b4ea7/html5/thumbnails/23.jpg)
Basic idea
![Page 24: VodQA_Parallelizingcukes_AmanKing](https://reader035.vdocuments.net/reader035/viewer/2022070301/5466d60eaf795965358b4ea7/html5/thumbnails/24.jpg)
Reduced build time~ 5 minutes
![Page 25: VodQA_Parallelizingcukes_AmanKing](https://reader035.vdocuments.net/reader035/viewer/2022070301/5466d60eaf795965358b4ea7/html5/thumbnails/25.jpg)
Build time chart
![Page 26: VodQA_Parallelizingcukes_AmanKing](https://reader035.vdocuments.net/reader035/viewer/2022070301/5466d60eaf795965358b4ea7/html5/thumbnails/26.jpg)
What we did
![Page 27: VodQA_Parallelizingcukes_AmanKing](https://reader035.vdocuments.net/reader035/viewer/2022070301/5466d60eaf795965358b4ea7/html5/thumbnails/27.jpg)
Parallelization on a single multicore machine
used Ruby library parallel_tests
![Page 28: VodQA_Parallelizingcukes_AmanKing](https://reader035.vdocuments.net/reader035/viewer/2022070301/5466d60eaf795965358b4ea7/html5/thumbnails/28.jpg)
Report generation in parallelwrote custom report formatter
![Page 29: VodQA_Parallelizingcukes_AmanKing](https://reader035.vdocuments.net/reader035/viewer/2022070301/5466d60eaf795965358b4ea7/html5/thumbnails/29.jpg)
Isolating databaseseach process needs its own database
![Page 30: VodQA_Parallelizingcukes_AmanKing](https://reader035.vdocuments.net/reader035/viewer/2022070301/5466d60eaf795965358b4ea7/html5/thumbnails/30.jpg)
database.ymltest: &test adapter: mysql2 encoding: utf8 reconnect: false database: myapp_test<%= ENV['TEST_ENV_NUMBER'] %> pool: 5 username: root password:
![Page 31: VodQA_Parallelizingcukes_AmanKing](https://reader035.vdocuments.net/reader035/viewer/2022070301/5466d60eaf795965358b4ea7/html5/thumbnails/31.jpg)
Isolating external dependencies (Solr)
each process needs its own Solr instance
![Page 32: VodQA_Parallelizingcukes_AmanKing](https://reader035.vdocuments.net/reader035/viewer/2022070301/5466d60eaf795965358b4ea7/html5/thumbnails/32.jpg)
sunspot.ymltest: solr: hostname: localhost port: <%= 8982 - ENV['TEST_ENV_NUMBER'].to_i %> log_level: INFO #WARNING log_file: <%= File.join(::Rails.root, 'log', "solr_#{ENV['TEST_ENV_NUMBER'].to_i}.log") %> data_path: <%= File.join(::Rails.root, 'solr', 'data’, "#{ENV['TEST_ENV_NUMBER'].to_i}") %> pid_path: <%= File.join(::Rails.root, 'solr', 'pids', "#{ENV['TEST_ENV_NUMBER'].to_i}") %>
![Page 33: VodQA_Parallelizingcukes_AmanKing](https://reader035.vdocuments.net/reader035/viewer/2022070301/5466d60eaf795965358b4ea7/html5/thumbnails/33.jpg)
Handling multiple Firefox instances
and Selenium’s use of
shared ephemeral ports
![Page 34: VodQA_Parallelizingcukes_AmanKing](https://reader035.vdocuments.net/reader035/viewer/2022070301/5466d60eaf795965358b4ea7/html5/thumbnails/34.jpg)
Monitoring Memory and CPU usage
adjust number of parallel process accordingly
![Page 35: VodQA_Parallelizingcukes_AmanKing](https://reader035.vdocuments.net/reader035/viewer/2022070301/5466d60eaf795965358b4ea7/html5/thumbnails/35.jpg)
Auto rerunning failed testsnon-deterministic failures are likelier with
the CPU under stress
![Page 36: VodQA_Parallelizingcukes_AmanKing](https://reader035.vdocuments.net/reader035/viewer/2022070301/5466d60eaf795965358b4ea7/html5/thumbnails/36.jpg)
Beyond parallelization…
![Page 37: VodQA_Parallelizingcukes_AmanKing](https://reader035.vdocuments.net/reader035/viewer/2022070301/5466d60eaf795965358b4ea7/html5/thumbnails/37.jpg)
Stubbing external callsreduce dependency-related delays where
avoidable
![Page 38: VodQA_Parallelizingcukes_AmanKing](https://reader035.vdocuments.net/reader035/viewer/2022070301/5466d60eaf795965358b4ea7/html5/thumbnails/38.jpg)
Consolidating scenarioscombine similar test scenarios into single
runs instead of separate tests
![Page 39: VodQA_Parallelizingcukes_AmanKing](https://reader035.vdocuments.net/reader035/viewer/2022070301/5466d60eaf795965358b4ea7/html5/thumbnails/39.jpg)
Maintaining conventions when writing automated tests
avoid time-based wait statements, use test framework APIs that take lesser time, etc
![Page 40: VodQA_Parallelizingcukes_AmanKing](https://reader035.vdocuments.net/reader035/viewer/2022070301/5466d60eaf795965358b4ea7/html5/thumbnails/40.jpg)
Conclusion
![Page 41: VodQA_Parallelizingcukes_AmanKing](https://reader035.vdocuments.net/reader035/viewer/2022070301/5466d60eaf795965358b4ea7/html5/thumbnails/41.jpg)
Resourceshttp://cukes.info
http://rubygems.org/gems/capybara
http://seleniumhq.org/projects/remote-control
http://rubygems.org/gems/parallel_tests
http://selenium-grid.seleniumhq.org
http://test-load-balancer.github.com
http://github.com/bblimke/webmock
http://testing.thoughtworks.com