Download - Avoiding test hell
![Page 2: Avoiding test hell](https://reader036.vdocuments.net/reader036/viewer/2022070513/5886cf3d1a28abcc7d8b801f/html5/thumbnails/2.jpg)
2
Overview• Why Test?• Sample Application• BDD and FitNesse• What would you consider hell to be like?• How to get into hell• And how to make it less painful
![Page 3: Avoiding test hell](https://reader036.vdocuments.net/reader036/viewer/2022070513/5886cf3d1a28abcc7d8b801f/html5/thumbnails/3.jpg)
3
Sample Price Reporting App
• We are a financial service provider• Loads security prices from a client• Produces internally calculated
security prices• Compares the two prices and
selects a price based on a rule• Produces a valuation report
Client Reporting Module
Client Prices
Price Comparator Valuations
File
Valuation Generator
Internal Price
Calculator
Client Price Loader
![Page 4: Avoiding test hell](https://reader036.vdocuments.net/reader036/viewer/2022070513/5886cf3d1a28abcc7d8b801f/html5/thumbnails/4.jpg)
4
Testing Basics• Why
• To prove what you’ve done works and is what they asked for• Reduce risk when doing deployment to production
• How• Manual vs Automated• Different types of tests, unit tests, component tests, integration tests and so on
• When• After development is complete? (Waterfall)• Before you do any development? (TDD)
• Who• Dedicated team of testers?• Developers, BAs, users?
![Page 5: Avoiding test hell](https://reader036.vdocuments.net/reader036/viewer/2022070513/5886cf3d1a28abcc7d8b801f/html5/thumbnails/5.jpg)
5
BDD: Behaviour Driven Development• Focus on the behaviour • As a … I want … so that• Given … when … then …
![Page 6: Avoiding test hell](https://reader036.vdocuments.net/reader036/viewer/2022070513/5886cf3d1a28abcc7d8b801f/html5/thumbnails/6.jpg)
6
FitNesse
Wiki FixturesSystem Under Test
Tables
Java Code (System Under
Test)
![Page 7: Avoiding test hell](https://reader036.vdocuments.net/reader036/viewer/2022070513/5886cf3d1a28abcc7d8b801f/html5/thumbnails/7.jpg)
7
How to know whether you’re in Test Hell
• Your tests costs you more than they save you• How long until you can release the next release into production?• If you don’t do any manual testing, are you good to go?• If a test fails, do you believe that it has failed?• If a test passes, do you trust it?• How long until a new joiner can get up and running writing tests?• A function needs to be changed, how long until the tests can be updated?
![Page 8: Avoiding test hell](https://reader036.vdocuments.net/reader036/viewer/2022070513/5886cf3d1a28abcc7d8b801f/html5/thumbnails/8.jpg)
8
Holding up Releases due to Testing
• How long does it take to arrive at a go / no go decision?• Manual testing• Release process not tested and automated• Solution: automate environment deployments• Reduce manual steps• Co-locate services for faster deployments
![Page 9: Avoiding test hell](https://reader036.vdocuments.net/reader036/viewer/2022070513/5886cf3d1a28abcc7d8b801f/html5/thumbnails/9.jpg)
9
When a test fails
• Did it really fail?• How long does it take to verify it’s a genuine break?• Was the failure due to system state?• Have you got some wait condition so that an action can complete?• Are you running tests in parallel but have some shared data?
• Solutions:• Do not hide testing data (next slide)• Isolate tests that exhibit random behavior (next slide)• Retry?
![Page 10: Avoiding test hell](https://reader036.vdocuments.net/reader036/viewer/2022070513/5886cf3d1a28abcc7d8b801f/html5/thumbnails/10.jpg)
10
![Page 11: Avoiding test hell](https://reader036.vdocuments.net/reader036/viewer/2022070513/5886cf3d1a28abcc7d8b801f/html5/thumbnails/11.jpg)
11
![Page 12: Avoiding test hell](https://reader036.vdocuments.net/reader036/viewer/2022070513/5886cf3d1a28abcc7d8b801f/html5/thumbnails/12.jpg)
12
Testing Boundaries
PriceComparisonEngineThread
Price Loader
extractPrices(source)
Price Rules Persister
loadRules(rule)
comparePrices()
persistPrices()
• Do you want to test the whole engine?
• Or just a couple of features?
![Page 13: Avoiding test hell](https://reader036.vdocuments.net/reader036/viewer/2022070513/5886cf3d1a28abcc7d8b801f/html5/thumbnails/13.jpg)
13
When a test passes
• Have you got enough coverage to be confident enough to say the functionality works• Vertical vs horizontal coverage
• Coverage tools
![Page 14: Avoiding test hell](https://reader036.vdocuments.net/reader036/viewer/2022070513/5886cf3d1a28abcc7d8b801f/html5/thumbnails/14.jpg)
14
How easy is it to maintain tests?• Adding new tests• How easy is it for these people?
• Experienced? New? Haven’t touched the system in a while?
• Modifying tests for new functionality?• How many tests do they have to update?
• Solution• Documentation and libraries• DRY principle
![Page 15: Avoiding test hell](https://reader036.vdocuments.net/reader036/viewer/2022070513/5886cf3d1a28abcc7d8b801f/html5/thumbnails/15.jpg)
15
Technical Debt in Tests
• Packaging of test code isn’t maintained as well as code
• Less time spent refactoring and reducing duplication
• Often you will find similar methods that differ by one or two parameters
• Solution• Refactor and remove duplicate methods• Create libraries or APIs that create test
objects
![Page 16: Avoiding test hell](https://reader036.vdocuments.net/reader036/viewer/2022070513/5886cf3d1a28abcc7d8b801f/html5/thumbnails/16.jpg)
16
Mocking
• Simulate behavior of components and interfaces so that we can isolate that one component we want to test
• Faster than connecting to a file system or a database
• Can be brittle and a nightmare to maintain
• Can alleviate some issues if you create a test library that hides some of the unnecessary details
PriceComparisonEngineThread
Price Loader
extractPrices(source)
Price Rules Persister
loadRules(rule)
comparePrices()
persistPrices()
![Page 17: Avoiding test hell](https://reader036.vdocuments.net/reader036/viewer/2022070513/5886cf3d1a28abcc7d8b801f/html5/thumbnails/17.jpg)
17
Test Objects
![Page 18: Avoiding test hell](https://reader036.vdocuments.net/reader036/viewer/2022070513/5886cf3d1a28abcc7d8b801f/html5/thumbnails/18.jpg)
18
Vendor Software and External Data Providers
• Can not control when API and interface changes
• Complex data sets• Potentially quite difficult to mock out
their interfaces
• Pitfalls of the following approaches• Capture and Replay: changes in
message format• Table dumps: what if table structure
changes due to version upgrade?• Solution
• Use the Vendor API where possible
![Page 19: Avoiding test hell](https://reader036.vdocuments.net/reader036/viewer/2022070513/5886cf3d1a28abcc7d8b801f/html5/thumbnails/19.jpg)
19
People Issues
• The largest contributor to Test Hell: the people in your team• What are their motivations? What do they get from doing more frequent releases?• Why do they want to do anything different? Why replace their manual tasks with
automation?• Structure of your team• Who is good at what role? Not everyone is the same• Personality types such as: MBTI
• What if they don’t believe in testing?• Gamification? • Who are management focusing their attention on? The poor performers at
the cost of good performers?
![Page 20: Avoiding test hell](https://reader036.vdocuments.net/reader036/viewer/2022070513/5886cf3d1a28abcc7d8b801f/html5/thumbnails/20.jpg)
20
Things to Google …
• Unit Testing• BDD• TDD• Jmock• EasyMock• PowerMock• Junit• Continuous Integration
• Object Mothers vs Test Builders• Test Coverage• DevOps• FitNesse• Concordion• Cucumber• JBehave