acceptance testing vs. unit testing: a developers perspective owen rogers [email protected]...
TRANSCRIPT
![Page 1: Acceptance Testing vs. Unit Testing: A Developers Perspective Owen Rogers orogers@thoughtworks.com Rob Styles rob@objectagile.com](https://reader035.vdocuments.net/reader035/viewer/2022062511/5518c279550346a61f8b5631/html5/thumbnails/1.jpg)
Acceptance Testing vs. Unit Testing:
A Developer’s Perspective
Owen Rogers
Rob Styles
![Page 2: Acceptance Testing vs. Unit Testing: A Developers Perspective Owen Rogers orogers@thoughtworks.com Rob Styles rob@objectagile.com](https://reader035.vdocuments.net/reader035/viewer/2022062511/5518c279550346a61f8b5631/html5/thumbnails/2.jpg)
2
What are Acceptance Tests?
• Tests owned and defined by the customer to verify that a story is complete and correct.
![Page 3: Acceptance Testing vs. Unit Testing: A Developers Perspective Owen Rogers orogers@thoughtworks.com Rob Styles rob@objectagile.com](https://reader035.vdocuments.net/reader035/viewer/2022062511/5518c279550346a61f8b5631/html5/thumbnails/3.jpg)
3
Why we use Acceptance Tests:
• Increase team confidence that the system is correct.• Structures communication between customer and
developers.• Help customer think through requirements.• Informs developer estimates.• Indicates how to split a story.• Concrete definition of story completion (“knowing when
we’re done”).• Early involvement of QA.• Automated regression testing for free.
![Page 4: Acceptance Testing vs. Unit Testing: A Developers Perspective Owen Rogers orogers@thoughtworks.com Rob Styles rob@objectagile.com](https://reader035.vdocuments.net/reader035/viewer/2022062511/5518c279550346a61f8b5631/html5/thumbnails/4.jpg)
4
Why other people don’t:
• Customer participation.• Management buy-in.• QA support.• More tests for developers to implement.• No common framework.
![Page 5: Acceptance Testing vs. Unit Testing: A Developers Perspective Owen Rogers orogers@thoughtworks.com Rob Styles rob@objectagile.com](https://reader035.vdocuments.net/reader035/viewer/2022062511/5518c279550346a61f8b5631/html5/thumbnails/5.jpg)
5
ATs and UTs are the same though, right?
• Both can and should be written first, before implementation.
• Both should be automated by containing assertions for validating expected results.
• Both typically use fixture setup and teardown code to run before and after the execution of the tests.
• Both can be implemented using an xUnit framework.
![Page 6: Acceptance Testing vs. Unit Testing: A Developers Perspective Owen Rogers orogers@thoughtworks.com Rob Styles rob@objectagile.com](https://reader035.vdocuments.net/reader035/viewer/2022062511/5518c279550346a61f8b5631/html5/thumbnails/6.jpg)
6
Difference: ATs are written by the customer.
• UTs only need to be read and understood by other developers.
• ATs must be defined in a language familiar to the customer.
• ATs require a customer meta-language:• Generic enough to capture requirements• Abstract enough to be maintainable• Simple enough to be easily understood
![Page 7: Acceptance Testing vs. Unit Testing: A Developers Perspective Owen Rogers orogers@thoughtworks.com Rob Styles rob@objectagile.com](https://reader035.vdocuments.net/reader035/viewer/2022062511/5518c279550346a61f8b5631/html5/thumbnails/7.jpg)
7
Example Acceptance Tests.
fit.ActionFixture
start eg.Page
enter location http://google.com
check title Google
enter link Jobs
check title About Google
enter link Press
enter link Review
check title Google Press Room
FIT Acceptance Test: FAT Acceptance Test:
• Request: “http://google.com”
• Check title: “Google”
• Click link: “Jobs”
• Check title: “About Google”
• Click link: “Press”
• Click link: “Review”
• Check title: “Google Press Room”
![Page 8: Acceptance Testing vs. Unit Testing: A Developers Perspective Owen Rogers orogers@thoughtworks.com Rob Styles rob@objectagile.com](https://reader035.vdocuments.net/reader035/viewer/2022062511/5518c279550346a61f8b5631/html5/thumbnails/8.jpg)
8
Difference: Customers don’t use an IDE.
• UTs are implemented in the language of development using an IDE.
• ATs should be defined using a tool accessible to the customer (Excel, web page, wiki).
• ATs must be retrieved from the tool and parsed.• Ideally, the customer should be able to run the ATs from
this environment.
![Page 9: Acceptance Testing vs. Unit Testing: A Developers Perspective Owen Rogers orogers@thoughtworks.com Rob Styles rob@objectagile.com](https://reader035.vdocuments.net/reader035/viewer/2022062511/5518c279550346a61f8b5631/html5/thumbnails/9.jpg)
9
Interacting with customer tool.
Excel
Wiki
AT Framework
Extract Test (.html)
Extract Test (.csv)
Execute TestWrite Test
![Page 10: Acceptance Testing vs. Unit Testing: A Developers Perspective Owen Rogers orogers@thoughtworks.com Rob Styles rob@objectagile.com](https://reader035.vdocuments.net/reader035/viewer/2022062511/5518c279550346a61f8b5631/html5/thumbnails/10.jpg)
10
Difference: ATs are not directly executable.
• UTs invoke code-under-test directly.• AT definition is abstracted from its implementation.• ATs specify what needs to be done, but not how to do it.• ATs require an implementation layer to invoke the
application.• AT definitions must be interpreted – parsed and mapped
onto corresponding implementation.• Tests must be definable and executable prior to
implementation.
![Page 11: Acceptance Testing vs. Unit Testing: A Developers Perspective Owen Rogers orogers@thoughtworks.com Rob Styles rob@objectagile.com](https://reader035.vdocuments.net/reader035/viewer/2022062511/5518c279550346a61f8b5631/html5/thumbnails/11.jpg)
11
Test Definition / Test Implementation
• Request: “http://google.com”
• Check title: “Google”
• Click link: “Jobs”
• Check title: “About Google”
• Click link: “Press”
• Click link: “Review”
• Check title: “Google Press Room”
Interpreter
public void Request(string url) { }
public void CheckTitle(string title) { }
public void ClickLink(string link) { }
Test: Verify Google Links Class: WebPageFixtureAT Framework
![Page 12: Acceptance Testing vs. Unit Testing: A Developers Perspective Owen Rogers orogers@thoughtworks.com Rob Styles rob@objectagile.com](https://reader035.vdocuments.net/reader035/viewer/2022062511/5518c279550346a61f8b5631/html5/thumbnails/12.jpg)
12
Difference: ATs test through the same interface.
• UTs test at level of a single unit (class).• ATs test at level of user/client interaction.• All tests go through a common gateway (Http, Soap, UI
controls or application façade)• Considerable opportunity for reuse across test
implementations.• ATs need a framework in the technology of the interface
it drives: Web, GUI, Web Service, Messaging etc.
![Page 13: Acceptance Testing vs. Unit Testing: A Developers Perspective Owen Rogers orogers@thoughtworks.com Rob Styles rob@objectagile.com](https://reader035.vdocuments.net/reader035/viewer/2022062511/5518c279550346a61f8b5631/html5/thumbnails/13.jpg)
13
Testing through the application gateway.
Application G
ateway
ApplicationDefinition Implementation
Application G
ateway T
est Client
Acceptance T
est Interpreter
ATD
ATD
ATimpl
ATimpl
ATimpl
ATD
![Page 14: Acceptance Testing vs. Unit Testing: A Developers Perspective Owen Rogers orogers@thoughtworks.com Rob Styles rob@objectagile.com](https://reader035.vdocuments.net/reader035/viewer/2022062511/5518c279550346a61f8b5631/html5/thumbnails/14.jpg)
14
Difference: ATs test the Users’ Experience.
• UTs should test all application logic in detail.• ATs encourage the customer to consider the all aspects
of user experience.• ATs should test:
• Interaction & Flow• Performance• Error Handling• Security
• ATs don’t test:• Usability• Look & Feel
![Page 15: Acceptance Testing vs. Unit Testing: A Developers Perspective Owen Rogers orogers@thoughtworks.com Rob Styles rob@objectagile.com](https://reader035.vdocuments.net/reader035/viewer/2022062511/5518c279550346a61f8b5631/html5/thumbnails/15.jpg)
15
Difference: ATs are end-to-end.
• UTs test single unit in isolation.• ATs test interaction of all layers in the system.• ATs require external systems to be operating correctly.
![Page 16: Acceptance Testing vs. Unit Testing: A Developers Perspective Owen Rogers orogers@thoughtworks.com Rob Styles rob@objectagile.com](https://reader035.vdocuments.net/reader035/viewer/2022062511/5518c279550346a61f8b5631/html5/thumbnails/16.jpg)
16
Difference: ATs are process-driven.
• Each UT typically tests a single public method.• ATs consist of a sequence of steps simulating a client
interacting with the system.• ATs may test the same steps repeatedly in different
variations with different data.• Considerable overlap across different ATs.• Single point of failure will break multiple ATs.
![Page 17: Acceptance Testing vs. Unit Testing: A Developers Perspective Owen Rogers orogers@thoughtworks.com Rob Styles rob@objectagile.com](https://reader035.vdocuments.net/reader035/viewer/2022062511/5518c279550346a61f8b5631/html5/thumbnails/17.jpg)
17
Difference: ATs have serious side-effects.
• For UTs, setup and teardown code should be common only to a single fixture.
• ATs interact extensively with external systems and the environment.
• ATs typically require separate setup and teardown per test as different tests will have different side-effects.
• Setup and teardown routines are commonly shared across tests and even across stories.
![Page 18: Acceptance Testing vs. Unit Testing: A Developers Perspective Owen Rogers orogers@thoughtworks.com Rob Styles rob@objectagile.com](https://reader035.vdocuments.net/reader035/viewer/2022062511/5518c279550346a61f8b5631/html5/thumbnails/18.jpg)
18
Difference: ATs suffer concurrency issues.
• UTs should be largely independent of their environment.• ATs require that the environment is setup as expected
by the tests. • Problems can result from different users running ATs
concurrently.• Ensure sufficient variation in the data used for each test
to prevent collisions.
![Page 19: Acceptance Testing vs. Unit Testing: A Developers Perspective Owen Rogers orogers@thoughtworks.com Rob Styles rob@objectagile.com](https://reader035.vdocuments.net/reader035/viewer/2022062511/5518c279550346a61f8b5631/html5/thumbnails/19.jpg)
19
Difference: Failing ATs don’t break the build.
• UTs must pass when checked in – failure breaks the build.
• ATs may be written at the start of a story and may not pass until the very end of the iteration.
• We mark ATs as [Ignore] until they have been fully implemented.
![Page 20: Acceptance Testing vs. Unit Testing: A Developers Perspective Owen Rogers orogers@thoughtworks.com Rob Styles rob@objectagile.com](https://reader035.vdocuments.net/reader035/viewer/2022062511/5518c279550346a61f8b5631/html5/thumbnails/20.jpg)
20
Difference: ATs don’t go quietly
• UTs are designed to run silently, with all verification encoded in the assertions.
• Customer requires visual feedback to give reassurance that test does what is expected.
• ATs need to provide step-by-step visibility of the contents and results of running the tests.
![Page 21: Acceptance Testing vs. Unit Testing: A Developers Perspective Owen Rogers orogers@thoughtworks.com Rob Styles rob@objectagile.com](https://reader035.vdocuments.net/reader035/viewer/2022062511/5518c279550346a61f8b5631/html5/thumbnails/21.jpg)
21
Difference: ATs are useful outside of dev environment.
• UTs are primarily useful to developers for ensuring the code does what was intended of it.
• ATs on the other hand can provide a good assessment of the overall health of the system.
• This makes them useful for assuring deployments to testing environments are sound.
• To do this usually requires environmental parameters to be extracted to a configurable location.
![Page 22: Acceptance Testing vs. Unit Testing: A Developers Perspective Owen Rogers orogers@thoughtworks.com Rob Styles rob@objectagile.com](https://reader035.vdocuments.net/reader035/viewer/2022062511/5518c279550346a61f8b5631/html5/thumbnails/22.jpg)
22
Difference: ATs mock out systems, not units.
• Introducing mocks for dependencies in UTs helps to decouple the code.
• We often have the same needs in ATs to mock third party interfaces that are not always available.
• Mocking is very hard because the AT is outside of the application context so a specific interface for the system being mocked has to be created.
• Mocking is still important to maintain customer’s trust in the solution.
• Introduce mocks to help simulate failure conditions.
![Page 23: Acceptance Testing vs. Unit Testing: A Developers Perspective Owen Rogers orogers@thoughtworks.com Rob Styles rob@objectagile.com](https://reader035.vdocuments.net/reader035/viewer/2022062511/5518c279550346a61f8b5631/html5/thumbnails/23.jpg)
Implementation
Mo
ck
Application Interface
Unit TestFixture
Setup
Teardown
UnitImplementation
impl
impl
impl
impl
impl
impl
UT
UT
UT
UT
UT
UT
AcceptanceTest Definition
ATimpl
ATD
ATimpl
ATD
ATimpl
ATD
setup
teardown
setup
teardown
setup
teardown
Acceptance T
est Client