验收测试驱动开发实战验收测试驱动开发实战Acceptance Test Driven Acceptance Test Driven Development in practiceDevelopment in practice

Steven Mak Steven Mak 麥天志麥天志steven@odd-e.comsteven@odd-e.com

What are we up to now?What are we up to now?

Lost in translationLost in translation

Do not explain whyDo not explain why

Gaps discovered only until coding startedGaps discovered only until coding started

Cumulative effects of small misunderstandingsCumulative effects of small misunderstandings

Inadequate and essentially flawed Inadequate and essentially flawed requirements and specificationsrequirements and specifications

Failing to meet actual Failing to meet actual needsneeds

are obvious things really obvious?are obvious things really obvious?

Fulfilling specifications does not guarantee Fulfilling specifications does not guarantee successsuccess

Imperative requirementsImperative requirements

Meeting the needs with Meeting the needs with Acceptance TDDAcceptance TDD

Drive implementation of a requirement through Drive implementation of a requirement through a set of automated, executable acceptance a set of automated, executable acceptance teststests

RequiremeRequirementnt

RequiremeRequirementnt

AcceptancAcceptance Teste Test

AcceptancAcceptance Teste Test

ImplementImplementationation

ImplementImplementationationFeedbackFeedback

ATDD in a NutshellATDD in a Nutshell

Real-world examples to build a shared understanding Real-world examples to build a shared understanding of the domainof the domain

Select a set of these examples to be a specification Select a set of these examples to be a specification and an acceptance test suiteand an acceptance test suite

Automate the verification of acceptance testsAutomate the verification of acceptance tests

Focus the software development effort on the Focus the software development effort on the acceptance testsacceptance tests

Use the set of acceptance tests to facilitate Use the set of acceptance tests to facilitate discussion about future change requests.discussion about future change requests.

customer documentationcustomer documentation

The ATDD cycleThe ATDD cycle

A-TDDA-TDDWorkshWorksh

opop

A-TDDA-TDDWorkshWorksh

opop

Featu

reFe

atu

reD

on

eD

on

eFe

atu

reFe

atu

reD

on

eD

on

e

DevelopersDevelopersTestersTesters

Product OwnerProduct OwnerArchitectArchitect

Technical writersTechnical writers

DevelopersDevelopersTestersTesters

Product OwnerProduct OwnerArchitectArchitect

Technical writersTechnical writers

Example Example teststests

codingcoding

testingtesting

architecturearchitecture

other activitiesother activities

Benefits of ATDDBenefits of ATDD

Comprehensible examples over complex Comprehensible examples over complex formulasformulas

Close CollaborationClose Collaboration

Definition of DoneDefinition of Done

Co-operative WorkCo-operative Work

Trust and CommitmentTrust and Commitment

Testing on system levelTesting on system level

Variations - an escape?Variations - an escape?

Behaviour-driven developmentBehaviour-driven development

Example-driven developmentExample-driven development

Executable specificationsExecutable specifications

Names do not matter, but underlying practices Names do not matter, but underlying practices mattermatter

Worthwhile to try if your business people do Worthwhile to try if your business people do not like “testing”not like “testing”

Ideal candidate to work Ideal candidate to work withwith

Shared interest in successShared interest in success

Authority to make decisionAuthority to make decision

Ability to understand implicationsAbility to understand implications

Ability to explain the domainAbility to explain the domain

Specification by Specification by ExamplesExamples

Use realistic examples to demonstrate Use realistic examples to demonstrate differences in possibilities instead of abstract differences in possibilities instead of abstract requirementsrequirements

Write specifications down as tablesWrite specifications down as tables

Workflows:Workflows:

PreconditionsPreconditions

Processing stepsProcessing steps

VerificationsVerifications

Examples, Tests, and Examples, Tests, and SpecSpec

can becomecan become

elaborate

elaborate verif

yve

rify

Specification workshopSpecification workshop

Ask the domain expertsAsk the domain experts

Developers and testers should suggest Developers and testers should suggest examples of edges or important issues for examples of edges or important issues for discussiondiscussion

Ubiquitous languageUbiquitous language

Organise feedback to ensure shared Organise feedback to ensure shared understandingunderstanding

Use facilitator to stay focused if neededUse facilitator to stay focused if needed

Acceptance criteriaAcceptance criteria

Write tests collaborativelyWrite tests collaboratively

Examples in a form close to what your automation tool Examples in a form close to what your automation tool can understandcan understand

Keep tests in a form that is human-readableKeep tests in a form that is human-readable

Specification over Scripting, describe WHAT, not howSpecification over Scripting, describe WHAT, not how

Acceptance tests to prevent defects, not to discoverAcceptance tests to prevent defects, not to discover

Not necessarily automate anythingNot necessarily automate anything

Acceptance tests only work when we can discuss themAcceptance tests only work when we can discuss them

Some considerationsSome considerations

User InterfaceUser Interface

Easy?Easy?

Fragile?Fragile?

Performance issues?Performance issues?

Boundary of StubBoundary of Stub

Sufficiently closeSufficiently close

Simulators?Simulators?

Business logicBusiness logic

Not from developer perspectiveNot from developer perspective

Acceptance Test smellsAcceptance Test smells

Long testsLong tests

Parameters of calculation tests that always have Parameters of calculation tests that always have the same valuethe same value

Similar test with minor differencesSimilar test with minor differences

Tests that reflect the way code was writtenTests that reflect the way code was written

Tests fail intermittently even though you didn’t Tests fail intermittently even though you didn’t change any codechange any code

Interdependent tests, e.g. setup for othersInterdependent tests, e.g. setup for others

ChangeChange

Use existing acceptance tests to discuss future Use existing acceptance tests to discuss future changeschanges

Seek advices from customer to determine if it Seek advices from customer to determine if it specifies obsolete functionality when test failsspecifies obsolete functionality when test fails

Automate periodic execution of regression Automate periodic execution of regression tests with CItests with CI

Keep tests in the same version control as codeKeep tests in the same version control as code

ToolsTools

Table-based frameworksTable-based frameworks

FIT, FIT, http://fit.c2.comhttp://fit.c2.com

RobotFramework, RobotFramework, http://robotframework.orghttp://robotframework.org

Text-based frameworksText-based frameworks

Exactor, Exactor, http://exactor.sourceforge.nethttp://exactor.sourceforge.net

TextTest, TextTest, http://texttest.carmen.sehttp://texttest.carmen.se

FITFIT

FIT stands for “FIT stands for “FFramework for ramework for IIntegrated ntegrated TTests”ests”

Most popular framework in-useMost popular framework in-use

Table-basedTable-based

Supporting languages like Java, C#, Python, or Supporting languages like Java, C#, Python, or RubyRuby

FIT in practiceFIT in practice

Customer writes a Customer writes a test document test document

containing containing examplesexamples

Customer writes a Customer writes a test document test document

containing containing examplesexamples

Technical staff Technical staff enhance the tables enhance the tables

in the docin the doc

Technical staff Technical staff enhance the tables enhance the tables

in the docin the doc

Technical staff Technical staff implements fixture implements fixture

classesclasses

Technical staff Technical staff implements fixture implements fixture

classesclassesExecutable TestExecutable TestExecutable TestExecutable Test

Test doc Test doc with with

tablestables

Test doc Test doc with with

tablestables

Test doc with Test doc with sanitised sanitised

tablestables

Test doc with Test doc with sanitised sanitised

tablestables

Test doc and Test doc and backing code backing code

(e.g. Java)(e.g. Java)

Test doc and Test doc and backing code backing code

(e.g. Java)(e.g. Java)

Robot FrameworkRobot Framework

Python-based Keyword-driven test automation Python-based Keyword-driven test automation frameworkframework

Test libraries implemented either in Python or JavaTest libraries implemented either in Python or Java

Test cases are written in tabular format, save in Test cases are written in tabular format, save in HTML or TSV filesHTML or TSV files

Syntax similar to natural languageSyntax similar to natural language

Users can create new keywords from existing ones Users can create new keywords from existing ones and contribute to the projectand contribute to the project

Preparing Test casesPreparing Test cases

Test CaseTest Case ActionAction ArgumentArgument

Valid LoginValid Login Open Login PageOpen Login Page

Input NameInput Name demodemo

Input PasswordInput Password modemode

Submit Submit CredentialsCredentials

Test case nameTest case name

Test procedure using keywordsTest procedure using keywords

Keyword argumentsKeyword arguments

Data-driven test casesData-driven test cases

Define a keyword which will take the input data Define a keyword which will take the input data and prepare a table with test casesand prepare a table with test cases

Test case organisationTest case organisation

Simple way: Single HTML file containing all test Simple way: Single HTML file containing all test casescases

Test case taggingTest case tagging

ExecutionExecution

Gathering test cases, reading and setting Gathering test cases, reading and setting variablesvariables

Executing all actions for every test caseExecuting all actions for every test case

Providing global statisticsProviding global statistics

Writing the output in XML formatWriting the output in XML format

Generating report and log in HTML formatGenerating report and log in HTML format

Sample execution resultSample execution result

Sample Test ReportSample Test Report

in HTML format, showing all actions executed in HTML format, showing all actions executed up to the failing action, with fail messageup to the failing action, with fail message

Tested application Tested application InterfaceInterface

Command lineCommand line

OperatingSystemOperatingSystem

SSHLibrarySSHLibrary

Telnet libraryTelnet library

WebWeb

Robot SeleniumRobot Selenium

GUIGUI

Swing GUI librarySwing GUI library

AdoptionAdoption

Sense of achievementSense of achievement

IntegrityIntegrity

OpennessOpenness

Right timingRight timing

Facilitate AdoptionFacilitate Adoption

EvangeliseEvangelise

Lower the barLower the bar

Train and educateTrain and educate

Share and infectShare and infect

Coach and facilitateCoach and facilitate

Involve others by giving them rolesInvolve others by giving them roles

Organisational Organisational ChallengesChallenges

Business AnalystsBusiness Analysts

TestersTesters

DevelopersDevelopers

ReferencesReferences

Bridging the Communication Gap Bridging the Communication Gap

Gojko AdzicGojko Adzic

Practical TDD and ATDD for Java DevelopersPractical TDD and ATDD for Java Developers

Lasse KoskelaLasse Koskela

Agile TestingAgile Testing

Lisa Crispin and Janet GregoryLisa Crispin and Janet Gregory

Thank you! Thank you! 多謝！多謝！

Steven Mak Steven Mak 麥天志麥天志Email: Email: steven@odd-e.comsteven@odd-e.com

Twitter: Twitter: http://twitter.com/stevenmakhttp://twitter.com/stevenmak