AGILE ACCEPTANCE TESTING WITH F#

Zach Bray2010

Outline

Test Driven Development (TDD) Implementing TDD

Behaviour Driven Development (BDD) Implementing BDD .NET BDD frameworks Case study: Extending a C++ project

Test Driven Development (TDD) cycle

Write tests

Verify tests fail

Write minimal code needed to

pass testsVerify tests pass

Clean up code

Unit Tests - Good vs. Bad

Unit Testing

“A good unit test needs both to illustrate and define the behavioral contract of the unit in question. Behavior is more than just individual methods…” – Kevlin Henney

Na’vi

Behaviour Driven Development

“Is an agile software development technique that encourages collaboration between

Developers QA Business Participants

in a software project.”

“It extends TDD by writing test cases in a natural language that non-

programmers can read.”

User Stories/Features

Small chunks of business value. Creation and priority driven by stake

holders.

Acceptance Tests/Scenarios

Write in the language of the business domain i.e., high-level of abstractionRefunded items should be returned to stock• given a customer buys a black jumper• and I have 3 black jumpers left in stock• when he returns the jumper for a refund• then I should have 4 black jumpers in stock

Prerequisites for executing scenarios

Define an interpretable language. Domain Specific Language (DSL) Ubiquitous language Natural language

Choose a BDD framework

The BDD Framework (Unit Test generation)

Takes feature specifications + language context assembly

Matches scenario lines to functions

Compiles scenarios into tests

Compiles features into test fixtures

Scenario description line matching example

Types are tagged as language contexts.

Each line in the file, e.g., “given a customer buys 2 black jumpers”

Matches a member function. Use double-ticked for great readability

and DRYness

State within scenarios

In the previous example state was changed inside the language context.

Contexts are created on a per scenario basis. State doesn’t pass through to the next

scenario

Context Layer: Benefits of F# Discriminated Unions & Active Patterns

Domain Modelling Object Expressions

Implement interfaces Terse syntax

Readability Units of measure

Correctness Double ticked method names

Clear & DRY Async Workflows & Agents

Concurrency

BDD Patterns: Templates

Keeping things DRY with table driven templates. Useful for driving many similar test cases.

Refunded items should be returned to stock

given a customer buys a <colour> jumperand I have <x> <colour> jumpers left in stockwhen he returns the jumper for a refundthen I should have <y> <colour> jumpers in stockexamples:

| x | y | colour || 1 | 2 | green || 3 | 4 | black || 5 | 6 | red |

BDD Patterns: Tables

Using tables makes data input much more readable.

Tables are passed into functions.given the following bids exist in the Cape/Panamax market| qty@price | credit || 1000@ 12.9 | good || 1000@ 12.7 | bad || 800 @ 12.5 | good |and the following offers exist in the Panamax market| qty@price | credit || 1000@ 12.9 | good |when I observe implied orders for the Cape marketthen I should see the following bids| qty@price | credit || 1000@ 0.0 | good |

BDD Patterns: Templated Tables

given the following bids exist in the Cape/Panamax market| qty@price | credit || 1000@ <price1> | <cred1> || 1000@ 12.7 | bad || 800 @ 12.5 | good |and the following offers exist in the Panamax market| qty@price | credit || 1000@ <price2> | <cred2> |when I observe implied orders for the Cape marketthen I should see the following bids| qty@price | credit || 1000@ <calPrice> | <calCred> |examples:| price1 | price2 | calPrice | cred1 | cred2 | calCred

|| 1.0 | 1.0 | 0.0 | good | good | good || 1.0 | 0.0 | -1.0 | good | good | good || 2.50 | 2.50 | 0.0 | bad | good | bad |

Representation of complex scenarios

Plain text is dull. Rich text is cool.

It makes the point of the scenario pop out. Tables don’t look as ugly!

Confluence BDD tool It downloads features from a Confluence

Wiki. Then transforms them into executable

features.

Confluence BDD

BDD Patterns: More

Lots of examples online. Search for gherkin. Look at http://cukes.info/

Benefits of using BDD

Shared understanding of goals and progress.

It is independent of the implementation. Use the same tests on many implementations.

Regression suite. Nightly regression tests.

Reduces QA lag at the start of the project. QA can start specifying behaviour straight

away.

Costs of BDD

Requires more interaction with stake holders.

Requires more development effort than not testing at all.

Costs of not using BDD

Cost of wrong requirements. Cost of poor design. Cost of changing implementation

and having to refactor all of the tests.

The role of Unit Tests

Primary coverage should come from behaviour specifications.

Unit Tests should be used pragmatically. Documentation Edge cases

BDD Frameworks

GENERAL

RSpec Cucumber JBehave JSpec

etc.

.NET SPECIFIC

NBehave Nspec Cuke4Nuke SpecFlow StorEvil

SpecFlow: Integrated environment

Case Study: Indirect Aggression Project

Support indirect aggression of implied prices.

Requires extension to C++ Library. Verification is biggest problem. Not suited to Unit Tests.

Behaviour can be tested at API. F# made it easy to model the API in few

lines. Language interpretation was reused.

Starting to build up a regression suite.

Example scenario

Example matched function

Further Reading