specification by example with gui tests - how …...hand coded hand coded bypass gui? domain...

Emily Bache, Bache Consulting & PageroGeoff Bache, Jeppesen

Specification by Example with GUI tests - How could that work?

@esconfs#esconfs

Specification by Example with Rich-Client application?

3

Jeppesen

Agile transition underway next-generation Crew Management System

4

5

Specification by Example

6

Specification by Example

Gojko Adzic’s book Collaborative workshops Define Example Scenarios BEFORE

build code

7

Agile Requirements

User Story Conversation+ Examples+

From “Agile Testing” by Crispin, Gregory

Requirement=

8

aka Scenarios

Scenario -> Regression test

BEFORE code is written, it’s a requirement specification

AFTER code is written it’s a regression test

Important to preserve the language & format

9

Living Documentation

GUI testing with Record - Replay

10

Recorded GUI Tests

Click around the GUI like a user would

Record a script Re-play the script to regression

test

11

Assumes the GUI is already built

Recorded Scripts

Easy to create, hard to maintain

12

Page Object Pattern

13

Test Case Scripts

System Under Test

Page Objects

Hand Coded

Hand Coded

GUI testing with Cucumber, Fitnesse etc

14

Using Cucumber

Feature

System Under Test (SUT)

Step Definitions

http://cukes.info

Using Fitnesse

16

http://fitnesse.org

“Domain Language” Tests

Domain LanguageScenario or Table

System Under Test

Test Fixture “Glue Code”“Step Definitions”

17

Hand Coded

Hand Coded

Bypass GUI?

Domain LanguageScenario

Test Fixture GUI

Bypass GUI, test core system directly

18

Under-the-GUI API

19

Demo: building a new feature

with StoryText

20

Open Source Development paid for by Jeppesen supports Java Swing, Eclipse RCP, PyGTK, Tkinter, wxPython

http://texttest.org

21

TextTest tests have two parts

Use Case GUI Log

List of user actions:Domain language

Recorded text:Contents of GUI

22

TextTest - test fixtures

Create “use case” part of text by Record session in UI Name your actions: snippets of domain language UI Map - domain language mapped to actions

23

Compose a Domain Language

24

Group several user actions and name them Match the language of the specification workshop Re-usable terms for many tests

Record the Test Fixture/Step definitions

Use Case(one per test)

“UI Map” + “shortcuts”(shared by whole suite)

System Under Test

25

Recorded

Configured

StoryText tests

Use Case GUI Log

List of user actions:Domain language

Recorded text:Contents of GUI

26

The GUI log

“Assert” part of the test Auto-generated by

StoryText ASCII art represents UI

state

27

StoryText tests

Use Case GUI Log

Plain Text: search, store, manipulate, diff, version...

28

29

Sample Use-Case : Undo Service Type changes

30

open default plan and show rosterschange service type of activity JPO 2537 for Brogan to Mundowait for completion of undoclose and discard changes

Crew Management System

31

Assert Everything by default

Log of absolutely everything that changes in the GUI Each test defines filters to focus on interesting parts Often catch errors the test wasn’t designed to find

32

Limitations of the approach

Only supports certain GUI libraries Maintainability of tests depends on good test design◦ filter GUI log carefully

33

Conclusions

Specification by Example - automating Domain Language scenarios needs fixture code

◦TextTest “records” the fixtures

GUI intensive applications - need to check carefully what’s on the screen

◦TextTest takes “screen shots” as ASCII art

34