With a hint of HP Quality Center

Agile development and functional testing: friend or foe?

Tom Vercauteren, June 26th, 2009

1. User stories

2. Sprint backlog

3. Daily stand-up

4. Sprint burn down

5. Other ideas

3

- User stories

- Sprint backlog

- Daily stand-up

- Sprint burn down

- Other ideas

User stories

We test “per User Story”, and these stories are “bundled” in Sprints:

In HP Quality Center

4

- User stories

- Sprint backlog

- Daily stand-up

- Sprint burn down

- Other ideas

User stories

Sprints contain a number of Stories:

In HP Quality Center

5

- User stories

- Sprint backlog

- Daily stand-up

- Sprint burn down

- Other ideas

User stories

When a Story is being developed, we write tests:

In HP Quality Center

6

- User stories

- Sprint backlog

- Daily stand-up

- Sprint burn down

- Other ideas

User stories

When a Story is developed, we execute tests:

In HP Quality Center

7

- User stories

- Sprint backlog

- Daily stand-up

- Sprint burn down

- Other ideas

User stories

What the customer tells you: “As a customer I want to open a document so that I can view it.”

What comes up along the Sprint:

- We only want to view PDF files.

- Large files should still be opened within 5 seconds.

- I want to open the file in my browser, or save it to my hard disk.

This information is usually “found” in the daily stand-up meeting, and told to the developer.

It is almost never formally captured.

It might be forgotten in the next sprint, or during testing.

And acceptance criteria

8

- User stories

- Sprint backlog

- Daily stand-up

- Sprint burn down

- Other ideas

User storiesAnd acceptance criteria

According to Scrum, we should add “acceptance criteria”:

After the sprint, add them to the functional documentation!

US-25504

Document viewer

2 story points

US-25504

Document viewer

2 story pointsOnly PDFOnly PDF

Open in IESave to disk

Open in IESave to disk

File of 10 MB:< 5 sec.

File of 10 MB:< 5 sec.

9

- User stories

- Sprint backlog

- Daily stand-up

- Sprint burn down

- Other ideas

User storiesAnd acceptance criteria

Who?

- Anyone who thinks of them / discovers them.

Why?

- Don’t forget them during development or testing.

- Might even end up in documentation / user manual.

What?

- Any functional or technical “need to know” item.

When?

- During daily stand-up

- Any time!

1. User stories

2. Sprint backlog

3. Daily stand-up

4. Sprint burn down

5. Other ideas

11

- User stories

- Sprint backlog

- Daily stand-up

- Sprint burn down

- Other ideas

Sprint backlog – Sprint planningAnd functional testing

US-25504

Document viewer

2 story points

US-25504

Document viewer

2 story points Sprint backlog

HP Quality Center

12

- User stories

- Sprint backlog

- Daily stand-up

- Sprint burn down

- Other ideas

Sprint backlog – Sprint planningAnd functional testing

Sprint backlog:

- Contains Spikes

- Contains Technical User Stories (e.g. “setup acceptance environment”, “write manual”, …)

- Contains Functional User Stories

HP Quality Center

- Does not contain Spikes

- Does not contain Technical User Stories

- Contains a set of tests for every Functional User Story

- Contains non-functional Acceptance Criteria (e.g. “UI should work on 7 different browsers)

13

- User stories

- Sprint backlog

- Daily stand-up

- Sprint burn down

- Other ideas

Sprint backlog – Test preparationAnd functional testing

US-25504

Document viewer

2 story points

US-25504

Document viewer

2 story points

Task 1

25504

Task 1

25504

Task …

25504

Task …

25504

14

- User stories

- Sprint backlog

- Daily stand-up

- Sprint burn down

- Other ideas

Sprint backlog – Test executionAnd functional testing

US-25504

Document viewer

2 story points

US-25504

Document viewer

2 story pointsTestable

Testable

Test this.

25504

Test this.

25504OR

Use “testable” for shared testers(or “finished” for Spikes)

Use tasks for dedicated testers

Note: test work should be subject to “planning poker”

(even if you use “testable”)

15

- User stories

- Sprint backlog

- Daily stand-up

- Sprint burn down

- Other ideas

Sprint backlog – Test executionAnd functional testing

US-25565

Importer Validation

2 story points

US-25565

Importer Validation

2 story pointsTestable

Testable

This may include regression tests.

16

- User stories

- Sprint backlog

- Daily stand-up

- Sprint burn down

- Other ideas

Sprint backlog – Defect trackingAnd functional testing

17

- User stories

- Sprint backlog

- Daily stand-up

- Sprint burn down

- Other ideas

Sprint backlog – Defect trackingAnd functional testing

US-25504

Document viewer

2 story points

US-25504

Document viewer

2 story points

2609No error msg. on import of DOC

25504

2609No error msg. on import of DOC

25504Testable

Testable

Decision make by team:

- Fix in this Sprint

- Add to Product Backlog, for later fixing

- “not a bug”

18

- User stories

- Sprint backlog

- Daily stand-up

- Sprint burn down

- Other ideas

Sprint backlog – Re-testingAnd functional testing

US-25504

Document viewer

2 story points

US-25504

Document viewer

2 story pointsTestable

Testable

Developer updates HPQC:-Fixed in what release?- What was the problem?

19

- User stories

- Sprint backlog

- Daily stand-up

- Sprint burn down

- Other ideas

Sprint backlog – When testing is “done”And functional testing

US-25504

Document viewer

2 story points

US-25504

Document viewer

2 story points

Q.C.

Passed

25504

Q.C.

Passed

25504

“Q.C. passed” means that no significant problems were found (given the limited time the tester spent on this story)

20

- User stories

- Sprint backlog

- Daily stand-up

- Sprint burn down

- Other ideas

Sprint backlog – When testing is “late”Bugs are found “during the next sprint”

Bugs previous sprintBugs previous sprint

Create a dummy User Story.

As this story will contain bugs that belong to the previous sprint, this has high business value.

2609No error msg. on import of DOC

25504

2609No error msg. on import of DOC

25504

1. User stories

2. Sprint backlog

3. Daily stand-up

4. Sprint burn down

5. Other ideas

22

- User stories

- Sprint backlog

- Daily stand-up

- Sprint burn down

- Other ideas

Daily stand-upAnd the tester

The tester should be present at the daily stand-up meetings.

If you share a tester between projects, he should still be present at least twice a week.

- Best practice: “all shared team members (tester, customer rep., project manager, …) attend our daily scrum at least on <weekday>”

- Best practice: “As a shared team member, I always inform the team of when I’ll (not) attend the daily scrum.”

1. User stories

2. Sprint backlog

3. Daily stand-up

4. Sprint burn down

5. Other ideas

24

- User stories

- Sprint backlog

- Daily stand-up

- Sprint burn down

- Other ideas

Sprint burn downAnd defects found

0

5

10

15

20

25

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

US-1 testable

Defects found

US-1 “Q.C. passed”

US-2 testable

Defects found

Add 15% story time

for bug fixes: add

“blanc” tasks

25

- User stories

- Sprint backlog

- Daily stand-up

- Sprint burn down

- Other ideas

Sprint burn downAnd defects found

The last user story can be tested on the last day of the Sprint

- Testing is probably not done at the end of the Sprint

- We end up with bugs that are not fixed in this Sprint.

Can we go LIVE with this situation?

26

- User stories

- Sprint backlog

- Daily stand-up

- Sprint burn down

- Other ideas

Sprint burn downAnd defects found

Solution: a Hardening Sprint

A short Sprint

No new user stories

Only bug fixes and re-testing

Perhaps time to finish documentation?

Sprint 4(3 User stories)

Sprint 5(2 User stories) Go LIVE

1. User stories

2. Sprint backlog

3. Daily stand-up

4. Sprint burn down

5. Other ideas

28

- User stories

- Sprint backlog

- Daily stand-up

- Sprint burn down

- Other ideas

Other ideas?Your opinion matters

US-25504

Document viewer

2 story points

US-25504

Document viewer

2 story pointsQ.C.

Passed

25504

Q.C.

Passed

25504

Demo versus Acceptance testing?

- A demo is not enough as “acceptance test”!

- The user representative should do more extensive testing (preferably the day before the Sprint Review)

- Create a “finished user story” list, and add “user acceptance testing passed” notes.

U.A.T.

Passed

25504

U.A.T.

Passed

25504

29

- User stories

- Sprint backlog

- Daily stand-up

- Sprint burn down

- Other ideas

Other ideas?Your opinion matters!

User Stories that do not deliver a UI (e.g. web services) should we always deliver a “test app” for testing purposes.

Definition of done:

- Task: tested by developer (unit & integration tests)

- User story: functionally tested, no significant defects left

- Defect: resolved by developer, re-tested and closed by tester

…

30

- User stories

- Sprint backlog

- Daily stand-up

- Sprint burn down

- Other ideas

Other ideas?Your opinion matters!

What did I forget?

What was unclear?

…

With special thanks to Syed Rayhan, who reviewed this presentation.Take a look at his presentation on ScrumAlliance.org: “A practical guide to implementing Agile QA process on Scrum Projects”