johan jonasson - test strategy, why should you care - eurostar 2013

Johan JonassonHouse of Test

Test Strategy

Why Should You Care?

www.eurostarconferences.com

@esconfs#esconfs

I question the usefulness of the vast majority of

test strategies and strategy documents I come

across

Nevertheless I believe (good) strategy is worth

caring about

However, strategy should never be a goal in itself,

but a tool that helps us reach our goal

“A high-level description of the test levels to be performed and

the testing within those levels for an organization or

programme (one or more projects).”

ISTQB syllabus

A document

A detailed plan

Concerned with the logistics of testing

A collection of test technique definitions

Defect management process

Test suspension/resumption criteria

Entry/exit critera

Any other boilerplate...

“A strategy is the overarching

direction or design of a campaign,

whether that’s a marketing

campaign, a football season, or a

campaign of war”.

– Fiona Charles

“The purpose of a test strategy is to

clarify the major tasks and

challenges of the test project.”

– James Bach

The Big Picture

◦ The direction we’re heading

◦ The ideas and the risks

◦ Tasks & challanges

◦ The thinking

Tools

◦ Heuristics

◦ Checklists

◦ Contingency plans for dealing with

likely problems

High LevelStrategy

Medium LevelStrategy

Low LevelStrategy

Product Model

Abstraction of strategy

Not to be confused with process, e.g. Scrum

As context-independent as possible

Heuristics based

Created by taking inventory of existing behaviors and

identifying heuristic methods and patterns in those behaviors

Inspect and adapt, improve the model

HTSM by James Bach

Define information goals

◦ What information do your stakeholders need?

◦ What do you need to know to be able to deliver that information?

◦ Based on your and their needs: What risks do you see? How can you work to

explore, mitigate or prevent the risks?

◦ What assumptions are you making, what questions do you have, what tasks

need doing, etc.

Find a way to visualize the evolving strategy

◦ Mindmaps

◦ Kanban

◦ Dashboards

Start thinking about coverage needed for your

high level strategy

◦ Requirements

◦ Structural, functional...

◦ Quality characteristics

◦ Coverage heuristics

Test ideas

Test cases

Test case:set of test case preconditions, inputs (including actions, where applicable), and expected results, developed to drive the execution of a test item to meet test objectives, including correct implementation, error identification, checking quality, and other valued information

- ISO/IEC/IEEE 29119

Might lead to over-focused testing. Use with care.

This is what you do every day

Review your test idea

Think about information goals

Think about coverage

Think about quality characteristics

Get our your heuristics

Pick your test techniques

Frame your testing

Make your own checklists

Mission

Models

Heuristics

Stopping criteria

Valuable information

Concise

◦ Fits on a page, presentation slide or a mindmap

Evolving

◦ From being assumptions, questions, uncertainty to being more

specific, intentional, practical

Justified

◦ Frames the testing. Connects information goals to tests with good

reasons

Practical

◦ Likely to succeed or be effective

Explainable

◦ If you can’t explain it, you don’t understand it; neither will ”they”

Flexible

◦ Adaptable to changing context. When the terrain change, the map

must change as well

Specific

◦ Be clear and to the point. Remove the boilerplate.

Encouraging of dialogue and communication

◦ The worst things you can do as a testers is shroud yourself in

mystery. What your stakeholders can’t understand, they will

question, and rightly so.

Strategy guides your planning

”Plans are useless, planning is indispensible.” – Eisenhower

If you don’t know your strategy, the everything that happens

in the project is according to your strategy...

A good strategy lets you capture and present your ideas early

on in the project and frame your testing later

It gives you a coverage model to help you decide what to

cover, how to cover it, and...

Johan JonassonHouse of Test

http://blog.johanjonasson.com

http://houseoftest.se

http://commonsensetesting.org

@johanjonasson

johan_jonasson

Blog:

Web:

Web:

Twitter:

Skype:

johan.jonasson@houseoftest.se