exploratory testing as a quest

Exploratory TestingExploratory Testing……enjoy the Questenjoy the Quest

Exploratory Testing Class Meeting #1

DefinitionExploratory testing is simultaneous learning, test design and test execution

Unlike the scripted testing, here the tester actively controls the design of the tests as those tests are performed, and uses information gained while testing to design new and better tests.

Ad-hoc vs Exploratory

Exploratory means:

thoughtfulfocuseddisciplined accurate

“accomplishing mission”

Ad-hoc sounds like:

randomchaoticuncontrollablecareless

“wandering around”

Exploratory testing is a Quest

Think of any Quest game you ever played.You have a mission, and a set of tools to accomplish it.You have no idea on what’s going to happen next.But all your further steps will depend on that.

It is not easy to solve Quests, it requires logical thinking, observation, accuracy and lot of patience sometimes...

Before we start our class…

Is that possible to learn being a better explorer?Is that possible to learn being a better Quest-solver?

Yes.But not through THEORY. The only way is PRACTICE.

Structure

Tester Mission

Product

ET

Time

[Quest]

+ Reporting!

Test design. The ideas.

Exploratory tester designs tests ‘on the fly’. Instead of randomly clicking he first thinks of possible tests. Tests for systematical exploration of the product.

Quest player has many-many steps and combinations he may try. But experienced player knows immediately which of them have sense in the current situation.

Authorization functionality

Ability to design tests is predicting what may go wrong and selecting strong test data to check it

Test idea

How about trying to pass some invalid data, looking as valid?..

Test data

How about trying to pass some invalid data, looking as valid?..

Test case

How about trying to pass some invalid data, looking as valid?..

Success!Bug in security system!

Observation. Details matter.

Exploratory tester is always looking for symptoms. Anything unusual may mean a problem.

Quest player starts every new screen from observation. Every clickable item may help him in resolving the puzzle. Even the tiniest detail is crucial here.

For the novice player this screen doesn’t say much. The only way is go clicking here and there…

But an expert will notice important details (even the smallest once) right away!

Moreover, expert player will notice interesting connections between things...

Critical thinking.

Exploratory tester should be able to review and explain his logic, looking for errors in his own thinking.

Quest player bases his strategy on assumptions and ideas. But when he gets stuck he has to re-think his ideas to be able to move on.

Sometimes to go forward you must go back first.

I must have missed something...???

Diverse ideas.

Exploratory tester needs to produce many testing ideas, easily use diverse testing techniques.

Quest tasks are rarely obvious, so the player should be quite creative and inventive to solve them and find the right way between many available.

Every problem usually has many possible solutions

How can I get that bird?

Similarly – there are always plenty of test cases you might think of…

1 – try throwing something to hit it!

Similarly – there are always plenty of test cases you might think of…

2 – try getting the ladder …

Similarly – there are always plenty of test cases you might think of…

3 – use cable as a lasso …

Similarly – there are always plenty of test cases you might think of…

4 – get some bird food …

Similarly – there are always plenty of test cases you might think of…

5, 6, 7,… – your ideas?

Rich resources.

Excellent exploratory testers build a deep inventory of tools, information sources, test data and friends to draw upon.

Query player might consider using external resources to get his mission completed: hints, walks through, other players advises.

Your testing won’t be effective if you’re not prepared well enough...

I know nothing about botany…

Go get some spec. Talk to those who have information. Prepare some tools.

spec

tools

Finally, go GOOGLE it.

ET Basics

Test DesignObservationCritical thinkingDiverse ideasRich Resources

Focus on the Mission

Scripted vs ExploratoryWhich is better?

predictablerepeatableeasier to executeeasier to reporteasier to measure

time-taking preparation

not very flexible

flexiblerapidmay be very

effectiveresult-oriented

harder to trainharder to control

Scripted vs ExploratoryWhich is better?

Depends on the context.

…but as long as your task looks like Quest the best option is to become an explorer ;)

ReferencesWhat is Exploratory testing and how it differs from Scripted testing? (HTML)

Exploratory testing explained (PDF)

Exploratory testing dynamics (PDF)

p.s.

I used screens from the brilliant Quest game called Machinarium. Find out more here: http://machinarium.net