1

Traditional Testing R.I.P.

Alternative Testing

Do We Have to Test Like We Always Have?

by Julian Harty

14 Nov 2010

2

Today's climate

We live in an era where quality has been replaced by speed - people are trained to expect little to none, and are perfectly happy getting it.

In such world, testing for quality must redefine itself and find a new purpose.

Goranka Bjedov (Google)

3

Cost of “doing the right thing”

“ the cost of "doing it right" is far higher than the cost of only responding when someone does find a problem”

John Viega: CTO McAfee: The myths of security

4

Business Reality...

Police, private security experts and internet service providers said on Wednesday they had shut down the Mariposa botnet , one of the largest networks of hacked personal computers discovered.

It had siphoned off financial and other information from inside half of the largest 1,000 US companies.

March 3rd, 2010http://www.ft.com/cms/s/0/f6960e5a-2711-11df-b84e-0 0144feabdc0.html

5

Are Testers impotent?

Why didn't any of the testers find the security flaw(s)?

What can we do to improve our testing competences?

6

Why test at all?

Testing slows projects down

It can mislead, or divert attention from other, more important work

Most testing is futile!

What's the risk of failure?

7

Some impacts of “testing”

Incorrect diagnosis

Incorrect changes

Irrelevant bugs

Still unfixed bugs

(what we know could hurt us...)

8

Testers as second-class citizens

“Anyone can test”

Therefore, one tester is as good as another, and no/little better than 'anyone'

Therefore, testers are nothing special...

9

Speed is the competitive advantage

Faster, iterative, releases

Testing for

vs Productivity Quality

10

What do you actually do?

(when you think you're testing...)

11

Checking vs. Testing

Checking Is Confirmation

Testing Is Exploration and Learning

Checks Are Machine-Decidable; Tests Require Sapience

http://www.developsense.com/blog/category/testing-v s-checking/

12

Some outputs of testing

Data

Error logs

Can we get these from other sources?

15

Use Canaries

What are they good for?

16

Canaries

What are they good for?

We expect some to die

Cheep(2010)

R.I.P.

17

Automate the Checking

When practical, create automated 'tests' that check:

Results

Data

Logs

The state of the software

18

Good uses of automated tests?

T.D.D.

Continuous Builds

Regression Tests

Security Tests

RED

GREENREFACTOR

19

Contributory Success Factors(for 'not' testing)

Quality and Speed of feedback

Perception of failure

Time to address and fix issues

Risk and cost of failure

The Art of Customer Relationships

20

What about the Testers?

“Traditional Testing” is waning

Change brings Opportunity

We can choose to apply our skills to more valuable & rewarding work

21

Now what?

What can we do with all the redundant testers?

22

Strengths of Testers

Recognizing there may be alternatives

Produces answers sooner

Reduce doubt / Increase confidence Advocate for: bugs, users, developers, software

Be Trusted Advisors

Debunk mysteries & superstitions

Test for productivity (better, faster, cheaper)

Test for quality (of the end product)

Be disruptive; yet likeable :)

23

Test to provide answers sooner

A good reason to test is to provide answers sooner

We need to decide:

What the aims are

What to measure, and how

How to run the test (environment)

Then we need to: run the tests, analyze the results, & present our assessment clearlySee: Gordon Bragg & Design of Experiments

24

Testing as a Leadership skill

Probe

Experiment

Execute

Decide

Move on...

25

What now?

Rethink what you do

Consider:

The value you offer

The value you deliver

Your competencies

How you work

Who you work for

Be willing to give up any “ kindergarten testing ”

26

Facets of Software Testing

0

2

4

6

8

10

Software design

Testing techniques

Test automation

Industry knowledge

Product knowledge

People skills

Ability to apply stuff

27

Improving in your current role

Sapient Testing:Engage your brain when testing software

http://www.satisfice.com/blog/archives/99

http://www.satisfice.com/blog/archives/358

Trinity Testing http://julianharty-softwaretesting.blogspot.com

Use automation appropriately

28

Honing our Testing Skills

Beta Testing for other software

“Be a better tester, be a Beta tester”

(Alan Richardson)

Work for a crowd-sourcing company

Mob4hire

uTest

Learn from others

Pairing

Exercises

29

Possible new roles?

Technical Due Diligence

Disruptive Agent of Change

Chief Engineer

Spy?

30

Read on... Gordon M. Bragg: Principles of Experimentation and Measurement

ISBN 0-13-701169-5

Lloyd Roden: Trusted Advisors

http://www.stickyminds.com/s.asp?F=S7849_CP_2

Alan Richardson: Be a better tester, be a Beta test er

http://www.compendiumdev.co.uk/eurostar2003/defa ult.php

Julian Harty: Trinity Testing

http://julianharty-softwaretesting.blogspot.com

Dr Stuart Reid + Julian Harty

Non Functional Testing (NFT) http://www.commercetes t.com/

P.T.M.M. http://julianharty-softwaretesting.blogsp ot.com

Tom Van Vleck: Three Questions About Each Bug You F ind

http://www.multicians.org/thvv/threeq.html

31

Q & A

Now?

Later?julianharty@gmail.com