test improvement - any place, anytime, any where

Welcome to the Roadshow

Test Improvement

Any place, any time, anywhere

@RuudTeunissen

Insanity:doing the same

thing over and over again and expecting

different results.

Albert Einstein

DevOpsDevOps

PrototypingPrototyping

Context Driven Testing

Context Driven Testing

Behavior Driven Development

Behavior Driven Development

Model Based Testing

Model Based Testing

LeanLean

W-modelW-model

Rapid Software Test Management

Rapid Software Test Management Spiral model

Spiral model

(Acceptance) Test Driven Development(Acceptance) Test

Driven DevelopmentExploratoryExploratory

Agile SCRUMAgile SCRUMContinuous Integration

Continuous Integration

V-modelV-model

Rapid Software Testing

Rapid Software Testing

Continuous Delivery

Continuous Delivery

KanBanKanBan

ISTQBISTQB

WaterfallWaterfall

Rapid Application

DevelopmentRapid Application

Development

IterativeIterative

XPXP

TMap (Next)TMap (Next)

Not on this list…Not on this list…

Internet of ThingsInternet of Things

MobileMobileVirtualization

Virtualization

SOASOA

LocalizationLocalization

CloudCloud

DevicesDevices

PackagesPackages

Social mediaSocial media

Big DataBig Data

WebWeb

APIAPI LegacyLegacy

Test Improvement Intake(Objective – Scope – Approach)

AssessmentImprovement

‘Plan’

ArchitectureArchitecture

Test improvement - Any place, any time, anywhere

Test Improvement Intake

• Bound– TPI Next– TMMI– STEP– CTP – GQM– ……

What are the options?

improvement models with a pre-defined

approach for assessment and implementation

improvement models with a pre-defined

approach for assessment and implementation

TPI Next

Clu

sters

Key areas

Initial Controlled

checkpoints

“Enablers”

Improvementsuggestions

Efficient

checkpoints

“Enablers”

Improvementsuggestions

Optimizing

checkpoints

“Enablers”

Improvementsuggestions

TMMi

• Bound– TPI Next– TMMI– STEP– CTP – GQM– ……

• Tailormade– TI4Agile– TI4Automation– Belbin– ……

What are the options?

Tailormade approaches for

specific situations

Tailormade approaches for

specific situations

TI4Agile

Forming Norming Performing

Ag

ile

te

sti

ng

le

ve

ls

Set the basis and the first

steps towards working in an Agile manner

Adopt a process that facilitates the Agile view on

working

Continuously improve the

way you work by living the Agile way

Key area Forming Norming Performing

1 Stakeholder commitment 1 2 3 4 1 2 3 4 1 2 3

2 Planning & Estimation 1 2 3 4 1 2 3 4 1 2 3 4

3 People 1 2 3 4 1 2 3 4 1 2 3

4 Interaction 1 2 3 4 1 2 3 4 1 2 3 4

5 Teamwork 1 2 3 1 2 3 4 1 2 3 4

6 Test process 1 2 3 1 2 3 4 1 2 3

7 Test management 1 2 3 4 1 2 3 1 2 3

8 Test profession 1 2 3 4 1 2 3 4 1 2 3

9 Test automation 1 2 3 1 2 3 4 1 2 3 4

10 Regression & E2E testing 1 2 3 4 1 2 3 1 2 3 4

11 Defect management 1 2 3 1 2 3 4 1 2 3

12 Test environment 1 2 3 1 2 3 1 2 3 4

Belbin

What are the options?

• Unbound– Exploring– Test along– Heuristics

• Discussing• Questioning

– Implicit knowledge– Experience based– Mind mapping– Idea raising sessions– Out-of-the-Box– Good Practices– …

based on the expertise and

experience of all involved

based on the expertise and

experience of all involved

Heuristics

How do you feel?Ever felt this before?How about your work?Doing sports?How is your family?

In the morning?When was that?Need to walk?At what level?Simular complaints?

Before breakfast?…………

When effective?

Impact, Effect?

Easy or hard?

Investment?

Term Impact Investment Ease of implementation

Now High Free Just do it

Choosing The Right Approach

• Bound– TPI Next– TMMI– STEP– CTP – GQM– ……

• Tailormade– TI4Agile– TI4Automation– CTPI– SFAI-web– Belbin– ……

Choose and combine whatever fits

• Unbound– Exploring– Test along– Heuristics

• Discussing• Questioning

– Implicit knowledge– Experience based– Mind mapping– Idea raising sessions– Out-of-the-Box– Good Practices– …

Bound (Focused) UnboundAsses according to the model Assess by means of

Collect improvement suggestions

…Interviews

Idea raising sessionsHeuristics

Study…

…Walk around

Test alongAttend meetings

Feedback and react…

Improvement actionsAssessment tasks

Improvement tasks

ImplementationImplementation

ImprovementBacklog

Test improvement - Any place, any time, anywhere

We want everybody to act like adults, quit playing games,

realize that it's not just my way or the

highway.

Barack Obama

Improvement epics & stories

As senior IT-director,

I want to increase dev & test efficiency,

so that we can dev & test more with

the same effort

As scrum team member,

I want to automate the regression tests,

so that we can test more features during a sprint

As product manager,I want to increase the release frequency,

so that we will be more competitive

If I tell you,you will forget

If I show you,you’ll remember

If I involve you, you will understand

Confucius

Towards Improvement Sprint Backlog

• Refine and prioritize– In line with the objectives (Time, Cost, Q(Product), Q(Test), …)

– Respecting the scope (Area, Size, Test, Development, …)

• The team defines tasks, effort, …– Investigate open source tools for regression testing– Assess the level of unit testing– Implement a testing dashboard– Hire a test automation specialist

ImprovementBacklog

SprintBacklog

2 - 4weeks

Sprint

Improvement

increment

(Bi)Weekly

DoD

Improvement actionsAssessment tasks

Improvement tasks

Test Improvement Intake(Objective – Scope – Approach)

AssessmentImprovement

‘Plan’

Continuous Test Improvement at two levels

ArchitectureArchitecture

ImplementationImplementation

ImprovementBacklog

Test Improvement Intake(Objective – Scope – Approach)

AssessmentImprovement

‘Plan’

Improvement epics & stories

ArchitectureArchitecture

Improvement SCRUM MasterOwner Team Member

Improvement SCRUM MasterOwner Team Member

ImprovementBacklog

SprintBacklog

2 - 4weeks

Sprint

Improvementincrement

(Bi)Weekly

DoD

Synchronize improvement & Business as Usual

ImprovementImprovement

Business as UsualBusiness as Usual

ImprovementBacklog

SprintBacklog

2 - 4weeks

Sprint

Improvementincrement

(Bi)Weekly

DoD

ProductBacklog

SprintBacklog

2 - 4weeks

Sprint

Workingincrement

Daily

DoD

Conclusion

• The situation sets the improvement approach

• Synchronize improvement with Business As Usual

Continuous Improvement is

better than delayed perfection.

Mark Twain

Top 20 – Test Improvement suggestions• Focus on the right things to do

– Daily “scrums” within testing• High frequency of brief meetings

– With only those directly involved• Don't get involved in everything, don't get everyone

involved in everything– Stay focused on the job at hand

• Assign less time / resources to tasks within your project– Whatever you assign will be used

Top 20 – Test Improvement suggestions• As a Test Manager make decisions rapidly

– 7 out of 10 decisions right is almost perfect• Organize walkthroughs and workshops to discuss and finalize

business requirements and designs– Build and test the right thing

• Test Design Techniques only when required by the risks– Requirement and Risk Based Testing as it should be

• Report problems immediately, everyone!– Don't save them for the next meeting, don't think it won't be that bad

this time, don’t assume they will solve it somehow

Top 20 – Test Improvement suggestions• Advice scope reduction

– Scope of the whole project, not of testing• Create a checklist to help you focus on cost awareness

– Use this list every day!• Don't produce waste! Think before you produce something!

– Don't document because you're supposed to according to your process/manual/guideline

– That's how we do things here, run on autopilot– Is it necessary? Can we reuse it? What is the benefit?

• Assign one team member to cost reduction / awareness– (S)He’ll save more than (s)he costs

Top 20 – Test Improvement suggestions• Reduce the number of meetings and participants

– Meetings tend to be time consuming and unproductive

• Execute the plan - nothing more, nothing less– Watch for test scope creep!

• Force (external) decision making– If possible, without becoming a nuisance

• Stop nagging, start testing!– Look at the big picture, not just testing

Top 20 – Test Improvement suggestions• Be an active, omnipresent manager

– Focus on tasks, targets, getting the job done– Management by involvement instead of by spreadsheet

• 2 up to 4 metrics, no more!– Avoid metrics mania

• Enable business testers to focus on testing tasks and targets– Get them out of the business as usual!

• Use standard tools like Excel, Word, …– In stead of expensive test tools

Test Improvement

Any place, any time, anywhere

@RuudTeunissen