Test Management for Very Large ProgramsA Survival Kit

akaProgram Test Management - A Survival Kit

Graham ThomasIndependent Software Testing Consultant

STARWEST, Anaheim, October 2008

ABSTRACTIn large organizations with multiple, simultaneous, and related projects, how do you

coordinate testing efforts for better utilization and higher quality? Some organizations have opened Program Test Management offices to oversee the multiple streams of testing projects and activities, each with its own test manager.

Should the Program Test Manager be an über-manager in control of everything, or is this office more of an aggregation and reporting function? Graham Thomas examines the spectrum of possible duties and powers of this position.

He also shares the critical factors for successful program test management, including oversight of the testing products and deliverables; matrix management of test managers; stakeholder, milestone, resource and dependency management; and the softer but vital skills of influence and negotiation with very senior managers.

Relating experience gained on several large testing programs, Graham shares a practical model—covering the key test management areas of organization, people, process, tools, and metrics—that your organization can adapt for its needs.

2© Graham Thomas 2008

AGENDA Introduction Definitions Role Attributes & Skill Set Survival Kit

o Organisationo Peopleo Processo Toolso Metrics

Critical Success Factors Lessons Learnt Summary

© Graham Thomas 2008 3

INTRODUCTION Why Program Test Management?

o Large Organisationso Big Programs of worko Many inter-related projects and programs

Why Program Test Managers?o To represent testing at the program levelo To manage multiple streams of testing projects and activitieso At a level which is outside the sphere of influence for individual

projects or testing activities

4© Graham Thomas 2008

DEFINITIONTEST MANAGEMENT

• planning

• analysis

• design

• preparation

• execution

• validation

• reporting

“The direct management of testing activities”

5© Graham Thomas 2008

DEFINITIONPROGRAM MANAGEMENT

What is a Program?

“A broad effort encompassing a number of projects and/or functional activities with a common purpose.”

Association for Project Managers - Web site Definitions

What is Program Management?

“The management of multiple streams of development activities (including testing), each with their own teams carrying out detailed planning, management and execution activities.”

6© Graham Thomas 2008

DEFINITIONPROGRAM TEST MANAGEMENT

“The management of multiple streams of testing, each with their own test management in place which is responsible for detailed planning, management and control activities.”

7© Graham Thomas 2008

PgTM ROLE CONTINUUM Über Test Manager

o Micro-managemento Disempoweringo Decision making bottlenecko Stifle creativityo Perpetuates hero culture

Aggregating Viewo Detached and remoteo Spy in the campo Collation and aggregationo Viewed as overheado No ‘test’ value add

© Graham Thomas 2008 8

IDEAL PgTM ROLE RESPONSIBILITIES

PgTMs are not responsible for managing the testing activities. That is what we have Test Managers for!

So, what are PgTMs responsible for if they are not managing the testing?

Responsibilities vary from program to program: Testing oversight Analogous to Program

Management Looking for gaps

across the program

Getting the development groups to work together with the testers

Defining and managing the program testing approach

9© Graham Thomas 2008

ATTRIBUTES OF A GOOD PgTM

Honesty, Integrity, Experience, Credibility

Pro-active, (Dynamic - if you think rushing round helps

without causing panic)

Politically aware and astute

Focussed on the program and organisational goals

Independence

10© Graham Thomas 2008

ESSENTIAL SKILL SET In addition to:

o Good test managemento Good project management skills

Stakeholder managemento Influence & negotiation skills

Program Managemento Milestone managemento Resource & Dependency managemento Delivery through otherso Oversight and awareness

Motivate and enthuse

11© Graham Thomas 2008

SURVIVAL TOOLKIT 3 Good Things

Swiss Army Knife - That tool that no-one knows what it is for!

What to avoid doing, is as important to know, as what to do

12© Graham Thomas 2008

13

Video 1

THREE GOOD THINGSA few years ago I was invited to a corporate Clay

Pigeon shooting day. I had never shot before, but it sounded like fun so I thought that I would give it a go.

In the morning we had a practice at the six disciplines. I had never shot before so I was put into the absolute beginners group.

I was hopeless. I couldn’t hit anything.The coach was very good. He had a look at what I

was doing, and for each discipline, just told me 3 simple things, that I could relate to, that would help me.

In the afternoon I won the individual competition – with a high score, and two 6’s.

And that got me to thinking about 3 good things. If only we could do that for Software Testing.

14

ORGANISATION

BEST PRACTISE Define clear test organisation structure with matrixed

relationships Clear and agreed interface with stakeholders and sponsors Maintain independent reporting for testing

ANTI-PRACTISE Testing only reports into delivery Testing perceived as secondary activity

15© Graham Thomas 2008

16

Video 2

PEOPLE

BEST PRACTISE Ideally want people who want to be testers Identify your best people and give them the right

opportunities Create a collaborative environment to share test knowledge

and experience

ANTI-PRACTISE Unclear testing roles Lack of training and investment in testers

17© Graham Thomas 2008

18

Video 3

PROCESS

BEST PRACTISE Align development and testing activities Consistent processes - standards, methods and approach Fit for purpose, flexible and effective

ANTI-PRACTISE Testing in silos – UT, ST, SIT, UAT, OAT etc. Quality police

19© Graham Thomas 2008

TOOLS

BEST PRACTISE You will get as big a benefit from common process Match tools and processes e.g. Test Planning Not just automation - but Config, Release & Change Mgmt

ANTI-PRACTISE Developers get tools, testers don’t You have Quality Centre, what more can you want? No Silver bullets

20© Graham Thomas 2008

METRICS

BEST PRACTISE Align your measures and metrics to your stakeholders and

sponsors needs Keep your measures and metrics consistent Trend measures are very powerful

ANTI-PRACTISE If it moves measure it! Inappropriate use of metrics can drive blame culture

21© Graham Thomas 2008

CRITICAL SUCCESS FACTORS To ensure that you put in place a testing organisation that

meets the testing needs of the program

To ensure that you have an efficient and effective testing process that support and integrate with your development approach

To ensure that your stakeholders and sponsors clearly understand what testing is doing for them

To be able to articulate the program testing message

22© Graham Thomas 2008

SUMMARY & QUESTIONS Defined the Term Program Test Management Examined the PTM role continuum Looked at responsibilities, attributes and skills Worked through a model for Program Test Management Identified Critical Success Factors

Any Questions

23© Graham Thomas 2008

CONTACT DETAILS

24

Graham ThomasIndependent SoftwareTesting Consultant

graham@badgerscroft.com +44 7973 387 853 www.badgerscroft.com

© Graham Thomas 2008