jan jaap tmmi facts and figures v2

Experiences with TMMi: facts and figures

Jan Jaap Cannegieter

Vice President SYSQA B.V.

© S

YS

QA

Alm

ere

2

Agenda

0102030405060708090

100

What is

A maturity model

and

an assessment model

© S

YS

QA

Alm

ere

4

Test Maturity Model integration

© S

YS

QA

Alm

ere

5

TMMi levels

1. Initial

2. Managed

Test Policy and StrategyTest PlanningTest Monitoring and ControlTest Design and ExecutionTest Environment

3. Defined

Test OrganizationTest Training ProgramTest Life Cycle and IntegrationNon-Functional TestingPeer Reviews

4. Management and Measurement

Test MeasurementSoftware Quality EvaluationAdvanced Peer Reviews

5. OptimizationDefect PreventionTest Process OptimizationQuality Control

© S

YS

QA

Alm

ere

6

© S

YS

QA

Alm

ere

9

Assessment type: Quick Scan

Test manager 1

Tester 1

QA manager

Test environment administrator

Test manager 2

Tester 2

Project manager

You

8.30 Opening & TMMi

9.00 Rating PA’s level 2

12.00 Lunch

13.00 Rating PA’s

14.00 Conclusions

14.30 Determine improvement actions

16.00 Making action plan

16.30 Evaluation

(20.30 On the couch watching TV)

© S

YS

QA

Alm

ere

10

The tool

© S

YS

QA

Alm

ere

11

TimeStart

I I IQS QS QS QS QS

F

QS

Assessments and TMMi implementations

Level 2

© S

YS

QA

Alm

ere

12

Assessment types compared

0 100

Formal Informal Quick scan

Accuracy

Costs

100% Use at SYSQA

© S

YS

QA

Alm

ere

13

TMMi today

htt

p:/

/w

ww

.tm

mi.

org

/

The maturity of testing in The Netherlands

© S

YS

QA

Alm

ere

15

TMMi levels

1. Initial

2. Managed

Test Policy and StrategyTest PlanningTest Monitoring and ControlTest Design and ExecutionTest Environment

3. Defined

Test OrganizationTest Training ProgramTest Life Cycle and IntegrationNon-Functional TestingPeer Reviews

4. Management and Measurement

Test MeasurementSoftware Quality EvaluationAdvanced Peer Reviews

5. OptimizationDefect PreventionTest Process OptimizationQuality Control

© S

YS

QA

Alm

ere

16

How I got my data

• Goals

• 20 organizations• Telecommunication: 2

• Government: 7

• Financial services: 9

• Industry: 2

• Approach assessments: quick scans

• Analysis

© S

YS

QA

Alm

ere

17

Test policy and strategy - goals

• Establish a test policy

• Establish a test strategy

• Establish test performance indicators

© S

YS

QA

Alm

ere

18

Test policy and strategy: A: 30% V: 14%

0

10

20

30

40

50

60

70

80

90

100

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

Test policy and strategy

Organizations

Percen

tag

e o

f sp

ecif

ic g

oals

met

© S

YS

QA

Alm

ere

19

Test planning - goals

• Perform a product risk assessment

• Establish a test approach

• Establish test estimates

• Develop a test plan

• Obtain commitment to the test plan

© S

YS

QA

Alm

ere

20

Test planning: A: 41%, V: 13%

0

10

20

30

40

50

60

70

80

90

100

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

Test planning

Organizations

Percen

tag

e o

f sp

ecif

ic g

oals

met

© S

YS

QA

Alm

ere

21

Test monitoring and control - goals

• Monitor test progress against plan

• Monitor product quality against plan and expectations

• Manage corrective action to closure

© S

YS

QA

Alm

ere

22

Test monitoring and control: A: 47%, V: 15%

0

10

20

30

40

50

60

70

80

90

100

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

Test monitoring and control

Organizations

Percen

tag

e o

f sp

ecif

ic g

oals

met

© S

YS

QA

Alm

ere

23

Test design and execution - goals

• Perform test analysis and design using test design techniques

• Perform test implementation

• Perform test execution

• Manage test incidents to closure

© S

YS

QA

Alm

ere

24

Test design and execution: A: 60%, V: 8%

0

10

20

30

40

50

60

70

80

90

100

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

Test design and execution

Organizations

Percen

tag

e o

f sp

ecif

ic g

oals

met

© S

YS

QA

Alm

ere

25

Test environment - goals

• Develop test environment requirements

• Perform test environment implementation

• Manage and control test environments

© S

YS

QA

Alm

ere

26

Test environment: A: 59%, V: 9%

0

10

20

30

40

50

60

70

80

90

100

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19

Test environment

Organizations

Percen

tag

e o

f sp

ecif

ic g

oals

met

© S

YS

QA

Alm

ere

28

My conclusion

60%: Test design and execution

30%: Test policy and strategy

41%: Test planning

47%: Test monitoring and control

59%: Test environment

Inside

Outside

What about Romanian testers;should they care about TMMi?

© S

YS

QA

Alm

ere

31

Outsourcing and the Allen curve

‘The distance between engineers’ offices affected the frequency of their communication’ – Prof. Thomas Allen MIT

jan jaap tmmi facts and figures v2

Documents