software development and delivery metrics that matter

Software Development and Delivery

Wm. Barrett Simmsbarrett@wbsimms.com

@wbsimms

What are you interested in?

New Product Construction

Product Maintenance

Manage Cost

IT Visibility

SuccessEmployee

ManagementCustomer

Satisfaction

What’s in it for you?• Techniques• Managing technical product delivery• Managing product maintenance

• Metrics• Measure Constraints for Visibility

• Philosophy• The big picture has tons of little details

About Me

Software Developer

Agile Team Member

Team Lead

Agile Advocate

SDLC Implementer

Why do we want to measure?

Product

Development/QA

Physical Infrastructure

Project Management

COST

What are your variable costs?

Requirements Quality

Defects / Testing

Technical Skill

Infrastructure

These are under your control

The measurement process

Take ActionIdentify ProblemsMeasure

What to measure?

Development

•Code Coverage•Team Velocity•Developer Monitoring•Defect Kill Rate

QA•Defects Found•Testing Time : Manual vs. Automated•Kickback Rate•Quality Tracing

Business•Story Creation Rate•Story Unblock Rate•Release Progress

Actionable Stories

Itera

tion 1

Itera

tion 2

Itera

tion 3

Itera

tion 4

Itera

tion 5

Itera

tion 6

Itera

tion 7

Itera

tion 8

Itera

tion 90

20

40

60

80

100

120

140

160

180

Actionable

Added Unblocked

Itera

tion 1

Itera

tion 2

Itera

tion 3

Itera

tion 4

Itera

tion 5

Itera

tion 6

Itera

tion 7

Itera

tion 80

20

40

60

80

100

120

140

160

Blocked

Added Blocked

Unblock Rate

Iteration 1 Iteration 2 Iteration 3 Iteration 4 Iteration 5 Iteration 6

-1.5

-1

-0.5

0

0.5

1

1.5

Unblock Score Unblock Rate

New Story Count Story Blocked Story Unblocked10 3 710 10 010 6 410 0 1010 5 510 7 3

The Hawthorne Effect

• Knowing you are being monitored will change your behavior

Quality Tracing• Add findable defects in product

each week• Track how long it takes to find all

defects• Reveal the find/miss rate each

week

• Automated UI Testing• QA Define Requirements

1 2 3 4 5 6 7 8 9 10 11 12 13 14 150

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Trace Defects Found

Iteration Flow (QA Constraint)

1 2 3 4 5 6 70%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Defined In Progress Completed Accepted

1 2 3 4 5 6 70%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Defined In Progress Completed Accepted

Team Velocity

Developer Monitoring

1 2 3 4 5 6 7 8 9 10 11 12 13 140

200

400

600

800

1000

1200

Developer #1

Code Added Code Modified

1 2 3 4 5 6 7 8 9 10 11 12 13 140

200

400

600

800

1000

1200

Developer #2

Code Added Code Modified

Defect Kill Rate𝐴𝑣𝑒𝑟𝑎𝑔𝑒(𝐷𝑒𝑓𝑒𝑐𝑡𝐶𝑙𝑜𝑠𝑒𝑑−𝐷𝑒𝑓𝑒𝑐𝑡𝑂𝑝𝑒𝑛𝑒𝑑)

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 160

1

2

3

4

5

Difference

1 2 3 4 5 6 7 8 9 10 11 12 13 140

2

4

6

8

10

12

14

Average

Average

Implementation of Unit Testing!

SDLC

Code Coverage

Velocity versus Coverage

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 250%

10%

20%

30%

40%

50%

60%

70%

0

2

4

6

8

10

12

14

16

18

Coverage Velocity Linear (Velocity)

Putting it all together