1 productive engineering teams watts s. humphrey software engineering institute carnegie mellon...

1

Productive Engineering TeamsWatts S. Humphrey

Software Engineering InstituteCarnegie Mellon UniversityPittsburgh, PA 15213-3890

Sponsored by the U.S. Department of Defense© 2000 by Carnegie Mellon University

Carnegie Mellon University

Software Engineering Institute

© 2000 by Carnegie Mellon University Version # Productive Engineering Teams - page 2

The Improvement StrategyManagement comes first: without sound management • cost and schedule performance is unsatisfactory• quality engineering work is unlikely

With the CMM, management performance improves.• Cost and schedule commitments are met.• Processes are defined and data gathered.• Quality engineering work may or may not be done.

The PSP and TSP provide disciplined engineering.• Engineers use defined processes and data.• With sound management and disciplined practices,

quality engineering is standard.


ObjectivesEngineering organizations strive to produce• quality products• on aggressive schedules• for lowest cost

To do this, they need motivated and productive teams.

The methods for building such teams are well known but not obvious.

This talk describes how to build, manage and motivate productive engineering teams.


Characteristics of Productive TeamsProductive teams • are attacking important problems • have aggressive but realistic goals• work to defined plans and processes

The team members• are skilled and trained for the job• know their roles and responsibilities• are personally committed to the work

The Team Software Process (TSP) builds productive teams.

SM

SMTeam Software Process, and TSP are service marks of Carnegie Mellon University.


Costs and Benefits - 1 The benefits of the TSP are substantial.

Productivity has more than doubled.

Teams have been consistently on or ahead of schedule.

Testing time was cut by 10 or more times.

Delivered products have been defect free.

Engineering turnover has been zero.


Costs and Benefits - 2Results from three projects in one company show that• Test defects were reduced from an average of 20/KLOC

to 1/KLOC.• At an average cost of 12 hours per defect, the engineers

paid for the TSP investment with the first 1000 LOC written.

• By using TSP, the company estimates they have saved an estimated $5.3 Million in two years.

• Customer acceptance test time was reduced from many months to a few weeks.


Costs and Benefits - 3

Engineers like working on productive teams:• This really feels like a tight team.• This process forces you to design, to think the whole

thing out.• Design time is way up but code time decreased to

compensate.• Tracking your time is an eye opener.• Really good teamwork on this project - no duplication of

effort.• I’m more productive.• Gives you incredible insight into project performance.• Wonderful to have team members assigned specific

roles.• Team really came together to make the plan.• I feel included and empowered.


Costs and Benefits - 4The introduction costs for TSP are significant.

Seminars and courses• executives - 1 1/2 days• managers - 4 days• engineers - 125 to 150 hours• instructors/coaches - engineer training plus 2 weeks• team launches, relaunches, and coaching

Culture change• executives - leadership and participation• managers - coaching and support• engineers - commitment and ownership


The CMM and the TSPThe SEI has addressed process improvement from two perspectives.

The CMM® establishes the management framework.• policies and practices• systems, methods, and facilities

The TSP shows engineers and their managers how to • use a defined, measured, and planned process• establish the conditions for effective teamwork• manage, track, and report on the work

® Registered with the U.S. Patent and Trademark Office.


CMM, TSP & PSP Relationship

CMM - Builds organizational

capability

TSP - Builds quality products

on cost and schedule

PSP - Builds individual skill and

discipline


Requirements ManagementSoftware Project PlanningSoftware Project TrackingSoftware Quality AssuranceSoftware Configuration ManagementSoftware Subcontract Management

Defect PreventionTechnology Change ManagementProcess Change ManagementQuantitative Process ManagementSoftware Quality Management

Organization Process FocusOrganization Process DefinitionTraining ProgramIntegrated Software ManagementSoftware Product EngineeringIntergroup CoordinationPeer Reviews

The CMM KPAs and TSP/PSP

5 Optimizing

4 Managed

3 Defined

2 Repeatable

Continuous ProcessImprovement

Product and ProcessQuality

Engineering Process

Project Management

Level Focus Key Process Areas (KPA)

Indicates the CMM KPAs that are fully or partially addressed at the team level in the TSP

Requirements Management

Software Quality AssuranceSoftware Configuration Management

Intergroup Coordination

Software Project PlanningSoftware Project Tracking

Defect PreventionTechnology Change ManagementProcess Change ManagementQuantitative Process ManagementSoftware Quality ManagementOrganization Process FocusOrganization Process Definition Integrated Software ManagementSoftware Product Engineering

Peer Reviews

Indicates the CMM KPAs that are fully or partially addressed at the personal level in the PSP

Version # Course or Lecture or Module Info. - page 12

TSP Teams Need Proper Training

The PSPCourse

The TSPLaunch

The TSPProcess

TeamManagement

EngineeringSkills

TeamDisciplines

Personal measuresProcess disciplineEstimating & planningQuality management

Project goalsTeam rolesTeam processProject planBalanced plan

Risk analysisTeam communicationTeam coordinationStatus trackingProject reporting

A ProductiveEngineering Team


Introducing Engineering DisciplinesUntil they try it, most people do not believe disciplined engineering will work for them.• They won’t try it without evidence.• They can’t get evidence without trying it.

The TSP engineering course is called the PSP (Personal Software Process).• Engineers gather data while writing 10 programs.• They start with their current methods.• They advance in steps to the full PSP.

They see from their personal data that sound engineering works.

SM

SMPersonal Software Process and PSP are service marks of Carnegie Mellon University.


The Engineering Course OverviewThe course focuses on building engineering disciplines. • to consistently meet commitments• to measure and manage quality• to understand personal performance

The course instructors have all completed the engineering course themselves.

The students must be competent software engineers.


Building the Team EnvironmentEngineers take the PSP/TSP training as teams.

This starts the teambuilding process.

Shortly after completion, the trained teams launch their projects.


The Project Launch

TSP projects can starton any phase.

Each phase starts with a launch or relaunch step.

The strategy is to• overlap phases • develop in increments • balance team workload• accelerate tasks wherever

possible

Postmortem

Requirements

Launch

High-LevelDesign

Relaunch

Implementation

Relaunch

Integrationand Test

Relaunch


The TSP Launch

Every TSP project starts with a launch.

The launch takes 4 days.• It is part of the project.• It is led by a trained team coach.• It immediately follows PSP training.

In the launch, the engineers do essential project work.


The Launch Process Meetings

Day 1

1. Establish Product and

Business Goals

2. Assign Rolesand Define Team Goals

Day 2

4. Build Top-down and

Next-Phase Plans

5. Developthe Quality

Plan

6. Build Bottom-up and

ConsolidatedPlans

Day 3

7. ConductRisk

Assessment

8. PrepareManagementBriefing and

Launch Report

LaunchPostmortem

Day 4

9. HoldManagement

Review

3. Produce Development

Strategy


Launch ProductsDuring the launch, the teams produce important products.• documented team goals• defined team-member roles• an overall development strategy• a defined team process• a list of the project’s planned products• size estimates for the planned products• an overall project schedule• a quantitative quality plan• a detailed and balanced next-phase plan• a prioritized evaluation of the project’s key risks


Building Motivated TeamsWhile these products are important, the key launch objective is to build a motivated team.

Motivated teams believe their goals are challenging, important, and achievable.

To believe the goals are achievable, they must have• a plan they are committed to meeting• the skills and resources to do the job

When teams make their own plans, they are committed to meeting them.


The Team RolesThe roles distribute team management among the engineers.

These roles establish responsibility for managing the teamworking environment.

The members select their roles during the team launch.

The standard roles cover planning, process, quality, support, user interface, design, implementation, and test.


Planning a TSP Project In the TSP, planning is done at three levels.

The team first makes overall project and quality plans.

Then, the team makes a detailed plan for the next project phase.

The engineers next produce detailed personal plans for the following phase.

Finally, the engineers balance their plans to evenly distribute the work and minimize the schedule.


TSP Planning

ProduceConceptual

Design

Make Size Estimates

Define Processand Tasks

Estimate Tasks

Estimate Weekly Time

EstimateDefectsInjected

EstimateYields

Make theEngineers’

Plans

BalanceTeam

Workload

ConceptualDesign

OverallProject

Plan

QualityPlan

DetailedEngineer

Plans

BalancedTeamPlans


Tracking a TSP ProjectThe detailed team and individual plans facilitate precise project tracking.

The team members regularly reassess project risks and devise mitigation actions.

In weekly team meetings, the engineers • report on task status• review the key risks• rebalance the workload

The team produces weekly management status reports.


TSP Project Tracking

QualitySummary

ScheduleStatus

Engineer A

ProductSummary

Enter Defects by Component

and Phase

Enter Size by

Component

Enter Week TaskCompleted

Enter Time by

Task

Updated Team andEngineer Task, Schedule,

and Quality Plans

Team Task and Schedule

Summary

Task StatusEngineer A


Team Leader Support

The Team Leader periodically reports to management on project status and risks.

The team leader supports the team by• obtaining staff and arranging for training• protecting team resources• interfacing with other groups• resolving problems and issues• maintaining process discipline• overseeing process and product quality• sustaining the team’s energy and drive


Management Support The PSP/TSP will not work unless the engineers are fully supported by all management levels.

During the project launch the team reviews its plan with management.• answers questions• resolves issues• explores alternatives

To sustain the team’s motivation, management must • periodically review the project• probe the team’s data• focus on quality


Engineer and Team Performance By learning TSP skills, engineers improve their personal performance.

More important, however, the teams also build a defined process and working framework.

They now have a common language and the ability to manage their work with facts and data.

This combination produces extraordinary team performance.


TSP ExperienceMany TSP projects have been launched.

Project size has ranged from 2 up to 50 team members.

Products have been imbedded systems, real-time controllers, and commercial applications.

Program size has ranged from a few hundred LOC to nearly half a million LOC.

The following charts show some of the results to date.


PSP/TSP Change BehaviorMany engineers have now been taught the required PSP/TSP skills.

These skills both provide the foundation for effective teamwork and improve personal performance.

The following data show the results from 298 engineers in PSP courses.• In the course, the engineers write 10 programs.• For program 1, they use their current practices.• By program 10, they are practicing the PSP/TSP

disciplines.


The PSP Course for Engineers

PSP0•Current process•Basic measures

PSP1•Size estimating

•Test report

PSP2•Code reviews

•Design reviews

PSP3Cyclic development

Team Software Process

•Teambuilding •Risk management

•Project planning and tracking

PSP2.1 Design templates

PSP1.1•Task planning

• Schedule planning

PSP0.1•Coding standard

•Process improvementproposal

•Size measurement


Effort Estimation Results

111098765432100.2

0.3

0.4

0.5

0.6

0.7

Mean Time Misestimation

PSP Level Average

Effort Estimation Accuracy Trend

Program Number


Est

ima

ted

Min

ute

s –

Act

ua

l Min

ute

s/E

stim

ate

d M

inu

tes


Design Time Results

111098765432100.0

0.2

0.4

0.6

0.8

1.0

1.2

1.4

Design

Code

Compile

Test

Time Invested Per (New and Changed) Line of Code

Program Number

Mea

n M

inut

es S

pen

t P

er L

OC


Quality Results

111098765432100

10

20

30

40

50

60

70

80

90

100

110

120

Mean Compile + Test

PSP Level Mean Comp + Test

Defects Per KLOC Removed in Compile and Test

Program Number


Mea

n N

um

ber

of

Def

ect

s P

er K

LOC


Productivity Results

1110987654321020

22

24

26

28

30

Mean LOC/Hour

PSP Level Average

Lines of (New and Changed) Code

Produced Per Hour of Total Development Time

Program Number

© 2000 by Carnegie Mellon University Version # Productive Engineering Teams - page 35© 2000 by Carnegie Mellon University Version # Productive Engineering Teams - page 35

Me a

n L O

C/H

o ur


Engineering Performance in Industry From the PSP/TSP course, engineers learn how to • plan and track their personal work• measure and manage product quality

This significantly improves their performance when working alone.

It also has a positive impact on their performance when working on a team.


Predictable SchedulesSchedule Deviation Individual Value Control Chart -

Commercial Systems

-150-100

-500

50100150

200250

300350

01/8801/89

01/9001/91

01/9201/93

01/9401/95

01/9601/97

01/98

Date of Project Start

% D

evia

tio

n

Individual Data Points Mean Upper Natural Process Limit

Lower Natural Process Limit One Standard Deviation

Pre-CMM CMM PSP


With Less Test Time, Productivity Improves with PSP

0

0.5

1

1.5

2

2.5

3

May-90

Sep-91Jan-93

Jun-94Oct-95

Mar-97

Jul-98Dec-99

Days/KLOC

Linear (Days/KLOC)


Team Performance in Industry Even with the proper training, engineers do not generally practice the TSP methods.

To obtain the benefits of the TSP, organizations need to provide their engineers with• professional training • a defined teambuilding process• a supportive team environment• consistent management and coaching

Under these conditions, engineering teams can be highly productive.


TSP Benefits

# of Defect Detected

Release # 6 Release # 7 Release # 8 Release # 9

75% lower Defect

PSP/TSP trained

(Boeing Pilot #1)

2.36X moreSloc count

Software Size


TSP Benefits

System Test time

Release # 6 Release # 7 Release # 8 Release # 9

PSP/TSP trained

(Boeing Pilot #1)

2.36X moreSloc count

32 days 41 days28 days

4 days

94% less time


TSP Team Task Hours

Task hours directly produce products.Task hours initially averaged under 10 per week.Task hours were improved through quiet times, process documentation, more efficient meetings, etc.

Average Task Hours Per Week

15.1

13.312.6

9.6

0

2

4

6

8

10

12

14

16

18

04/2

0/19

98

04/2

7/19

98

05/0

4/19

98

05/1

1/19

98

05/1

8/19

98

05/2

5/19

98

06/0

1/19

98

06/0

8/19

98

06/1

5/19

98

06/2

2/19

98

06/2

9/19

98

07/0

6/19

98

07/1

3/19

98

07/2

0/19

98

07/2

7/19

98

08/0

3/19

98

08/1

0/19

98

08/1

7/19

98

08/2

4/19

98

08/3

1/19

98

09/0

7/19

98

09/1

4/19

98

09/2

1/19

98

09/2

8/19

98

10/0

5/19

98

10/1

2/19

98

10/1

9/19

98

10/2

6/19

98

11/0

2/19

98

11/0

9/19

98

11/1

6/19

98

11/2

3/19

98

11/3

0/19

98

12/0

7/19

98

12/1

4/19

98

12/2

1/19

98

12/2

8/19

98

01/0

4/19

99

01/1

1/19

99

01/1

8/19

99

01/2

5/19

99

02/0

1/19

99

02/0

8/19

99

02/1

5/19

99

02/2

2/19

99

03/0

1/19

99

03/0

8/19

99

03/1

5/19

99

03/2

2/19

99

03/2

9/19

99

04/0

5/19

99

04/1

2/19

99

04/1

9/19

99

04/2

6/19

99

05/0

3/19

99

05/1

0/19

99

05/1

7/19

99

05/2

4/19

99

05/3

1/19

99

06/0

7/19

99

06/1

4/19

99

06/2

1/19

99

06/2

8/19

99

Tas

k H

ours

Avg. Task Hours - Week

Avg. Task Hours - Phase



Unbalanced Team Workload

0 20 40 60

I

G

E

C

A

Eng

inee

r

Schedule Weeks


Balancing Team Workload

0 20 40 60

I

G

E

C

A

En

gin

eer

Schedule Weeks

UnbalancedBalanced


A TSP Government ProjectAir Force flight planning system• 6 software engineers• 5 were PSP trained

Integration and system test time was reduced from an average of 22% of schedule to 2.7%.

Productivity was 2.33 times the same team’s prior project.

Projects A B C TSPSize - LOC 67,291 7,955 86,543 25,820Test Days 63 23 92 6System test defects/KLOC 2.21 4.78 2.66 0.54Acceptance test defects/KLOC N.A. 1.89 0.07 0.15


A TSP Industrial Project - 1Communications test equipment• 12 software and 3 hardware engineers• not all software engineers PSP trained

Plan Actual

Size - KLOC 110 89.9Effort - hours 16,000 14,711Schedule - weeks 77 71Defects per KLOC Integration test 1.0 0.2 System test 0.1 0.4 Field trial 0.0 0.02


A TSP Industrial Project - 2Comparison with a prior project of 9,500 LOC.• Engineers not PSP trained.• Communications system controller

TSP non-TSP

Size - KLOC 89.9 9.5Field testing duration engineering trial 2 weeks 3 months acceptance trial 3 weeks 6 monthsTrial defects/KLOC 0.02 5+


A TSP Maintenance ProjectA company applied TSP with 2-engineer teams on 14 maintenance projects.

Average results showed.

TSP non-TSP

Time estimate errors +- 7.5% Not trackedSize estimate errors +- 10.0% +-23.4%Defects/page 0.14 0.35Review yield 57.1% Not trackedProductivity - pages/hour 2.28 2.08


Getting Started with TSPSprinkling a few PSP-trained engineers around an organization will not produce noticeable results.

Installing TSP in an organization requires• a team-based improvement focus• careful planning• senior management involvement and sponsorship• a change in the behavior of both the engineers and their

managers


The TSP Introduction StrategyThe TSP introduction strategy has these steps:• identify key areas for initial introduction• hold executive kickoff and planning seminar• identify and train affected managers • train the engineers in teams • train and authorize PSP instructors and TSP coaches• establish needed support for pilot projects• conduct PSP/TSP pilot projects • plan and initiate rollout across the organization


Introducing the TSP Installing TSP in an organization can be done in a few months. It requires• substantial training• support from all involved managers• management emphasis on completing PSP course work

before the first team launch

Training and transition support• PSP courses are available from the SEI and its transition

partners.• Some university PSP courses are now offered.• The TSP is only available from the SEI.


Preparation ActivitiesAn executive kickoff and planning seminar takes 1 1/2 days.

Management training is 4 days.

Engineer training: 125 - 150 hours of PSP training.• teaches personal project management• engineers learn to measure and manage quality• required for effective team participation

PSP/TSP coaches need additional training.


Example Introduction TimelineExample timeline• based on 9-12 month pilot projects• initial results are available in the first 6-12 months• final results available within 12-18 months• demonstrates costs and benefits with PSP/TSP• rapidly builds high-performance teams

Task Q1 Q2 Q3 Q4 Q5 Q6

Executive training/kickoff session X

Select participants, develop schedule X

Train managers, engineers, instructors X X X

Conduct TSP pilots, train coaches X X

Plan and initiate roll-out X


ConclusionThe TSP shows engineers how to • plan and direct their own work• meet aggressive schedules• produce superior products

The TSP shows management how to • build world-class teams• lead and support these teams

When properly trained and led, engineers can do extraordinary work.


For More InformationVisit the PSP or TSP web sites http://www.sei.cmu.edu/psp/ http://www.sei.cmu.edu/tsp/

Contact a PSP transition partner http://www.sei.cmu.edu/collaborating/partners/trans.part.psp.html

Contact SEI customer relations Software Engineering Institute Carnegie Mellon University Pittsburgh, PA 15213-3890 Phone, voice mail, and on-demand FAX: 412/268-5800 E-mail: [email protected]


TSP Applications - Domain

The PSP/TSP has been used in a wide range of application domains.

TSP programs have also been developed for Windows, Unix, and imbedded systems.

AutomotiveAvionicsCommunications controlDisplay systemsFinancial

ImbeddedInternetOffice systemsReal timeSpacecraft


TSP Applications - SizeThe size of TSP systems has varied widely.

The smallest new programs of several hundred to a few thousand LOC have been developed by students.

The largest programs have been for imbedded communications systems and integrated financial applications of 100,000 LOC up to about 500.000 LOC.

Both small and large modifications have been made to very large legacy systems.


TSP Applications - UsersSome of the organizations that are introducing the TSP are

AISAllied SignalBahnBoeingComnetEBSEDSEricksonHoneywellIomegaJPL

KaiserLittonSAICSDRCTeradyneTrilogyUnited DefenseUSAF: Hill AFBUSN: China LakeUSN: NAVOCEANOXerox

1 productive engineering teams watts s. humphrey software engineering institute carnegie mellon...

Documents

disciplined engineering

engineering turnover

team software process

management performance

tsp investment

design time

sound management cost

sm team software process