what works best with tspi for small team productivity and quality · 2014-07-18 · tspi for small...
TRANSCRIPT
1W. L. Honig, TSPi Symposium 2006 San Diego, CA
What Works Best withTSPi for Small Team
Productivity and Quality
William L. Honig, Ph.D.Associate Professor, Department of Computer Science
2W. L. Honig, TSPi Symposium 2006 San Diego, CA
TSPi Effectiveness withSmall Teams
• TSPi impact on software teams– 23 teams of 7 to 12 graduate students on real world
developments– Software process awareness and impact
• Productivity coupled with quality• Result of planning and analysis• Extensive data collection
• Bringing real world software experience to theclassroom– R&D leadership in communications companies– Land line, wireless, satellite, private and public
networks• Voice, data, land line, mobile, satellite, network
management
3W. L. Honig, TSPi Symposium 2006 San Diego, CA
What Results?• Data Summary – Productivity
– Source Lines of Code (LOC) per Person Hour• High 47.4• Average 13.5• Low 1.8(complete Cycle 2 development, including reuse – all phases)
• Data Summary – Quality– Defects Injected per Total KLOC
• Low 2.8• Average 24.1• High 86.3
4W. L. Honig, TSPi Symposium 2006 San Diego, CA
How do the teams work?
• Team composition• Students assigned to Team
» Based on From INFO
• Roles matched to background• Demographic mixture• Well trained individual programmers
• Learning environment• 14 to 17 weeks of class• Strict enforcement of team discipline• Face to face team meetings required• Students ? Employees, but can be “fired”
Team Phoenix
Fall 2001
Dr. William L. Honig2005
COMP 474 Software Engineering
Team Roles at a Glance(Five Specialized Roles)
• Support Manager
• Quality/ProcessManager
• Planning Manager
• DevelopmentManager
• Team Leader
6W. L. Honig, TSPi Symposium 2006 San Diego, CA
Team Productivity – Cycle 2Source Lines Of Code (LOC) per Hour
0.000
5.000
10.000
15.000
20.000
25.000
30.000
35.000
40.000
45.000
50.000
To
tal L
OC
/ T
ota
l Tim
e in
Ph
ase
Ram
bler
Ice
Coo
l
Luci
d
Pho
enix
Blu
e B
ee
Dim
Sum
Doc
Max
Soc
rote
s
Kite
s
Tita
ns
G10
eUph
oria
Bet
a
Sea
ls
Bee
s
Sha
rp
Sili
con
Rai
ders
Evo
lutio
n
Ava
lanc
he T3
Cod
e W
arrio
rs
Pho
enix
II
Vol
ki
Teams
Productivity
Productivity of Each Team
Good
7W. L. Honig, TSPi Symposium 2006 San Diego, CA
How is TSPi used in theclassroom?
• Student teams complete two cycles ofof development
• Same team assignment for both cycles• Some switch roles for cycle two
• “Customer” provides starting point• Product Needs Statement (not full requirements)• 2 to 4 meetings with customer to clarify needs and
review requirements and plans
• Teams present key milestones and demonstrateproduct to faculty, research assistants, customer
Dr. William L. Honig2005
COMP 474 Software Engineering
Strategy
The Process at a Glance(TSPi)Plan
Requirements
Design
Implementation
TestPostmortem
Repeat
A controlled, data driven,step-by-step process for
software life cycle
Launch
9W. L. Honig, TSPi Symposium 2006 San Diego, CA
How do students learnPSP first?
• Personal Software Process (PSP)– Required for individuals– Prerequisite for TSPi
• PSP trial introduction– Undergraduate programming course– Plan (estimate time), track defects, record time spend
• Only some TSPi student teams have thisexperience before TSPi begin– Quick two day introduction– One programming project
10W. L. Honig, TSPi Symposium 2006 San Diego, CA
Development Projects
• “Real World” Development– University staff groups as customer
• working system or, • prototype or,• requirements clarification,…
• Wide range of applications– Prospect tracking for Graduate School– Summer visit registration for College of Arts and Sciences– Student Portal for Information Technology– Grant Approval and Tracking for VP Research
• Many technologies– C++, Java, XML, ColdFusion, …
Titans
Fall 2002
11W. L. Honig, TSPi Symposium 2006 San Diego, CA
How are data collected?
• Textbook: Watts S. Humphrey, Introduction tothe Team Software Processsm
• Key data entered weekly into 21 forms:– Product Summary (SUMP)– Quality Summary (SUMQ)– Work Tasks/Effort (TASK)– Schedule and Earned Value (SCHEDULE)– Defect Identification and Correction (LOGD)– Inspection Reports (INS)– Time Recording Log (LOGT) Phoenix
Fall 2001
Dr. William L. Honig2002
Initial Findings, FEB 2002
TSPi Plan Summary: Form SUM P
Name Date Team Instructor Part/Level Cycle
Product Size Plan Actual Requirements pages (SRS) Other text pages High-level design pages (SDS) Detailed design lines Base LOC (B) (measured) Deleted LOC (D)
(Estimated) (Counted)
Modified LOC (M) (Estimated) (Counted)
Added LOC (A) (N-M) (T-B+D -R)
Reused LOC (R) (Estimated) (Counted)
Total New and Changed LOC (N) (Estimated) (A+M)
Total LOC (T) (N+B-M -D +R) (Measured)
Total New Reuse LOC Estimated Object LOC (E) Upper Prediction Interval (70%) Lower Prediction Interval (70%)
Time in Phase (hours) Plan Actual Actual % Management and miscellaneous Launch Strategy and planning Requirements System test plan Requirements inspection High-level design Integration test plan High-level design inspection Implementation planning Detailed design Detailed design review Test development Detailed design inspection
If it’s notdocumented, it’s notthere…
If you can’t measureit, it’s not there…
13W. L. Honig, TSPi Symposium 2006 San Diego, CA
What Results?Defects Injected per LOC
0
0.01
0.02
0.03
0.04
0.05
0.06
0.07
0.08
0.09
Tota
l Def
ects
Inje
cted
/ To
tal L
OC
Ram
bler
Ice
Coo
l
Luci
d
Pho
enix
Blu
e B
ee
Dim
Sum
Doc
Max
Soc
rote
s
Kite
s
Tita
ns
G10
eUph
oria
Bet
a
Sea
ls
Bee
s
Sha
rp
Sili
con
Rai
ders
Evo
lutio
n
Ava
lanc
he T3
Cod
e W
arrio
rs
Pho
enix
II
Vol
ki
Teams
Quality--a
Quality(a) of EachTeam
Good
14W. L. Honig, TSPi Symposium 2006 San Diego, CA
Quality Results fromCycle Testing ONLY
• In Cycle Testing determines the qualitynumbers– No “production” use recorded
• “Testing can only show the presence ofbugs, not their absence”– Fault Seeding– Bug Density / Arrival Rate Analysis
15W. L. Honig, TSPi Symposium 2006 San Diego, CA
Where are the Hours Used?Total Time by Phase
Total Cycle 2 Hours by Phase
20%
3%
9%
8%
16%
24%
15%
5% Mgmt&Misc
Launch
Strat&Plan
Requirements
Design
Implementation
Test
PostMortem6396 Total Hours to Date
16W. L. Honig, TSPi Symposium 2006 San Diego, CA
Student Outcomes• Student Perceptions – Popular Course
• Team work experiences very positive learning• Understand process – appreciation varies• Data collection a struggle
– Volume of data needed– Needed for timely team cooperation
• My Viewpoint• Students well equipped to join industrial teams;
larger team sizes work well• TSPi textbook is great on metrics and quality, limited
on coverage of design, testing,…• Volume of “paper work” can lead to cybercrud
Volki
Spring 2005
Pot Luck
Dr. William L. Honig2002
Initial Findings, FEB 2002
Students “Value” Forms
StudentSurvey:Choosetheformsusefulto yourteam.
Question 15
0%
25%
50%
75%
100%
CCR INS
TASKLO
GDSCHEDULE
SUMS
LOGT
SUMP
SUMPQCSR IT
LLO
GTESTSTR
ATW
EEKPEER
INFO
Greatest perceived value in forms that manage change anddefects (red) and project plan creation and tracking (blue)
18W. L. Honig, TSPi Symposium 2006 San Diego, CA
How do these findingsapply to industry?
• Student teams approximate small industry task teams /development groups– Importance of (self) policing team behavior– Specialized roles help (in addition to developer role)
• Training / Coach / Observer role is critical to rapidintroduction of process such as TSPi– Get through one cycle quickly to speed learning– Need Process Coach / Facilitator
• Face to face regular meetings– Weekly cycle of data, analysis, action– Emphasis on analysis and quality is key
• Lead teams to analysis (not just data generation)
• Historical data a real help for getting started– If none, BEGIN NOW!
19W. L. Honig, TSPi Symposium 2006 San Diego, CA
What about TSPi andSmall Teams?
• Team data for 23 student teams show industrylevel productivity early in learning TSPi- Quality *always* needs focus
• TSPi can be learned efficiently andapplied rapidly– Team composition and coaching
• The “academic” learning approach likelyapplicable to other types of organizations– Value of discipline, data collection, metrics
G10
Fall 2002
20W. L. Honig, TSPi Symposium 2006 San Diego, CA
LOC Vary GreatlyTotal LOC and Its Max. Min. and Avg.
0
2000
4000
6000
8000
10000
12000
14000
Ramble
rIce
Coo
lLu
cidPho
enix
Blue B
eeDim
Sum
Doc M
axSoc
rotes
Kites
Titans
G10eU
phor
iaBet
aSea
lsBee
sSha
rp
Silicon
Raid
ers
Evolut
ionAva
lanch
e T3
Code W
arrio
rsPho
enix
IIVolk
i
Teams
To
tal L
OC Total LOC of Each Team
MaxMinAverage
Dr. William L. Honig2002
Initial Findings, FEB 2002
Ramblers Team Metric Chart
Planned Value vs. Earned Value
11%
32%
93%100%
0%
20%
40%
60%
80%
100%
120%
Week1 Week2 Week3 Week4
Time
Val
ue PV
EV
Week1 Week2Week3
Week4
PHAH
90 85
141
82
89 8989 89
-
50
100
150
Val
ue
Time
Planned Hours vs. Actual Hours
PH
AH
Defects Injected vs. Removed
5
5
4
5
5
4
Week1
Week2
Week3
Week4
Removed
Injected
CCR Tracking Chart
-7
16
--7
16
-- - - --
5
10
15
20
Week1 Week2 Week3 Week4
Time
No.
Submitted Approved Rejected
No. INS Finished
5
4
6
0 2 4 6 8
Week1
Week2
Week3
Week4
No.
Time
No. INS
Ramblers Team Metric Chart
5-UpChart
Pert Chart
Lau Strategy Plan Request Design Implement Test Postmortem
Size and time Estimation
(All members)15 hrs
Set goals (All members)
5 hrs
Updaterisk & issues (Support M.)
5 hrs
Updateconfiguration management
procedure(Support M.)
5 hrs
UpdateProduct listand size
estimation(Plan M.)14 hrs
Allocate tasks among members
(Plan M.)3 hrs
Estimating the defects
(Quality M.)10 hrs
Produce SRS(Develop M.)
11 hrs
Produce STP(Develop M.)
11 hrs
Inspect SRS(Quality M.)
6 hrs
Produce SDS(Develop M.)
9 hrsInspect SDS(Quality M.)
8 hrs
Detaileddesign
(All members)14.5 hrs
Test plan and development (Develop M.)
8 hrs
Build & Integration
(Support M.)9 hrs
Documentation (Support M.)
11.5 hrs
System test(Develop M.)
9.5 hrs
Week 1 Week 2 Week 4Week 3
Proj
ect
Task
s
Task
Dependency
Time
Assign roles(All members)
3 hrs
ITL
SUMSTASK
SCHEDULESUMPSUMQ
Pert Chart
30 hrs
Inspect STP(Quality M.)
6 hrs
SRSSTP
SRS-INSSTP-INS
Produce ITP(Develop M.)
7 hrsInspect ITP(Quality M.)
5 hrs
SDS-INSITP-INS
SDS-INSITP-INS
Implementationplanning
(Develop M.)11.5 hrs
Unit Test Plan(All members)
9.5 hrs
Detailed Design inspection
(Quality M.)11 hrs
Code(All members)
22 hrs
Code inspection
(Quality M.)9 hrs
Quality review(All members)
7.5 hrs
Code, CCRSUMS, SUMPSUMQ, LOGTLOGD, INS
LOGDLOGTEST
SUMPSUMQ
Finish documentations (Support M.)
24 hrs
Update douments(All memebers)
22 hrs 17.5 hrs 58 hrs
PIIC
CYCLE 2
Dr. William L. Honig2002
Initial Findings, FEB 2002
Larger Team Size Works• Flexibility in Roles:
– Some ability to switch roles– Easier to recover from “drop outs”
• Student Feedback:– Students identified the problems their team encountered– 20% felt a smaller team size of 5 would lessen the problems
Dr. William L. Honig2002
Initial Findings, FEB 2002
What are some next steps?
Expand Focus on Analysis Metrics for In cycle QualityImprovement
Ease Data Gathering Travail Mobile Tool
Incorporate Teaching Materials on Technique Best Practices
Effectiveness of TSPi to Accelerate Transition to CMMI
Questions, follow-ups, ideas…. contact
William L. [email protected], 1-312-915-7988