cmmi current state and future plans · cmmi current state and future plans ... cmmi adoption •...
TRANSCRIPT
Carnegie Mellon UniversitySoftware Engineering Institute
November 2004 Page 1© 2004 by Carnegie Mellon University
®
CMMICurrent State
and Future PlansBob Rassa
Industry CMMI Chair,Raytheon
Clyde ChittisterChief Operating Officer,
Software Engineering Institute
November 16, 2004
Carnegie Mellon UniversitySoftware Engineering Institute
November 2004 Page 2© 2004 by Carnegie Mellon University
®
IntroductionCMMI Adoption
• CMMI Appraisal Results• CMMI Adoption Trends
CMMI Current Activities
• CMMI-AM (CMMI Acquisition Module)• CMMI Product Suite v1.2 Update
Carnegie Mellon UniversitySoftware Engineering Institute
November 2004 Page 3© 2004 by Carnegie Mellon University
®
Current Appraisal SynopsisBased on SCAMPI v1.1 Class A appraisalsconducted since the April 2002 release throughAugust 2004
• 424 appraisals• 385 organizations• 206 participating companies• 33 reappraised organizations• 1,704 projects• 54.8% Non-USA organizations
For more information go to: http://www.sei.cmu.edu/sema/profile_cmmi.html
Carnegie Mellon UniversitySoftware Engineering Institute
November 2004 Page 4© 2004 by Carnegie Mellon University
®
Reporting Organizational Types
Commercial/In-house
Contractor forMilitary/Government
59.0%
47.3%
34.5%
45.1%
6.5%
7.7%
Military/GovernmentAgency
Blue 385 organizations 2004Red 91 organizations 2003
0%
% of Organizations
10% 20% 30% 40% 50% 60% 70%
Carnegie Mellon UniversitySoftware Engineering Institute
November 2004 Page 5© 2004 by Carnegie Mellon University
®
Based on 178 organizations reporting SIC code. For more information visit: http://www.osha.gov/oshstats/sicser.html
Business Services36.5%
Engineering & ManagementServices15.7%
Retail Trade0.6%
Transportation,Communication, Electric, Gas
and Sanitary Services1.7%
Finance, Insurance and RealEstate8.4%
Wholesale Trade1.1%
Public Administration(Including Defense)
13.5%
Fabricated Metal Products0.6%
Primary Metal Industries0.6%
Industrial Machinery AndEquipment
1.1%
Transportation Equipment5.1%
Instruments And RelatedProducts
6.7%
Electronic & Other ElectricEquipment
5.6%
Health Services1.1%
Services, Nec1.7%
Organizations TypeBased on Primary Standard Industrial Classification (SIC) Code
Manufacturing19.7%
Services55.1%
Carnegie Mellon UniversitySoftware Engineering Institute
November 2004 Page 6© 2004 by Carnegie Mellon University
®
Based on 376 organizations reporting size data
25 or fewer10.4%
101 to 20015.4%
201 to 30011.2%
76 to 1008.2%
51 to 759.8%
26 to 5011.4%
301 to 50011.2%
501 to 10009.0%
1001 to 20009.6%
2000+3.7%
Organizational SizeBased on the total number of employees within the area of the organization that was appraised
1 to 10039.9%201 to 2000+
44.7%
Carnegie Mellon UniversitySoftware Engineering Institute
November 2004 Page 7© 2004 by Carnegie Mellon University
®
Disciplines Selected for Appraisals
0
50
100
150
200
250
300
SE/SW SW SE/SW/IPPD/SS SE/SW/SS SE SE/SW/IPPD SW/IPPD SW/SS SE/SS SE/IPPD/SS
424
46.2%
Based on 424 appraisals reporting coverage
SE = System EngineeringSW = Software EngineeringIPPD= Integrated Product and
Process DevelopmentSS = Supplier Sourcing
Num
ber o
f App
rais
als
34.2%
4.5%2.1% 0.9% 0.9% 0.5%
5.2%
0.2%
5.2%
Carnegie Mellon UniversitySoftware Engineering Institute
November 2004 Page 8© 2004 by Carnegie Mellon University
®
Based on most recent appraisal of 385 organizations
Maturity Profileby All Reporting Organizations
Num
ber O
f App
rais
als
0
50
100
150
200
Not Given Initial Managed Defined QuantitativelyManaged
Optimizing
11.7%
4.7%
31.7%
25.2%
4.7%
22.1%
Carnegie Mellon UniversitySoftware Engineering Institute
November 2004 Page 9© 2004 by Carnegie Mellon University
®
Countries where Appraisals have beenPerformed and Reported to the SEI
Argentina Australia Belarus Brazil Canada Chile ChinaColombia Denmark Finland France Germany Hong Kong IndiaIsrael Japan Korea, Republic of Malaysia New Zealand Philippines RussiaSingapore South Africa Spain Sweden Switzerland Taiwan ThailandUnited Kingdom United States
Red country name: New additions with this reporting since Nov. 2003
Carnegie Mellon UniversitySoftware Engineering Institute
November 2004 Page 10© 2004 by Carnegie Mellon University
®
16.7%
4.0%
32.2%28.7%
1.1%
17.2%
7.6%5.2%
31.3%
22.3%
7.6%
26.1%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Not Given Initial Managed Defined QuantitativelyManaged
Optimizing
USA Non-USA
Based on 174 U.S. organizations and 211 Non-USA organizations
Maturity Profiles by All ReportingUSA and Non-USA Organizations
%O
f Org
aniz
atio
ns
Carnegie Mellon UniversitySoftware Engineering Institute
November 2004 Page 11© 2004 by Carnegie Mellon University
®
0
100
200
300
400
500
600
700
800
1987 1988 1989 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004
SPA SEI CBAIPI SCAMPI vXClass A
Number of Appraisals Conducted by YearReported as of 1 November 2004
Carnegie Mellon UniversitySoftware Engineering Institute
November 2004 Page 12© 2004 by Carnegie Mellon University
®
0
20
40
60
80
100
120
140
160
180
200
1999 2000 2001 2002 2003 YTD 2004
StagedContinuous
Number of SCAMPI v1.x Appraisals Conductedby Year by Model Representation*
Reported as of 1 November 2004*Where Representation is reported
Carnegie Mellon UniversitySoftware Engineering Institute
November 2004 Page 13© 2004 by Carnegie Mellon University
®
0
5
10
15
20
25
30
35
40
45
50
Jun-
02
Jul-0
2
Aug-
02
Sep-
02
Oct
-02
Nov
-02
Dec
-02
Jan-
03
Feb-
03
Mar
-03
Apr-
03
May
-03
Jun-
03
Jul-0
3
Aug-
03
Sep-
03
Oct
-03
Nov
-03
Dec
-03
Jan-
04
Feb-
04
Mar
-04
Apr-
04
May
-04
Jun-
04
Jul-0
4
Aug-
04
Sep-
04
Number of SCAMPI v1.1 AppraisalsConducted by MonthReported as of 1 November 2004
Carnegie Mellon UniversitySoftware Engineering Institute
November 2004 Page 14© 2004 by Carnegie Mellon University
®
Appraisal Results Summary424 appraisals have been reported to the SEI in 27monthsCommercial organization percent is increasingcompared to the government contractor community• higher percentage of the commercial organizations
are outside of the U.S.A.• higher percentage of the government contractors
are located in the U.S.A.Of U.S. organizations, the services and manufacturingindustries reported most appraisalsCompared to the early reports of the SW-CMMmaturity profile, the early data reflects a relativelymore mature CMMI® profile
Carnegie Mellon UniversitySoftware Engineering Institute
November 2004 Page 15© 2004 by Carnegie Mellon University
®
CMMI Transition Status1Reported as of 30 September 2004
TrainingIntroduction to CMMI – 24,091 trainedIntermediate CMMI – 1,166 trainedIntroduction to CMMI Instructors – 349 trainedSCAMPI Lead Appraisers - 470 trained
AuthorizedIntroduction to CMMI V1.1 Instructors – 219SCAMPI V1.1 Lead Appraisers – 344
Carnegie Mellon UniversitySoftware Engineering Institute
November 2004 Page 16© 2004 by Carnegie Mellon University
®
SEI (Transition) Partners123 Introduction to CMMI Partners
105 commercial offerors15 internal-use only3 government-use only
172 SCAMPI Appraiser Services Partners158 commercial offerors10 internal-use only
4 government-use only
For more info go to: http://www.sei.cmu.edu/partners
CMMI Transition Status2
Reported as of 30 September 2004
Carnegie Mellon UniversitySoftware Engineering Institute
November 2004 Page 17© 2004 by Carnegie Mellon University
®
Number of Lead Appraisers Authorized(Cumulative) Reported as of 31 October 2004
0
50
100
150
200
250
300
350
400
450
500
1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 YTD2004
Lead Evaluators(SCE)Lead Assessors(CBA IPI)Lead Appraisers(SCAMPI)
10-31-04
Carnegie Mellon UniversitySoftware Engineering Institute
November 2004 Page 18© 2004 by Carnegie Mellon University
®
0
5000
10000
15000
20000
25000
1995 1996 1997 1998 1999 2000 2001 2002 2003 YTD2004
CMM Intro
CMMI Intro
CMMI Intermediate
9-30-04
Intro to the CMM and CMMI Attendees(Cumulative) Reported as of 30 September 2004
Carnegie Mellon UniversitySoftware Engineering Institute
November 2004 Page 19© 2004 by Carnegie Mellon University
®
CMMI Adoption Trends: Website Visits
There were 392 organizations that visited the CMMIwebsite more than 200 times during September2004
• 56 defense contractors
• 9 DoD activities
• 28 government agencies
• 253 commercial companies
• 46 universities
Carnegie Mellon UniversitySoftware Engineering Institute
November 2004 Page 20© 2004 by Carnegie Mellon University
®
CMMI Adoption Trends: Website Visits
Carnegie Mellon UniversitySoftware Engineering Institute
November 2004 Page 21© 2004 by Carnegie Mellon University
®
Performance Results SummaryReported as of 12 November 2004
23 organizations reported credible quantitative evidence inconference presentations and via direct communicationwith the SEI• initial CMMI benefits and ROI report, October 2003• CMMI Technology Conference, November 2003• SEPG and European SEPG, March and June 2004• confidential communication with SEI
14 of these organizations reported results from which wecan show percent change over time
Future results will come from• externally conducted case studies• collaborative case studies• community benchmarking
Carnegie Mellon UniversitySoftware Engineering Institute
November 2004 Page 22© 2004 by Carnegie Mellon University
®
Performance Results Summary2Reported as of 12 November 2004
43.8 : 12 : 113 : 1Return on Investment
333%10%55%Customer Satisfaction
647%33%72%Quality
411%75%Productivity
1055%15%90%Schedule
826%5%83%Cost
# ofdata
points
MedianLowHighImprovements
28%
Carnegie Mellon UniversitySoftware Engineering Institute
November 2004 Page 23© 2004 by Carnegie Mellon University
®
CMMI Acquisition Module (CMMI-AM)Draft released February 2004Focuses on effective acquisition activities andpractices that are implemented by first-levelacquisition projects (e.g., System ProjectOffice/Program Manager)Acquisition practices drawn and summarized fromexisting sources of best practices• Software Acquisition Capability Maturity Model
(SA-CMM)• Capability Maturity Model Integration (CMMI)• FAA Integrated Capability Maturity Model (iCMM)• Section 804
Intended to be used in conjunction with the CMMIas an acquisition “lens” for interpreting the CMMI inacquisition environments
Carnegie Mellon UniversitySoftware Engineering Institute
November 2004 Page 24© 2004 by Carnegie Mellon University
®
CMMI Acquisition Module
CMMI Acquisition Module
Engineering SupportProjectManagement
• Project Planning• Project Monitor and
Control• Integrated Project
Management• Risk Management• Integrated Teaming• Solicitation and
Contract Monitoring
• RequirementsManagement
• RequirementsDevelopment
• Verification• Validation
• ConfigurationManagement
• Process and ProductQuality Assurance
• Measurement andAnalysis
• Decision Analysis andResolution
• Transition toOperations andSupport
• OrganizationalEnvironment forIntegration
Carnegie Mellon UniversitySoftware Engineering Institute
November 2004 Page 25© 2004 by Carnegie Mellon University
®
CMMI-AM Phase III Progress• pilot program assessments complete.
– 8 pilots conducted by OSD
• draft report prepared by OUSD; reviewed byassessment team members who hadparticipated on more than one assessment (10people)
• report, including recommendations, deliveredto OUSD management and CMMI–AM authorson October 8th
• report, including recommendations for CMMI-AM Update, to be presented to Steering Groupand Sponsor at November meeting
Carnegie Mellon UniversitySoftware Engineering Institute
November 2004 Page 26© 2004 by Carnegie Mellon University
®
CMMI-AM Pilot Results: Positive• no show stoppers to prevent immediate use
when accompanied with training
• applicable to all life cycle phases,applications, and program size within pilotstudy with appropriate interpretation
• more robust process areas: Project Planning,Project Monitoring and Control, RiskManagement, Requirements Management, andRequirements Development
Carnegie Mellon UniversitySoftware Engineering Institute
November 2004 Page 27© 2004 by Carnegie Mellon University
®
CMMI Version 1.2 PlanPrime Rule: Maintain CMMI Stability• No Major Changes in v1.2 that will affect
previous appraisals
New Framework Architecture
New Content Opportunities
Single Course Training
Model Simplifications
Carnegie Mellon UniversitySoftware Engineering Institute
November 2004 Page 28© 2004 by Carnegie Mellon University
®
CMMI v1.2 Architecture• Revised (internal) Architecture will allow better
accommodation of content- In CMMI v1.1, “IPPD” and “SS” are not
“disciplines” in the same sense as SE andSW
– Defined as CMMI-SE/SW/IPPD/SS becauseof architecture constraints
- V1.2 Architecture will allow proper treatmentof focus elements such as IPPD and SS, andthe easy addition of other elements such assafety or security, for example
– Existing PAs can now be changed whenadding such additional focus elements
- New designation: CMMI-SE/SW+IPPD+SS
Carnegie Mellon UniversitySoftware Engineering Institute
November 2004 Page 29© 2004 by Carnegie Mellon University
®
Additional Content OpportunitiesNew v1.2 architecture allows additional constructs• new framework will support Services and Operations
content, such as maintenance/support, help desk, etcvia new “services constellation” of CMMI- proposal for “Services” Constellation already received by
CMMI Steering Group
• allows present “development constellation” to bebetter structured- i.e. CMMI-SE/SW+IPPD+SS- allows for additional focus elements such as safety or
security– NOTE these are NOT disciplines in the sense of SE
and SW
• additional content will follow release of v1.2
Carnegie Mellon UniversitySoftware Engineering Institute
November 2004 Page 30© 2004 by Carnegie Mellon University
®
“Single Book, Single Course” Strategy
• CMMI v1.2 will consolidate both staged andcontinuous representations
• single course for “Intro to CMMI” has been created- first public offering here last week
– 1/3 of “staged” proponents converted to“continuous”
- new instructors will only be trained in single course- existing instructors will receive upgrade training- staged and continuous courses will be available until
planned upgrade training is complete
• SCAMPI refinements will complement the strategy
Carnegie Mellon UniversitySoftware Engineering Institute
November 2004 Page 31© 2004 by Carnegie Mellon University
®
Model SimplificationsTo address size and complexity• “single book” approach to model
- not eliminating staged or continuous, just packaging better• eliminate concept of advanced practices.• eliminate concept of common features.• improve model-method interactions for artifacts.• looked at combining select PAs such as VAL & VER
- detailed analysis revealed that no PA combinations hadvalidity or merit
To address hardware design applicabilityadding “hardware” amplifications to provide bettercoverage of hardware design in CMMI
- NOT adding “HW” discipline! Design of hardware is alreadycovered in CMMI, just not a well as we would have liked
Carnegie Mellon UniversitySoftware Engineering Institute
November 2004 Page 32© 2004 by Carnegie Mellon University
®
CMMI v1.2 Schedule• several thousand change requests were analyzed
since January 2004- content, clarifications, errata, etc
• development of Change Packages (CPs) underway• Configuration Control Board (CCB) starts meeting
now- approve CPs for development into
Implementation Packages (IPs)- training update will occur to match model
• pilot v1.2 in early FY06 (i.e. Fall 2005)• release v1.2 late FY06 (i.e. June 2006)• Release date of “Services” constellation will follow
within 3 to 6 months
Carnegie Mellon UniversitySoftware Engineering Institute
November 2004 Page 33© 2004 by Carnegie Mellon University
®
SummaryCMMI is in widespread adoption• more people CMMI-trained in past 2-1/2 years than
in 12 years of SW-CMM
• CMMI Appraisal results flowing fast & furious- 1st USAF site achieved CMMI-SE/SW Level 5
– Warner Robins AFB, Oct 2004
CMMI is already demonstrating substantial ROI andmeasurable benefits to adopting organizations
CMMI v1.2 will bring needed, but gentle,improvements
CMMI v1.2 “Services” constellation will provide fulllife cycle and enterprise coverage
http://www.sei.cmu.edu/cmmi