pruebas scampi b y c

Scalable and Tailorable Appraisal Methods -SCAMPI B&C

Presented to the Silicon Valley SPINApril 26, 2006

by Diane McDonald

This material was adapted from the SEI’s Handbook for Conducting SCAMPI B and C Appraisals, version 1.1 and overview material® CMMI and Capability Maturity Model Integration are registered in the U.S. Patent and Trademark Office by Carnegie Mellon University.

® StepUp Solutions is registered in the U.S. Patent and Trademark Office.sm SCAMPI is a service mark of Carnegie Mellon University.

http://www.stepupsolutions.com

PDF created with pdfFactory trial version www.pdffactory.com

http://www.stepupsolutions.com

http://www.pdffactory.com


SCAMPI B & C Overview - SV SPIN April, 2006 StepUp Solutions, Inc.

Topics

• Standard CMMI Appraisal Method for Process Improvement (SCAMPI) Family of Appraisals

• SCAMPI B and C Characteristics

• Tailoring Considerations

• Usage Modes

• Core Concepts

• Appraisal Phases

• Summary





SCAMPI Appraisals

• SCAMPI - Standard CMMI Appraisal Method for Process Improvement - SEI’s official methodology for conducting CMMI-based appraisals- “SCAMPI” B & C are distinguished from “Class” B and C

appraisals, which are defined by Lead Appraisers using their own techniques

• The SCAMPI method is embodied in 3 standard variants based on the class structure defined in the Appraisal Requirements for CMMI (ARC V1.1)

• The SCAMPI family of appraisals provides a variety of solutions to accommodate various appraiser and organizational needs, e.g., - Self-assessments, initial appraisals, quick-look or mini-appraisals,

incremental appraisals, benchmarking appraisals- Shared definitions of inputs, activities, outputs (upward compatible

results)- Wide variety of possible tailorings to fit different situations





SCAMPI A, B, & C Comparison

range of tailoring

depth of investigation

SCAMPI C - provides broad flexibility in addressing the approach planned (or taken) to implement processes.

SCAMPI B - provides flexibility in model & organizational scope and is performed on deployed (implemented) practices.

SCAMPI A - the most comprehensive and rigorous method, providing ratings (goal, maturity level rating, capability level ratings).

The tailoring and scaling permitted in each class is so wide that the 3 methods can overlap in many respects.





SCAMPI A, B, & C Comparison (cont.)

Characteristics Class C Class B Class AResource requirements (organization/appraisal team)

Low Medium High

Amount objective evidence Low Medium High

Characterizations generated

Optional Yes Yes

Ratings generated No No Yes





SCAMPI B & C Tailoring Examples• Model scope/focus

- Number of Process Areas (PAs), coverage of PAs (e.g., specific practices only, single goal only)

- Level of granularity (SCAMPI C), e.g., practice level, goal level• Organizational unit scope

- 1 project, several projects- Disciplines (e.g., SW Engineering only, SW and Systems

Engineering)• Appraisal team vs. individual; team size• Approach vs. deployment

- Review of processes only, review of processes and implementation

• Appraisal data sources - documentation, interviews, both• Appraisal output options

- For SCAMPI C, findings only, or findings and characterizations (color-coded charts)

- Findings/charts at project level or rollup at organizational level





Example Usage ModesGap Analysis• Initial detailed appraisal to baseline an improvement plan• Appraisal used to bridge to new model, or new PAs, or new parts of

an organization• Detailed look at target areas found weak in a previous appraisal -

anticipating a future benchmark appraisal

Mini-Appraisal• Interview-intensive first-time experience for new group learning

about process improvement and appraisals• Full coverage dress rehearsal for benchmarking appraisal in a group

that has a past history of process improvement

Improvement Monitoring• Internal progress appraisal using EPG staff to monitor deployment

of new practices• Contract incentive award determination based on external auditor• Consultants may perform incremental appraisals building PIIDs for a

future benchmark appraisal





Core Concept 1: SCAMPI Family ArchitectureSCAMPI family of appraisals can cover the following range:• SCAMPI C “approach,” - looking at the approach planned to satisfy

process improvement goals, to • SCAMPI B “deployment” - examining deployment of processes in

selected projects in an organizational unit, to • SCAMPI A “institutionalization” - benchmarking the

institutionalization of CMMI in an organizational unit

Approach Deployment Institutionalization

A

B

C

Reliability, rigor & cost

Accuracy of results





Core Concept 1: SCAMPI Family Architecture (cont.)

Example Scenario• Phase 1 - SCAMPI C is conducted on the proposed processes => Approach

- Using the results from the SCAMPI C, the organization could then make adjustments to these processes to increase their confidence of fidelity to the CMMI.

- Question being addressed: If these proposed processes are implemented, would they satisfy the goals and meet the intent of the CMMI?

• Phase 2 - Implement the processes in pilot projects => Deployment- A SCAMPI B could then be conducted to examine the results of executing

these processes. - Question being addressed: If the processes examined in the pilot

projects were broadly implemented, would they satisfy the goals and meet the intent of the CMMI?

• Phase 3 - The organization could then, with a higher level of confidence, broadly implement the processes => Institutionalization- A SCAMPI A could then be used to validate institutionalization and

establish a benchmark for the organization.





Core Concept 2: Objective Evidence

• Objective Evidence (OE) includes Direct Artifacts, Indirect Artifacts, and Affirmations- Direct Artifacts - tangible outputs resulting directly from

performance of a CMMI practice (e.g., CMMI typical work products)

- Indirect Artifacts - a secondary consequence of performing a CMMI practice (e.g., meeting minutes, reviews, logs, and reports)

- Affirmations - oral or written statements confirming practice implementation

• The organization can build a set of OE gradually, through a series of SCAMPI C and/or B appraisals - Practice Implementation Indicator Descriptions (PIIDs) provide an

organized structure for storing and presenting the objective evidence needed for the appraisal

• OE examined during an appraisal are related to the CMMI practices (or other components, such as goals, for a SCAMPI C)





Core Concept 3: Data Sources

• Primary data sources:- Documentation - in hardcopy or electronic provide evidence

for a wide range of practices- Interviews - used to corroborate the documentation and gain

insight into how the work is actually performed• Note: For a SCAMPI B, evidence of implementation (not just

intended approach, as in SCAMPI C) must be examined

OE





Core Concept 4: Practice Characterizations

• Each SCAMPI class has a unique characterization scale designed to match the minimum standards of data sufficiency for that method - SCAMPI B - uses red, yellow, green

- Each practice within scope is characterized indicating the risk of CMMI goal satisfaction if the observed practices were deployed across the organizational unit.

- Model scope is not limited to entire Process Areas, but could include sets of related practices (e.g., associated with 1 goal)

- SCAMPI C - uses low, medium, high- The intent of the model practice is characterized indicating the risk of

CMMI goal satisfaction if the proposed processes were implemented.- Data may be examined at a higher level of granularity than an

individual practice (e.g., goal, process area, maturity level)- Note: Rating of goals are not permitted in SCAMPI B or C

appraisals





Core Concept 5: Appraisal Outputs

• Outputs associated with each class of method are matched to the conclusiveness supported by the appraisal process. Outputs:- Findings (weaknesses, strengths)- Characterizations - Standard for SCAMPI B; Optional for

SCAMPI C





SCAMPI B & C Phases

Three phases involved in all SCAMPI appraisals:• Plan and Prepare for Appraisal• Conduct Appraisal• Report Results





Phase 1 - Plan and Prepare for Appraisal

Requirement SCAMPI C SCAMPI BLed by SEI-authorized Team Lead Yes Yes

Communication between Team Lead and sponsor

Yes Yes

Sponsor approves Appraisal Input and Plan and any changes

Yes Yes

Minimum team size 1 2

Team members complete Intro to CMMI and appraisal training

Yes, if a team is used

Yes

Criteria for team experience in engineering and management

No Yes

Objective evidence mapped to CMMI Yes Yes

Data Collection Plan Yes Yes

Readiness Review Yes Yes





Primary Contents of Appraisal Input

• Sponsor identity• Appraisal purpose, including alignment with business objectives• CMMI model used and scope• Organizational unit appraised and process context• Appraisal constraints• Identity and affiliation of the Team Leader, appraisal team

members, participants and support staff, and their responsibilities • Planned appraisal activities and outputs• Appraisal resources, schedule, and logistics• Risks and mitigation plans• Anticipated follow-on activities• Planned tailoring of SCAMPI and associated tradeoffs, including

sample size or coverage of the organizational unit





Example SCAMPI C Readiness Review Document Count Chart

Engineering Category Results

0%10%

20%30%

40%50%60%

70%80%

90%100%

RM-D RM-I RD-D RD-I TS-D TS-I PI-D PI-I VER-D VER-I VAL-D VAL-I

Process Areas

% P

ract

ices

# Not Review ed

# Missing

# Incomplete

# Complete

RM-D RM-I RD-D RD-I TS-D TS-I PI-D PI-I VER-D VER-I VAL-D VAL-I# Complete 25 0 31 4 34 0 25 0 26 1 19 0

# Incomplete 19 0 37 0 23 0 29 1 30 0 23 0# Missing 7 0 4 0 12 0 9 0 4 0 9 0

# Not Reviewed 0 51 0 68 0 69 0 62 0 59 0 51





Phase 2 - Conduct Appraisal

Requirement SCAMPI C SCAMPI B

Direct artifacts, indirect artifacts, affirmations identified

Yes Yes

Objective evidence mapped to CMMI & projects/org Yes Yes

Direct artifacts reviewed, interviews conducted either both

Sources of objective evidence (interviews, documentation, instruments)

1 2

Interviews conducted by at least 2 team members No YesObservations corroborated No YesValidation of preliminary findings No YesFinal Findings include strengths & weaknesses (gaps wrt CMMI)

Yes Yes





Phase 2 Steps - Conduct Appraisal

Examine Objective Evidence

DocumentObjective Evidence

VerifyObjective Evidence

Validate Prelim. Appraisal Outputs

Generate Appraisal Outputs

•Direct Artifacts•Indirect Artifacts

•Affirmations

•Findings•Characterizations*

*Optional for SCAMPI C

•accuracy•model relevance•consistency•corroboration

•notebooks•spreadsheets

•feedback•optional for SCAMPI C





Characterization Options - SCAMPI B

• Every practice in the appraisal scope is characterized on a 3 point scale (red, yellow, green) indicating risk of CMMI goal satisfaction if the observed practices were deployed across the Organizational Unit (OU)

• Model scope is not limited to entire PAs, but could include sets of related practices





Practice Characterization - SCAMPI B

Label Meaning

RedThe intent of the model practice is judged to be absent or poorly addressed in the set of implemented practices – gaps or issues that will prevent goal achievement, if the deployment occurred in this way across the organizational unit, were identified

YellowThe intent of the model practice is judged to be partially addressed in the set of implemented practices – some gaps or issues were identified, which might threaten goal achievement if the deployment occurred in this way across the organizational unit.

GreenThe intent of the model practice is judged to be adequately addressed in the implemented set of practices examined – in a manner that would support goal achievement, if the practice were deployed across the organizational unit.





Aggregation of Practice Characterizations - SCAMPI B

**OU Characterization Instance Characterizations

Red All instantiations characterized Red

Yellow All instantiations characterized Yellow

Green All instantiations characterized Green

Team judgment subject to below *

All conditions not included above

* The OU characterization shall not be Red unless at least one instance characterization is Red* The OU characterization shall not be Green unless at least one instance characterization is Green

**OU = Organizational Unit





Data Sufficiency Criteria - SCAMPI B

For each practice within the scope of the appraisal:• Rule 1: At least one Direct Artifact or at least one Oral

Affirmation for each instantiation• Rule 2: In addition, at least one Direct Artifact and at least one

Oral Affirmation from the set of instantiations mapped to a goal

For SCAMPI B appraisals focused on a single instance:• A Direct Artifact for each practice• An Oral Affirmation for at least one practice within each set of

practices mapped to a goal





SCAMPI B Coverage: Example Acceptable Combinations

DirectArtifacts

OralAffirmations

project A project B project C project D





Characterization Options - SCAMPI C

• The characterization must be a 3-point scale• Every characterization generated must be supported by

relevant objective evidence• The characterization scale must not overstate the

conclusiveness of the outcome (e.g., referencing goal satisfaction)

• Can be scoped at any level of granularity, and the scale can be tailored to the appraisal objectives, e.g.,- Fidelity of observed practices to model goal achievement- Return on Investment to the organization from implementing

practices





Example Practice Characterization for SCAMPI C: Fidelity to Model

Label Meaning

LowThe intent of the model practice is judged absent, or inadequately addressed in the approach – goal achievement is judged unlikely because of this absence or inadequacy.

MediumThe intent of the model practice is judged to be partially addressed in the approach – and only limited support for goal achievement is evident.

HighThe intent of the model practice is judged to be adequately addressed in the set of practices (planned or deployed) – in a manner that supports achievement of the goal in the given process context.





Appraisal: ACME ORGANIZATIONProject Name: PLUTO

REQM PP PMC SAM M&A PPQA CMSpecific Goal 1SP 1.1 H L L H H L HSP 1.2 M M H H H H HSP 1.3 L H H H H MSP 1.4 H H M MSP 1.5 H MSP 1.6 MSP 1.7 M

Specific Goal 2SP 2.1 H M H L H HSP 2.2 H H H M H HSP 2.3 M H M HSP 2.4 H L HSP 2.5 HSP 2.6 HSP 2.7 H

Specfic Goal 3SP 3.1 H HSP 3.2 M HSP 3.3 MSP 3.4SP 3.5

Specific Goal 4SP 4.1SP 4.2SP 4.3

Generic Goal 2GP 2.1 L H H L H L HGP 2.2 M M M M M M MGP 2.3 H H H L H L HGP 2.4 H H H H H H HGP 2.5 M M M M M M MGP 2.6 H H H M H M HGP 2.7 H H H H H H HGP 2.8 H H H H H H HGP 2.9 M M M H M H MGP 2.10 M M M M M M M

Generic Goal 3GP 3.1 H H H H H L LGP 3.2 H L L H L H L

Example SCAMPI C Characterization Chart





Data Sufficiency Criteria - SCAMPI C

At least one item of Objective Evidence must be examined for every model component in the scope of the appraisal• If the appraisal is scoped at the practice level, then this means

one item of OE for each practice





Other Characterization Values - SCAMPI B & C

• Not Yet - The practice under consideration has not been implemented yet, because the project has not reached the appropriate point in the life cycle.

• Out of Scope - No characterization was assigned because the appraisal did not gather data to support characterization of this practice





Phase 3 - Report Results

Requirement SCAMPI B SCAMPI CDesignated appraisal results provided to the stakeholders identified in the appraisal plan

Yes Yes

Appraisal record delivered to the appraisal sponsor

Yes Yes

Appraisal data package submitted to the CMMI Steward

Yes Yes





Contents of the CMMI Steward Data Package

• Record of Entry - information related to appraisal entity, participants, team,

etc.• Appraisal input and plan• Appraisal findings – strength and/or weakness statements• Appraisal disclosure statement

- summary statements regarding the appraisal results, conditions, and constraints

• Team member and sponsor feedback





Summary

• SCAMPI B and C provide a wise range of choices and customizations to be made by the organization and Lead Appraiser - Broad flexibility to define an appraisal event to match

business drivers• Used in a progression, all 3 methods in the SCAMPI family can

be tailored to enhance integration of data from one appraisal event to another- Supports focused individual applications, as well as use in a

multi-event, on-going program of improvement




pruebas scampi b y c

Documents

scampi method

c appraisals

scampi b c overview

level of granularity

c characteristics

investigationscampi

stepup solutions

c comparison cont