high maturity workshop @eurospi...
TRANSCRIPT
HIGH MATURITY WORKSHOP @EUROSPI CONFERENCEChristian Hertneck
June 2014
> Motivation
> Statistical Thinking
> Capability and Maturity Level 4 & 5
> Overview of Statistical Tools
> Workshop Topics> Examples
> Exercises
> Discussions
> Workgroups
2
CONTENT
3
4
TYPICAL REPORTING SYSTEM
QualityJuly
Actual Value
Monthly Average
Value% Diff % Diff from
July Last Yr Actual Plan or Average % Diff
This YTD as % of
Last YTDOn-time Shipments (%) 91.0 91.3 -0.3 -0.9 90.8 91.3 -0.6 -0.3First Time Approval (%) 54.0 70.0 -23.0 -10.0 69.3 70.0 -1.0 -0.4Pounds Scrapped (per 1000 lbs production) 124.0 129.0 -3.9 0.0 132.0 129.0 2.3 1.5
Production
Production Volume (100s/lbs) 34.5 36.0 -4.2 -2.0 251.5 252.0 -0.2 -8.0Material Costs ($/100 lbs) 198.29 201.22 -1.5 -1.9 198.5 201.2 -1.4 -3.6Manhours per 100 lbs 4.45 4.16 7.0 4.5 4.5 4.2 7.2 9.3Energy & Fixed Costs/100 lbs 11.34 11.27 0.6 11.3 11.02 11.27 -2.2 9.2Total Production Costs/100 lbs 280.83 278.82 0.7 0.9 280.82 278.82 0.7 0.4In-process Inventory (100's lbs) 28.00 19.70 + 42.0 + 12.0 21.6 19.7 + 9.6 + 5.9
Yr-to-Date
Monthly Report
5
TIME SERIES FOR MONTHLY IN-PROCESS INVENTORY
Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov DecYr 1 19 27 20 16 18 25 22 24 17 25 15 17Yr 2 20 22 19 16 22 19 25 22 18 20 16 17Yr 3 20 15 27 25 17 19 28
In-Process Inventory
5
10
15
20
25
30
35
J F M A M J J A S O N D J F M A M J J A S O N D J F M A M J J
20.04
Average
No long term trendsNo other systematic patterns
Does not say whether July value is exceptional Is the July value a signal---or is it just noise?
In-process Inventory (100's lbs) 28.00 19.70 + 42.0 + 12.0 21.6 19.7 + 9.6 + 5.9
July Actual Value
Monthly Average
Value% Diff
% Diff from July Last Yr
ActualPlan or
Average% Diff
This YTD as % of
Last YTD
Yr-to-Date
Need to filter out the month to month variation.
28
6
0
5
10
15
J F M A M J J A S O N D J F M A M J J A S O N D J F M A M J J
4.35
14.2 = upper limit for difference between Monthly values
Moving range directly measures the month-to-month variation.Upper control limit for Moving Range = 3.27 x Avg = 14.2
Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec8 7 4 2 7 3 2 7 8 10 2
3 2 3 3 6 3 6 3 4 2 4 13 5 12 2 8 2 9
Result: If the amount of in-process inventory changes (up or down) by more than 1420 lbs from one month to the next, then one should look for an explanation.
UPPER CONTROL LIMIT FOR MOVING RANGE CHART
Result of monitoring the process behavior chart: No problem! 7
TIME SERIES FOR MONTHLY IN-PROCESS INVENTORY
Size of % difference partially depends upon the magnitude of the base number10 unit change from 100 to 110 is 10% change10 unit change from 300 to 310 is 3.3% change
Comparing lines in a monthly report by comparing the size of the percent differences assumes that all lines should show the same amount of relative variation month-to-month
Differences based upon comparison of the current value with last year’s value, a large % difference may be due to an unusual value in the past rather than an unusual value in the present
8
PROBLEMS WITH PERCENT DIFFERENCES AS A BASIS FOR INTERPRETING VALUES
Month-to-month variation
Inventory change > 1420 lbs from one month to the next, one should look for an explanation…likely due to the direct result of some special / assignable cause.
Actual Values Monthly Chart
Limits on the individual values chart define how large or small a single monthly value must be before it represents a definite departure from the historic average…value in excess of 31.6 would be a signal that the amount of inventory had shifted upward.
Value of 28 is not, by itself, a signal. There is not real
evidence of any real change in the in-process inventory.
9
INTERPRETATION
10
QualityJuly
Actual Value
Monthly Average
Value% Diff % Diff from
July Last Yr Actual Plan or Average % Diff
This YTD as % of
Last YTDOn-time Shipments (%) 91.0 91.3 -0.3 -0.9 90.8 91.3 -0.6 -0.3First Time Approval (%) 54.0 70.0 -23.0 -10.0 69.3 70.0 -1.0 -0.4Pounds Scrapped (per 1000 lbs production) 124.0 129.0 -3.9 0.0 132.0 129.0 2.3 1.5
Production
Production Volume (100s/lbs) 34.5 36.0 -4.2 -2.0 251.5 252.0 -0.2 -8.0Material Costs ($/100 lbs) 198.29 201.22 -1.5 -1.9 198.5 201.2 -1.4 -3.6Manhours per 100 lbs 4.45 4.16 7.0 4.5 4.5 4.2 7.2 9.3Energy & Fixed Costs/100 lbs 11.34 11.27 0.6 11.3 11.02 11.27 -2.2 9.2Total Production Costs/100 lbs 280.83 278.82 0.7 0.9 280.82 278.82 0.7 0.4In-process Inventory (100's lbs) 28.00 19.70 + 42.0 + 12.0 21.6 19.7 + 9.6 + 5.9
Yr-to-Date
Monthly Report
TYPICAL REPORTING SYSTEM
11
TIME SERIES FOR PERCENTAGE ON-TIME SHIPMENTS
12
INTERPRETATION
Six of the individual values and one of the moving ranges fall outside the limits.
The six values below the limit shouldbe treated as signals. The process is trying totell you it has a problem.
If you concentrate on the percent differences in the monthly report, you are not likely to ever be aware of this problem until it is already too late…
Control Charts are about separating signals from noise.
> Motivation
> Statistical Thinking
> Capability and Maturity Level 4 & 5
> Overview of Statistical Tools
> Workshop Topics> Examples
> Exercises
> Discussions
> Workgroups
13
CONTENT
14
PROCESS VARIATION
Shewhart’s notion of dividing variation into two types:
> common cause variation > variation in process performance due to normal or
inherent interaction among process components (people, machines, material, environment, and methods)
> represents the “noise” of the process
> assignable/special cause variation > variation in process performance due to events that are not part of the
normal process. > represents sudden or persistent abnormal changes in one or more of the
process components> represents a “signal” of the process
15
COMMON CAUSES
16
BY CHANGING THE PROCESS YOU HAVE TO ADAPT THE CONTROL CHARTS
17
PREDICTABILITY IS THE KEY FOR HIGH MATURITY
> A process is said to be predictable when, through the use of past experience, we can describe, at least within limits, how the process will behave in future.
> A unpredictable process will display exceptional variation that is the result of assignable/special causes.
> A predictable process will display routine variation that is characteristic of common causes.
Image courtesy of Microsoft Cliparts
18
REMOVING SPECIAL CAUSES OF VARIATION
> Difficult to do well!> Identify special/assignable causes as soon as possible
> checks/alerts/triggers in data entry/measurement tools> checks within the measured process> use charts/diagrams for daily review> assign individuals to monitor process data> provide template/form to fill out initial details
of special variance> Maintain data base of special cause investigations> Use data base reports to monitor corrective actions> Train individuals for the investigations and problem
solving techniques.
19
IDENTIFYING SPECIAL CAUSES OF VARIATION - EXAMPLE
20
FROM PROCESS STABILITY TO PROCESS PREDICTABILITY
Process is predictable!
Process is stable!
Derive prediction model based on an appropriate*probability model
appropriate* means the model fits the reality
Determine stability of the process
21
Managing Quantitatively? - Example
Therefore, just having nice charts and diagrams DOES NOT mean you are statistically controlling anything.
22
SIMPLE DETECTION TESTS FOR INSTABILITIES
23
REMOVING SPECIAL CAUSES OF VARIATION - REVISITED
> Use traditional, effective, problem solving techniques to select corrective action
> team of experts> addressing root causes of problems> obtain approval, pilot, deploy, train, monitor, inform
> Beware of “false alarms”> data errors, inconsistencies> improperly calculated limits, wrong assumptions> lack of data grouping> too many tests> nothing wrong with the process
> Beware of a lack of any special variation> control limits may be too wide
> Improving data quality and data grouping is a continuous effort
24
LISTENING TO VOICES
Process Capability
DOES NOT EQUATE TO a capable process.
Voice of the process = the natural bounds of process performance
Stable
UCL
LCL
Capable
UCL LCL
USL
LSL
USL/LSL = Upper/Lower Specification LimitUCL/LCL = Upper/Lower Control Limit
• Capable process = stable process + product conformance
• Voice of the customer = the goals established for the product and process performance (e.g. specifications)
USL
LSL
25
ALIGNING PROCESS PERFORMANCE TO PROCESS REQUIREMENTS
Change the Specs
Upper Spec
Shift the AimUpper Spec
Mean
Reduce VariationUpper Spec
> Motivation
> Statistical Thinking
> Capability and Maturity Level 4 & 5
> Overview of Statistical Tools
> Workshop Topics> Examples
> Exercises
> Discussions
> Workgroups
26
CONTENT
PA 4.1 Process measurement attribute> Measure of the extent to which measurement results are used to
ensure that performance of the process supports the achievement of relevant process performance objectives in support of defined business goals.
> Result of full achievement> process information needs in support of relevant defined business goals are
established;> process measurement objectives are derived from process information needs;> quantitative objectives for process performance in support of
relevant business goals are established;> measures and frequency of measurement are identified and
defined in line with process measurement objectives and quantitative objectives for process performance;
> results of measurement are collected, analyzed and reported in order to monitor the extent to which the quantitative objectives for process performance are met;
> measurement results are used to characterize process performance. ISO/IEC 15504-2
CAPABILITY LEVELS 4 & 5 (SPICE)
0 Incomplete
5 Optimizing
4 Predictable
3 Established
2 Managed
1 Performed
27
PA 4.2 Process control attribute
> Measure of the extent to which the process is quantitatively managed to produce a process that is stable, capable, and predictable within defined limits.
> Result of full achievement> analysis and control techniques are determined and applied where applicable;> control limits of variation are established for normal process performance;> measurement data are analyzed for special causes of variation;> corrective actions are taken to address special causes of variation;> control limits are re-established (as necessary) following corrective action.
ISO/IEC 15504-2
CAPABILITY LEVELS 4 & 5 (SPICE)
0 Incomplete
5 Optimizing
4 Predictable
3 Established
2 Managed
1 Performed
28
PA 5.1 Process innovation attribute> Measure of the extent to which changes to the process are identified
from analysis of common causes of variation in performance, and from investigations of innovative approaches to the definition and deployment of the process.
> Result of full achievement> process improvement objectives for the process are defined that support the
relevant business goals;> appropriate data are analyzed to identify common causes of variations in process
performance;> appropriate data are analyzed to identify opportunities for best practice and
innovation;> improvement opportunities derived from new technologies and process concepts
are identified;> an implementation strategy is established to achieve the process improvement
objectives.
CAPABILITY LEVELS 4 & 5 (SPICE)
0 Incomplete
5 Optimizing
4 Predictable
3 Established
2 Managed
1 Performed
29
PA 5.2 Process optimization attribute
> Measure of the extent to which changes to the definition, management and performance of the process result in effective impact that achieves the relevant process improvement objectives.
> Result of full achievement> impact of all proposed changes is assessed against the objectives of the defined
process and standard process;
> implementation of all agreed changes is managed to ensure that any disruption to the process performance is understood and acted upon;
> effectiveness of process change on the basis of actual performance is evaluated against the defined product requirements and process objectives to determine whether results are due to common or special causes.
ISO/IEC 15504-2
CAPABILITY LEVELS 4 & 5 (SPICE)
0 Incomplete
5 Optimizing
4 Predictable
3 Established
2 Managed
1 Performed
30
CMMI – CONNECTING CAPABILITY AND MATURITY ON ORGANIZATIONAL LEVEL
Maturity Level 5Optimizing
Organizational Performance ManagementCausal Analysis and Resolution
Capability Level 1 2 3
Maturity Level 4Quant. Managed
MaturityLevel
3Defined
Organizational Process PerformanceQuantitative Project Management
Requirements Management Project PlanningProject Monitoring and ControlSupplier Agreement ManagementMeasurement and AnalysisProcess and Product Quality AssuranceConfiguration Management
Requirements DevelopmentTechnical SolutionProduct IntegrationVerificationValidationOrganizational Process FocusOrganizational Process DefinitionOrganizational Training Integrated Project ManagementRisk ManagementDecision Analysis and Resolution
MaturityLevel
2Managed
31
32
WHAT AN ORGANIZATIONAL LEVEL 3 HAS ACHIEVED …
» Managing a company by means of the monthly report is like trying to drive a car by watching the yellow line in the rear-view mirror «
[Myron Tribus]
33
HIGH MATURITY REQUIRES LEADING INDICATORS
34
SUMMARY OF TERMINOLOGY
Improve process continually to reduce variability and improve quality, cost and cycle time.
Improvement
Capability
Predictability
Stability
Performance
Ensure that process behaviour is stable by removing assignable causes.
Make the process capable by changing the process performance.
Focu
s of
Lev
el 4
A process is said to be predictable when through the use of past experience, you can describe, at least within limits, how the process will behave in future.
Focu
s of
Lev
el 5
Measuring of attributes of the process
Focu
s of
ML
3
> Motivation
> Statistical Thinking
> Capability and Maturity Level 4 & 5
> Overview of Statistical Tools
> Workshop Topics> Examples
> Exercises
> Discussions
> Workgroups
35
CONTENT
36
PURPOSE AND BENEFITS OF SPC
„Provide quantitative information that improves decision making in time to positively affect the business outcome
Decisions regarding adaptations may be necessary due to changing circumstances on project, product, process, or business level
Suitable information is necessary to be able to decide what needs adaptation on business, process, or project level Early identification of critical project situations and problems
Early identification of critical project situations and problemsObtain leading (measures to decide) instead of lagging
(measures to learn) indicatorsConsistent prediction and monitoring across different levels
(project, multi-project, organizational)
37
WHAT IS STATISTICAL PROCESS CONTROL?
Typical questions to be answered using SPC
Is process x in control and predictable?(Is the quality of work products generated in the design phase predictable?)
What is the average value and range for process x?(How many hours do we typically spend on peer reviews of design documents?)
Is the latest measurement typical or did something unusual happen (need for corrective action)?
SPC is an analytical / statistical technique used to identify and understand sources of process performance variations in order to quantitatively monitor, control, and predict the process.
QUANTITATIVE/STATISTICAL TOOLS AND TECHNIQUES - 1
Statistics
Inferential Statistics Descriptive StatisticsPrecondition
Watching Statistics monitor variation, i.e. distinguish between usual random
and abnormal change.
Inferential statistics comprises the use of statistics to make inferencesconcerning some unknown aspect (usually a parameter) of a population
Descriptive statistics is a branch of statisticsthat denotes any of the many techniques used to summarize a set of data. In a sense, we are usingthe data on members of a set to describe the set.
Watching statistics are applied to assess the nature of variation in a process and to facilitate forecasting and management (monitor variation, i.e. distinguish between usual random from abnormal change).
39
QUANTITATIVE/STATISTICAL TOOLS AND TECHNIQUES - 2
Statistics
Inferential Statistics Descriptive StatisticsPrecondition
Numerical Graphical
Mean MedianMode
Pareto ChartHistogramRun ChartContingency T.Scatter PlotFishboneBox Plot
Location Dispersion
Range, IQRStandard DeviationVariance
Distributions(the shape of the process)
Hypothesis Testing
Regression Analysis
Prediction Models
Parameter Estimation
Watching Statistics monitor variation, i.e. distinguish between usual random from abnormal change.
> If we measure more things, and involve more people in reviewing and using the measures, we will eventually achieve High Maturity...
> The key to achieving high maturity is measuring the right things, and using the correct techniques to analyze and interpret the measures...
> We need to wait until we have more of the right kind of data before we can attempt to implement High Maturity Practices...
40
MISPERCEPTIONS ABOUT HIGH MATURITY - 1
Image courtesy of Microsoft Cliparts
> Adding the use of Control Charts to the practice of measurement and analysis results in High Maturity…
> All I need to do is to use Control Charts to analyze the outcome of our critical subprocesses and we can control them...
> Using threshold based on specification limits is a high maturity practice..
41
MISPERCEPTIONS ABOUT HIGH MATURITY - 2
Image courtesy of Microsoft Cliparts
> All the things we need to understand about high maturity practices can be adequately explained in a single conference presentation or workshop. [10]
42
MISPERCEPTIONS ABOUT HIGH MATURITY - 3
Image courtesy of Microsoft Cliparts
43
Clarify business goals
Identify and prioritize issues
Select and define measures
Collect, verify, and retain data
Analyze process behaviour
Processstable?
Processcapable?
New goals,strategy?
New issues?
Newmeasures?
Remove assignable causes
Change process
Continually improve
Yes
Yes
YesNo
No
No
No
No
Yes
Yes
Leve
l 5
Leve
l 4
Leve
l 3
SUMMARY –PROCESS BEHAVIOR MEASUREMENT FRAMEWORK
44
HIGH MATURITY IS ABOUT IDENTIFYING WASTE AND IMPROVEMENT OPPORTUNITIES QUANTITATIVELY
> Motivation
> Statistical Thinking
> Capability and Maturity Level 4 & 5
> Overview of Statistical Tools
> Workshop Topics> Examples
> Exercises
> Discussions
> Workgroups
45
CONTENT
> High Maturity case studies
> Selecting processes and data for statistical analysis
> Connecting business improvement to quantitative data
> Creating performance models and their use in project management.
> Tools for use in statistical analysis of data
46
SAMPLES FOR WORKGROUP DISCUSSIONS
47
QUESTIONS
THANK Y0U!
Christian Hertneck Anywhere.24 GmbHLindberghstr. 1182178 Puchheimwww.anywhere24.com
VersionDraft /
for review / released
Date Comments/Change History Author
0.01 Draft 06.02.2014 Layout; Contents; Summary... C.Hertneck
0.02 For review 17.06.2014 Contents C.Hertneck
1.00 Released 23.06.2014 Final touches C.Hertneck
® Capability Maturity Model, Carnegie Mellon, CMM, andCMMI are registered in the U.S. Patent and Trademark Office by Carnegie Mellon University.
sm CMM Integration; IDEAL; Personal Software Process; PSP; SCAMPI; SCAMPI Lead Assessor/ Appraiser; SEPG; Team Software Process; and TSP are service marks of Carnegie Mellon University.
> [1] Measuring the Software Process, William Florac and Anita Carleton, Addison-Wesley, 1999.
> [2] Understanding Variation: Key to Managing Chaos, Donald Wheeler, SPC Press, 1993.
> [3] Understanding Statistical Process Control, Donald Wheeler and David Chambers, SPC Press, 1992.
> [4] Practical Software Metrics for Project Management and Process Improvement, Robert Grady, Englewood Cliffs, 1992.
> [5] Statistical Methods for Software Quality - Using Metrics to Control Process and Product Quality, Adrian Burr and Mal Owen, International Thomson Computing Press.
> [6] Goal-Driven Software Measurement - A Guidebook, Robert Park, WolfhartGoethert and William Florac, CMU/SEI-96-HB-002, Carnegie Mellon University.
> [7] Metrics and Models in Software Quality Engineering, Stephen Kan, Addison Wesley, 1995.
> [8] Software Metrics: A Rigorous & Practical Approach, Norman Fenton, Shari Pfleeger, Thomson, 1997.
> [9] Capability Maturity Model® Integration (CMMI-DEV v1.3).
> [10 ]SEPG Presentations, e.g., High Maturity Misconceptions, Will Hayes, 2007
51
REFERENCES