risk based decision making - rev 4
TRANSCRIPT
Supplier RisksRisk Based Decision Making
Julie M. RodriguesGlobal Manager of Supplier
Development
Introduction What is Risk? Risk Assessment Methods Risk Matrix Development Integrated Risk Based Decision Making Questions
Agenda
What are we going to do today?
Risk in Supplier Quality systems Assessment methods Some ways to apply risk assessment to supplier
quality systems◦ Easy◦ Efficient◦ In alignment with organizational needs
Introduction
Quality System Standards◦ ISO 9001:2008◦ ISO 13485:2003◦ ISO 14971:2007
Regulatory standard◦ 21 CFR 820
Guidance Documents
Global Harmonization Task Force (GHTF) Documents◦ SG3/N17/2008, Quality Management System – Medical
Devices – Guidance on the Control of Products and Services Obtained from Suppliers
◦ SG3/N15R8/2005 , Implementation of Risk Management Principles and Activities within a QMS
◦ SG4/N30R20/2006, Guidelines for Regulatory Auditing of QMS of Medical Device Manufacturers, Part 2, Regulatory Auditing Strategy
Guidance Documents (cont.)
Introduction What is Risk? Risk Assessment Methods Risk Matrix Development Integrated Risk Based Decision Making Questions
Agenda
Product or Service is an item, tangible or intangible, which is purchased or otherwise obtained by the manufacturer. - A Product is the result of a process. - Service is the result of at least one activity necessarily performed at the interface between the supplier and customer and is generally intangible (ISO 9000:2005, Clause 3.4.2)
A Supplier is anyone that is independent from the manufacturer’s quality management system from whom a Product or Service is purchased. (ISO 9000:2005, Clause 3.3.6)
Definitions (from GHTF)
Risk is the combination of the probability of occurrence of harm and the severity of that harm. (ISO/IEC Guide 51:1999, definition 3.2)
What can go wrong? How bad is it? How likely is it to happen?
What is Risk?
When identifying risk:
◦ Do it to the best of our ability! - don’t do this in isolation
◦ Removed from the emotional impact◦ Without being alarmist: Cassandra Syndrome◦ With a value meaningful to the organization
Identifying Risk
Risk Assessment is the overall process comprising a Risk Analysis [identification of hazards and estimate of risk] and a Risk Evaluation [judgment as to whether a risk is acceptable]. (ISO/IEC Guide 51:1999, definition 3.12)
Process that identifies:What is the risk level?Based on the risk level, what happens?
What is Risk Assessment?
Supplier Risk: ◦ Risk to the organization resulting from a supplier
component, service, or process ◦ Risk associated with Supplier Quality Processes.
This includes◦Selection◦Evaluation◦Control◦Re-Evaluation
What is Supplier Risk
Not hard at all! ◦We do it all the time, within and without our
QMS
◦Examples: Who should I select as my primary care
physician? What brand of tires should I use? Should my next car be foreign or domestic? What cereal should I buy? Should I buy organic fruits and vegetables?
How hard is this really?
Why don’t we do this already?◦ We do! Intuitively
So what is the problem?◦ Consistency◦ Objective evidence – auditor wants to know why◦ Documented activities◦ Buy in from the entire organization
Not Difficult!
Introduction What is Risk? Risk Assessment Methods Risk Matrix Development Integrated Risk Based Decision Making Questions
Agenda
What are some desirable characteristics of a Risk Assessment?◦ Consistent◦ Easy to use and easy to understand◦ Provides objective evidence◦ Flexible◦ Subsequent activities can be based on results◦ Single assessment to satisfy all Supplier Quality
activities
Characteristics
Risk Assessment = Decision Making
We can use the same toolset
How?
There are lots of decision making tools.◦Build consensus through discussion◦Decision Tree◦Decision Table◦Decision Matrix
Each method has pros and cons
Decision Making Tools
Technique:- Discuss the options- Use good meeting practices to formulate options and achieve agreement
Method: Building Consensus
Pros• Easy• Familiar• No training required• Meeting minutes
serve as documentation
Cons• Lack of consistency• May take a while• Emotional
Technique:- Follow decision tree- Use good meeting practices to formulate options and achieve agreement
Method: Decision Tree
Pros• Single path• Minimal training• Sequential, some paths
eliminate options• Used frequently in
medical field
Cons• Can be complicated• May not be flexible,
limited to existing path• Decision process can be
emotional
Decision Tree, Example
From GHTF/SG3/N15R8
Technique:- Search for appropriate action in the table- Use good meeting practices to formulate options and achieve agreement
Method: Decision Table
Pros• More flexible• Non-linear• Visual
Cons• Training required,
complex• More options than needed• Decision process can be
inconsistent
Decision Table, Example
From GHTF/SG3/N15R8
Technique:- Combines decision tree and decision table methodologies- Answer questions, calculate weighted sums, look up result in table.
Method: Decision Matrix
Pros• Fact based decisions• Minimal training to use• Consistent results• Tool itself is documentation• Can be automated
Cons• Initial setup is complex• Calculations can be tedious
if not automated
Decision Matrix, Example
Material #:
Supplier Type Rati
ng
Complexity Rati
ng
1 - Fully Certified 1 - Simple, not critical (DIN)2 - Existing Supplier 2 - Nominal difficulty3 - New Supplier 3 - Complex/Critical
Similarity of parts Exposure1 - Modification of existing part 1 - Inside Device2 - Family part with characteristic differences 2 - Exposed to operating environment3 - No part similarities 3 - Patient contact/Critical Component
Delivery (last 12 weeks) Cosmetic1 - 98-100 1 - None/Internal2 - 90-97 2 - Process variable3 - <=89 3 - High visibility
Quality (last 12 weeks) Replacement Risk1 - 98-100 1 - Easily replaceable/Minimal rework2 - 90-97 2 - Reworkable3 - <=89 3 - Difficult or no rework/Must replace
Qualification Performance Detectable 1 - 5 of last 5 complete 1 - Visibly detectable2 - 4 of last 5 complete 2 - Detectable in assembly3 - <= 3 of last 5 complete 3 - Not detectable
Total 20 Total 27
4 - 11 = low, 12 - 24 = Nom, 25 - 36 = High Overall Risk Matrix
Overall Risk Assessment High
1 2
3 2
3 2
Risk Assessment123456
Supply Chain Risk Quality Risk
Nom
inal
Hig
h
2 3
3 3
Each process has pros/cons Selection will depend on the organization
KSE uses the Decision Matrix model
There is no “right” or “wrong” method
Introduction What is Risk? Risk Assessment Methods Risk Matrix Development Integrated Risk Based Decision Making Questions
Agenda
Internationally recognized organization that trains management in rational decision making.
Originally part of the Rand Corporation Integral in the development of 8D process and
other problem solving tools
Kepner-Trego Decision Making
Steps:◦ Establish decision criteria◦ Give each criteria a weight (numerical)◦ Score each option against the criteria◦ Weighted sum calculation establishes “best”
solution
Decision Making
Speed( 5 )
(1-10)
Cost( 3 )
(1-10)
Effectiveness( 9 )
(1-10)
Weighted Sum
200% Inspection 9 6 1
(9*5)+(6*3)+(1*9)= 72
Simple EP 5 5 8(5*5)+(5*3)+(8*9)=112
Vision System 2 1 10
(2*5)+(1*3)+(10*9)=
103
Example:
9*5=45 6*3=18 1*9=45
5*5=25 5*3=15 8*9=72
2*5=10 1*3=3 10*9=90
45+18+45 =108
25+15+72 =112
10+ 3+90 =103
Leverage the KT decision making approach Team based development
◦ QA◦ Purchasing◦ Manufacturing Engineering◦ Manufacturing◦ R&D
Advantages:◦ Tough decisions are made up front◦ Buy-in and comprehension across the organization◦ Can be automated in Excel
Risk Assessment team
Select Risk Decision Scale◦Low (1), Nominal (2), High (3)
All decisions in the matrix made using this scale
Why? ◦Typical of decision processes already used in
the organization (green/yellow/red)
Weight/Scale selection
Split into 2 decision path, and combine the results◦ QA◦ Supply chain
Why?◦ Both organizations need something from the
system◦ The needs sometimes conflict, but are real
Decision Paths
Select Options
Purchasing – Supplier Capability◦ Low: Certified Supplier (1)◦ Nominal: Existing approved supplier (2)◦ High: New supplier (3)
Options
Quality – Part Complexity◦ Low: Off the shelf/DIN/ISO component (1)◦ Nominal: No particular difficulty, special process or
other complication (2)◦ High: Complex component or assembly, or a critical
component (3)
Options (cont.)
Select Decision Criteria (Low/Medium/High risk):
Similarity of Parts◦ Modification of an existing part made by the supplier◦ One of a family of similar parts◦ No part similarities
Delivery Score◦ 98-100◦ 90-97◦ <89
Purchasing Criteria
Quality Score◦ 98-100◦ 90-97◦ <89
Success in completing qualification◦ Completed 5 of 5 most recent qualifications◦ Completed 4 of 5 most recent qualifications◦ Completed 3 or fewer of most recent 5 qualifications
Purchasing Criteria (cont.)
Purchasing Risk Matrix
Supplier Type Rati
ng
Complexity1 - Fully Certified 1 - Simple, not critical (DIN)2 - Existing Supplier 2 - Nominal difficulty3 - New Supplier 3 - Complex/Critical
Similarity of parts Exposure1 - Modification of existing part 1 - Inside Device2 - Family part with characteristic differences 2 - Exposed to operating environment3 - No part similarities 3 - Patient contact/Critical Component
Delivery (last 12 weeks) Cosmetic1 - 98-100 1 - None/Internal2 - 90-97 2 - Process variable3 - <=89 3 - KST acceptable
Quality (last 12 weeks) Replacement Risk1 - 98-100 1 - Easily replaceable/Minimal rework2 - 90-97 2 - Reworkable3 - <=89 3 - No rework/Must replace
RIS Performance Detectable 1 - 5 of last 5 complete 1 - Visibly detectable2 - 4 of last 5 complete 2 - Detectable in assembly3 - <= 3 of last 5 complete 3 - Not detectable
Total 16 Total
1
3
3
Supply Chain Risk Quality Risk
Nom
inal
2
1
Options
Criteria
CalculatedRisk
Purchasing Risk4 - 11 = Low 12 - 24 = Nom 25 - 36 = High
Exposure◦ Internal component◦ Exposure to operating environment◦ Patient Contact/Critical component
Cosmetics◦ None/Internal◦ Process variable◦ High visibility
Quality Criteria
Replacement Risk◦ Easy to replace, standard practice◦ Re-workable◦ Difficult or no rework/must replace
Detectable◦ Visually detectable◦ Detectable in process◦ Not detectable
Quality Criteria (cont.)
Quality Risk Matrix
Options
Criteria
CalculatedRisk
Complexity Rati
ng
1 - Simple, not critical (DIN)2 - Nominal difficulty3 - Complex/Critical
Exposure1 - Inside Device2 - Exposed to operating environment3 - Patient contact/Critical Component
Cosmetic1 - None/Internal2 - Process variable3 - High Visibility
Replacement Risk1 - Easily replaceable/Minimal rework2 - Reworkable3 - No rework/Must replace
Detectable 1 - Visibly detectable2 - Detectable in assembly3 - Not detectable
Total 30
2
2
3
Quality Risk
Hig
h
3
3
Quality Risk4 - 11 = Low 12 - 24 = Nom 25 - 36 = High
Combine Risk
L N HH High High High
N Low Nominal High
L Low Low NominalPurc
hasi
ng R
isk
Quality RiskOverall Risk
Risk Matrix
Material #:
Supplier Type Rati
ng
Complexity Rati
ng
1 - Fully Certified 1 - Simple, not critical (DIN)2 - Existing Supplier 2 - Nominal difficulty3 - New Supplier 3 - Complex/Critical
Similarity of parts Exposure1 - Modification of existing part 1 - Inside Device2 - Family part with characteristic differences 2 - Exposed to operating environment3 - No part similarities 3 - Patient contact/Critical Component
Delivery (last 12 weeks) Cosmetic1 - 98-100 1 - None/Internal2 - 90-97 2 - Process variable3 - <=89 3 - High visibility
Quality (last 12 weeks) Replacement Risk1 - 98-100 1 - Easily replaceable/Minimal rework2 - 90-97 2 - Reworkable3 - <=89 3 - Difficult or no rework/Must replace
Qualification Performance Detectable 1 - 5 of last 5 complete 1 - Visibly detectable2 - 4 of last 5 complete 2 - Detectable in assembly3 - <= 3 of last 5 complete 3 - Not detectable
Total 20 Total 27
4 - 11 = low, 12 - 24 = Nom, 25 - 36 = High Overall Risk Matrix
Overall Risk Assessment High
1 2
3 2
3 2
Risk Assessment123456
Supply Chain Risk Quality Risk
Nom
inal
Hig
h
2 3
3 3
Do we have these characteristics?◦ Consistent?◦ Easy to use and easy to understand?◦ Provides objective evidence?◦ Flexible?
We don’t have these yet…◦ Subsequent activities can be based on results◦ Single assessment to satisfy all Supplier Quality
activities
Characteristics
Introduction What is Risk? Risk Assessment Methods Risk Matrix Development Integrated Risk Based Decision Making Questions
Agenda
Is it possible to apply a single risk assessment model to all aspects of purchased product control?
Integration
Risk Assessment
Selection
Evaluation
Control
Re-Evaluation
Consider ◦ all components provided by the supplier or◦ all those components will be provided by the
supplier, Select the worst case for each option over all
components to get the overall supplier risk level.
i.e. If the supplier provides 5 components, but 4 are nominal, but 1 of them is a critical component, then the score for part complexity is High
Uses: Supplier Audit plans, improvement plans, critical supplier list
Evaluation – Supplier Risk
Risk assessment is completed for a particular component to get component risk level for a particular supplier
This can be done for ◦ New parts◦ Parts that have changed
Scale requirements based on risk level:◦ Part qualification requirements◦ Need for a supplier corrective action◦ Receiving inspection requirements (STS eligibility,
etc)
Control – Component Risk
Supplier Risk level can be used as part of supplier selection. ◦ Based on the score for each element of the
assessment, strengths and weakness become apparent
Re-Evaluation can be completed each time a new or modified component is assessed◦ New component (i.e. more complex or critical)◦ New data (i.e. on-time delivery)◦ This new assessment can be compared to the
existing one as part of Re-Evaluation
Selection and Re-Evaluation
Documented evidence of risk assessment may be an initialed copy of the completed Risk Matrix itself.
Subsequent review of the matrix clearly indicates the decision process
Results are easy to understand and auditable
Documentation
Can a single assessment method be used to satisfy multiple systems?
Yes! And subsequent activities can be scaled based on the result
As Part of the QMS
Risk Assessment
Selection
Evaluation
Control
Re-Evaluation
Questions?
KSE Supplier Quality Systems
Thank you