designing out unacceptable risksocalqualityconference.com/wp-content/uploads/2018/01/e02.pdf•...

Auditing Your Risk Management System Southern California Annual Quality Conference

November, 2017

© B. Craner 2017 1

Lecture 17 – Auditing Risk Management Process, Procedures, Documentation

Designing Out Unacceptable Risk

Barrett C. Craner

© B. Craner 2004-2017

Goals of Audit Session •  Your Risk Management system WILL be audited. •  Key targets of the audit and today’s topics:

–  What is Risk? –  Where is Risk? –  Why would your Risk Management Process be audited? –  Risk Management Program and Planning –  Risk Management File/Index (and Risk Analyses) –  Risk Benefit Analysis –  Risk Management Report (in RMF) –  Risk Management Review –  Field problems à Complaints, MDR’s à Risk Errors –  Summary – Prep for Audit! –  19011 – Audit Standard Process overview –  Workshop questions

2


November, 2017

© B. Craner 2017 2

© B. Craner 2004-2017

High

Medium Low

What is Risk?

A B C

D E

F

G H

I

J

3

© B. Craner 2004-2017

Risk – AS9100, ISO14971 •  (AS9100:2009, clause 3.1) An undesirable situation or

circumstance that has both a likelihood of occurring and a potentially negative consequence.

•  (ISO14971:2012) “It is accepted that the concept of risk has two (three) components: –  a) the probability of occurrence of harm, that is, the frequency

in which the harm can occur; –  b) the consequences of that harm, that is, how severe it might

be. –  c) the detectability of the emergence of the harm.

•  The acceptability of a risk to a stakeholder is influenced by these components and by the stakeholders’ perceptions of the risk.”

4


November, 2017

© B. Craner 2017 3

© B. Craner 2004-2017

Where is Risk?

•  Everywhere! Defense, Aerospace, Medical Devices, Food, Drug, Cosmetics, Finance, Investments, Automotive, Real Estate, Appliances, ISO13485...

•  And, Risk Management is Middle-Aged, but Auditing Risk is in Puberty

5

© B. Craner 2004-2017

Saturn V – Risk??? Yes, Huge risk capability •  Power of 85 Hoover Dams •  60’ Taller than Statue of Liberty •  13x weight of Statue of Liberty •  Six successful Moon landing missions •  13 successful launches from Kennedy

–  No loss of crew of payload •  >3,000,000 parts

–  (>700,000 components) •  Software 1,000,000’s of lines •  20,000 Contractors •  Fuel and containment •  Stage Separation process •  Launch strategy •  Launch vectors, calculations •  Telemetry •  Atmospheric conditions

6


November, 2017

© B. Craner 2017 4

© B. Craner 2004-2017

SpaceX Falcon 9 ... Falcon Heavy •  Huge risks:

Load to orbit, then land 1st stage back at LZ1!

7

© B. Craner 2004-2017

Appliances: Refrigerator

•  Risks here? – Shock (AC cord bared of insulation) – Suffocation (child trapped inside) – Food spoiled (Fridge failed [several causes]

•  Over-cooled (frozen, not to be frozen) •  Over-heated (fridge fails, and heats food) •  Non-cooled (fridge fails, food too long at room temp •  Non-cooled (human error sets temp too warm)

– Natural Gas powered – risks? – Single mission – to keep food from spoiling. – Others?

8


November, 2017

© B. Craner 2017 5

© B. Craner 2004-2017

Finance Industry – Sarbanes Oxley

•  Risk Grid – Summary – For discussion only

Compliance Risk

Fina

ncia

l Ris

k

9

© B. Craner 2004-2017

Risk Management – Regulatory Everywhere! •  FDA GMP 1976

•  PPQA 1989 • Medical Device Directives 1991 •  cGMP’s 1996 (QSR) •  Design Control 1998 • Medical Devices (EN/ISO 14971) •  Food, Drug, Cosmetics (Pharma Q9) •  Finance, Investments (Sarbanes-Oxley) •  Aerospace, Defense (FAA-H-8083-2, AS9100) •  Automotive (ISO 31000, TS 16949, QS9000...) •  Toys, Fire Safety, Asbestos, Furniture, Clothing...

Design

45%

Mfg.

45%

10%

Other

Design-Related Recalls 45% 1983-1988

Medical Device

10


November, 2017

© B. Craner 2017 6

© B. Craner 2004-2017

In Medical Device Directive 2016 Ed ANNEX I "General Safety and Performance

Requirements” Section I •  Reduce Risks As Far As Possible •  Safe Design Manufacturing Process •  Safe Ergonomic Features Section II •  Safe Design •  Safe Manufacturing Process •  Safe Installation •  Safe Hazard Levels (Radiation, etc.) Next four slides shows increase in Risk Req’ts! We Will be audited!

11

© B. Craner 2004-2017

MDD ANNEX I "General Safety and Performance Requirements”

Section I 1aa: "The requirements in this annex to reduce risks AS FAR AS POSSIBLE mean reduce risks AS FAR AS POSSIBLE without adversely affecting the risk-benefit ratio." 2 (b): "eliminate or reduce risks as far as possible through safe design and manufacture;" 2b: "reducing as far as possible the risks related to the ergonomic features of the device for the environment in which the device is intended to be used...”

12


November, 2017

© B. Craner 2017 7

© B. Craner 2004-2017

Section II 7.4.1: Devices shall be designed and manufactured in such a way as to reduce as far as possible the risks posed by substances or particles, ..." 7.5: "Devices shall be designed and manufactured in such a way as to reduce as far as possible risks posed by the unintentional leaking of substances from the device..." 7.6: "Devices shall be designed and manufactured in such a way as to reduce as far as possible the risks of substances or particles leaking into the device..." 8.1: "Devices and manufacturing processes shall be designed in such a way as to eliminate or to reduce as far as possible..." 8.1 (aa): "reduce as far as possible and appropriate..." 8.1 (b): "reduce as far as possible...”

13


© B. Craner 2004-2017


Section II 11.2: "Devices shall be designed and manufactured in such a way as to reduce as far as possible..." 13.1: "...shall be reduced as far as possible..." 13.1 (b): "reducing the risks inherent to installation as far as possible..." 13.3: "...radiation is reduced as far as possible..." 14.1: "...eliminate or reduce as far as possible..." 15.1: "...eliminate or reduce as far as possible..." 15.5: "...reduce as far as possible..." 15.7: "...to avoid, as far as possible, the risk...”

14


November, 2017

© B. Craner 2017 8

© B. Craner 2004-2017


Even more – the point is clear. •  AFAP (As Far As Possible) •  This language is quite different from the current

MDD ALARP (As Low As Reasonably Practical) language which takes into account the economic considerations of risk reduction, as mentioned in the Medical Device Directive

•  We'll have to see how the the Competent Authorities & Notified Bodies interprets this, but it is a tightening of the requirements.

•  We WILL be audited.

15

© B. Craner 2004-2017

Why Would Your Risk Management Program

Be Audited?

16

Pareto Head. What is this text you sent this morning?

You mean about the ISO auditor

being here?

It says that the “isolated android” is here.

I guess I should turn off Autocorrect.


November, 2017

© B. Craner 2017 9

© B. Craner 2004-2017

Reasons to be Audited? 1.  Your product is causing problems.

–  Harming People, Property, Environment –  Perception high-impact product (autos) might cause harms.

2.  Regulatory Compliance issues –  Mistake seen during audit –  Mistake seen during document review

3.  Intended: part of a regular audit –  Auditors are now trained. –  Auditors are not yet fully trained but curious “Danger, Will Robins”

4.  Unintended: comes up in a product/process review –  Risk Management File part of Design Process

5.  Holes, errors in your Risk Management System? –  Review your RM policy, process, documents, training

17

© B. Craner 2004-2017

OK, now show me a Risk Management Process or Two

18


November, 2017

© B. Craner 2017 10

© B. Craner 2004-2017

ISO/EN 14971:2012 (Medical Device)

Step 1

Step 2

Step 3

Step 4

Step 5a

Step 6 Step 7

Step 9

Step 10

Step 8

Step 11

Step 12

Step 13

START

Intended Use / Intended Purpose Identify Characteristics related

to Safety (4.2)

Identify Known and Foreseeable Hazards (4.3)

Estimate Risk(s) for Each Hazardous Situation (4.4)

Identify Appropriate RISK CONTROL Measures, Record

Risk Control Requirements (6.2)

Implement, Record and Verify Appropriate Measures (6.3)

Prepare Risk Management Report (8)

Review Production and Post-Production Information (9)

Is Risk Reassessment Necessary?

(9)

UNACCEPTABLE (Spec, Redesign,

Protection, Training, etc.)

Does Medical Benefit

Outweigh Residual Risk?

(6.5)

Is Risk Reduction

Necessary? (5)

Is The Risk Reducible?

(6.2) Is the Residual Risk Acceptable?

(6.4)

Are New Hazards or Hazardous

Situations Introduced, or Existing Affected

(6.6)

Is Overall Residual Risk

Acceptable? (7)

Yes

No No

No

No

No

No

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Step 5b

No

No

Risk Analysis

Risk Evaluation

Risk Control

Production, Post-Production Information

Overall Residual Risk Evaluation

All Identifiable Hazards

Considered? (6.7)

Do Medical Benefits

Outweigh overall

residual risk? (7)

No

Yes

EOL

END

19© B. Craner 2004-2016

© B. Craner 2004-2017

6 Steps in Risk Management Process

RiskIden4fica4on

RiskAnalysis

RiskMi4ga4on(Control)

RiskMonitoring

RiskRepor4ng

Howdoweknowtherisks?

Whatrisksneedreducing?

Howdowereduce/controltherisks?Howtomonitorrisks?

Whattoreport?

RiskFeedback!(Wheredidwegowrong?)

20


November, 2017

© B. Craner 2017 11

© B. Craner 2004-2017

Example Risk Management File Sequence •  Intro Docs

•  Plans

•  Analyses

•  Designs

•  Analyses

•  Mitigations

•  Verifications

•  Reports

•  File

•  Updating

Start Risk Management

Plan Start

Product Concept Available

Risk Management

Report

Preliminary Hazard

Analysis

Hazard Analysis

Preliminary Fault Tree

Analysis (2)

Further Hazard

Analysis

FMEA Process

Start Risk Management

File Start

Create Mitigations

Design with Risk

Controls

FMEA Use

Risk Benefit Analysis

FMEA Design

FTA Product

FTA Process HACCP

Preliminary Specification

Update FMEA’s & FTA’s, etc.

Create Mitigations

Update FMEA’s &

FTA’s, RMF

Approve Spec

Prelim FTA (2)

Hazard Analysis

21

Hazard ID

© B. Craner 2004-2017

Examples of Risk Management Programming and Planning?

22


November, 2017

© B. Craner 2017 12

© B. Craner 2004-2017

A Risk Management Program •  Policy, Procedures Approved •  Planning and Analysis Templates (all approved) •  Program timing templates (approved scheduling) •  Product Specification – Derive Risks / Hazards •  Personnel trained, competent (or not...?)

•  Part of your development plan, task sequencing •  Partnership agreements (who will assist outside)? •  Planned documents: Plan, Analyses, reports… •  Post-Market follow-up (surveillance, complaints...) •  Are these ready for audit?

23

© B. Craner 2004-2017

Plan: Annexes (Informative) 14971:2012 F.  Risk management plan

A.  General B.  Scope of the plan C.  Assignment of responsibilities and authorities D.  Requirements for review of risk management activities E.  Criteria for risk acceptability when probability of

occurrence of harm cannot be estimated F.  Verification activities G.  Method or methods of obtaining relevant feedback

production and post-production information

24


November, 2017

© B. Craner 2017 13

© B. Craner 2004-2017

u  ISO 14971 Standard

Text excerpted from Annex F

à

à

à

25

© B. Craner 2004-2017

u  ISO 14971 Standard

Text excerpted from Annex F

à

à

à

à

26

Is your Risk Mgt Plan reasonable, ready to audit, and have you followed it?


November, 2017

© B. Craner 2017 14

© B. Craner 2004-2017

What is a Risk Management File/Index?

27

© B. Craner 2004-2017

Plan/List Completion (info filled out) Plan approved Good to Go Completed Dates Done, Approvals

Plan Document File/List/Index and Approval

28


November, 2017

© B. Craner 2017 15

© B. Craner 2004-2017

Plan/List/Index

Plan Document File/List/Index and Approval

29

© B. Craner 2004-2017

Simple Risk Management Plan (Framework) –  General: Reduce risk for the Innovate Infusion Pump –  Scope: Plan (document listing and plan approval - seen in previous

slides) perform Hazard Identification, Analysis, FMEA (product, process, application), Risk/Benefit Analysis, Risk Mgt Report.

–  Responsibilities and authorities: Project Manager functional representatives listed in RM Plan approval document.

–  Review of risk management activities: Will be performed by project personnel and at Risk Management Review.

–  Risk acceptability: When probability of occurrence of harm cannot be estimated performed per Risk Mgt SOP 110-014.

–  Verification activities: Performed and documented per Verification and Validation SOP 116-001.

–  Relevant Feedback: Production / post-production information will be acquired by Post Market Surveillance procedure SOP 114-002 and Complaint/MDR/AE Procedure 114-001.

30


November, 2017

© B. Craner 2017 16

© B. Craner 2004-2017

Some Major Risk Management Concepts?

31

© B. Craner 2004-2017

Concept of Risk Level

Broadly Acceptable (Low) Risk Region

Intolerable (High) Region

Medium Risk Region (AFAP?)

Increasing Severity of harm

Increasing probability of occurrence

Example: Generic three-region risk chart

32


November, 2017

© B. Craner 2017 17

© B. Craner 2004-2017

Concept of a Risk Level Chart

Increasing Severity of Harm

Increasing Probability of Occurrence

1

2

3

4

5

1 2 3 4 5

ACC

ACC

ACC

ACC

Undes Intol Intol Intol

ACC

ACC ACC

Intol

Intol

Intol Intol

Example: Specific 3-Level risk chart*

33

Undes Undes

Undes Undes

Undes Undes

Undes Undes

Undes

Undes

© B. Craner 2004-2017

Concept of a Risk Level Chart

Increasing Severity of Harm

Increasing Probability of Occurrence

1

2

3

4

5

1 2 3 4 5

IV

IV

IV

III

III II I I I

II

III

IV

IV IV III II

I

I

I

II

II I

II

II III

Example: Specific 2-region 4-Level risk chart*

Either the risk is acceptable...

34


November, 2017

© B. Craner 2017 18

© B. Craner 2004-2017

Concept of “True” Risk Levels

Acceptable Region

Unacceptable Region

Increasing Severity of harm

Increasing probability of occurrence

Example: Generic (final) two-region risk chart

Either the risk is acceptable, or it is not!

35

© B. Craner 2004-2017

Your Risk Impact Table(s): Can you define/defend it/them?

Inconceivable P < 1 x 10-8

Remote 10-6 < P < 10-8

Possible 10-5 < P < 10-6

Probable 3 x 10-2 < P < 10-5

Frequent P > 3 x 10-2

Severity Scale

Neglig Minor Moderate Critical Catastr.

1 2 3 4 5 Frequency Scale

Frequent 5 Acc Acc Intol Intol Intol

Probable 4 Acc Acc Intol Intol Intol

Possible Remote

3

2

Acc Acc Acc Acc

Acc

Acc

Intol

Intol

Intol

Intol

Inconceivable 1 Acc Acc Acc Acc Intol

Probabilities? Severities? Risk Acceptability's?

Impact on People, Property, Environment & Product / Process Performance 36

Can you defend your Risk Impact Table 1-5 (above), 1-6, or 1-6, or 1-N?


November, 2017

© B. Craner 2017 19

© B. Craner 2004-2017

Severities by Level

Scale Definition

Use and Design Clinical Effects; Process End Effects (Customer) Process Local Effects

1 Negligible/ Cosmetic

(Negl)

No / virtually no injury to people; no / virtually no negative effect on the environment or property.

No impact on product performance, might not be noticed.

No effect up to minor disruption to production line; some rework, little scrap.

2 Moderate (Mod)

Moderate injury to people; moderate negative effect on the environment or property.

Reduced product performance or user confidence in the product/company (e.g. customer is very annoyed or dissatisfied).

Minor disruption to production line; product entered into the non-conforming material system with a portion to all being scrapped.

3 Serious (Ser)

Serious injury (reversible) to people; severe negative effect on the environment or property.

Serious loss of customer confidence Also, labeling that could lead to a field action must be

ranked at a minimum of 4.

Major disruption to production line; product to NCMR system, maybe all scrapped. Labeling or trace issues might be at level 4.

4 Catastrophic (Cat)

Serious injury (irreversible) or death of people; very severe negative effect on the environment or property.

Full loss of performance, and huge loss of customer base and cost to company, shareholders

May endanger machine/ assembly operator; or noncompliance with gov’t safety regulations. Failure may occur with or without warning.

Damage (Harm)

Performance, Customer Confidence

Mfg Process

37

© B. Craner 2004-2017

Severities by Level (Can We Define/Defend?)

Scale Definition

Use and Design Clinical Effects; Process End Effects (Customer) Process Local Effects

1 Negligible/ Cosmetic

(Negl)

No / virtually no injury to people; no / virtually no negative effect on environment or property.

No impact on product performance, might not be noticed.

No effect up to minor disruption to production line; some rework, little scrap.

2 Moderate (Mod)

Moderate injury to people; moderate negative effect on the environment or property.

Reduced product performance or user confidence in the product/company (e.g. customer is very annoyed or dissatisfied).

Minor disruption to production line; product entered into the non-conforming material system with a portion to all being scrapped.

3 Serious (Ser)

Serious injury (reversible) to people; severe negative effect on the environment or property.

Serious loss of customer confidence Also, labeling that could lead to a field action must be

ranked at a minimum of 4.

Major disruption to production line; product to NCMR system, maybe all scrapped. Labeling or trace issues might be at level 4.

4 Catastrophic (Cat)

Serious injury (irreversible) or death of people; very severe negative effect on the environment or property.

Full loss of performance, and huge loss of customer base and cost to company, shareholders

May endanger machine/ assembly operator; or noncompliance with gov’t safety regulations. Failure may occur with or without warning.

Damage (Harm)

Performance, Customer Confidence

Mfg Process

38


November, 2017

© B. Craner 2017 20

© B. Craner 2004-2017

Probabilities by Level

Scale Definition Use and Design Clinical Effects; Process End Effects (Customer) Process Local Effects

1 Inconceivable (Inc)

Does not happen for nearly any device during its service life for this model. (4* 10-6) (< 1 in 250,000 Rx)

Not practical that such an event happens. Extreme surprise this would happen.

Extreme low expectation this will happen during manufacturing.

If it does it would be catastrophic, would stop production, and may do so with or without warning. Costs >$100,000?

2 Remote (Rem)

May happen one per device during useful life. (2x10-4) or (< 1 in 50,000 Rx)

On average, some surprise if this happens for any device in field, with slight customer slightly annoyed, inconvenience.

Very low expectation this would happen. If it does, Moderate disruption to production line; product entered into the non-conforming material system with a portion to all being scrapped, <$2000 scrapping.

3 Probable (Pos)

Happens only a few times in a product lifecycle. (<10-3) or (< 1 in 1000 Rx)

Expected by user to be rare (maintenance). If not expected, when it does happen, results in reduced user confidence in the product/company (e.g. customer is very annoyed or dissatisfied).

When occurring, would result in a Minor disruption to production line; possible decrease in yield, such as need for sort or repair. So little scrapping that it max one unit or <$100 per year.

4 Frequent (Freq)

Should be expected (>5*10-3) (> 1 in 200 Rx) Likely to happen often during service life of

each instrument, such as small Alkaline batteries replaced on a monthly or bi-monthly bases. Disaster if not expected.

This does happen, and is be a local process issue, but should not stop production line; no rework, no scrap. Such as notice and replace bad part quickly with no process delay.

Probability Customer View Mfg Process

39

© B. Craner 2004-2017

Probabilities by Level (Can We Define/Defend?)

Scale Definition Use and Design Clinical Effects; Process End Effects (Customer) Process Local Effects

1 Inconceivable (Inc)

Does not happen for nearly any device during its service life for this model. (4* 10-6) (< 1 in 250,000 Rx)

Not practical that such an event happens. Extreme surprise this would happen.

Extreme low expectation this will happen during manufacturing.

If it does it would be catastrophic, would stop production, may do so with or without warning. >$100,000?

2 Remote (Rem)

May happen one per device during useful life. (2x10-4) or (< 1 in 50,000 Rx)

On average, some surprise if this happens for any device in field, with slight customer slightly annoyed, inconvenience.

Very low expectation this would happen. If it does, Moderate disruption to production line; product entered into the non-conforming material system with a portion to all being scrapped, <$2000 scrapping.

3 Probable (Pos)

Happens only a few times in a product lifecycle. (<10-3) or (< 1 in 1000 Rx)

Expected by user to be rare (maintenance). If not expected, when it does happen, results in reduced user confidence in the product/company (e.g. customer is very annoyed or dissatisfied).

When occurring, would result in a Minor disruption to production line; possible decrease in yield, such as need for sort or repair. So little scrapping that it max one unit or <$100 per year.

4 Frequent (Freq)

Should be expected (>5*10-3) (> 1 in 200 Rx) Likely to happen often during service life of

each instrument, such as small Alkaline batteries replaced on a monthly or bi-monthly bases. Disaster if not expected.

This does happen, and is be a local process issue, but should not stop production line; no rework, no scrap. Such as notice and replace bad part quickly with no process delay.

Probability Customer View Mfg Process

40


November, 2017

© B. Craner 2017 21

© B. Craner 2004-2017

Risk Analysis #1

Hazard Identification, and Hazard Analysis

Process Orientation

41

© B. Craner 2004-2017

Hazard Example Comments u  Identification of the Hazards – Potential for Harm

l What types of hazards are possible? l Product Hazards for example:

l  Potential for Patient Harm l  Potential for User Harm l  Potential for Facility Damage l  Potential for Equipment Damage

l Process Hazards for example: l  Potential for Operator Harm l  Potential for Equipment Damage l  Potential for Facility Damage l  Potential for stock loss

l Use Hazards (to patient, Operator by Use/IFU Error)?

Did you do this Hazard

Identification with sufficient

thoroughness?

All Hazards Identified?

“Possible sources of Harm”

42


November, 2017

© B. Craner 2017 22

© B. Craner 2004-2017

u  Table E.1 Examples of Hazards Create your own checklists from this table

43

© B. Craner 2004-2017

Hazard Assessment Record u  Table E.1 Examples of Hazards Example checklist - Template

Was this an appropriate

list of potential hazards

for this product or

process?

44


November, 2017

© B. Craner 2017 23

© B. Craner 2004-2017

Hazard Assessment Record u  Table E.1 Examples of Hazards Example checklist - Completed

Were the right

hazards

identified? Actions

documented

properly?

45

© B. Craner 2004-2017

Product Hazard à Hazardous Situation à Harm… Hazards listed, appropriately analyzed for Risk?

As with other analyses, many formats possible


November, 2017

© B. Craner 2017 24

© B. Craner 2004-2017 47

!!!! !

!

Sabotage Broken or Leaking

Dispensing Nozzle

Broken or Leaking Solvent

Container

Poor Ventilation

Around Solvent

Over- Dispensing of Bonding

Solvent

Improper Setup of

Evac System

Wrong “Titrator” System

Nozzle cracked

(worn out)

O-Ring (Seal) Failure

Valve Over Pressure

Spec

HVAC out of Spec

Nozzle Improperly Installed

Dropped Container

Container Cracked

under stress

High Pressure in Container

Wrong Workstation

Configuration

Operator over-

dispenses

Titrator Volume Limit

Defective

Operator over-

dispenses

Operator High Solvent

Exposure

Wrong Instructions

Training Not

Effective

Kitting Error

Specification Error

Risk Analysis #2

Fault Tree Analysis Product

Top Down

Hazard-Based

Top Level Causes

Next Level Causes

© B. Craner 2004-2017

Hazard à Fault Tree Example •  Analyze the Single Hazard

–  From the Hazard Analysis (and list of hazards) –  Example “Loss of Sterility”

•  Results of Brainstorming possible causes of Hazard –  List all causes derived (will rank them later) examples

•  Torn packaging •  Centrifuge Seal Failure •  Failed filter •  Used kit installed •  Improper antiseptic practices •  Improper aseptic procedure •  Failed relieve valve •  Disposable damaged while installing •  Centrifuge Drive mech/electr damage •  Improper installation of centrifuge •  Solvent Contaminated •  Improper sterilization •  Unintended puncture of disposable •  Unintended extrusion of disposable

u  Punctured packaging u  Used canister installed u  ReverseflowfromVentSystemu  Mfg process failure to decontaminate u  Compromisedfiltermembraneu  Improper collection of canisters u  Brokencomponentu  Contaminated vent system u  Excessive pressure from vacuum pump u  Membrane damaged during installation u  Hose damaged (faulty pump or valve) u  Improper packaging, handling, shipping u  Contaminateddisposablesetu  Unintended kink / fracture of disposable

Did the Fault Trees have

the right Hazards?

48


November, 2017

© B. Craner 2017 25

© B. Craner 2004-2017

Hazard Causes à Fault Tree Example •  Analyze the Single Hazard

–  From the Hazard Analysis (and list of hazards) –  Example “Loss of Sterility”

•  Next, Using brainstorm results (all causes): –  Highlight all “top causes” (directly causing hazard)


u  Punctured packaging u  Used canister installed u  Reverse flow from Vent System u  Mfg process failure to decontaminate u  Compromised filter membrane u  Improper collection of canisters u  Broken component u  Contaminated vent system u  Excessive pressure from vacuum pump u  Membrane damaged during installation u  Hose damaged (faulty pump or valve) u  Improper packaging, handling, shipping u  Contaminated disposable set u  Unintended kink / fracture of disposable

Top causes

Lower causes

Hazard

Did they have the right

Top Causes?

49

© B. Craner 2004-2017

•  Analyze the Single Hazard –  From the Hazard Analysis (and list of hazards) –  Example “Loss of Sterility”

•  Next, Select lower causes from the list of causes –  List of all causes derived:


u  Punctured packaging u  Used canister installed u  Reverse flow from Vent System u  Mfg process failure to decontaminate u  Compromised filter membrane u  Improper collection of canisters u  Broken component u  Contaminated vent system u  Excessive pressure from vacuum pump u  Membrane damaged during installation u  Hose damaged (faulty pump or valve) u  Improper packaging, handling, shipping u  Contaminated disposable set u  Unintended kink / fracture of disposable

Top causes

Lower causes

Hazard

Did you catch all of the

right lower-causes?

Hazard Causes à Fault Tree Example

50


November, 2017

© B. Craner 2017 26

© B. Craner 2004-2017

!!!! !

!


Dispensing Nozzle


Container

Poor Ventilation

Around Solvent


Solvent

Improper Setup of

Evac System


Nozzle cracked

(worn out)


Valve Over Pressure

Spec

HVAC out of Spec


Dropped Container

Container Cracked

under stress


Wrong Workstation

Configuration

Operator over-

dispenses


Defective

Operator over-

dispenses


Exposure

Wrong Instructions

Training Not

Effective

Kitting Error

Specification Error

Process FTA Example Hazard to Operator, or Property, or Environment First Branch

Process Fault Tree

Reasonable?

!!!! !

!


Dispensing Nozzle


Container

Poor Ventilation

Around Solvent


Solvent

Improper Setup of

Evac System


Nozzle cracked

(worn out)


Valve Over Pressure

Spec

HVAC out of Spec


Dropped Container

Container Cracked

under stress


Wrong Workstation

Configuration

Operator over-

dispenses


Defective

Operator over-

dispenses


Exposure

Wrong Instructions

Training Not

Effective

Kitting Error

Specification Error

51

Operator High Solvent Exposure

© B. Craner 2004-2016

© B. Craner 2004-2017

!!!! !

!


Dispensing Nozzle


Container

Poor Ventilation

Around Solvent


Solvent

Improper Setup of

Evac System


Nozzle cracked

(worn out)


Valve Over Pressure

Spec

HVAC out of Spec


Dropped Container

Container Cracked

under stress


Wrong Workstation

Configuration

Operator over-

dispenses


Defective

Operator over-

dispenses


Exposure

Wrong Instructions

Training Not

Effective

Kitting Error

Specification Error

Process FTA Example Hazard to Operator, or Property, or Environment

First Branch Process Fault Tree

Reasonable?

52

Same for Product Fault Tree: Was Product Fault Tree Reasonable?


November, 2017

© B. Craner 2017 27

© B. Craner 2004-2017

Risk Analysis #3 Failure Modes and Effects

Analysis Product, Process, Use

Bottom Up Component/Event - Based

53

© B. Craner 2004-2017

Failure Modes and Effects Analysis (FMEA) •  “Bottom-Up Analysis” •  Analyzing each (Can you explain/defend them?)

–  dFMEA: Component/Subsystem (part fails – device error) –  pFMEA: Process Step (Process heating failure – too hot) –  uFMEA: Use Step (Nurse performs step out of order)

•  Potential low level failures –  Impact on Local, Larger System –  Estimation of Severity, Frequency, Detectability –  Determination of Threshold:

•  Failure Modes needing Risk Reduction •  Failure Modes not needing Risk Reduction

54


November, 2017

© B. Craner 2017 28

© B. Craner 2004-2017

Product FMEA Example - Generic

55

Numeric Rankings

© B. Craner 2004-2017

Product FMEA Example - Generic

56

CAT

SER

CAT

CAT

CAT

SER

REM

REM

I NC

I NC

I NC

FREQ

FREQ

FREQ

REM

REM

REM

REM

I NTOL

I NTOL

I NTOL

ACC

ACC

ACC

Attribute Rankings


November, 2017

© B. Craner 2017 29

Remember the Example Risk Management Review?

Company ABC Quality System

Management Presentations October, 2016

Did you do such a review?

5710/2016 Company Confidential10/2016 Company Confidential© B. Craner 2004-2017

© B. Craner 2004-2017

General Agenda GeneralAgenda–ManagementReviewQ1-Q22016

§  GeneralManagementOverview

§  RiskManagementSystemEffec8venessß§  Equipment/FixturesCalibra8on&Maintenance§  DurableDeviceManufacturingProcess§  SingleUse-DeviceManufacturingProcess§  ManufacturingMetrics§  SupplierQuality§  Non-ConformingMaterials§  ServiceProcess§  CAPA§  ProductDevelopment§  QualitySystemMetrics§  QualitySystemImprovementProject/PlansfromPreviousReview§  ChangestoQualitySystemandRegulatoryReq’ts§  QualityPolicyandSummary–Effec8venessofQualitySystem

10/2016 Company Confidential

àA#endeesPresidentVPR&D

VPOpera4onsVPofQA/RA(Mgt.RepresentaDve)

ControllerHumanResourcesDirector

ServiceManager MaterialsDirector

ManufacturingEngineeringManagerOthers...

Right topics?

Right Attendees?

58


November, 2017

© B. Craner 2017 30

© B. Craner 2004-2017

RiskManagementEffec4veness

ABCRiskManagementSystemRiskManagementReview

ManagementRepresenta8ve

Right presenter?


© B. Craner 2004-2017

RiskManagement–Reminder:RegulatoryChanges

ISO 14971:2012 Revision of Risk Management Standard

§  Change 1: ALARP may not be used as a risk management policy. It is not possible to include the use of economics in determining acceptable risk. (ALARP à ACC)

§  Change 2: Each risk must be reduced until it is not possible to reduce the risk anymore, e.g., That is multiple risk controls must be applied until it may be shown (through documentation) that no further risk reduction is possible (AFAP).

§  Change 3: Information for Use cannot be used to reduce risk. §  IFU/Labeling (However, Training may be used to reduce risk)

§  Change 4: Each risk and the overall risk must have a documented benefit-risk analysis.

Bottom Line: Notified Body is source for final judgment on risk management, acceptance of company risk management systems.

Right Discussions?



November, 2017

© B. Craner 2017 31

© B. Craner 2004-2017

u Risk Management (product, process, use) l  Risk Mgt Planning l  Hazard Analysis l  Fault Tree Analysis l  Failure Modes and

Effects Analysis l  HACCP l  Risk:Benefit Analysis l  Risk Mgt Rpt

•  Production and field product performance •  Legacy product engineering •  New Product Design

–  Bench Studies –  Prototype Studies –  Final Studies

•  CAPA •  Field action •  Product investigation •  Complaints/MDR/AE •  Complaint investigation •  Calibration, Maintenance •  Nonconforming materials •  Standards and Regulations •  Competitive product information •  Clinical information (literature, etc.)

Metrics: based on these interactions and goals

RiskManagement:Contribu4ngsystems

Right Information,

Metrics?


© B. Craner 2004-2017

Product Realization & Risk Management

Product Realization Risk Analysis Tools

Product Development

Risk Analysis Inputs

Animal/Cadaver Study

Bench Study

Clinical Study

Hazard Analysis

Regulatory Standards

Performance Standards

Literature, Competitive Product Info

Launched Product (Legacy & New)

Risk Mgt Rpt

Design FMEA

Process FMEA

Use FMEA

Sustaining Product Engineering




Literature, Competitive Product Info

Complaints/MDRs

Internal & External Audits

Process Data (NCMRs, Yields)

CAPA

Product Realization Product

Specifications

Requirement Specifications

Device Master Record

FeedbackfromthesesourcestoRiskManagementProcess

Were These Analyzed? Were These Analyzed? Did the information guide Design and Improvement?


Process Information


November, 2017

© B. Craner 2017 32

© B. Craner 2004-2017

Test Results as Input to Product Development Risk Management

Product Development


Animal/Cadaver Study

Bench Study

Clinical Study



Literature, Competitive Product

Info

Launched Product (Legacy & New)

§  Bench testing of Product A output: Modified math for increased accuracy

§  Bench testing of Product B: S/W change to prevent data loss when power disconnected prior to completion of write cycle

§  Distribution Testing on Product A Tubing Set: Changed to heavier pouch materials

§  Animal Testing with Product C Hand-piece: Addition of indicators for adequate electrode deployment

§  10 more slides from the other Risk Mgt inputs à


Were there other information sources missed? If product problems, where would the audit focus?

© B. Craner 2004-2017

Suggested Action Items §  IEC 62304: Capitalize on the good work already done for SW

Development, by completing the process for SW Maintenance: •  Establish a feedback procedure •  Update Software Change Request Procedure, or •  Create new Problem Reporting SOP to deal with Problem

Requests and Change Requests (for SW, HW, & Mechanical; not just SW)

§  IEC 60601-1: Streamline process to make it efficient for the PM & QE to complete & faster for the Safety Agency to review •  Revise the RMF SOP to address issues at a global level. •  Create IEC 60601-1 Main Checklist & ISO 14971

Checklist templates populated with pointers to mandatory IEC 60601-1 items

§  Add a likelihood of occurrence scale for disposables to RMF

Were there Actionable Tasks?

Progress on these tasks?



November, 2017

© B. Craner 2017 33

© B. Craner 2004-2017

Conclusion on Effectiveness of Risk Management System

The Risk Management System at Company ABC is deemed effective.

Based on complaint trends, no recommendations for field action based on risk analysis have been made during Q1 & Q2 2016.

Were there Appropriate Conclusions?


© B. Craner 2004-2017

What and Why Audited?

66


November, 2017

© B. Craner 2017 34

© B. Craner 2004-2017

Reasons to be Audited? 1.  Your product is causing problems.

–  Harming People, Property, Environment –  Perception high-impact product (autos) might cause harms.

2.  Regulatory Compliance issues –  Mistake seen during audit –  Mistake seen during document review

3.  Intended: part of a regular audit –  Auditors are now getting trained. –  Auditors are not yet fully trained but curious “Danger, Will Robins”

4.  Unintended: comes up in a product review –  Risk Management File part of Design Process

5.  Holes, errors in your Risk Management System? –  Review your RM policy, process, documents, training

67

© B. Craner 2004-2017

Examples of Auditable Risk Management Items1

•  Specs, Contracts: RFQ and Contract Review Process, e.g., any special requirements, critical requirements.

•  Top Management: Does Top management clearly understands their “Risks” and what is being done to ensure mitigating those “Risks”? In Management Review?

•  Risk Mgt Tools: Are the Risk Management Tool effective? •  Risk Communicated: Are “Risks” communicated and

managed throughout the organization e.g. Design, Planning, Purchasing, Suppliers, Manufacturing, Inspection, Delivery and Post Delivery – within and from Management Review?

•  Design Control: Design inputs, Design Hazard Analyses, Design Fault tree Analyses, Design FMEAs, Design Verification and Validation.

(Adopted from Kimberly Maggie, Ron Tarach, QUAL-TECH, INC., 2010)

68


November, 2017

© B. Craner 2017 35

© B. Craner 2004-2017


•  Critical Product Characteristics: Critical characteristics across the product lifecycle, ensuring the Process FMEAs and Control Plans are linked.

•  Risk Feedback Information Channels: Processes in place for capturing leading and lagging indicators related to Design Quality Performance.

•  Continual Improvement Sought / Needed? –  Evaluate whether the organization has closed loop

Continual Improvement Processes that captures and sustains Product and Process Quality.

–  Organization is using Lessons Learned and Best Practices. (Adopted from Kimberly Maggie, Ron Tarach, QUAL-TECH, INC., 2010)

69

© B. Craner 2004-2017


•  Change Management Attuned: –  Does the organization’s Change Management Process

involve the right people at the right time with the right process/product.

–  Has Change Management been integrated with assessments to ensure correct consideration of “Risk”.

•  Risk Assessment Follow Through: Is “Risk Assessment” tracked, recommended controls to completion and ensuring “Risks” were mitigated as prescribed.

•  Residual Risk Followed: Are controls in place and followed for “Risk” still remaining after mitigation actions. (Adopted from Kimberly Maggie, Ron Tarach, QUAL-TECH, INC., 2010)

70


November, 2017

© B. Craner 2017 36

© B. Craner 2004-2017

ISO 19011 Quality Management System Auditing

•  ISO 19011 – Provides guidance on: – Auditing principles – Managing audit programs – Conducting internal & external audits – Competence of auditors –  Input, Output, Process – Plan Do Check Act

71

© B. Craner 2004-2017

Audit Process Flow - 19011 Establishing the Audit

Program

- objectives and extent

- Responsibilities

- Resources

- Procedures

Implementing the Audit Program

-  Scheduling the audits

-  Evaluating auditors

-  Selecting audit teams

-  Directing audit activities

-  Maintaining records

Improving the Audit Program

Competence and evaluation of

auditors

Audit Activities

Authority for the audit program

Act

Plan

Do

Check

Homework:

Read ISO 19011

Monitoring and Reviewing the Audit Program

-  Monitoring and Reviewing

-  Identifying needs for CAPA

-  ID opportunities for imprvt

72


November, 2017

© B. Craner 2017 37

© B. Craner 2004-2017

Quality System Regulation •  October 1996; Design Control 6/98; UDI 2016 •  Current U. S. Requirements for Quality Systems •  Audits based on this document alone for FDA •  Notified Body audits >90% content from QSR •  Risk Management Audit by NB, soon by FDA •  Note FDA description of GMP/QSR

–  Excellent description –  Excellent resource list –  http://www.fda.gov/medicaldevices/

deviceregulationandguidance/postmarketrequirements/qualitysystemsregulations/

73

© B. Craner 2004-2017

Audit Plan •  Procedures: Review Risk Mgt procedures, specifications and work

instructions;

•  Personnel: Interview personnel responsible for / involved with audited elements;

•  Procedural Compliance: Examine work areas and work in progress for evidence of compliance to Risk Management procedures and work instructions;

•  Process Controls: Examine process controls (e.g., cited in pFMEA) and records to determine compliance with requirements;

•  Gather Objective Evidence: needed to support the audit results. The Internal Auditor ensures that objective evidence has sufficient depth necessary to determine if Risk Management elements are effectively implemented;

•  Communicate: Share the outcome of the audit all people involved, including Risk Management Review. à

74

Audits, each of these areas appropriate?

Critical – were outcomes communicated (good/ and needs for improvement)?

Next: Workshops for You and Your Company


November, 2017

© B. Craner 2017 38

© B. Craner 2004-2017

Workshop Question 1 •  Risk Management System Scenario: Product

design process for Invasive radio frequency ablation system (electrode catheter connected to an RF Ablation Generator) in which several risk management tools will be used (Hazard Analyses, Fault Tree Analyses, Failure Modes and Effects Analyses, Risk Benefit Analyses, Risk Management Plan, and Risk Management Report).

•  Workshop Question: What actions would you take to ensure your documents were "audit-proofed", that is how would you ensure your risk analyses and other documents would be clear, competent, audited without observation?

75

© B. Craner 2004-2017

Workshop Question 2 •  Risk Management Problem Scenario: There are

three alleged problematic cases of over-destruction of left ventricle cardiac tissue by an Invasive Catheter, energized by Radio Frequency Ablation delivery system product in the field, two resulting in deaths, and one requiring the implantation of a cardiac pacemaker.

•  Workshop Question: Where do we focus in the risk management system audit? –  Select two risk management analyses –  Suggest what might have happened in the RM system to

miss catching this risk before product launch. What should we have done differently?

76


November, 2017

© B. Craner 2017 39

© B. Craner 2004-2017

Workshop 3 – Question 1 •  Risk Management Audit – Basic: Your Risk

Management System will be audited for ~2-hours. •  Workshop Question: Where will the auditor focus

with no previous knowledge of product or process risk issues? Critique the intensity below: –  Risk Management Procedure – does it meet the

requirements of the FDA and ISO 14971:2012,and of the directives?

–  Risk Management Review – was it performed during the Management Review?

–  Briefly assess an FMEA at random (product, process, or use/applicability).

–  Other items, fewer??

77

© B. Craner 2004-2017

Workshop 3 – Question 2 •  Risk Management Audit – Moderate: Your Risk

Management System will be audited for ~4-hours. •  Workshop Question: Where will the auditor focus

with some previous knowledge of product or process risk issues? Critique the intensity below: –  Risk Management Procedure –  Risk Management Review –  Assess a Hazard Analysis from a known product of

interest. –  Assess a design/product FMEA from a known product of

interest. –  Assess a process FMEA from a known process created

for a product of interest. 78


November, 2017

© B. Craner 2017 40

© B. Craner 2004-2017

Workshop 3 – Question 3 •  Risk Management Audit – Thorough: Your Risk

Management System will be audited for a day. •  Workshop Question: Where will the auditor focus

with previous knowledge of product or process risk issues (significant complaints/MDR/AE in the field)? Critique the intensity below: –  Risk Management Procedure, Risk Management Review –  Assess a Hazard Analysis from a known product of

interest at some depth, focusing on reported/unreported significant harms from hazards identified/unidentified.

–  Assess design/product FMEAs from the known product(s) of interest, looking for controls seen as inadequate.

–  Assess process FMEA’s from known process created for product(s) of interest, looking for inadequate controls.

79

© B. Craner 2004-2017

Workshop Question 4 •  Risk Management Audit: Order of most to least

critical Risk Management Items to be audited. •  Workshop Question: Rank the items listed below

(and others) as High (1), Moderate (2), Low (3): q Preliminary Hazard Analysis q Risk Management Review (latest) q Risk Management Review (just earlier) q Risk Management Procedure(s) q Current approved Hazard Analysis q Current approved Hazard Analysis with heavy complaints/MDR/AE q Design FMEA randomly selected q Product FMEA randomly selected q Use/Application FMEA randomly selected q Design FMEA from the known product(s) of interest, looking for controls seen as inadequate. q Product FMEA from the known product(s) of interest, looking for controls seen as inadequate. q Use FMEA from the known product(s) of interest, looking for controls seen as inadequate. q Assess process FMEA’s from known process created for product(s) of interest, looking for

inadequate controls. q Other __________________________ q Other__________________________

80

designing out unacceptable risksocalqualityconference.com/wp-content/uploads/2018/01/e02.pdf•...

Documents