process validation (stage 2) acceptance criteria and ... · process validation (stage 2) acceptance...

Process Validation (Stage 2)

Acceptance Criteria and

Sampling Plans

Jacalyn L. Schroeder

Statistician, Validation/Quality

Engineering Specialist

J.L. Schroeder , 3M, June 2015 4

• Sampling Plans to Meet Validation Stage 2

Objectives

• Understanding of sampling plan performance

levels, confidence levels, and associated risks

• Utilization of product and process understanding

studies (PPU) to determine sample size and

acceptance criteria needed for validation study

• Confidence statements to be made when passing a

validation study

• Interactive Exercises (Throughout)

AGENDA

Sampling Plans

• A sampling plan is a

procedure for making

accept/reject decisions:

– Take 300 samples from a lot

and accept if there are zero

defects

• Includes both a sample size

and acceptance criteria

Resource: Dr. Wayne Taylor

Validation Sampling Plan Class -

2008


•Accept = Pass validation study and go into production

•Reject = Fail validation study and must improve the product or process

Sampling Plans - Validation



2008

Objective of Validation

• Establishing by objective

evidence that a process

consistently produces a result

or product meeting its

predetermined specifications

(21 CFR Part 820)



2008

J.L. Schroeder , 3M, June 2015

Validation Stages

A Lifecycle Approach

Process Design

Stage 1

Process Qualification

Stage 2

Continued Process Verification

Stage 3


• Per 21 CFR 820.250 – “Sampling plans,

when used, shall be written and based

on a valid statistical rationale”

• A sampling plan can be used to

demonstrate that a required

performance level (protection for

consumer) is met

Sampling Plans to Meet

Validation Stage 2 Objective(s)



Validation Stage 2 Objectives

• CGMP regulations regarding sampling set

forth a number of requirements for

validation:

• Samples must represent the batch under analysis

(§ 211.160(b)(3))

• Sampling plan must result in statistical confidence

(§ 211.165(c) and (d))

• Batch must meet its predetermined specifications

(§ 211.165(a)).”


• For validation we want a sample to represent production

• Need Representative Sample: Stratified and periodic sampling is desired over complete random sampling. Keep samples in time order for analysis.

• “Handful Sampling” is not representative sample

• Remember that validation is confirmation:

• Should not enter validation until you have an understanding that process is capable

– Design verification should verify

– Estimate of Ppk > 1




Statistical Based Sampling Plans

(Based on OC Curve and Two Points)

(AQL, 1-α)

RQL, β)

Each sampling plan has it’s own OC curve


• The AQL of a sampling plan

is a level of quality (percent defective, defects

per hundred units, etc.) routinely accepted by

the sampling plan

• If you want to be assured of passing the

validation study, your process percent

defective needs to be below the sampling

plan’s AQL

What is AQL?

AQL

What is routinely accepted 0.95


• The LTPD of a sampling plan is a level of quality

(percent defective, defects per hundred units, etc.)

routinely rejected by the sampling plan

• Let LTPD0.05 (LTPDβ) be designated as the level of

performance (or performance level) that we want to

be below in order to pass the validation

What is LTPD (AKA RQL)?

LTPD

.05 or .10

What is routinely rejected

Why is AQL/LTPD Important?

• Why is AQL important?

– Effects chances of passing (Higher defective, need higher AQL to pass)

– Effects overall cost of validation

• Why is LTPD important?

– Products or processes at or worse than the LTPD fail most of the time

– Provides plan protection

• Example of Different Plans

– All these plans have different chances of passing (AQL) but provide the

same protection (LTPD0.05)

Sampling Plan AQL LTPD0.05

n=300, a=0 0.017% 1.0%

n=470, a=1 0.076% 1.0%

n=630, a=2 0.13% 1.0%

n=770, a=3 0.18% 1.0%23

Steps To Choose Statistical Based

Validation Sampling Plans to Meet



Steps to Choose Statistical Based

Sampling Plan for Validation


1. Determine Protection Required

a. Select Performance Level (What will FAIL

plan?). Select sampling plans that have

Performance level = LTPDβ =LTPD.05

b. Select Confidence Level (1-β)

2. Select Best Sampling Plan based on

estimated AQL (Ppk for variables data) and

sample size

Steps To Choose Validation Sampling Plans

Resource: Dr. Wayne Taylor Validation

Sampling Plan Class – 2008

Step 1a:

Select Performance Level

• Performance Level should be selected

based on risk

• Should be consistent with AQLs used in

manufacturing

– Validated performance level should also be

consistent with on-going inspection performed

during manufacturing



2008

Step 1a: Performance Levels

[LTPDβ] for PQ

• Commonly used in pharmaceutical and medical device industries for disposablesare:

– For Defect (Health and Safety)0.065%, 0.1%

– For Defects (Functional)0.25%, 0.4%, 0.65%, 1.0%

– For Defects (Cosmetic)2.5%, 4.0%



2008 29

Step 1a: Performance Levels

[LTPDβ] for PQ

• Commonly used in medical device industry for hardware and very expensive devices are:

– For Critical Defect (Health and Safety)0.1% (99.9% reliability), 0.3% (99.7% reliability), 1% (99% reliability)

– For Major Defects (Functional)1% (99% reliability), 3% (97% reliability), 5% (95% reliability)

– For Minor Defects (Cosmetic)5.0% (95% reliability), 10% (90% reliability)



2008

Step 1a: Performance Levels at LTPD.05

Risk OQ PQ

Critical 5% (95% Conformance) 1% (99% Conformance)

Major 10% (90%

Conformance)

3% (97% Conformance)

Minor 20% (80%

Conformance)

5% (95% Conformance)

31

Resource: Taylor Enterprises

Statistical Procedures For

Medical Device Industry 2013


�Confidence Level Recommendations:

�Recommend using Consumer Risk (β) of

� β=5% or 10%

�Equates to (1-β)

�95% or 90% Confidence Level

Step 1b:

Determine Confidence Level (1-β)

to make CONFIDENCE STATEMENT

Confidence Statements

• Conclusion when pass a sampling plan

• Sampling plans are designed to demonstrate that the product tested meets a specified performance level (LTPDβ)with certain confidence (typically 1 – β or 95%)

– Passing the plan n=300, a=0 allows one to state: “With 95% confidence the defect level is below 1% defective.”

34



2008 LTPD


�For Variables Data: Use estimated

Ppk to estimate AQL

�For Attribute Data: Estimate quality

level (typically in percent defective)

that customer will find acceptable

95% of the time

Step 2: Determine BEST Sampling

Plan based on ESTIMATED AQL


• Choose Attribute or Variables Plans

• Use Validation Sampling Plan Tables

(www.variation.com)

Step 2:

Select Validation Sampling Plans

36

Tables of Plans

• From Design Verification and Process Validation Sampling Plans SOP

• LTPD:

– 20%, 10%, 6.5%, 5%, 4%, 3%, 2.5%, 1.5%, 1%, 0.65%, 0.4%, 0.3%, 0.25%, 0.15%, 0.1%, 0.065%

• Tables:

– Attribute 90% Confidence

– Attribute 95% Confidence

– Variables 1-Sided 95% Confidence

– Variables 2-Sided 95% ConfidenceResource: Dr. Wayne Taylor


2008


LTPD = 1%

Attribute Plans with 90% Confidence Attribute Plans with 95% Confidence

Type Parameters AQL LTPD 0.1 Type Parameters AQL LTPD 0.05

Single n=230, a=0 0.02% 1% Single n=300, a=0 0.02% 1%


Double n1=250, a1=0, r1=2, n2=400, a2=2 0.11% 1% Double n1=325, a1=0, r1=2, n2=400, a2=2 0.09% 1%





Variables – 1-sided – 95% Confidence Variables – 2-sided – 95% Confidence

Parameters AQL LTPD 0.05 Parameters AQL LTPD 0.05

n=15, Ppk=1.17 0.00017% (Ppk=1.55) 1% (Ppk=0.78) n=15, Ppk=1.17, Pp=1.17 0.00016% (Ppk=1.55) 1% (Ppk=0.78)







n=100, Ppk=0.89 0.13% (Ppk=1.01) 1% (Ppk=0.78) n=100, Ppk=0. 90, Pp=0.97 0.11% (Ppk=1.02) 1% (Ppk=0.78)

Tables Developed from Validated “Sampling Plan Analyzer”

– Version 2.0

Copyright 2001 Taylor Enterprises Inc.


Summary:

Steps to Choose Statistical Based

Sampling Plan for Validation


• In order to pass a validation study ,

the process must be significantly

better than the validation performance level

• Two Questions You Should Always Ask:

– Answered by LTPD:

• If I pass the validation study, what confidence

statement can I make?

– Answered by AQL:

• If my process is good, what are the chances of

passing the validation study?

Key Points



200840

• Desire to validate a product characteristic (disposable medical

device) with risk of MAJOR.

• Per local SOP, to validate a characteristic that could have an effect

on functionality (MAJOR risk), the protection required is

– Performance Level: LTPD0.05 =1%

– Confidence Level (1 – β = .95) or 95%

• To determine BEST Sampling plan, SOP states to estimate AQL:

– Variables data: An estimated AQL may be determined by

estimated process Ppk.

– Attributes data: An estimated AQL may be determined from

history or estimated from experimental runs.

• Engineer will use sampling plan tables in local SOP to determine

sample size and acceptance criteria for validation

Exercise

41


• Variables data

• Disposable medical device with risk of MAJOR.

• Experimentation determined estimated Ppk for CQA of 1.26

• SOP requires Performance Level of LTPD0.05 =1%

• Engineer goes to LTPD0.05 =1% sampling plan tables to

determine sample size and acceptance criteria

Exercise

Sampling Plan AQL LTPD 0.05

n=15, Ppk=1.17 0.00017% (Ppk=1.55) 1% (Ppk=0.78)

n=20, Ppk=1.10 0.0012% (Ppk=1.41) 1% (Ppk=0.78)

n=30, Ppk=1.02 0.008% (Ppk=1.26) 1% (Ppk=0.78)

Sample Size &Ppk to pass plan

Ppk value that will fail

Process Ppk estimate (accept 95% of time)

Resource: Dr. Wayne Taylor Validation

Sampling Plan Class - 2008


LTPD = 1%





















– Version 2.0



• Suppose:

– 2-sided Spec

– LTPD.05 = 1%

– Have processwherePpk = 1.28

• What sample size should be selected to provide 95% of passing?

• What is corresponding AQL?

• What confidence statement can be made?

Interactive Exercise


• Suppose:

– 1-sided Spec

– LTPD.05 = 1%

– Have processwherePpk = 1.05

• What sample size should be selected to provide 95% of passing?

• What is corresponding AQL?

• What confidence statement can be made?


Using Selected Plan in Stage 2

Validation Protocol


• Document Sampling Plan and Analysis Methods in Protocol

• Document PPU work to determine estimated process capability. Ppk should be > 1.0

• Make sure that work has been completed to show that good product can be made at extremes; validate at nominal

• Select Sampling Plan based on AQL or Ppkestimate for variables data

• Document Confidence Statements to meet Validation Objectives.

Document and Execute


• “Per 95% Confidence Sampling Plan for

LTPD.05 = 1%, the validation will pass for

fill volume if the Ppk is 1.11 or greater.”

• If this validation passes one can state

with 95% confidence that the process for

controlling fill volume is less than 1%

defective.”

49

Document Confidence StatementIn Validation Protocol


Example – PQ Protocol



Prod_W

Prod_X

Prod_Y

Prod_Z

All Greater than

Ppk of 1.0

53

LTPD = 1%




















Tables Developed from Validated “Sampling Plan Analyzer” – Version 2.0



All

Greater

than

Ppk of 1.0


• Data

• Analysis and Conclusions

–Was acceptance criteria met?

–Did validation pass?

–What confidence statement can be

made if criteria was met?

Results: Validation Report


Example – PQ Report

Prod_W

Prod_X

Prod_Y

Prod_Z


Example – PQ Report

Prod_W


• Critical to Quality Attribute – Attribute Data

• Validation Plan requires LTPD0.05 = 1% and

95% Confidence Level

• AQL estimate from PPU: 0.09%

• Determine:

– Sample Size?

– Ppk to Pass Validation?

– What confidence statement can be made for

Attribute CQA if validation passes?

Break Out Exercise


LTPD = 1%





















– Version 2.0

Copyright 2001 Taylor Enterprises Inc.AQL estimate from PPU: 0.09%


Break Out Exercise

• Critical to Quality Attribute – Variables Data (2-

Sided Specification)

• Validation Plan requires LTPD0.05 = 0.4% and 95%

Confidence Level

• Estimated Ppk : 1.31

• Determine:

• Sample Size?

• Ppk to Pass Validation?

• What confidence statement can be made for Variables

CQA if validation passes?

60Estimated Ppk : 1.31


Questions?


Jacalyn L. Schroeder – 3M Company

Title: Quality Engineering Specialist; MS

Applied Statistics 2013

E-mail: [email protected]

Work Phone: 605-696-1355

Contact Information


1. Dr. Wayne A. Taylor, “Guide to Acceptance Sampling”, version 1, www.variation.com, Taylor Enterprises,Inc. Note: Version 2 of new book 2014-2015.

2. GHTF/SG3/N99-10:2004 (Edition 2) , “Quality Management Systems - Process Validation Guidance”, January 2004

3. Validation Sampling Plan Course, Wayne Taylor, March 2008

4. Vladimir, Veselov, Helen Roytman, Lori Alquier, “Medical Device Regulations for Process Validation: Review of FDA, GHTF, and GAMP Requirements”, Journal of Validation Technology, spring 2012

References

process validation (stage 2) acceptance criteria and ... · process validation (stage 2) acceptance...

Documents