FDA Process Validation Guidance, Jan 2011

• Statistics mention 15 times• “statistical”• “statistics”• “statistically”• “statistician” – as a suggested team member

• Clear that FDA expects more statistical thinking in validation

• Some statisticians asked to be a team member may not be familiar with Quality Assurance applications and jargon• Acceptance Sampling• Statistical Process Control (SPC)• Process Capability

22

FDA Process Validation Guidance Overview

Process Validation: The collection and evaluation of data, from the process design stage through commercial production, which establishes scientific evidence that a process is capable of consistently delivering quality product.

33

FDA Process Validation Guidance Overview

The new Guidance specifies a lifecycle approach:

• Stage 1 – Process Design• Statistically designed experiments (DOE)

• Stage 2 – Process Qualification• Design of facility and equipment/utilities qualification• Process Performance Qualification (PPQ)

• SPC; Variance components; Acceptance Sampling; CUDAL, etc.• Number of lots required is no longer specified as three

• Must complete Stage 2 before commercial distribution

• Stage 3 – Continued Process Verification (CPV)• Continual assurance the process is operating in a state of control• Data trending, SPC, Acceptance Sampling, etc.• Guidance recommends scrutiny of intra- and inter-batch variation

44

Statistical Acceptance Criteria for Validation

Statistical confidence required may be based on…• Risk• Scientific knowledge• Criticality of attribute (AQL, etc.)• Prior / historical knowledge

• Stage 1 knowledge• Revalidation

• Is test an abuse test?

5

Provide X% confidence that the requirement has been met

Requirements: Process performance to consistently meet attributes related to identity, strength, quality, purity, and potency

5

Three Common Situations

Provide statistical confidence that…

1. A high percent of the population is within specification2. A population parameter is within specification

- Mean; Standard Deviation; RSD; Cpk/Ppk3. A standard test (UDU, Dissolution, etc.) will pass

66

Assure a High % of Population in Spec

“90% confidence at least 99% of population meets spec”“90% confidence nonconformance rate <1%”“90% confidence for 99% reliability”

Common statistical methods• Continuous data: Normal Tolerance Interval• Discrete data: High Confidence Binomial Sampling Plan

77

A Word about Confidence…

Which sampling plan provides more confidence?

1. n=90, accept=0, reject=1

2. n=300, accept=0, reject=1

3. n=2300, accept=0, reject=1

99% confidence ≥95% conforming

95% confidence ≥99% conforming

90% confidence ≥99.9% conforming

What you want to be confident of is usually more important than

how confident you want to be

88

Validation: High Degree of Assurance• Phrase “high degree of assurance” mentioned four times• “…the PPQ study needs to be completed successfully and

a high degree of assurance in the process achieved before commercial distribution of a product.”

ICH Q7A GMP for APIs:• “A documented program that provides a high degree of

assurance that a specific process, method, or system will consistently produce a result meeting pre-determined acceptance criteria.”

• Suggest 90% or 95% confidence is acceptable• This confidence is more related to Type II error and Power• Although α=0.05 / 95% confidence is common for Type I error, it is

not as common for power, where 80% and 90% also common.

99

Assure a High % is Within Spec

Variables data: Normal Tolerance Interval*

• Example: Show with 90% confidence that at least 99.6% of powdered drug fill weights meet spec of 505-535mg.

• Test n=50 bottles; 1 every 5 minutes for 4 hrs• Acceptance criterion: 90% confidence ≥99.6% meet spec• Variables data with average, s.d.: use tolerance interval method• must be within specification limits

• Why 99.6%? Production AQL is 0.4% for fill weight.

*other methods may be used, such as variables sampling; may give lower Type I error

ksMean

1010

Fill Weight Example

11

464136312621161161

530

520

510Indiv

idual V

alu

e

_X=519.31

UCL=529.10

LCL=509.52

464136312621161161

10

5

0

Movin

g R

ange

__MR=3.68

UCL=12.03

LCL=0

50403020100

522

516

510

Observation

Valu

es

532528524520516512508

LSL USL

LSL 505USL 535

Specifications

525520515510

Within

Overall

Specs

StDev 3.26386Cp 1.53Cpk 1.46

WithinStDev 3.24963Pp 1.54Ppk 1.47Cpm *

Overall

Filler Validation RunI Chart

Moving Range Chart

Last 50 Observations

Capability Histogram

Normal Prob PlotAD: 0.464, P: 0.245

Capability Plot

11

Fill Weight Example

• Process is in statistical control, normality not rejected• 90% confidence / 99.6% coverage tolerance interval:

• Pass: Tolerance interval lies within spec of 505 - 535• We can be 90% confident ≥99.6% of containers meet spec

• Will be able to pass in-process 0.4% AQL sampling• If process is stable, 90% confidence ≥95% of lots will pass

• Engineer friendly: tables or software can be used

)2.5304.508(25.335.331.519

ksx

1212

2-Sided Normal Tolerance Interval Factors

Two-Sided Normal Tolerance Limit k-Factors

n 95% 99% 99.6% 99.9% n 95% 99% 99.6% 99.9%

20 2.56 3.37 3.76 4.30 20 2.75 3.62 4.04 4.6130 2.41 3.17 3.55 4.05 30 2.55 3.35 3.74 4.2840 2.33 3.07 3.43 3.92 40 2.45 3.21 3.59 4.1050 2.28 3.00 3.35 3.83 50 2.38 3.13 3.49 3.9960 2.25 2.96 3.30 3.77 60 2.33 3.07 3.43 3.9270 2.22 2.92 3.27 3.73 70 2.30 3.02 3.38 3.8680 2.20 2.89 3.24 3.70 80 2.27 2.99 3.34 3.8190 2.19 2.87 3.21 3.67 90 2.25 2.96 3.31 3.78

100 2.17 2.85 3.19 3.65 100 2.23 2.93 3.28 3.75

% Coverage % Coverage

90% Confidence 95% Confidence

Mean ± 3.35s must be within spec limits

1313

1-Sided Normal Tolerance Interval Factors

14

One-Sided Normal Tolerance Limit k-Factors

n 95% 99% 99.6% 99.9% n 95% 99% 99.6% 99.9%

20 2.21 3.05 3.46 4.01 20 2.40 3.30 3.73 4.3230 2.08 2.88 3.27 3.79 30 2.22 3.06 3.47 4.0240 2.01 2.79 3.17 3.68 40 2.13 2.94 3.33 3.8750 1.97 2.73 3.11 3.60 50 2.07 2.86 3.25 3.7760 1.93 2.69 3.06 3.55 60 2.02 2.81 3.19 3.7070 1.91 2.66 3.02 3.51 70 1.99 2.77 3.14 3.6480 1.89 2.64 3.00 3.48 80 1.96 2.73 3.10 3.6090 1.87 2.62 2.97 3.46 90 1.94 2.71 3.07 3.57

100 1.86 2.60 2.96 3.44 100 1.93 2.68 3.05 3.54

90% Confidence 95% Confidence

% Coverage % Coverage

14

A problem with most normality tests

• Need to check for normality to use normal tolerance interval• Process quality data is often rounded

• Or data is “granular”

• Most normality tests will interpret rounding as non-normality• Example: n=100 from N(100,1.52)

15

Unrounded Rounded

100.071 100

98.238 98

99.122 99

99.190 99

100.623 101… , n=100 … , n=100

15

A problem with most normality tests

16

105.0102.5100.097.595.0

99.9

99

959080706050403020105

1

0.1

Unrounded

Perc

ent

Mean 99.78StDev 1.502N 100AD 0.379P-Value 0.400

Normal Probability Plot of Unrounded

Unrounded data: normality not rejected by Anderson-Darling test16

A problem with most normality tests

17

1041021009896

30

25

20

15

10

5

0

Rounded

Frequency

Mean 99.84StDev 1.536N 100

n=100, N(100, 1.5^2) Rounded to 0 Decimals

17

A problem with most normality tests

18

• Rounding data causes most normality tests to fail• SAS 9.2 Proc Univariate Tests:

Unrounded DataTest --Statistic--- -----p Value------Shapiro-Wilk W 0.98874 Pr < W 0.5643 Kolmogorov-Smirnov D 0.063184 Pr > D >0.1500Cramer-von Mises W-Sq 0.06213 Pr > W-Sq >0.2500Anderson-Darling A-Sq 0.378299 Pr > A-Sq >0.2500

Rounded Data (to whole numbers)Test --Statistic--- -----p Value------Shapiro-Wilk W 0.956808 Pr < W 0.0024Kolmogorov-Smirnov D 0.148507 Pr > D <0.0100Cramer-von Mises W-Sq 0.358655 Pr > W-Sq <0.0050Anderson-Darling A-Sq 1.926991 Pr > A-Sq <0.0050

OK

Rejectnormality

18

A problem with most normality tests

19

• Two normality tests not substantially affected by granularity• Ryan-Joiner test (Minitab 16)• Omnibus skewness/kurtosis test

105.0102.5100.097.595.0

99.9

99

959080706050403020105

1

0.1

Rounded

Perc

ent

Mean 99.84StDev 1.536N 100RJ 0.999P-Value >0.100

Probability Plot of Rounded and Ryan-Joiner Test

For more information, see Seier, E. “Comparison of Tests for Univariate Normality.” http://interstat.statjournals.net/YEAR/2002/articles/0201001.pdf

19

Confidence for Conformance Proportion• Usual 2-sided normal tolerance interval controls both tails

• This can present a problem for an uncentered process

20

105104103102101100999897969594

0.6%

USLLSL

2-Sided Normal Tol Int for 99% Coverage Will FailBoth tails controlled to 0.5% , half of the non-coverage

20

Estimation for Conformance Proportion

21

Example: Removal Torque, Spec = 5.0 – 10.0 in-lbs95% conf / 99% coverage tolerance interval: (4.85, 8.62) FAILS

10.09.59.08.58.07.57.06.56.05.55.0

5 10

.

Tol_Int

1098765

8.58.07.57.06.56.05.5

99

90

50

10

1

Perc

ent

N 30Mean 6.738StDev 0.561

Lower 4.852Upper 8.623

AD 0.281P-Value 0.617

Statistics

Normal

Normality Test

Normal Probability Plot

Torque: 95% Confidence / 99% Coverage Tolerance Interval

21

Estimation for Conformance Proportion

• Usual 2-sided normal tolerance interval controls both tails• This can present a problem for an uncentered process

• Can use estimation for proportion conforming• Also called bilateral conformance proportion

• Reduce probability of failing for uncentered processes

• Similar method used by ANSI Z1.9 for routine production sampling

22

)(..)Pr(],[

AQLtheusuallyvalueacceptanceisforICupperifPassBYAproportioneconformancbilateralThe

BAionspecificatwithsticcharacteriqualitythebeYLet

22

Estimation for Conformance Proportion

23

Estimation for Conformance Proportion for Removal Torque:95% confidence ≥ 99.07% conforming: PASS

Lee, H., and Liao, C. “Estimation for Conformance Proportions in a Normal Variance Components Mode.” Journal of Quality Technology, Jan., 2012.Taylor, W. Distribution Analyzer, version 1.2.

TorqueNo Transformation (Normal Distribution)

Sample Size = 30Average = 6.737513Standard Deviation = 0.561204Skewness = 0.47Excess Kurtosis = 0.72

Test of Fit: p-value = 0.2808 (SK All) Decision = Pass (SK Spec) Decision = Pass

Pp = 1.48Ppk = 1.03Est. % In Spec. = 99.901940%

With 95% confidence more than 99% of the values are between 4.8576399 and 8.6173854With 95% confidence more than 99.0666% of the values are in spec.

5.5723100 8.17047006.8713900

LSL = 5 USL = 10

23

Random Effects Process Tolerance Limits

24

1Krishnamoorthy, K. and Mathew, T. “One-Sided Tolerance Limits in Balanced and Unbalanced One-Way Random Models Based on Generalized Confidence Intervals.” Technometrics, Vol. 46, No. 1, Feb. 2004.

24

• Overall process tolerance limits may be constructed to take between-lot variation into account

• Example: Impurities n Mean StDev Min MaxLot 1 20 0.047 0.0113 0.018 0.069Lot 2 20 0.054 0.0055 0.046 0.065Lot 3 20 0.050 0.0087 0.035 0.068

• Approx 90% confidence / 95% coverage tolerance interval1:

• Usual 90/95% tolerance interval for all data combined: 0.07

12.0)1(

.. ),(1 1

nkk

sstx k

Appears conservative

What is an AQL?

25

• AQL = "Acceptance Quality Limit“• The quality level that would usually (95% of the time) be

accepted by the sampling plan

• RQL = "Rejection Quality Limit“• The quality level that will usually (90% of the time) be rejected

by the sampling plan • Also called LTPD (Lot Tolerance Percent Defective)• Also called LQ (Limiting Quality)

25

AQL / RQL

26

AQL: Pr(accept)=0.95RQL: Pr(accept)=0.10

AQL and RQL (LTPD) for n=50, a=1

0.0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1.0

0 1 2 3 4 5 6 7 8 9 10

Percent Defective

Pro

ba

bil

ity

of

Ac

ce

pta

nc

e

AQL = 0.72% RQL = 7.56%

26

What is an AQL?

27

Can be cast as a hypothesis test or confidence interval

For routine acceptance sampling…Ho: p ≤ Assigned AQL

H1: p > Assigned AQLα=0.05, “accept” lot if Ho not rejected

But for validation…Ho: p > Assigned AQL

H1: p ≤ Assigned AQL or desired performance level

α =1-confidence; i.e., 1-.90 = .10 for 90% confidencePass validation if Ho rejected

27

Typical AQLs in Pharma / Medical Devices

Product Attribute Typical Assigned AQLs

Critical 0.04%, 0.065%, 0.1%, 0.15%, 0.25%

Major Functional 0.25%, 0.4%, 0.65%, 1.0%

Minor Functional 0.65%, 1.0%, 1.5%

Cosmetic Visual 1.5%, 2.5%, 4%, 6.5%

28

• For validation, suggest 90% confidence that process ≤ assigned AQL• Why 90%?

• Traditional probability used for RQL/LTPD/LQ• This means for validation the assigned AQL is treated as an RQL• If nonconforming rate is at AQL, will fail validation 90% of the time• Selection of the AQL more important than confidence selected• Much tighter than ANZI Z1.4/Z1.9 tightened (10-20% confidence)

28

Assure high % within spec: attributes data

Example: n=230, a=0 provides 90% confidence ≥ 99% conforming;

90% confidence ≤1% nonconforming

Conforming n a n a99.9% 2300 0 3000 0

3890 1 4745 15320 2 6295 2

99.6% 575 0 750 0970 1 1185 1

1330 2 1575 299.0% 230 0 300 0

390 1 475 1530 2 630 2

97.5% 90 0 120 0155 1 190 1210 2 250 2

95.0% 45 0 29 077 1 45 1

105 2 60 2

90% Confidence 95% Confidence

Attributes data is binomial pass/fail data

Attributes data is binomial pass/fail data

2929

Attributes Example for AQL: Fill Volume PPQ

• Production assigned AQL is 1.0% • AQL = “Acceptance Quality Limit”• Assigned based on risk assessment• If process is better than AQL, almost all mfg lots will be accepted

• Validation: Show with 90% confidence that the process produces ≤1.0% nonconforming units• Multi-head filler; we know data are non-normal• 90% confidence ≥99% are in spec

• Medical devices: 90% confidence for 99% “reliability”• Assures that future AQL production sampling can be passed• If process is at the AQL, ~95% of lots will pass AQL sampling

3030

Example: Fill Volume• Attributes plans: 90% confidence ≤1.0% nonconforming

• If the validation sampling plan passes…• We have 90% confidence the nonconforming rate is ≤1.0%• ANSI Z1.4 plans provide far less than 90% confidence

• Normal sampling: typically about 5% confidence• Tightened: typically about 15% confidence

• Note: RQL=“Rejection Quality Limit”• Also called LTPD (Lot Tolerance Pct Defective) or LQ (Limiting Quality)

Sampling Plan RQL0.10

n=230, acc=0, rej=1 1.0%

n=390, acc=1, rej=2 1.0%

n1=250, a1=0, r1=2n2=250, a2=1, r2=2

1.0%

31

Z1.4 normal: n=80, acc=2Z1.4 tightened: n=80, acc=1

31

PV Acceptance Criteria for Attribute TypesAttribute type Comment

AQL attributes• Fill volume• Tablet defects• Extraneous matter, etc.

≥90% confidence that• Nonconformance rate ≤ assigned AQL

Non-AQL attributes• Dissolution / UDU / Batch Assay• Other tests

≥90% confidence that…• USP test will be met ≥95% of the time• ≥99% of results in spec (critical)• ≥95% of results in spec (non-critical)

Statistical Parameters• Mean / sigma / RSD(CV)• Cpk, Ppk

≥90% confidence that…• Mean / sigma / RSD in spec• Ppk ≥1.0, 1.33 or related to % coverage

No within batch variation expected• pH of a solution• Label copy text

Statistics not usually necessary• May consider 3X-10X testing• Assess between lot variation

3232

Show Population Parameter Meets Spec• Show confidence interval for parameter in spec• Example: API mean potency; spec of 98.0-102.0

• n=30 test results (3 from each of 10 drums)• 95% C.I. for mean is traditional

101.2100.8100.4100.0

Median

Mean

100.6100.5100.4100.3100.2100.1

1st Quartile 100.08Median 100.293rd Quartile 100.75Maximum 101.13

100.26 100.54

100.14 100.62

0.30 0.50

A-Squared 0.70P-Value 0.059

Mean 100.40StDev 0.37Variance 0.14Skewness 0.440565Kurtosis -0.997480N 30

Minimum 99.84

Anderson-Darling Normality Test

95% Confidence I nterval for Mean

95% Confidence I nterval for Median

95% Confidence I nterval for StDev95% Confidence I ntervals

Summary for API 95% C.I. for mean is100.26 – 100.54; pass.95% C.I. for mean is100.26 – 100.54; pass.

Also need to analyze data across drums!Also need to analyze data across drums!

3333

Process Capability/Performance Statistic Ppk

Measures process capability of meeting the specifications

34

LTLTpk

LSLxxUSLMinP

3,

3

σLT is long-term sd, usual formula, includes variation over time;Cpk uses short-term estimate of sd

95

96

97

98

99

100

101

102

103

104

105

Ppk=2.0 Ppk=1.33Ppk=1.0 Ppk=1.0 Ppk=0.9

USL

LSL

34

Ppk vs Percent Nonconforming

• In statistical control• Normally distributed• Centered in spec• If1-sided: half % shown

3535

Example: Show Ppk Meets Requirement• Provide 90% confidence process Ppk≥1.3• n=15 assay tests were obtained across each of 3 PPQ lots• No significant difference in mean/variance in the 3 lots; pool data?

105.0103.5102.0100.599.097.596.0

LSL USL

LSL 95Target *USL 105Sample Mean 101.298Sample N 45StDev(Within) 0.637147StDev(Overall) 0.760421

Process Data

PPU 1.62Ppk 1.62Lower CL 1.39Cpm *Lower CL *

Cp 2.62Lower CL 2.24CPL 3.30CPU 1.94Cpk 1.94Lower CL 1.66

Pp 2.19Lower CL 1.88PPL 2.76

Overall Capability

Potential (Within) Capability

% < LSL 0.00% > USL 0.00% Total 0.00

Observed Performance% < LSL 0.00% > USL 0.00% Total 0.00

Exp. Within Performance% < LSL 0.00% > USL 0.00% Total 0.00

Exp. Overall Performance

WithinOverall

Process Capability of Assay Result(using 90.0% confidence)

Pass

3636

Pooling OK if Process in Statistical Control

95

96

97

98

99

100

101

102

103

104

105

Batch 1 Total process variation

Total V

ariation

Process in Classical Statistical ControlCommon Cause Variation Only

Within batch

Batch 2 Batch 3 Batch 4 Batch 5

3737

Intra-batch and Inter-batch Variation

Intra=Within batch: σw Inter=Between batch: σb

3838

Special Cause Variation

95

96

97

98

99

100

101

102

103

104

105

Total process over time

With

in lo

t

Process not in Statistical Control - Special Cause Variation

?Batch 1 Batch 2 Batch 3 Batch 4 Batch 5

3939

Ppk if Process is Not in Statistical Control

• Use of Ppk is controversial if process not in statistical control1• “Ppk has no meaningful interpretation”• “statistical properties are not determinable”• “a waste of engineering and management effort”• Note: between-batch variation means not in classic statistical control

• If variance components model holds, estimate Ppk with σw and σb

• Usual confidence intervals from standards/software not applicable• Confidence interval must take degrees of freedom for σb into account• Difficulty in proving variance components model assumptions with

small number of lots

1 Montgomery, Introduction to Statistical Quality Control 6th edition, p 363

4040

Potential problems with Ppk over multiple lots

• Usual Ppk confidence interval assumes normal distribution and process stable / in statistical control• Any changes/trends within or between lots invalidates assumption• Often differences in mean between batches

• Usual Ppk C.I. does not consider variance components• Example: 30 samples from each of 5 lots

• (30-1)x5 = 145 degrees of freedom for within lot variation• (5-1) = 4 degrees of freedom for between lot variation• ASTM reference E2281 does not address this

• Alternative: Show Ppk for each lot meets requirement for• 3 lots: ~87% overall confidence median process Ppk meets spec• 4 lots: ~94% confidence• 5 lots: ~97% confidenceOr calculate a modified Ppk based on variance components C.I.

4141

Example: Ppk if process not in statistical control

54321

101

100

99

Sam

ple

Mean

__X=100.037UCL=100.342

LCL=99.733

54321

0.8

0.6

0.4Sam

ple

StD

ev

_S=0.5509

UCL=0.7689

LCL=0.3330

54321

102

100

98

Sample

Valu

es

104.0102.4100.899.297.696.0

LSL USL

LSL 95USL 105

Specifications

10410210098

Within

Overall

Specs

StDev 0.5557Cp 3.00Cpk 2.98PPM 0.00

WithinStDev 0.9321Pp 1.79Ppk 1.77Cpm *PPM 0.08

Overall

1

1

1

1

Process Capability for ValidationXbar Chart

S Chart

Last 5 Subgroups

Capability Histogram

Normal Prob PlotAD: 0.512, P: 0.192

Capability Plot

Ppk=1.77; lower 95% C.I. for Ppk using Minitab is 1.60.But should PPQ pass? Scientific understanding of trend?

Plot your data!

Plot your data!

4242

Assure a Standard Test will Pass

• Example: Uniformity of Dosage Units (Content Uniformity)

• Requirement: Pass USP‹905› Uniformity of Dosage Units

• ≥90% confidence USP test would be passed ≥95% of the time (coverage)• See Bergum1 for specifics to determine acceptance criteria• Why 90% confidence? Comparable to RQL probability.• Why 95% coverage? Comparable to AQL probability for single test.• Bayesian approach also available2

43

1Bergum, J. and Li, H. “Acceptance Limits for the New ICH USP 29 Content-Uniformity Test”, Pharmaceutical Technology , Oct 2, 2007

2Leblond, D., and Mockus, L. “Posterior Probability of Passing a Compendial Test.” Presented at Bayes-Pharma 2012, Aachen, Germany.

43