Statistical Issues With Linear Extrapolation of Stability Data

1

Jason Marlin, MS/T Statistics, Eli Lilly & Co.

Define OOT & PTE OOT—Out of Trend: the FDA considers any

atypical test result or slope to be an “out of trend.”

PTE—Project to Exceed (Lilly term): linear extrapolation of stability data that indicates a lot may exceed the regulatory limit prior to expiry

2

FDA Resource for OOT PTE From Q1E: The Evaluation of Stability Data

(June 2004) The FDA expects industry to monitor stability

batches and to react to trends or changes. Where appropriate, determine if batches may not

stay within the regulatory specifications during shelf-life.

Inform agency in advance if product will fail Modeling an appropriate fit is clearly important to

this work.

3

Additional Resources

4

PTE Resource From April 2003, Identification of Out of Trend

Stability Results “The extrapolation of OOT should be

limited and scientifically justified, just as the use of extrapolation of stability data is limited in regulatory guidance (ICH, FDA). “

5

WIIFM Every pharmaceutical company is required to

evaluate stability data for trends and respond accordingly.

Every product has different challenges—e.g. assay (RMSE of common slopes), process mean, markets, possibly different stability timepoints, possibly different sites of manufacture, etc.

Intent: provide some measure of accounting for these differences in the process of evaluating stability data for projection to exceed. What is not covered is a method for controlling the Type

II error rate—namely, failing to signal a batch that will exceed regulatory prior to expiry. (A rough simulation has been performed but further refinement is needed).

6

WIIFM-continued PTE’s are often evaluated by a stability

coordinator (often non-statistician) A false signal=wasted resources (deviation,

investigation, re-assay, etc.) Generic computer algorithms don’t account

for necessary variables (how many batches are manufactured, assay, process mean, etc.)

Thus, some “decision-science” is needed to allow the coordinator to determine how many stability timepoints are required to obtain a reliable projection

7

Example: Actual Example ( w/coded data)

87 stability batches, all with data through at least 50% of expiry

Example: Actual Product w/coded data continued

9

Expiry

A lot with 6 timepoints PTE. 6th timepoint at 18 months

Sources of OOT Reasons for a false signal:

Non-linear data Limited data Process mean deviation from label claim Assay variability

What data requirements are needed to limit the probability of a false signal?

Problem: for products with no practical change on stability and relatively high assay variability, three or four points are not sufficient to prevent false signals

10

simulation

Simulation Properties with Single-sided and Dual-sided

limits are evaluated. For single-sided limits, the mean bias and the assay

standard deviation are expressed as a portion of the regulatory limit

For dual-sided limits, the mean bias and the regulatory limit are expressed in actual units as % of label claim but they can also be expressed as k-sigma units of the assay standard deviation

For the sake of simplicity, both single-sided and dual-sided properties are expressed in k-sigma units as “distance to nearest specification.” This eliminates the need for presenting two sets of results.

11

Simulation-cont’d

12

94

96

97

98

99

104

106

LRAC

URAC

Target

Mean

+1s

-1s

+2s

-2sRe

sult

-5 0 5 10 15 20 25 30 35 40

Time (months)

Mean Bias=1.5% of Target. Assay Standard Deviation=0.85% of Target

0.6

0.8

1

1.2

1.4

1.6

URAC

Mean

+1s

-1s

+2s

Re

sult

-5 0 5 10 15 20 25 30 35 40

Time (months)

Mean=14.3% of RAC, Assay Standard Dev=14.7% of RAC

One-sided Limit

Two-sided Limit

Simulation—continued Population means and assay standard deviations

(assumed known) simulated relative to target. For single-sided and dual-sided limits,1500 simulations

are performed for 40 different combinations of true mean, regulatory limits and assay standard deviation

Assumptions: 1) No Change on Stability 2) 36-month expiry 3) Typical Stability timepoints (0,3,6,9,12,18,24,30) A reduction in alpha (α)—all else held constant—could affect beta (β).

**The assumption that there is not change on stability MUST be based on the science of the molecule/packaging and supported by available stability data

13

Errors in PTE Type I—designating a batch as PTE when the

true property is not outside regulatory limits at expiry.

Type II—failing to detect a batch with a true property value outside regulatory limits at or before expiry.

14

Goal In general, identify the minimum n such

that no PTE’s are issued >95% of the time for properties which are practically stable.

Also evaluated n (minimum # of stability timepoints) for 90 and 99%.

Allow manufacturing sites to select a minimum # of stability time-points to achieve the desired alpha.

Allow manufacturing sites to see the impact of mean bias and assay standard deviation on the likelihood of falsely PTE Target products are those with large assay error and with

process means close to the Regulatory limits (small R values)

15

Brief JMP Simulation Overview

16

R

17

Question: Is an R=4 the same for different combinations?

R1=4=100-90/2.5R2=4=99-95/1

Simulation shows the results are effectively the same for a given n.

18

Rn=4(0,3,6,9,12,18,24months)

Bivariate Fit of % Not PTE By R n=4

RMSE=2.79%

Each point represents a different combination of assay, process mean and regulatory.

80

82

84

86

88

90

92

94

96

98

100

% N

ot P

TE

0 5 10 15 20

R

19

Rn=5(0,3,6,9,12,18,24months)

Bivariate Fit of % Not PTE By R n=5

RMSE=2.16%

80

82

84

86

88

90

92

94

96

98

100

% N

ot P

TE

0 5 10 15 20

R

20

Rn=6(0,3,6,9,12,18,24months)

Bivariate Fit of % Not PTE By R n=6

RMSE=0.99%

80

82

84

86

88

90

92

94

96

98

100

% N

ot P

TE

0 5 10 15 20

R

21

Rn=7(0,3,6,9,12,18,24months)

Bivariate Fit of % Not PTE By R n=7

RMSE=0.31%

80

82

84

86

88

90

92

94

96

98

100

% N

ot P

TE

0 5 10 15 20

R

22

R (4, 5, 6 & 7 stability timepoints)

50

60

70

80

90

100

% N

ot

PT

E

0 5 10 15 20

R

n=4

n=5

n=6

n=7

Label

23

R Summary Tables

90% TableR n

≤2.4 72.5-3.2 63.3-7.4 5>7.5 4

95% TableR n

≤3.2 73.3-4.8 64.9-9.4 5>9.5 4

99% TableR n

≤4.8 74.9-7.5 67.6-9.5 5

90% TableR n

≤1.9 72.0-2.4 6

2.5-3.3 5>3.4 4

95% TableR n

≤2.4 7

2.5-3.2 63.30-4.8 5

>4.8 4

99% Table

R n≤3.3 7

3.4-7.4 6

7.5-9.4 5>9.4 4

0,3,6,9,12,18,24 months 0,3,6,12,18,24,30 months

Summary

• Simulation provides a method to protect against falsely PTE for products which are stable• Increasing the number of data-points required to perform a PTE evaluation can—for instances where actual product stability changes occur—result in an increased likelihood of failing to detect a PTE for a product which will have a true property value outside the regulatory specifications• PTE evaluations need to account for assay variability, mean bias and the scientific basis of product stability.

24

Summary

•Next Steps—simulate with linear slope to determine at what expense the reduction in false PTE’s (by increasing n) is achievedDetermine how the PTE changes as a result of true change on stabilityInitial look (slopes of 0.05%/month and 0.10%/month ) for 36-month dating and for a product with ±10% regulatory limits, as long as the mean was within 2% of target, these slope changes would not result in a true property value outside the regulatory limits.

25

Talking Points

26

• Distance of projection to expiry is very important• Alpha should be a ƒn of # of lots manufactured and historical data• In general, 4 stability timepoints is marginal for 95% probability of not falsely PTE• Beta dependent on assumption of stability

Questions

27

One-Sided Simulation For different true, but unknown property mean values and for

different true but unknown assay variabilitys, simulate the probability of PTE given the seven assumed stability time-points for two different stability evaluations: 0, 3, 6, 9, 12, 18 and 24 months (standard validation timepoints) 0, 3, 6, 12, 18, 24 and 30 months (alternative timepoint approach)

1500 simulations are performed for 20 different combinations of true mean and assay standard deviation Assay stdev levels=10,20, 30, 40 and 50 % of regulatory Mean Bias Levels=0, 5, 10, 20 and 25 % of regulatory Regulatory Level is not a factor for the one-sided simulation as

both variables are expressed in terms of a given regulatory limit (this is verifed by simulation to work for different regulatory limits)

Assumptions: 1) No Change on Stability 2) 36-month expiry 3) Typical Stability timepoints 4) A reduction in alpha (α)—all else held constant—could affect beta (β).

28

Two-Sided Simulation For different true, but unknown property mean values (in

% of label claim but also easily expressed in k-sigma units from target) and for different true but unknown assay variabilities, simulate the probability of PTE given the seven assumed stability time-points for two different stability evaluations: 0, 3, 6, 9, 12, 18 and 24 months (standard validation

timepoints) 0, 3, 6, 12, 18, 24 and 30 months (alternative timepoint

approach) 1500 simulations are performed for 40 different

combinations of true mean, regulatory limits and assay standard deviation Assay stdev levels=0.50, 0.75, 1.00, 1.25 and 1.50 % of target Mean Bias Levels=0.0, 0.5, 1.0, and 1.5 % from target Regulatory Levels=±5% from target, ±10% from target

Assumptions: 1) No Change on Stability 2) 36-month expiry 3) Typical Stability timepoints 4) A reduction in alpha (α)—all else held constant—could affect beta (β).

29

30

Summary Table for One-sided and Two-sided

One-sided V1 One-sided V2   Two-sided V1 Two-sided V290% Table 90% Table   90% Table 90% TableR n R n   R n R n

≤1.9 7 ≤2.4 7   ≤2.3 6 ≤3.2 62.0-2.4 6 2.5-3.2 6   2.4-3.5 5 3.3-4.7 52.5-3.3 5 3.3-7.4 5   >6.7 4 >4.7 4>3.4 4 >7.5 4    

       95% Table 95% Table   95% Table 95% TableR n R n   R n R n

≤2.4 7 ≤3.2 7   ≤2.3 7 ≤2.7 72.5-3.2 6 3.3-4.8 6   2.4-3.2 6 2.8-4.5 6

3.30-4.8 5 4.9-9.4 5   3.3-4.5 5 4.6-5.7 5>4.8 4 >9.5 4   >4.5 4 >5.7 4

       

99% Table 99% Table  99% Table  

99% Table  

R n R n   R n R n≤3.3 7 ≤4.8 7   ≤3.3 7 ≤4.0 7

3.4-7.4 6 4.9-7.5 6   3.4-4.5 6 4.1-5.3 67.5-9.4 5 7.6-9.5 5   4.6-6.7 5 5.4-8.5 5

>9.4 4       >6.7 4 >8.6 4

31

Summary graphs for R by simulation

75

80

85

90

95

100

% N

OT

PT

E

0 10 20

R

one-sided v1

two-sided v1

Simulation

75

80

85

90

95

100%

NO

T P

TE

0 10 20

R

one-sided v2

two-sided v2

Simulation

0, 3, 6, 9, 12, 18, 24 months

0, 3, 6, 12, 18, 24, 30 months