comparability of an icief test method performed on ... · acceptance criteria for comparability...

CE Pharm; Seattle, WA; October 14, 2014 Yasushi Ogawa

David Guillon, Caroline Schackemy, Lan Dai, Thomas Aldick

Global Biologics QC, Methods Management & Technology, Analytical Chemistry & Lifecycle Management, Genentech, South San Francisco, CA

Comparability of an ICIEF Test Method Performed on Different Instrument Models Among Multiple QC Sites

Background Information

ICIEF method addition to the control system

• A test method was developed on ProteinSimple iCE280 to determine the levels of unconjugated antibody present in antibody-drug conjugate (ADC) product.

• Co-validated between QC sites A and B.

• Specification was developed based on the data generated using iCE280.

• Recently, QC sites C and D became additional testing sites for the product. QC sites C and D have ProteinSimple iCE3.

• Test method was adopted onto iCE3 and validated.

• Method validation data (intermediate precision, system suitability standard, limit of quantitation) from iCE280 and iCE3 were used to show comparability between the two ICIEF models.

Test Method

Test Method for ICIEF

1. Samples and reference standard are diluted. 2. Digest with CpB to remove C-terminal Lys:

• reduces charge heterogeneity due to variable C-terminal truncation • simplifies profile • enables better quantitation of charge variants, which are due to different

numbers of drug attached. 3. Mix with ampholyte and inject for IEF.

ICIEF Method is not Stability-Indicating

• No change in Peak 1 area or the profile at 25°C up to 3 month • Under these conditions, the drug:antibody linkage is known to be unstable. There

are other methods available that can detect the changes under these conditions.

• The ICIEF method is not used in stability studies

Comparability between iCE280 and iCE3

Comparability Study Between iCE280 and iCE3

• Test method performance with iCE280 and iCE3 were compared for:

• Accuracy and precision

• Used method validation data

• Compared at 3 levels of unconjugated antibody:

• ~150 units (high; exceeds upper specification limit 103 units): used intermediate precision data

• ~50 units (typical level in DS and DP batches): used intermediate precision data

• System suitability standard (N= approximately 90), 60 units

• LOQ (approx. 25 units unconjugated antibody)

• Identical samples of 50 and 150 units unconjugated antibody and system suitability standard were used in both method validations performed using iCE280 and iCE3.

Acceptance Criteria for Comparability

• The results generated on 50 and 150 unit samples (intermediate precision), system suitability standard, and at LOQ were compared between iCE280 and iCE3 .

• Comparison done in 2 stages (absolute and statistical): • Accuracy

1. Mean values were compared as a ratio 2. Student’s t-test (confirmation and understanding of data

distribution) • Precision

1. SDs were compared as a ratio 2. F-test

• If the acceptance criterion based on absolute difference is met, the result is considered comparable even if statistical evaluation fails

Comparability Acceptance Criteria and Justification: Accuracy

Acceptance criteria: For 50 and 150 units samples and SS standard Ratio of mean: 0.86 to 1.14

Based on two justifications: • Method Variability

• RSD: 6.8% • 2 x RSD = 13.6%, rounded to 14%

• System suitability data (N=93) from method validation performed using iCE280

• Based on practical significance

• Unconjugated antibody in DS and DP is typically 50 to 63 units • A relative difference of ±14% (7 to 9 units) is not considered of practical

significance compared the specification limit of 103 units and process variability.

Diagram of Comparability Acceptance Criteria

Actual differences between iCE280 and iCE3 data are 2 to 5%.

Comparison of Intermediate Precision Results for

50 units Sample: Accuracy

Unconjugated Antibody (units) iCE280 iCE3

Mean 53.94 54.82 SD 2.11 2.29

RSD 3.90 4.19 N 12 12

Comparability Criteria Result Conclusion Mean ratio (iCE3/iCE280):

0.86 < ratio < 1.14 1.02 Comparable

t-test: p > 0.05 0.34 Comparability confirmed


150 units Sample: Accuracy

Unconjugated Antibody (units) Assay no. iCE280 iCE3

Mean 156.7 160.0 SD 5.18 1.82

RSD 3.31 1.14 N 12 12



t-test: p > 0.05 0.05 Comparability confirmed

Comparison of System Suitability Results: Accuracy

Given the large number of data (N=93, 94), a relatively small difference between the two means is statistically significant. The difference of 3 units is not of practical significance compared to the mean unconjugated antibody (59 to 62 units). The result is accepted as comparable.


Mean 58.79 61.70 SD 4.00 2.64

RSD 6.81 4.28 N 93 94



t-test: p > 0.05 0.00 Means are statistically different,

but the difference is not of practical significance

Comparability Acceptance Criteria and Justification: Precision

• 50 and 150 units samples: • Acceptance criterion: ≤1.89 (SD ratio, iCE3/iCE280) • Ratios >1.89; p≤0.05, F-test, N=12

• System suitability standard:

• Acceptance criterion: ≤1.24 (SD ratio, iCE3/iCE280) • Ratios >1.24; p<0.05, F-test, N=80 for iCE3, N=93 for iCE280 • Better precision preformance for iCE3 is considered acceptable


50 units Sample: Precision


Mean 53.94 54.82 SD 2.11 2.29

RSD 3.90 4.19 N 12 12

Comparability Criteria Result Conclusion SD ratio (iCE3/iCE280):

< 1.89 1.09 Comparable

by F-test: p > 0.05 0.78 Comparability confirmed

Comparison of Intermediate Precision Results for 150 units

Sample: Precision

The F-test failure is due to significantly smaller SD of the iCE3 data than iCE280 (SD: 1.82 vs 5.18) and suggests iCE3 may have a better precision than iCE280. This apparent higher instrument performance is acceptable and is not of practical significance. The result is accepted as comparable.


Mean 156.73 160.01 SD 5.18 1.82

RSD 3.31 1.14 N 12 12


< 1.89 0.35 Comparable

F-test: p > 0.05 0.00 Precisions are statistically different,

but not of practical significance

Comparison of System Suitability Results: Precision

The F-test failure is due to large number of data (N=93, 94), which can detect relative small difference as statistically significant. The SD of iCE3 (2.64) is smaller than of iCE280 (4.00) and suggests iCE3 may have a better precision than iCE280. This apparent better instrument performance is acceptable and is not of practical significance. The result is accepted as comparable.


Mean 58.79 61.70 SD 4.00 2.64

RSD 6.81 4.28 N 93 94


SD ratio < 1.89 0.66 Comparable

by F-test: p > 0.05 0.00 Precisions are statistically different,

but not of practical significance

Comparability Acceptance Criteria for Limit of Quantitation

• Acceptance criterion: LOQ for iCE3 must be ≤ 30 units unconjugated antibody

• 20% higher than the LOQ for iCE280

• LOQ for iCE280 from the method validation is 25 units unconjugated antibody (RSD = 7.6%).

• >30 units is significantly different from 25 units (p<0.05, t-test for N=3).

LOQ (units Unconjugated Antibody)

Outcome iCE280 iCE3 Conclusion 1 25 < 30 Comparable

2 25 > 30 Fail

Comparison of Limit of Quantitation Results

Unconjugated Antibody (units)

iCE280 iCE3

25.1 27.3

Comparability Criteria Result Conclusion

LOQ for iCE3:

<30 units uAb 27 Comparable

Summary of Comparability Results

50 units Sample 150 units Sample SS Standard Accuracy Ratio of means Pass Pass Pass

t-test Pass Pass Fail

Conclusion Comparable Comparable Comparable

Precision SD ratio Pass Pass Pass

F-test Pass Fail Fail

Conclusion Comparable Comparable Comparable

LOQ Value at LOQ Pass

Conclusion Comparable

Conclusion: Accuracy and precision performances of the two ICIEF instruments are comparable.

Conclusion

• Although not all acceptance criteria were met, the instruments are considered comparable.

• Statistical differences observed between the instruments are • due to the influence of large number of data has on relatively small

differences • possibly better precision on iCE3 • not of practical significance when considering the impact on specification

range and manufacturing process capability • When designing instrument comparability studies, factors to consider are

• Impact of mean differences on generating potential initial OOS results • Process capability, based on historical lot release data • Stability data and specification limit • Number of data points needed to achieve desired level of statistical

significance • Implementation of the new instrument, considering change controls and

regulatory filing where applicable

Acknowledgement

Roche-Genentech, South San Francisco, CA Yan Chen Lan Dai Fred Jacobson Paul Motchnik Roche-Kaiseraugst, Switzerland David Guillon Caroline Schackemy Nadine Dietrich Roche-Grenzach, Germany Thomas Aldick

Doing now what patients need next

Comparability Acceptance Criteria for Accuracy:

mean difference and statistical evaluation

Ratio of mean values (iCE3/iCE280)

Outcome 50 and 150 units Samples System Suitability Standard Conclusion 1

Both: 0.86 < ratio < 1.14

0.86 < ratio < 1.14

Comparable

2

Pass, Fail

Pass

Fail

3 Both pass or fail Fail Fail

Comparison of mean values by ratios Comparison by t-test Outcome Ratio Conclusion t-test Conclusion

1

0.86 < ratio < 1.14 Comparable p > 0.05 Comparability confirmed

2 Pass Comparable p < 0.05 Statistically different, but not of practical significance

3

Fail Fail p > 0.05 Data scattered wider than expected

4 Fail Fail p < 0.05 Confirms means are different

Mean ratio:

t-test:

Acceptance Criteria for Comparability of Precision:

SD difference and statistical evaluation

Ratio of SDs (iCE3/iCE280) Outcome 50 and 150 units samples System suitability standard Conclusion

1

Both: < 1.89 < 1.24 Comparable

2

Both pass Fail

3

Either or both fail

Pass Fail

4

Either or both fail Fail

Comparison of SD ratios Comparison by F-test Outcome Ratio Conclusion F-test Conclusion

1

Pass Comparable p > 0.05 Comparability confirmed

2

Pass Comparable p < 0.05 Precisions are statistically different, but not of practical significance

3

Fail Fail p > 0.05 Data may be scattered wider than expected

4

Fail Fail p < 0.05 Confirms SDs are different

SD ratio:

F-test:

comparability of an icief test method performed on ... · acceptance criteria for comparability...

Documents