ibc biological assay development & validation 2011 gra presentation

of 66/66
Developing Bioanalytical Methods Balancing the Statistical Tightrope

Post on 21-Jan-2015



Health & Medicine

2 download

Embed Size (px)


Presentation at IBC's Biological Assay Development and Validation conference 12 May 2011


  • 1. Developing Bioanalytical MethodsBalancing the Statistical Tightrope

2. Lee: can I use this number?Process DevelopmentGSK, 19972 3. its about 40 about 40? probably...3 4. Enlightenment? 5. 5 6. Blooms Taxonomythe 4 stages of competenceIncompetent CompetentConsciousConsciousnessUnconscious6 7. A Statistical GodMe 8. Using Statistics 9. Why? Six Reasons1. Potency assays are key in making medicines2. Bioassays are very variable3. Statistics will help you understand your data4. Understanding your data willreveal if control exists5. Your level of control allows you to judge RISK6. Regulators globally require it9 10. The Regulator & Assay Control Regulators have been asking for this for years! QbD1. Pharmaceutical cGMPs for the 21st Century2. PAT3. ICH Q2: Validation of Analytical Procedures4. ICH Q8: Pharmaceutical Development5. ICH Q9: Quality Risk Management6. ICH Q10: Quality Pharmaceutical Systems 10 11. StatisticsThe complete solution? 12. Or this? Your assay? 12 13. Or this? or your assay?13 14. Statistics - an Amazing Transition 14 15. Bioassays will always be variableYou can improve it- by understanding it- Focusing effort in right places- This brings control- You can manage expectations- This is understood by regulators 15 16. Why assay variation matters? product variation + A few unsatisfactory assay variation + batches may even inaccuracypass specification due to a combination of assay method and process variability Many satisfactory OOS batches likely to fail (potentially costing Ms)because of combination of assay method & process inaccuracy & variation16 17. Our Control StrategyWhat does the scientist need to achieve?Definei.e. selectivity, accuracy, precision linearity Identify & prioritise analytical CNX parametersMeasureControlNoiseeXperimentalparameters parametersparametersAnalyseFix & control e.g., MSA,e.g., DoE Input intoPrecision RegressionMethodMethodImproveRuggednessRobustnessMethod Control Strategy & reduce Risk prior toControl Validation Routine Use & Continuous Improvement17 18. Generating Bioassay Data18 19. The Rules1. Speak with your statistician before generating data2.See Rule 1 19 20. Lots data Value 20 21. 21 22. Statistics are a tool22 23. QC Which Tools? UCL Stage 4 TechnologyQC Tools CELLULA, Shewhart chart,YES Transfer LCLCUSUMNO TIMEStage 1:Qualification ToolStage 3:Fishbone, MinitabValidation ToolsNested, CELLULAPrecisionStage 2:AccuracyLinearity etc.Development ToolsDX8, JMP, MinitabDesign 24. Whats Appropriate Knowledge? Learning takes time Will you use it often enough? Its not an academic pursuit Activities must add value do whats necessary24 25. Scope&Design 26. Define & ScopeHow is the assay performing? Prec/TOL2-sided = 6 x 16.76 100 = 1.01 26 27. Parameters (e.g. 15)pDNANaClpHTube LengthTimeSeeding DensityRatio of TransfectionTemperatureAgitation and levelVector type, concAddition Order 28. Q. How Many parameters?Q. Which parameters?Q. What ranges?A. Existing knowledgeA. Common senseA. Practical limits 29. Define & ScopeDrill down - map out assay - build understanding & scope Assay Flow 29 30. Define & Scope Drill down & map out assay to build understanding & scopeAttention is focusedtoward key stepsand the parametersinvolved in thesesteps Cause & Effect Diagram (Fishbone) helps think your assay through Identify & prioritise analytical CNX parameters 30 31. Scope & ScreenScope ranges with simple experimentsScoping Experiments Explore mildest to most forcing conditions 31 32. Revealing the Big Hitters32 33. Temptation 34. Building UnderstandingOFATProvides estimatesof effects at setconditions of theNaClother factors andno interaction pDNAeffects.pH34 35. Building UnderstandingFactorial Design2400 2600 1300900 1800Estimates effects atdifferent conditions toestimate interactions350 600250300 500Design of ExperimentsDOE35 36. Optimisation Optimise the parameters that survived the initial screeningwork towards aRobust Optimum36 37. Simulations The tools allow you to simulate scenarios using the data youve built upVisual simulation of expected performance relative to specification37 38. Is the Model Correct?38 39. Validate & Verify The evaluation of robustness should be considered during the development phase and should show the reliability of an analysis with respect to deliberate variations in method parametersICH Q2B, 1994Method stretchwhat if?Ideal SettingsControl SpaceDesign Space39 40. Assay Control: control the parameters inside boundaries40 41. Working within the controlboundaries will keep theassay under control Even if you go outside the control boundaries, the assay will have enough flexibility to deal with it without an OOS 41 42. Summary - Data Driven DevelopmentScope Screen Optimize VerifyQC/TT Transfer to QC to validate on batches & bring into routine useExplore mildest Identify few potential Estimate & utilizeto most forcing key parameters interactions to move Rattle the cage toconditionsFocus on vital few & towards optimumdeliver a designnarrow rangesconditions space 43. 43 44. PrecisionIt may be considered at three levels:1. Repeatability2. Intermediate precision3. ReproducibilityICH Q2A, 1994 45. Repeatability1 analyst in 1 laboratory on 1 day injecting 6 times Summary StatisticsNumber of Standard Coefficient Lower 95% CI Upper 95% Values MeanDeviationof Variationfor MeanCI for Mean t30 PS 6223.27 6.43 2.88%216.52230.02 45 46. Intermediate Precision 1 analyst in 1 laboratory on 1 day injecting 6 samples each tested 6 timesAs well as sample variation, this study still providesinformation on repeatability 46 47. Intermediate PrecisionSo we compare the mean values for each sample(over replicate results per sample) Variance Components Factor df Variance % Total Sample527.853521% Repeat 30 102.636179%35 130.4896 100% Standard MeanDeviationRSD 47216.2411.42325.28% 48. and the others..?Precision within a laboratory but withdifferent analysts, on different days, withdifferent equipmentreflects the realconditions within one laboratory ICH Q2A 199548 49. Intermediate PrecisionData collect using several analysts using several instrumentsover several days: Y56000555005500054500Peak Area54000535005300052500520000 5 1015 2025 Sample49 50. Intermediate PrecisionPotentially misleading: large analyst-to-analyst variationpresent:Y56000555005500054500Peak Area54000535005300052500520000 5 1015 20 25 SampleAnalyst 1Analyst 2Analyst 350 51. Intermediate Precisionbetter examined looking at multiplesources of variation within an assaywant to understandmajor sources ofvariation such assample, prep,analyst etc. 51 52. Intermediate Precision 52 53. Intermediate PrecisionCan also perform Unbalanced designsOne operator performs multiple injections on singlepreparation;Two operators perform single injections on multiplepreparations53 54. Reproducibility multiple laboratories; typically run as an inter- laboratory cross-over study, with each participating lab sending samples to every other lab and analysing all samples (including own) . sent to and analysed by other lab A BCSamples fromA laboratory:B C 54 55. ReproducibilityCan use analysis of variance (ANOVA) to look fordifferences or biases between labsAlternatively look for analytical equivalence 56. Risk ManagementThe level of effort, formality and documentation....should be commensurate with the level of risk ICH Q9Evaluation of the risk to quality should be based onscientific knowledge & ultimately link to theprotection of the patientIs the bioassay fit for purpose and under control?56 57. Before & AfterHow is the assay performing? P/TOL2-sided =6 x 16.76100= 1.01 57 58. Before & AfterBetter P/TOL2-sided = 6 x 6.99100= 0.42 58 59. Risk ManagementMethod Understanding, Control and Capability (MUCC) Understand impact of variation upon riskRiskUnderstanding?Capable? ManagementLoop Statistical CapabilityProcess Control& Precision (SPC) Charts Control? 59 60. Risk ManagementUnderstanding? Understanding?Capable? P/TOL2-sided =6 x 16.76Capable?100 Control?= 1.01 Capability& Precision60 61. Risk Management P/TOL2-sided =6 x 6.99 100 I-MR Chart of t30 PS Summary Report = 0.42Is the process mean stable? I ChartEvaluate the % of out-of-control points. Investigate out-of-control points.0% > 5%225 UCL=220.77Yes No 2100.0%t30 PS _ X=199.87 195Comments 180 LCL=178.96 The process mean is stable. No data points are out of control 1 6 11 16 21 26 31 36 41 46 on the I chart.61Observation 62. Summary1.Build a good basic understanding ofstats but dont need to become guru2.Involve a statistician, at least at the beginning3.Build understanding of your bioassay (QbD) its a must4.Get to grips with Bioassay Variability62 63. Lee: can I use this number?63 64. Yes its 42 0.05 with 95% Confidencefor the statisticians in the audience64 65. AcknowledgmentsDr. Paul Nelson Prism TC LtdPictures from The Cartoon Guide to StatisticsLarry Gonick & Woollcott Smith65