PREDICTUM INC. WWW.PREDICTUM.COM ©2014
DOE challenges & opportunitiesWAYNE J LEVIN PREDICTUM INC. [email protected] !
WWW.PREDICTUM.COM
1
agenda
About Predictum Use the right methods (and there are more of them) Build a system Assume nothing
2
About Predictum
we increase productivity more exploitable insights in less time, effort, cost less frustration
3
DOE purpose
look at numerous effects comprehensively yet isolate each effect’s influence independently of the other effects
in less time, effort and cost
4
use latest & greatest methods
Definitive Screening Designs latest in computer-generated optimal designs
Split-plot designs hard to change factors
5
definitive screening
more independence of effects
6
Some correlation among main effects and 2-way interactions
Definitive screening - 21 runsOptimal Design -20 runs
Zero correlation among main effects and 2-way interactions
7
Sacrifice some D-efficiency, but not much
Definitive screeningOptimal Design
8
split-plot designs
it’s a multivariate universe
9
The concept of hard to change is broader than you might think. !Here multiple pipetting constitutes hard to change where materials are the same across wells
10
ScenarioThe tool is typical of HTS, in that there is limited ways a chemical can be varied within a plate.
11 12
generating errors
Under REML highlighted items show type I errors if analyzed traditionally !Under Traditional highlighted items show type II errors if analyzed traditionally
13
generating errors
DOE can be a lot of work take the time, make the effort longer more complex experiment designed right often yields more correct insights than a series of small experiments
if not done correctly, DOEs will generate type I and type II errors confusion and frustration
14
build a system
15Copyright © 2013 Predictum Inc. Confidential
time
R&D and Improvement Initiatives are launched and completed in isolation.
16
Copyright © 2013 Predictum Inc. Confidential
time
What if they were connected? What if they started with all relevant information previously acquired and paid for?
17Copyright © 2013 Predictum Inc. Confidential
time
This is typically what happens. Insufficient institutional memory. Each project is isolated. !
Slippage on retaining acquired, prior insights. Must pay to re-acquire what was previously known.
18
assume nothing
be methodical
19
Step 8 Wait Several DaysAll 6 plates Do steps 1-7 again
Step 7 Read all 6 plates
Step 6
Step 5
Step 4
Step 3All 6 plates
Step 2All 6 plates
BSA
concentration
solution
0.2%
water
PK Substrate
PK Substrate
combined with
Kinase Glo
400nM
40nM
400nM
40nM
Timing 5 min
10 min
Kinase Glo 4 uL
nothing
Plate
1 2 3
4 5 6
Plate
1 2 3
4 5 6
Plate
1 2 3
4 5 6
Nozzle
12
345678
Magnesium
solution
5 uM
water
Column within plate
1 2 3 4 5 6 7 8
1 2 3 4 5 6 7 8
Utilizing Design of Experiment Statistical Models to Improve Assay Developmentin High Throughput Biology
Diana Ballard of Predictum Inc., Samuel Hasson of NIH, Wayne Levin of Predictum Inc.
Overview
Introduction
handled in classical full and fractional factorial designs leading them to provide
MethodsOBJECTIVES:
FACTORS: All eight of the following factors are difficult to vary on the Thermo Multi Drop
RESPONSE:
DESIGN NOTES:
Nozzle Buffer
Type
pH PK Enzyme
concentration
1 HEPES 8 100 uM
2 Tris 7 10 uM
3 Tris 8 10 uM
4 HEPES 7 10 uM
5 Tris 8 100 uM
6 HEPES 8 10 uM
7 Tris 7 100 uM
8 HEPES 7 100 uM
similar tests assume that the entire experiment
Conclusions
experimental design and we found a straightforward method to implement its use in DOE
Visualize
Buffer Type *
Magnesium *
PK Substrate
[Nested
Packaging
of Assay]
Factor / Effect Experimental unit (EU) Error Term in the model PK Enzyme
concentration
The wells on all plates that had unique bottle setups for PK Enzyme.
Step 1 had 8 unique bottles, repeating step 1 in iteration 2. Across
both iterations, there are 16 EU.
PK Enzyme concentration
* Buffer Type * pH *
Iteration
Buffer Type The wells on all plates that were treated by one of the 16 bottles
made for step 1.
PK Enzyme concentration
* Buffer Type * pH *
Iteration
Magnesium ion
concentration
The wells on all plates that had unique bottle setups for Step 2 and
Magnesium concentration. Across both iterations, there are 4 EU.
Magnesium concentration
* Iteration
Nested packaging
of assay
Changed on a per plate basis. Any plate could have gotten either
setting. Across both iterations, there are 12 EU.
Plate * Iteration
PK Enzyme
concentration *
Magnesium ion
concentration
The intersection of the EU for PK Enzyme and the EU for
Magnesium ion concentration. 16 x 4 = 64 EU across
both iterations.
PK Enzyme concentration
* Buffer Type * pH *
Magnesium concentration
* Iteration
Buffer Type *
Magnesium ion
concentration
The intersection of the EU for Buffer Type and the EU for
Magnesium ion concentration. 16 x 4 = 64 EU across
both iterations
PK Enzyme concentration
* Buffer Type * pH *
Magnesium concentration
* Iteration
Buffer Type *
Nested packaging
of assay
The intersection of the EU for Buffer Type and the EU for
Nested packaging of assay. 16 EU x 12 EU = 192 EU
across both iterations.
PK Enzyme concentration
* Buffer Type * pH * plate *
Iteration
Source (Partial List of 85) REML Prob > F
ANOVA Prob > F
Buffer Type <.0001 <.0001
PK Enzyme concentration*Buffer Type*Nested Packaging
of assay
<.0001
<.0001
pH <.0001 <.0001
Buffer Type*pH *PK Substrate [Nested Packaging of assay] <.0001 0.0186
Buffer Type*PK Enzyme concentration <.0001 0.0111
pH*PK Substrate[Nested Packaging of assay] <.0001 0.2577
PK Enzyme concentration <.0001 0.0377
PK Enzyme concentration*Timing*Nested Packaging of assay <.0001 0.0432
Magnesium ion concentration*pH <.0001 0.0651
pH*Buffer Type <.0001 0.0669
pH*Timing*Nested Packaging of assay 0.0004 0.1765
Buffer Type*Magnesium ion concentration*PK Substrate
[Nested Packaging of assay
0.0006 0.2379
The inclusion of two iterations is critical
instead of the experimental unit to
the experimental unit for this model
Step 1All 6 plates
2550
2600
2650
2700
2750
2800
2550
2600
2650
2700
2750
2800
PK Enzyme
Tris
HEPES
10 20 30 40 50 60 70 80 90 100 110
10
100
Tris HEPES
Buffer Type
FIG 1
FIG 2
Copyright © 2008 Predictum Inc.
PLAN•insights sought
(specify model: main effects, interactions, quadratics)
•responses & goals (maximize, minimize, match target)
•factors & levels•identify difficult to
change factors•constraints on factors
& levels•design experiment
(evaluate properties)•detail expectations•operational
definitions of run changes and response measurement
•list all assumptions
STUDY•what happened that
was expected?•what did not happen
that was expected?•what happened that
was not expected?
ACT•what was learned,
that if proven valid, can be implemented?
•new questions/insights sought?
•what next?
know
ledg
e
time