PREDICTUM INC. WWW.PREDICTUM.COM ©2014

DOE challenges & opportunitiesWAYNE J LEVIN PREDICTUM INC. LEVIN@PREDICTUM.COM !

WWW.PREDICTUM.COM

1

agenda

About Predictum Use the right methods (and there are more of them) Build a system Assume nothing

2

About Predictum

we increase productivity more exploitable insights in less time, effort, cost less frustration

3

DOE purpose

look at numerous effects comprehensively yet isolate each effect’s influence independently of the other effects

in less time, effort and cost

4

use latest & greatest methods

Definitive Screening Designs latest in computer-generated optimal designs

Split-plot designs hard to change factors

5

definitive screening

more independence of effects

6

Some correlation among main effects and 2-way interactions

Definitive screening - 21 runsOptimal Design -20 runs

Zero correlation among main effects and 2-way interactions

7

Sacrifice some D-efficiency, but not much

Definitive screeningOptimal Design

8

split-plot designs

it’s a multivariate universe

9

The concept of hard to change is broader than you might think. !Here multiple pipetting constitutes hard to change where materials are the same across wells

10

ScenarioThe tool is typical of HTS, in that there is limited ways a chemical can be varied within a plate.

11 12

generating errors

Under REML highlighted items show type I errors if analyzed traditionally !Under Traditional highlighted items show type II errors if analyzed traditionally

13

generating errors

DOE can be a lot of work take the time, make the effort longer more complex experiment designed right often yields more correct insights than a series of small experiments

if not done correctly, DOEs will generate type I and type II errors confusion and frustration

14

build a system

15Copyright © 2013 Predictum Inc. Confidential

time

R&D and Improvement Initiatives are launched and completed in isolation.

16

Copyright © 2013 Predictum Inc. Confidential

time

What if they were connected? What if they started with all relevant information previously acquired and paid for?

17Copyright © 2013 Predictum Inc. Confidential

time

This is typically what happens. Insufficient institutional memory. Each project is isolated. !

Slippage on retaining acquired, prior insights. Must pay to re-acquire what was previously known.

18

assume nothing

be methodical

19

Step 8 Wait Several DaysAll 6 plates Do steps 1-7 again

Step 7 Read all 6 plates

Step 6

Step 5

Step 4

Step 3All 6 plates

Step 2All 6 plates

BSA

concentration

solution

0.2%

water

PK Substrate

PK Substrate

combined with

Kinase Glo

400nM

40nM

400nM

40nM

Timing 5 min

10 min

Kinase Glo 4 uL

nothing

Plate

1 2 3

4 5 6

Plate

1 2 3

4 5 6

Plate

1 2 3

4 5 6

Nozzle

12

345678

Magnesium

solution

5 uM

water

Column within plate

1 2 3 4 5 6 7 8

1 2 3 4 5 6 7 8

Utilizing Design of Experiment Statistical Models to Improve Assay Developmentin High Throughput Biology

Diana Ballard of Predictum Inc., Samuel Hasson of NIH, Wayne Levin of Predictum Inc.

Overview

Introduction

handled in classical full and fractional factorial designs leading them to provide

MethodsOBJECTIVES:

FACTORS: All eight of the following factors are difficult to vary on the Thermo Multi Drop

RESPONSE:

DESIGN NOTES:

Nozzle Buffer

Type

pH PK Enzyme

concentration

1 HEPES 8 100 uM

2 Tris 7 10 uM

3 Tris 8 10 uM

4 HEPES 7 10 uM

5 Tris 8 100 uM

6 HEPES 8 10 uM

7 Tris 7 100 uM

8 HEPES 7 100 uM

similar tests assume that the entire experiment

Conclusions

experimental design and we found a straightforward method to implement its use in DOE

Visualize

Buffer Type *

Magnesium *

PK Substrate

[Nested

Packaging

of Assay]

Factor / Effect Experimental unit (EU) Error Term in the model PK Enzyme

concentration

The wells on all plates that had unique bottle setups for PK Enzyme.

Step 1 had 8 unique bottles, repeating step 1 in iteration 2. Across

both iterations, there are 16 EU.

PK Enzyme concentration

* Buffer Type * pH *

Iteration

Buffer Type The wells on all plates that were treated by one of the 16 bottles

made for step 1.

PK Enzyme concentration

* Buffer Type * pH *

Iteration

Magnesium ion

concentration

The wells on all plates that had unique bottle setups for Step 2 and

Magnesium concentration. Across both iterations, there are 4 EU.

Magnesium concentration

* Iteration

Nested packaging

of assay

Changed on a per plate basis. Any plate could have gotten either

setting. Across both iterations, there are 12 EU.

Plate * Iteration

PK Enzyme

concentration *

Magnesium ion

concentration

The intersection of the EU for PK Enzyme and the EU for

Magnesium ion concentration. 16 x 4 = 64 EU across

both iterations.

PK Enzyme concentration

* Buffer Type * pH *

Magnesium concentration

* Iteration

Buffer Type *

Magnesium ion

concentration

The intersection of the EU for Buffer Type and the EU for

Magnesium ion concentration. 16 x 4 = 64 EU across

both iterations

PK Enzyme concentration

* Buffer Type * pH *

Magnesium concentration

* Iteration

Buffer Type *

Nested packaging

of assay

The intersection of the EU for Buffer Type and the EU for

Nested packaging of assay. 16 EU x 12 EU = 192 EU

across both iterations.

PK Enzyme concentration

* Buffer Type * pH * plate *

Iteration

Source (Partial List of 85) REML Prob > F

ANOVA Prob > F

Buffer Type <.0001 <.0001

PK Enzyme concentration*Buffer Type*Nested Packaging

of assay

<.0001

<.0001

pH <.0001 <.0001

Buffer Type*pH *PK Substrate [Nested Packaging of assay] <.0001 0.0186

Buffer Type*PK Enzyme concentration <.0001 0.0111

pH*PK Substrate[Nested Packaging of assay] <.0001 0.2577

PK Enzyme concentration <.0001 0.0377

PK Enzyme concentration*Timing*Nested Packaging of assay <.0001 0.0432

Magnesium ion concentration*pH <.0001 0.0651

pH*Buffer Type <.0001 0.0669

pH*Timing*Nested Packaging of assay 0.0004 0.1765

Buffer Type*Magnesium ion concentration*PK Substrate

[Nested Packaging of assay

0.0006 0.2379

The inclusion of two iterations is critical

instead of the experimental unit to

the experimental unit for this model

Step 1All 6 plates

2550

2600

2650

2700

2750

2800

2550

2600

2650

2700

2750

2800

PK Enzyme

Tris

HEPES

10 20 30 40 50 60 70 80 90 100 110

10

100

Tris HEPES

Buffer Type

FIG 1

FIG 2

Copyright © 2008 Predictum Inc.

PLAN•insights sought

(specify model: main effects, interactions, quadratics)

•responses & goals (maximize, minimize, match target)

•factors & levels•identify difficult to

change factors•constraints on factors

& levels•design experiment

(evaluate properties)•detail expectations•operational

definitions of run changes and response measurement

•list all assumptions

STUDY•what happened that

was expected?•what did not happen

that was expected?•what happened that

was not expected?

ACT•what was learned,

that if proven valid, can be implemented?

•new questions/insights sought?

•what next?

know

ledg

e

time