never miss a discovery -...

1© 2018 Riffyn Inc

Never Miss a Discovery ®

PREP MEDIA

THAW CELLS INOCULATE ADD TREATMENT INCUBATE IMMUNO

ASSAY

MEASURE CELL COUNT

0.19540.2122

0.2397

0.1823 10.589

10.897

2.6349

10.732

2.57322.7589

11.344

2.6349

2© 2018 Riffyn Inc

“Data that goes into a database should obey what I call the CAP principle. It should be complete, it should be accurate, and it should be permanent, so you never have to do it again.

Otherwise, there’s no progress.”

— Sydney Brenner, Nobel Laureate, 2004

3© 2018 Riffyn Inc

* Error rate = R&D results that fail to reproduce, leading to wasted resources, failed tech transfers and missed discoveries.

Life Sciences

Chemicals/Materials

Energy

Food/AG

Mining and Metals

Pulp, Paper, Textiles

Water/Wastewater

Other

0 50 100 150 200 250 300

Sources:2013 R&D Magazine Global Funding ForecastNSF Science and Engineering Indicators 2012

CIA World Fact BookPrinz, et al., Believe it or not: how much can we rely on published data on potential drug targets?, Nat. Rev. Drug Disc., 2011

Begley & Ellis, Raise Standards for Preclinical Cancer Research, Nature, 483, 2012Ioannidis & Khoury, Improving Validation Practices in “Omics” Research, Science, 334, 2011

Halford, The Second Annual State of Translational Research, Sigma-Aldrich / AAAS, 2014Freedman, et al., The Economics of Reproducibility in Preclinical Research, PLOS Biology, 2015

R&D Spend

$420BR&D losses

$105B>25%error rate*

Annual R&D Spend ($B)

The problem in science today: >$100B of lost R&D productivity each year

4© 2018 Riffyn Inc

“It often takes us 2-3 months to assemble the data from a single development batch”

- senior scientist global pharmaceutical company

5© 2018 Riffyn Inc

Researchers spend 80% of their time cleaning and organizing data, instead of learning from it

20%learning

80%cleaning

6© 2018 Riffyn Inc

Wrangler

7© 2018 Riffyn Inc

Data Wrangler

8© 2018 Riffyn Inc

“Assessing the impact of batch variations on product quality is quite challenging.”

“We want integrated data for rapid correlative analysis across workflow &sample genealogy.”

“We are trying to escape Excel hell.”

“Our existing systems are too inflexible to support our changing processes.”

Statements of pain in R&D

Animal DMPK studies

Biologics Bioprocessing

Screening / Fermentation

Antibody Bio-panning

R&D data issues• quality• access• integration• interpretation• system flexibility

9© 2018 Riffyn Inc

But we, as a society, are failing to

to teach the principles,develop the tools, &

build the culture

of quality in R&D

Clean data starts with quality experiments

10© 2018 Riffyn Inc

Examples from the front lines of scienceThe bad, the ugly and the hopeful


Examples selected from stages of the typical biotech development lifecycle

animal DMPK studies

cell line screening

bioprocess process

development

downstream process

development

analytical chemistry


Example 2:“Regression to the mean” – screening hits don’t hold up over time

17-RBOS-002117-RSC-0012 Signal Peptide Robustness Testing

17-RSC-0013 Strain Testing17-RSC-007 ExoproteasesELN-17-RBOS-0014 1-190

ELN-17-RBOS-0014 191-380ELN-17-RBOS-0016ELN-17-RBOS-0017

ELN-17-RBOS-0020_AELN-17-RBOS-0020_C

ELN-17-RSC-0008ELN-17-RSC-0009 4994

ELN-17-RSC-0010test

TEST

6

8

10

12

14

16

<0A-

> H

PLC

Ana

lysi

s | o

| Fe

rmen

tatio

n |

etha

nol |

con

cent

ratio

n

Bridgeport/AMP Redfield Energy Redfield Energy/LpH

Valero Valero/AMP

<0A-> Prepare corn mash | i | corn mash | resource name

Collected aggregate experiment data set from Riffyn. Identified common controls

Assessed media lot-to-lot effects on control strains

Model fit to control for lot-effects and temperature effects. Most of variation in product titer attributable to these 2 factors, not the candidate strains

Loteffect Temperatureeffect

1 2

3

CONCLUSION: Candidate strains thought to be hits were not hits (indistinguishable from parent).

4

Baddata?(Excludedfromfurtheranalysis)

9.510.010.511.011.512.012.513.013.514.014.5

<0A-

> H

PLC

Ana

lysi

s | o

| Fe

rmen

tatio

n | e

than

ol |

conc

entra

tion

Leve

rage

Res

idua

ls

11.0 11.5 12.0 12.5 13.0 13.5 14.0<0A-> Prepare corn mash | i | corn mash |

resource name Leverage, P<.0001

10.0

10.5

11.0

11.5

12.0

12.5

13.0

13.5

14.0

14.5<0

A->

HPL

C A

naly

sis

| o |

Ferm

enta

tion

| eth

anol

| co

ncen

tratio

n Le

vera

ge R

esid

uals

31.0 31.5 32.0 32.5 33.0 33.5 34.0 34.5 35.0 35.5 36.0<0A-> Fermentation | i | incubator |

temperature Leverage, P<.0001

StrainExperiment 1Experiment 2Experiment 3...

Experiment 14

Lot 1 Lot 2 Lot 3 Lot 4 Lot 5

Lot

ProductTiter

Temp

ProductTiter

Strain 1Strain 2Strain 3...

Reference Strain 1

.

.

.

Reference Strain 2....Strain N

# Samples


Site 1 Site 2

MEAN

MEAN

Test

ing

resu

lt

Testing Day 1

Testing Day 3

Testing Day 2

Example 3:Pre-clinical testing results don’t match across independent sites

Same compound and protocol across 2 R&D sites

animal DMPK studies


2 L 2 0 0 , 0 0 0 L

YIEL

D

BEFORE

PD MFG

XX% yield drop$X lost product

12 months transfer time

0.5 L

Example 4:Fermentation scale-up—troublesome tech transfers to full-scale manufacturing

bioprocess process

development

analytical chemistry


Example 4:Fermentation process development—initial state

Time(h)

Yield

HIGH NOISE -> Unknowable process behavior

proc

ess

CV

TypicalPDBatchBEFOREQUALITY

%RecoveryofStand

ard

2.5%CV

98%accuracy

Quality improvement campaign

Original product assay

%RecoveryofStand

ard

0.3%CV

99.9%accuracy

Redeveloped product assay


Example 4: Fermentation process development—after quality campaign

Time(h)

Yield

HIGH NOISE -> Unknowable process behavior

proc

ess

CV

TypicalPDBatchBEFOREQUALITY

Time(h)

Yield

LOW NOISE -> Predictive process model

proc

ess

CV

TypicalPDBatchAFTERQUALITY


2 L 2 0 0 , 0 0 0 L

YIEL

D

BEFORE

PD MFG

XX% yield drop$X lost product

12 months transfer time

0.5 L

Exact performance matchacross scales

3-4 months transfer time

2 L 2 0 0 , 0 0 0 L

YIEL

D

AFTER

PD MFG

0.5 L

Example 4: Fermentation scale-up: before & after quality improvement and data integration


Example 5:High-throughput strain screening artifacts and unreproducible hits

96 well plate data

Assa

y va

lue

Microtiter plate #

Root cause


6X reduction in screening error doubles rate of strain improvement

1X R&D pace

2007 2008 2009 2010

2X R&D pace

6X error reduction

Year

Source: Gardner, TS (2013) Trends in Biotech. 31:3, 123-125.

BEFORE VARIANCE REDUCTION

AFTER VARIANCE REDUCTION

30% relative error5% relative error

Prod

uct y

ield

of a

str

ain

2X


Learning from historyReduced decision-making error delivers faster progress

Improved data quality and integration revolutionizes the auto industry

Total Quality Implementation

Toyota

Nissan

Average for GM, Ford, and Chrysler


Lessons of manufacturing quality

Define

Measure

Analyze / Improve

Control

DMAIC (6-sigma)(Mfg)

Design

Measure

Analyze / Improve

Share

Iterate

TRANSLATION TO R&D(R&D)


But we, as a society, are failing to

to teach the principles,develop the tools, &

build the culture

of quality in R&D

Clean data starts with quality experiments


We need to recognize our experiments as measurement systems

and teach our future scientists accordingly


Never Miss a Discovery™

Riffyn’s mission is to help scientists deliver research we can trust and build upon efficiently.

never miss a discovery -...

Documents