Model management tools for improved reproducibility in systems biology

Dagmar Waltemath, on behalf of the SEMS team

University of Rostock, Germany

10th International CellML Workshop Auckland, June 2016

2

On models and simulations

Model Simulation

Figs: BioModels (top) and DOI: 10.1073/pnas.88.16.7328 (bottom)

3

Most scientific discoveries rely on previous findings.

Model

Fig.: Tyson 2001 (BIOM195)

Fig.: Tyson 1991 (BIOM005)

Successor

Fig.: History of Cell Cycle models in BioModels

4

Can we rely on findings that we ourselves cannot evaluate? (Probably not!)

“only in ~20–25% of the projects were the relevant published data completely in line with our in-house findings (Fig. 1c). In almost two-thirds of the projects, there were inconsistencies [..] that either considerably prolonged the duration of the target validation process or, in most cases, resulted in termination of the projects because the evidence [..] was insufficient to justify further investments into these projects.” Prinz et al (2011)

5

We identified key challenges of reproducibility insystems biology and systems medicine.

Lack of data standards – Lack of data quality and quantity – Lack of data availability – Lack of transparency

6

A lack of data availability makes it impossible for researchers to reproduce results.

● Model code in BioModels, including supplemental with a how-to reproduce the figures given in the original paper

● Online tool makes data available and browseable

TriplexRNA

Recon 2Recon 2

● Publication backed up with a website containing the supplemental material

● Model code in (non-curated) BioModels● Visualisation of the model can easily

be explored● References to original works

How can we support scientistswho wish to share model-based results?

Issues– Simulation studies comprise

of several files

– Data is heterogeneous, distributed, complex

– Data changes over time

– Documentation of the how the study was performed often missing

7

A lack of data availability makes it impossible for researchers to reproduce results.

How can we support scientistswho wish to share model-based results?

Issues– Simulation studies comprise

of several files

– Data is heterogeneous, distributed, complex

– Data changes over time

– Documentation of the how the study was performed often missing

Our solutions– Tool support for the

COMBINE Archive – lowering the effort to share reproducible models

– Graph-based storage of model-related files – integrated & searchable virtual experiments

– Model version control –towards a provenance of models

8

The COMBINE archive bundles all files necessary to reproduce a simulation study.

COMBINE archive toolkit

● manage COMBINE archives

– Explore

– Edit

– Share

– Publish● Used in: PMR 2, JWS Online,

SED-ML Web Tools, OpenCor …

WebCAT, Scharm et al 2014

9

STON, SED-ML DB & MASYMOS

Integrated storage & retrieval system (MASYMOS)

doi: 10.1093/database/bau130

doi: 10.1186/s13326-015-0014-4

Search across heterogeneous data, ontologies, and structures→poster

Tailor-made storage systems (STON, SED-ML DB)

Using graph databases to integrate standardised model-based data

https://dx.doi.org/10.6084/m9.figshare.3382993.v1

SED-ML DB in JWS Online

BioModelsPhysiome Model repository

10

BiVeS & COMODI

Model version control (BiVeS, COMODI) Provenance-to-be (COMODI)

Tracking the evolution of a CellML/SBML model over time

doi: 10.1093/bioinformatics/btv484

Tracking the evolution of simulation studies and biological systems.

https://dx.doi.org/10.6084/m9.figshare.2543059.v5

Physiome Model repository

doi: 10.1093/bioinformatics/btv484

11

What's next? Models for the clinic, or: Bridging the gap between standards for systems biology & systems medicine

Fig. courtesy Atalag et al (2015) http://hdl.handle.net/2292/27911

Thank you for your attention.

m n @SemsProject

Martin ScharmBiVeS, COMODI, COMBINE Archive Video master

Fabienne LambuschPattern & structure search in SBML models

Mariam NassarRank aggregation

Tom GebhardtSBGN-compliant diffs

Martin PetersM2CAT, COMBINE Archive, SED-ML database

Vasundra ToureSTON, SBGN-ED, SBGN symbol of the month

Ron HenkelMASYMOS, MORRE

www.sems.uni-rostock.de

References

Atalag et al (2015) http://hdl.handle.net/2292/27911

Bergmann et al. (2014) F.T. Bergmann, R. Adams, S. Moodie, J. Cooper, M. Glont et al.: COMBINE archive and OMEX format: one file to share all information to reproduce a modeling project. BMC Bioinformatics (2014)

Prinz et al. (2011) Prinz, Florian, Thomas Schlange, and Khusru Asadullah. "Believe it or not: how much can we rely on published data on potential drug targets?." Nature reviews Drug discovery 10.9 (2011): 712-712.

Schmitz et al. (2014) Schmitz, Ulf, et al. "Cooperative gene regulation by microRNA pairs and their identification using a computational workflow." Nucleic acids research (2014): gku465.

Thiele et al. (2013) Thiele, Ines, et al. "A community-driven global reconstruction of human metabolism." Nature biotechnology 31.5 (2013): 419-425.

Waltemath & Scharm (2014) D. Waltemath and M. Scharm: Extracting reproducible simulation studies from model repositories using the CombineArchive Toolkit. Workshop on Data Management for the Life Sciences (2014), Hamburg, BTW 2014.

Waltemath & Wolkenhauer (2016) D. Waltemath and O. Wolkenhauer: How modeling standards, software, and initiatives support reproducibility in systems biology and systems medicine. IEEE Transactions on Biomedical Engineering (2016) in the press.