Development and deployment ofa cross-platform open-source software package:the Infobiotics Workbench

Jonathan Blakes26/10/2010Supervised by Professor Natalio Krasnogor

Outline

Introduction to InfobioticsOrganisation of software

Supporting technologies

Tour of the Infobiotics Dashboard UIInterface considerations

Deployment on Windows, Mac and Linux

Finishing touches and availability

Conclusions

Infobiotics Workbench

Infobiotics Models (syntax)

Infobiotics Models (semantics)

Infobiotics Experiments

Command line tools with a common parameter file format:simulation (mcss)

model checking (pmodelchecker)

model structure and parameter optimisation (poptimizer)

Desktop applications (not cloud-based)

Written in C++ for speed

Infobiotics Dashboard

User friendly graphical user interface for perform Infobiotics experimentssetting, loading and saving of parameters

running experiments and reporting progress

plotting output

Present options in a clear and understandable manner, setting limits on parameter values

Give feedback when things go wrong!

Written in Python

Why Python?

Dynamic language: rapid prototyping

Large stdlib: tempfile, XML parsers, itertools

3rd party Libraries:NumPy n-dimensional arrays and ufuncs

pytables HDF5 (simulator output)

PyQt Qt GUI bindings (GPL), QScintilla text editor

Matplotlib 2D plotting

Mayavi 3D VTK-based visualisation (TVTK)

Traits...

Traits = Reactive Programming

Simulation Parameters (file)

Simulation Parameters (GUI)

Model checker parameters (1)

MC2

Model checker parameters (2)

Simulator Results

Simulation output amounts of each species in each compartment at each timepoint for each run - quickly become very large (gigabytes), but only some data is of use to the modeller.selecting important information

scaling amounts (molecules) to biologist-friendly units (e.g. nanomolar)

calculating statistics across runs (and other dimensions)

plotting timeseries, exporting data, etc.

SimulatorResultsDialog

Selecting data (filtering)

Filter compartments by name using wildcards

Using set class difference method

Custom ListWidget is a factory for filter LineEdit

Selecting data (sorting)

By providing custom QListWidget with regular expression and function map

Sort species by name

Sort compartments by name and/or positions

Selecting data (sampling)

Reducing data size by sampling reduces memory requirements, speeds up calculations and unclutters plots100 of 10,000 runs

Every 100th recorded timepoint

Transforming data (scaling)

Converts time, substance quantities and volumes from simulator data units into user-friendly display units, using python-quantities

For calculating concentrations, use compartment volumes (at each timepoint) if recorded, otherwise use one volume for all compartments at all timepoints

Exporting data

Selected data can be exported in several common formats for use with other tools:text (.csv)

Excel (.xls)

NumPy (.npz)

Plotting timeseries (old)

Plotting timeseries (improved)

Exporting figures

Added ability to save figures resized to specific screen dimensions (pixels), as surprisingly, matplotlib only supports physical dimensions (inches)

Model checker results

Visualising simulations (Mayavi)

Cross-platform issues

A lot of communication in Workbench uses temporary files these behave differently on Windows to Linuxmust close temp files before use, which can delete them!

Pseudo-terminal emulation library pexpect used to gather progress information not for Windowswinpexpect port not functioning at time of release

fall back to 'working' progress indicatornot good for long experiments

Deploying Python apps

A lot of dependencies inc. Python

Python comes with most Linux distributions, but not with Windows (unlike JVM) and Mac system Python is crippled

Therefore we needed to bundle Python interpreter and all libraries for Windows/Mac

Possible solution: 'freezing' Python applications, creates a single executable (~100 megabytes!)

Need a solid base to start from and add missing libraries

also python(x,y)for Windows

Freezing Python applications

Freezing works well for small scripts, not so well for applications with many librariesuses static code analysis for determine libraries for bundling TraitsUI/Matplotlib use dynamic imports

Freezers: cx_freeze, bbfreeze, PyInstaller,

Wrap python setup.py py2exe/py2app with bash scriptscopy missed files, unzip zipped Python classes

patch Matplotlib

fix Mac library paths using install_name_tool

Windows and Mac non-Python

mcss, pmodelchecker and poptimizer need cross-compiling for 32/64-bit Linux (chroot), Windows (32-bit) and Mac (OSX Snow Leopard only)

pmodelchecker requires PRISM and MC2 which are both Java applications

InstallJammer builds cross-platform installer:

Linux (Debian)

Not trivial either no authorative packaging guideMake packages for mcss, ..., PRISM, MC2, libhdf5-1.8

Used python-support to install Dashboard for system Python(s)

(1) add apt repository to sources.list

(2) sudo apt-get install infobiotics-workbench

Automatic updates!

Fedora/Suse packages made using Alien

Documentation (User Guide)

Documentation (User Guide)

Produced using Sphinx:input is reStructuredText

output is high-quality formatted HTML, PDF,

Used Enthought reST editor for writing

Documentation (API)

HTML API generated from docstrings using Endo

Alternatives:Epydoc

Doxygen

Code availability

Soliciting user feedback

getsatisfaction.com Javascript added to Sphinx template

Conclusions

Cross-platform application development in Python is easy, deployment is hard

Need to become an expert in knowing the differences between platforms

50 / 50 development vs. the rest

Good to be part of a team :)

Publications

Acknowledgements

Jamie Twycross (mcss, packaging)

Fran Romero Campero (IML, pmodelchecker, docs)

Claudio Lima (poptimizer, docs)

Pawel (for turning me on to Python)

Natalio Krasnogor

Questions?

Slides available on web

Infobiotics

Muokkaa otsikon tekstimuotoa napsauttamalla

Muokkaa jsennyksen tekstimuotoa napsauttamallaToinen jsennystasoKolmas jsennystasoNeljs jsennystasoViides jsennystasoKuudes jsennystasoSeitsems jsennystasoKahdeksas jsennystasoYhdekss jsennystaso