taverna and myexperiment. scape presentation at a hack-a-thon
Post on 09-May-2015
308 Views
Preview:
DESCRIPTION
TRANSCRIPT
SCAPE
Alexandra Nenadic, David Withers University of Manchester
Practical Tools for Digital Preservation: A Hack-a-thon York, 28th September 2011
Taverna and myExperiment Tools for creating and sharing workflows
Taverna
SCAPE What is a workflow?
• Connecting a set of tools/services to create automated and repeatable processing/analysis
SCAPE
Design and run workflows
Taverna
SCAPE Taverna Features - Overview
• Access to remote, distributed or local services and resources
• Enables service interoperability and integration
• Automates data flow between services
• Implicit iteration over data sets, list handling and control links to determine order of service invocation
• Extensible • Large external developer base
• Various third party plugins available
• Data and provenance collection
SCAPE Taverna Workbench
• Graphical desktop tool
• Drag-and-drop services into diagram
• Connect services, run, reconnect, rerun
• Integrates diverse set of tools
SCAPE Workflow Design
Available services
Tree view of the workflow
structure
Workflow diagram
SCAPE Taverna Workflows – Features in Detail
• A set of (local and remote) services to analyze or manage data
• Data-links connects services • i.e. output from service A is input to
service B and C • Describes the desired dataflow
instead of process coordination • Nested workflows are also services • Automatic iterations • Parallelization • Can customize list handling and
control links • Fault tolerance
• Retry (with delay and back off) • Failover (alternate services)
SCAPE Supported Services
• SOAP/WSDL Web services • REST Web services • SoapLab Web services • R statistical services • Inline Beanshell scripts • External tools and scripts (via ssh or localhost) • Spreadsheet import • XPath and text manipulation services • SADI semantic Web services • Nested workflows (workflow within workflow) • BioMoby • BioMart • … your tool (write your own Taverna plugin)
SCAPE Workflow Results
Input data and results per port
Progress report
Previous runs
SCAPE Workflow Provenance
• Information about a workflow run • What happened?
• And when?
• Lineage tracing • Which input produced which output
• Intermediate data • Inputs and outputs for each workflow step
• Useful for debugging
• Saved in standard format (such as OPM)
SCAPE Taverna is Domain-Independent
• Bioinformatics
• Biomedicine
• Chemistry
Pharmacogenomics Association study of Nevirapine-induced skin rash in Thai Population
HIV and TB research in South Africa Sleeping Sickness in African Cattle
Systems Biology for Crop research, BioDiversity
SCAPE Taverna is Domain-Independent
• Astronomy
• Data and text mining
• Digital content preservation (IMPACT)
• Social simulations
Astronomy & HelioPhysics
Observing Systems Simulation Experiments JPL, NASA
Library Document Preservation British Library
SCAPE
Share, discover and reuse workflows
SCAPE myExperiment
• http://www.myexperiment.org
• Social networking for people to share workflows and collaborate
• Makes it easy for people to contribute to a pool of workflows, build communities and form relationships
• Enables people to share, describe, reuse and repurpose workflows, reduce time-to-production, share expertise and avoid reinvention
SCAPE myExperiment
SCAPE Workflow Sharing, Ownership and Attribution
• myExperiment can provide a central location for workflows from one community/group
• myExperiment allows you to say
– Who can look at your workflow
– Who can download your workflow
– Who can modify your workflow
– Who can run your workflow
• Workflow ownership and attribution
• Users do not need to start from scratch – reuse or modify existing workflows
• Attribute/credit original author
SCAPE Use myExperiment from Taverna
SCAPE Training
• Tutorials and Training • 58+ tutorials to >900 people • >20 Universities, institutes
and networks • Major conferences • Summer schools
• Developer and User Days • Annotation Jamborees
• Undergraduate and Postgraduate Bioinformatics in > 30 Universities
SCAPE Taverna and SCAPE
• SCAPE preservation components/actions as services in Taverna workflows
• Use Taverna Workbench to create and test SCAPE preservation workflows on local data
• Then scale-up and run the workflows on a parallelized platform using Hadoop MapReduce
• Share Taverna SCAPE workflows on myExperiment
top related