expeditionworkshop08192008.ppt

33
Grid Rapid Application Virtualization Interface (gRAVI) - Service Oriented Science Ravi K Madduri, Argonne National Laboratory/ University of Chicago Joshua Boverhof, Lawrence Berkeley Laboratory Kyle Chard, University of Wellington, NZ Dinanath Sulakhe, University of Chicago Cem Onyukusel, CMU Ian Foster, University of Chicago/ANL

Upload: cameroon45

Post on 11-May-2015

286 views

Category:

Technology


0 download

TRANSCRIPT

Page 1: ExpeditionWorkshop08192008.ppt

Grid Rapid Application Virtualization Interface

(gRAVI) - Service Oriented Science

Ravi K Madduri, Argonne National Laboratory/ University of Chicago

Joshua Boverhof, Lawrence Berkeley Laboratory

Kyle Chard, University of Wellington, NZ

Dinanath Sulakhe, University of Chicago

Cem Onyukusel, CMU

Ian Foster, University of Chicago/ANL

Page 2: ExpeditionWorkshop08192008.ppt

Agenda What is Service Oriented Science ?

Software as a service Examples from industry (google, salesforce.com etc) Examples of SoS from Scientific communities

Why are we excited about this ? How this helps Science

Sum of parts greater Sustainable infrastructure Virtualization

Different aspects of SoS

Authoring

Deploying and Securing

Discovery

Provisioning on demand

Composition

How does it help you ? End users perspective – Brian’s talk Bio workflow

Conclusions and Further resources

Q & A

2

Page 3: ExpeditionWorkshop08192008.ppt

Examples of SaaS from Industry

Salesforce.com Google Calendar, Google docs, Google

spell check etc.. Amazon S3, EC2 Flickr Facebook Microsoft Live And on and on….

3

Page 4: ExpeditionWorkshop08192008.ppt

Service-Oriented Science

People create services (data or functions) … which I discover (& decide whether to use) … & compose to create a new function ... & then publish as a new service.

I find “someone else” to host services, so I don’t have to become an expert in operating services & computers!

I hope that this “someone else” can manage security, reliability, scalability, …!!

“Service-Oriented Science”, Science, 2005 4

Page 5: ExpeditionWorkshop08192008.ppt

Creating Services

People create services (data, code, instr.) …

which I discover (& decide whether to use) …

& compose to create a new function ...

& then publish as a new service.

I find “someone else” to host services, so I don’t have to become an expert in operating services & computers!

I hope that this “someone else” can manage security, reliability, scalability, …

!!“Service-Oriented Science”, Science, 2005 5

Page 6: ExpeditionWorkshop08192008.ppt

Creating Services

Introduce + gRAVIShannon Hastings

Scott OsterDavid Ervin

Stephen Langella

Joshua BoverhofKyle Chard

Ravi Madduri

Page 7: ExpeditionWorkshop08192008.ppt

Introduce Overview A framework which enables fast and easy creation of Globus

based grid services Provide easy to use graphical service authoring tool. Hide all “grid-ness” from the developer Utilize best practice layered grid service architecture Integration with other core grid services and architecture

components GAARDS Security Infrastructure (Dorian, GridGrouper, CSM) Globus Index Service Global Model Exchange (GME) Cancer Data Standards Repository

Extension Framework for integrating with other architecture components

7

Page 8: ExpeditionWorkshop08192008.ppt

Inside the Introduce created service

Services have many moving and configurable parts which support features such as: Advertisement Discovery Invocation Security (Authentication/Authorization) Stateful Resources

The Introduce Toolkit can keep all these features in sync as the developer creates and modifies the grid service

8

Page 9: ExpeditionWorkshop08192008.ppt

Introduce Features Supports modification of operations

Adding operations Removing Operations Updating Operations Importing Operations

Graphical Configuration Advertisement Security Service Metadata Specification Service Metadata Editing Service Configuration Properties

Auto Generates Code for Service Auto generates a client API for service. Graphical Deployment of Service

Globus Tomcat JBoss

9

Page 10: ExpeditionWorkshop08192008.ppt

ApplnService

Create

Index service

Store

Repository ServiceAdvertize

Discover

Invoke;get results

Introduce

Container

Transfer GAR

Deploy

gRAVI

Grid Remote Application Virtualization Interface

Builds on Introduce Define service Create skeleton Discover types Add operations Configure security

Wrap arbitrary executables

10

Page 11: ExpeditionWorkshop08192008.ppt

Discovering Services

People create services (data or functions) …

which I discover (& decide whether to use) …

& compose to create a new function ...

& then publish as a new service.

I find “someone else” to host services, so I don’t have to become an expert in operating services & computers!

I hope that this “someone else” can manage security, reliability, scalability, …

!!“Service-Oriented Science”, Science, 2005 11

Page 12: ExpeditionWorkshop08192008.ppt

Discovering ServicesWS-MDS+ Taverna

Laura Pearlman

Mike Darcy

myGrid Team

Page 13: ExpeditionWorkshop08192008.ppt

The ultimate arbiter?

Types, ontologies

Can I use it?

Billions of servicesDiscovering Services

Assume success

Semantics

Permissions

Reputation

A B13

Page 14: ExpeditionWorkshop08192008.ppt

Discovery (1):Registries

Globus

Page 15: ExpeditionWorkshop08192008.ppt

Composing Services

People create services (data or functions) …

which I discover (& decide whether to use) …

& compose to create a new function ...

& then publish as a new service.

I find “someone else” to host services, so I don’t have to become an expert in operating services & computers!

I hope that this “someone else” can manage security, reliability, scalability, …

!!“Service-Oriented Science”, Science, 2005 15

Page 16: ExpeditionWorkshop08192008.ppt

Taverna

caGrid Scavenger with semantic/metadata based caGrid service query

A sample caGrid workflow

16

Page 17: ExpeditionWorkshop08192008.ppt

Hosting Services

People create services (data or functions) …

which I discover (& decide whether to use) …

& compose to create a new function ...

& then publish as a new service.

I find “someone else” to host services, so I don’t have to become an expert in operating services & computers!

I hope that this “someone else” can manage security, reliability, scalability, …

!!“Service-Oriented Science”, Science, 2005 17

Page 18: ExpeditionWorkshop08192008.ppt

Provisioning ServicesWS-GRAM + VWS

Martin FellerStuart Martin

Kate KeaheyTim Freeman

Joshua Boverhof

Page 19: ExpeditionWorkshop08192008.ppt

Provisioning using WS-GRAM

gRAVI uses JSDL for Application Description Generates a method on the generated service

called <appName>GRAM Generates implementation that creates a

GramJob from Application Description Uses the bootstrapped community credential

to run the application as a grid job Used widely in realizing the usecases from

caBIG community

19

Page 20: ExpeditionWorkshop08192008.ppt

Repository Service

Using VWS/Cloud Computing

gRAVI service Wrap the application as

Service Create the GAR and put in

repository Provision resources

Use clouds (VMM) start service

Transfer gar, deploy Index Service

Grid service registers itself

nimbus stratus

Index service

Transfer,deploy

Registerservice

Create VM

provision

discovery

portal GAR

20

Page 21: ExpeditionWorkshop08192008.ppt

Cloud Computing

21

Page 22: ExpeditionWorkshop08192008.ppt

SoS Deployments

Page 23: ExpeditionWorkshop08192008.ppt

cancer Biomedical Informatics Grid (caBIG™)

A national, NCI-funded program with participants from cancer centers and research institutions Improve cancer research and accelerate efforts to find a cure for

cancer Make cancer research data and tools (maintained at different sites)

more efficiently accessible to researchers Create scalable, federated, actively managed biomedical

informatics network that will connect members of the cancer research enterprise

Informatics support for basic, clinical, and translational research “National Standards, Local Management”*

caBIG community 50+ Cancer Centers, 30+ Organizations, over 900 participants

23

Page 24: ExpeditionWorkshop08192008.ppt

Example Scenario

Location AMicroarray, Protein, Image data

Location BMicroarray, Protein, Image data

Location CMicroarray, Protein, Image data

Location CImage Analysis

Location DImage Analysis

Microarray and protein databases at other institutions

caGrid Service Interfaces

caGrid Environment

Registered Object Definitions

Advertisement

Log on, Grid credentials

Query and Analysis Workflow

Discovery

24

Page 25: ExpeditionWorkshop08192008.ppt

Users Experience Early Adopters

Love it ! 45 services from 50+ cancer centers (and growing)

Some services are lot more popular than others Bio-informatics group at Argonne – Details in subsequent

slides APS plans to pursue this model – Brian Tieman will talk more

about this End users

caBIG hack-a-thon participants were happy about workflows Cancer Centers

Not too happy right now Would like to see some RoI

25

Page 26: ExpeditionWorkshop08192008.ppt

Integration of caBIG-geWorkbench-Teragrid

26

Page 27: ExpeditionWorkshop08192008.ppt

caGrid Security (GTS, Grid Grouper, Dorian, CDS)

http://www.cagrid.org/mwiki/index.php?title=GAARDS:Main

27

Page 28: ExpeditionWorkshop08192008.ppt

Transposon Workflow Details Transposons are small mutant sequences of

DNA that move between positions within the genome of a cell

Gene functions can be identified by inserting mutant sequences into a genome and analysing where the sequence resides (between genes or on a gene) and how it affects its phenotype

BLAST is used to compare the genome after insertion against the original genome This identifies where the transposon resides

28

Page 29: ExpeditionWorkshop08192008.ppt

Realizing the workflow

Format DB

BLAST

Create Report

Create a fasta file representing the Genome sequences

Compare these sequences against the original genome

Find Transpos

onFind sequences that have the given transposon

NeighbouringGenes

NeighbouringGenes

BLASTMissesBLASTMisses

BLASTHits

BLASTHits

Compile a report summarising where the transposon was inserted and results of the BLAST search

Loop Until there are no misses or all genomes have been searched

Genome SequencesGenome

Sequences

29

Page 30: ExpeditionWorkshop08192008.ppt

30

Page 31: ExpeditionWorkshop08192008.ppt

Center for Enabling Distributed Petascale Science

Workflow Automation at DOE Facilities

AutomationReproducibilitySecurityReusability

StorageMetadataAnalysis

Visualization

Advanced Photon Source

31

Page 32: ExpeditionWorkshop08192008.ppt

Lessons Learned

Nice higher level abstraction Suitable for a subset of scientific computing

usecases Sustainable infrastructure Work well with hospital/cancer center/APS IT

infrastructure Workflows Scalability Provenance No Vendor lock-in (if you are careful)

32

Page 33: ExpeditionWorkshop08192008.ppt

Further Resources

Further Details on gRAVI http://dev.globus.org/wiki/Incubator/gRAVI

Further Details on Introduce www.cagrid.org

Details on Taverna + gRAVI http://dev.globus.org/wiki/Using_gRAVI_Servi

ces_in_Taverna gRAVI users mailing list

[email protected]

33