servo grid: solid earth research virtual observatory grid/web services and portals supporting...

37
Solid Earth Research Virtual Solid Earth Research Virtual Observatory Observatory Grid/Web Services and Grid/Web Services and Portals Supporting Portals Supporting Earthquake Science Earthquake Science August 26 2004 August 26 2004 Beijing China Beijing China Geoffrey Fox, Marlon Pierce Geoffrey Fox, Marlon Pierce Community Grids Lab, Community Grids Lab, Pervasive Technologies Laboratories Pervasive Technologies Laboratories Indiana University Indiana University Current SERVOGrid is USA Project led by JPL (Jet Propulsion Laboratory) but next is iSERVO with International Collaboration between Australia, China, Japan and USA

Upload: camilla-henry

Post on 05-Jan-2016

219 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: SERVO Grid: Solid Earth Research Virtual Observatory Grid/Web Services and Portals Supporting Earthquake Science August 26 2004 Beijing China Geoffrey

SERVO Grid:SERVO Grid: Solid Earth Research Virtual ObservatorySolid Earth Research Virtual Observatory

Grid/Web Services and Portals Grid/Web Services and Portals Supporting Earthquake ScienceSupporting Earthquake Science

August 26 2004 August 26 2004 Beijing China Beijing China

Geoffrey Fox, Marlon PierceGeoffrey Fox, Marlon PierceCommunity Grids Lab,Community Grids Lab,

Pervasive Technologies LaboratoriesPervasive Technologies LaboratoriesIndiana UniversityIndiana University

Current SERVOGrid is USA Project led by JPL (Jet Propulsion Laboratory) but next is iSERVO with International Collaboration between Australia, China, Japan and USA

Page 2: SERVO Grid: Solid Earth Research Virtual Observatory Grid/Web Services and Portals Supporting Earthquake Science August 26 2004 Beijing China Geoffrey

Solid Earth Science QuestionsSolid Earth Science Questions

From NASA’s Solid Earth Science Working Group Report, Living on a Restless Planet, Nov. 2002

1. What is the nature of deformation at plate boundaries and what are the implications for earthquake hazards?

2. How do tectonics and climate interact to shape the Earth’s surface and create natural hazards?

3. What are the interactions among ice masses, oceans, and the solid Earth and their implications for sea level change?

4. How do magmatic systems evolve and under what conditions do volcanoes erupt?

5. What are the dynamics of the mantle and crust and how does the Earth’s surface respond?

6. What are the dynamics of the Earth’s magnetic field and its interactions with the Earth system?

Page 3: SERVO Grid: Solid Earth Research Virtual Observatory Grid/Web Services and Portals Supporting Earthquake Science August 26 2004 Beijing China Geoffrey

The Solid Earth is:The Solid Earth is:Complex, Nonlinear, and Self-OrganizingComplex, Nonlinear, and Self-Organizing

Relevant questions that Computational Relevant questions that Computational technologies can help answer:technologies can help answer:

1.1. How can the study of strongly correlated solid earth How can the study of strongly correlated solid earth systems be enabled by space-based data sets?systems be enabled by space-based data sets?

2.2. What can numerical simulations reveal about the What can numerical simulations reveal about the physical processes that characterize these systems?physical processes that characterize these systems?

3.3. How do interactions in these systems lead to space-How do interactions in these systems lead to space-time correlations and patterns?time correlations and patterns?

4.4. What are the important feedback loops that mode-lock What are the important feedback loops that mode-lock the system behavior?the system behavior?

5.5. How do processes on a multiplicity of different scales How do processes on a multiplicity of different scales interact to produce the emergent structures that are interact to produce the emergent structures that are observed?observed?

6.6. Do the strong correlations allow the capability to Do the strong correlations allow the capability to forecast the system behavior in any sense?forecast the system behavior in any sense?

Page 4: SERVO Grid: Solid Earth Research Virtual Observatory Grid/Web Services and Portals Supporting Earthquake Science August 26 2004 Beijing China Geoffrey

Characteristics of Computing for Characteristics of Computing for Solid Earth ScienceSolid Earth Science

Widely distributed datasetsWidely distributed datasets in various formats in various formats• GPS, Fault data, Seismic data sets, InSAR satellite dataGPS, Fault data, Seismic data sets, InSAR satellite data• Many available in state of art tar files that can be FTP’dMany available in state of art tar files that can be FTP’d• Provenance problems: faults have controversial parameters Provenance problems: faults have controversial parameters

like slip rates which have to be estimated.like slip rates which have to be estimated. Distributed models and expertiseDistributed models and expertise

• Lots of codes with different regions of validity, ranging from Lots of codes with different regions of validity, ranging from cellular automata to finite element to data mining applications cellular automata to finite element to data mining applications (HMM) (HMM)

• Simplest challenges are just making these codes useable for Simplest challenges are just making these codes useable for other researchers.other researchers.

• And hooking this codes to data sourcesAnd hooking this codes to data sources• Some codes also have export or IP restrictionsSome codes also have export or IP restrictions• Other codes are highly specialized to their deployment Other codes are highly specialized to their deployment

environments.environments. Decomposable problemsDecomposable problems requiring interoperability for requiring interoperability for

linking full modelslinking full models• The fidelity of your fault modeling can vary considerablyThe fidelity of your fault modeling can vary considerably• Link codes (through data) to support multiple scalesLink codes (through data) to support multiple scales

Page 5: SERVO Grid: Solid Earth Research Virtual Observatory Grid/Web Services and Portals Supporting Earthquake Science August 26 2004 Beijing China Geoffrey

SERVOGrid RequirementsSERVOGrid Requirements Seamless AccessSeamless Access to data repositories and computing to data repositories and computing

resourcesresources Integration of multiple data sourcesIntegration of multiple data sources including including

databases, file systems, sensors, …, with databases, file systems, sensors, …, with simulation simulation codes.codes.

Core web servicesCore web services for common tasks like command for common tasks like command execution and file management.execution and file management.

Meta-dataMeta-data generation, archiving, and access with generation, archiving, and access with extending openGIS (Geography as a Web service) extending openGIS (Geography as a Web service) standards.standards.

PortalsPortals with component model (portlets) for user with component model (portlets) for user interfaces and web control of all capabilitiesinterfaces and web control of all capabilities

Basic Grid tools: Basic Grid tools: complex job managementcomplex job management and and notificationnotification

Collaboration Collaboration to support world-wide workto support world-wide work• ““Collaboration” can range from data sharing to Audio-video Collaboration” can range from data sharing to Audio-video

conferencingconferencing

Page 6: SERVO Grid: Solid Earth Research Virtual Observatory Grid/Web Services and Portals Supporting Earthquake Science August 26 2004 Beijing China Geoffrey

SERVOGrid ApplicationsSERVOGrid Applications Codes range from simple “rough estimate” codes to Codes range from simple “rough estimate” codes to

parallel, high performance applications.parallel, high performance applications.• DislocDisloc: handles multiple arbitrarily dipping dislocations : handles multiple arbitrarily dipping dislocations

(faults) in an elastic half-space.(faults) in an elastic half-space.• SimplexSimplex: inverts surface geodetic displacements for fault : inverts surface geodetic displacements for fault

parameters using simulated annealing downhill residual parameters using simulated annealing downhill residual minimization. minimization.

• GeoFESTGeoFEST: Three-dimensional viscoelastic finite element : Three-dimensional viscoelastic finite element model for calculating nodal displacements and tractions. model for calculating nodal displacements and tractions. Allows for realistic fault geometry and characteristics, Allows for realistic fault geometry and characteristics, material properties, and body forces. material properties, and body forces.

• Virtual CaliforniaVirtual California: Program to simulate interactions : Program to simulate interactions between vertical strike-slip faults using an elastic layer over between vertical strike-slip faults using an elastic layer over a viscoelastic half-space a viscoelastic half-space

• RDAHMMRDAHMM: Time series analysis program based on Hidden : Time series analysis program based on Hidden Markov Modeling. Produces feature vectors and probabilities Markov Modeling. Produces feature vectors and probabilities for transitioning from one class to another. for transitioning from one class to another.

Preprocessors, mesh generators: Preprocessors, mesh generators: AKIRA suiteAKIRA suite Visualization tools: Visualization tools: RIVARIVA, , GMTGMT,, IDL IDL

Page 7: SERVO Grid: Solid Earth Research Virtual Observatory Grid/Web Services and Portals Supporting Earthquake Science August 26 2004 Beijing China Geoffrey

SERVOGrid Codes, RelationshipsSERVOGrid Codes, Relationships

Elastic DislocationPattern Recognizers

Fault Model BEM

Viscoelastic Layered BEM

Viscoelastic FEMElastic Dislocation Inversion

This linkage called Workflow in Grid/Web Service parlance

Page 8: SERVO Grid: Solid Earth Research Virtual Observatory Grid/Web Services and Portals Supporting Earthquake Science August 26 2004 Beijing China Geoffrey

Role of WorkflowRole of Workflow

Programming the GridProgramming the Grid: Workflow describes linkage : Workflow describes linkage between servicesbetween services

As distributed, As distributed, linkage must be by messageslinkage must be by messages Linkage is two-way and has both control and dataLinkage is two-way and has both control and data Apply to multi-scale (Apply to multi-scale (complexitycomplexity) linkage, multi-) linkage, multi-

program linkage, link program linkage, link visualization to simulationvisualization to simulation, GIS to , GIS to simulations and viz filters to each othersimulations and viz filters to each other

Microsoft-IBM specification Microsoft-IBM specification BPELBPEL is current preferred is current preferred Web Service XML specification of workflowWeb Service XML specification of workflow

SERVOGrid uses ANT (well known XML build tool) to SERVOGrid uses ANT (well known XML build tool) to perform workflow and this works well in our relatively perform workflow and this works well in our relatively simple cases)simple cases)

Service-1 Service-3

Service-2

Page 9: SERVO Grid: Solid Earth Research Virtual Observatory Grid/Web Services and Portals Supporting Earthquake Science August 26 2004 Beijing China Geoffrey

(i)SERVO Web (Grid) Services(i)SERVO Web (Grid) Services ProgramsPrograms: : All applications wrapped as Services using proxy strategyAll applications wrapped as Services using proxy strategy Job SubmissionJob Submission:: support remote batch and shell invocations support remote batch and shell invocations

• Used to execute simulation codes (VC suite, GeoFEST, etc.), mesh Used to execute simulation codes (VC suite, GeoFEST, etc.), mesh generation (Akira/Apollo) and visualization packages (RIVA, generation (Akira/Apollo) and visualization packages (RIVA, GMT).GMT).

File managementFile management::• Uploading, downloading, backend crossloading (i.e. move files Uploading, downloading, backend crossloading (i.e. move files

between remote machines) between remote machines) • Remote copies, renames, etc.Remote copies, renames, etc.

Job monitoringJob monitoring WorkflowWorkflow: Apache Ant-based remote service orchestration: Apache Ant-based remote service orchestration

• For coupling related sequences of remote actions, such as RIVA For coupling related sequences of remote actions, such as RIVA movie generation.movie generation.

Data servicesData services: support remote data bases and query construction: support remote data bases and query construction• XML data model being adopted for common formats with XML data model being adopted for common formats with

translation services to “legacy” formats.translation services to “legacy” formats.• Migrating to Geography Markup Language (GML) descriptions.Migrating to Geography Markup Language (GML) descriptions.

Metadata ServicesMetadata Services: for archiving user session information.: for archiving user session information.

Page 10: SERVO Grid: Solid Earth Research Virtual Observatory Grid/Web Services and Portals Supporting Earthquake Science August 26 2004 Beijing China Geoffrey

Security: Authentication and Authorization• Authentication describes who the user is• Authorization describes what a given user can do

– What data and computers can be accessed– Basically a database

• Current portal uses password accounts and provides services for free for demonstration.– iSERVO should decide on “charging for” services

• We have (through Community portal effort OGCE) support for GSI and Kerberos authentication services.– These just plug in and replace the default login service.

• Authorization is currently simple: you can only reach your files. – iSERVO should develop an authorization policy

• Simultaneous Cross Administrative Domain access is a very hard Grid problem and no consensus as to good solution

• Systematic use of Services helps security/privacy/IP issues as “danger of misuse” is lower for services (which have limited privileges) than for direct computer access

Page 11: SERVO Grid: Solid Earth Research Virtual Observatory Grid/Web Services and Portals Supporting Earthquake Science August 26 2004 Beijing China Geoffrey

SERVO Data SourcesSERVO Data Sources

Fault DataFault Data • Developed as part of the projectDeveloped as part of the project• QuakeTables: http://infogroup.usc.edu:8080QuakeTables: http://infogroup.usc.edu:8080

Seismic data formatsSeismic data formats• Available from www.scec.orgAvailable from www.scec.org• SCSN, SCEDC, Dinger-Shearer, Haukkson SCSN, SCEDC, Dinger-Shearer, Haukkson

GPS data formatsGPS data formats• Available from www.scign.orgAvailable from www.scign.org• JPL, SOPAC, USGS JPL, SOPAC, USGS

Page 12: SERVO Grid: Solid Earth Research Virtual Observatory Grid/Web Services and Portals Supporting Earthquake Science August 26 2004 Beijing China Geoffrey

Applications and Observational DataApplications and Observational Data Several SERVO codes work Several SERVO codes work directly with directly with

observational dataobservational data. . Scenarios includeScenarios include

• GeoFEST, VirtualCalifornia, Simplex, and Disloc all GeoFEST, VirtualCalifornia, Simplex, and Disloc all depend upon fault models.depend upon fault models.

• RDAHMM and Pattern Informatics codes use seismic RDAHMM and Pattern Informatics codes use seismic catalogs.catalogs.

• RDAHMM primarily used with GPS dataRDAHMM primarily used with GPS data ProblemProblem: We need to provide a way to integrate : We need to provide a way to integrate

these codes with the online data repositories.these codes with the online data repositories.• QuakeTables Fault Database was developed QuakeTables Fault Database was developed • What about GPS and Earthquake Catalogs? What about GPS and Earthquake Catalogs? • Many formats, data available in tars or files, not Many formats, data available in tars or files, not

searchable, not easy to integrate with applicationssearchable, not easy to integrate with applications SolutionSolution: use databases to store catalog data; use : use databases to store catalog data; use

XML (XML (GMLGML) as exchange data format; use ) as exchange data format; use Web Web ServicesServices for data exchanges, invoking queries, and for data exchanges, invoking queries, and filtering data.filtering data.

Page 13: SERVO Grid: Solid Earth Research Virtual Observatory Grid/Web Services and Portals Supporting Earthquake Science August 26 2004 Beijing China Geoffrey

Geographical Information Service Geographical Information Service (GIS) Data Formats and Services(GIS) Data Formats and Services

OpenGISOpenGIS Consortium (OGC) is an international group Consortium (OGC) is an international group for defining GIS data formats and services.for defining GIS data formats and services.

Main data format language is the XML-based Main data format language is the XML-based GMLGML..• Subdivided into schemas for drawing maps, representing Subdivided into schemas for drawing maps, representing

features, observations, …features, observations, … First StepFirst Step: design GML schemas and build specialized : design GML schemas and build specialized

Web Services for GPS and Earthquake data.Web Services for GPS and Earthquake data. OGC also defines servicesOGC also defines services..

• Services include Web Features Services, Web Map Services, Services include Web Features Services, Web Map Services, and similar.and similar.

• These are currently pre-Web Service, based on HTTP Post, but These are currently pre-Web Service, based on HTTP Post, but they are being revised to comply with WS standards.they are being revised to comply with WS standards.

Next StepNext Step: Implement OGC compatible Web Services : Implement OGC compatible Web Services for this problem i.e. build a for this problem i.e. build a GIS GridGIS Grid• Also build services to interact with QuakeTables Fault DB.Also build services to interact with QuakeTables Fault DB.

Page 14: SERVO Grid: Solid Earth Research Virtual Observatory Grid/Web Services and Portals Supporting Earthquake Science August 26 2004 Beijing China Geoffrey

GML and Existing Data FormatsGML and Existing Data Formats

GPS or seismic data used in this project GPS or seismic data used in this project are retrieved from different URLs and are retrieved from different URLs and have different text formats.have different text formats.

Seismic data formatsSeismic data formats• SCSN, SCEDC, Dinger-Shearer, Haukkson SCSN, SCEDC, Dinger-Shearer, Haukkson

GPS data formatsGPS data formats• JPL, SOPAC, USGS JPL, SOPAC, USGS

We defined We defined 2 GML Schemas2 GML Schemas to unify to unify thesethese• http://http://grids.ucs.indiana.edu/~gaydin/servogrids.ucs.indiana.edu/~gaydin/servo

A summary of all supported formats and A summary of all supported formats and data sources can also be found there.data sources can also be found there.

Page 15: SERVO Grid: Solid Earth Research Virtual Observatory Grid/Web Services and Portals Supporting Earthquake Science August 26 2004 Beijing China Geoffrey

Prototype GML ServicePrototype GML Service

First version of the First version of the system availablesystem available• Tried XML databases Tried XML databases

but performance was but performance was awfulawful

• Currently database Currently database uses MySQLuses MySQL

Download results are Download results are in GML, but we can in GML, but we can convert to appropriate convert to appropriate text formats.text formats.

Page 16: SERVO Grid: Solid Earth Research Virtual Observatory Grid/Web Services and Portals Supporting Earthquake Science August 26 2004 Beijing China Geoffrey

Search DB Search DB For EarthquakeFor Earthquake

Catalogs Catalogs

Page 17: SERVO Grid: Solid Earth Research Virtual Observatory Grid/Web Services and Portals Supporting Earthquake Science August 26 2004 Beijing China Geoffrey

Search XML DB For GPS CatalogsSearch XML DB For GPS Catalogs

1

Page 18: SERVO Grid: Solid Earth Research Virtual Observatory Grid/Web Services and Portals Supporting Earthquake Science August 26 2004 Beijing China Geoffrey

openGIS Grid Semantics• Note GIS (Geographical Information System) Grid at heart of all

these Grids• Geography Markup Language (GML) is an XML encoding for the

specification of the geometry and properties of geographic features. GML utilizes the OpenGIS Abstract Specification geometry model which has been harmonized with the ISO geospatial geometry model. – We are building CI specific ontologies in terms of GML to define

faults, satellites etc.– http://ripvanwinkle.ucs.indiana.edu:4780/examples/download/schema/

• Styled Layer Descriptor (SLD) specifies the format of a map-styling language for portraying the output of Web Map Servers, Web Feature Servers and Web Coverage Servers etc. SLD will enable different communities in the Emergency Response area to develop a set of customized portrayal rules that best fit their mission requirements.– This becomes the specification of portals to different composite Grids

• Sensor Markup Language (SensorML) defines the information model for discovering, querying and controlling Web-resident sensors.

• Observations & Measurements (O&M) defines the information model for observations that are returned from the CrisisGrid sensors.

Page 19: SERVO Grid: Solid Earth Research Virtual Observatory Grid/Web Services and Portals Supporting Earthquake Science August 26 2004 Beijing China Geoffrey

GIS Grid Services I• Web Feature Service (WFS) supports the query and discovery of

geographic features delivering GML representations of simple geospatial features in response to queries from HTTP clients. WFS can access geographic features including critical infrastructure features, incident locations, and flood-related geographic features including inundation areas, watershed boundaries, and demographic feature.

• Web Coverage Service (WCS) supports the query and discovery of digital geospatial information such as digital elevation models, imagery, orthophotography, weather coverages (such as predicted rainfall, air pressure, wind speed and direction), and any other space-varying flood-related phenomena.

• Web Map Service (WMS) uses a SLD portrayal to generate "pictures" of georeferenced feature or coverage data. WMS will provide a means to portray geographic information independent of the underlying data model (WFS or WCS).

• Coverage Portrayal Service (CPS) defines a standard interface for producing visual pictures from coverage data typically accessed via WCS with a SLD portrayal.

Page 20: SERVO Grid: Solid Earth Research Virtual Observatory Grid/Web Services and Portals Supporting Earthquake Science August 26 2004 Beijing China Geoffrey

GIS Grid Services II• Web Terrain Service (WTS) augments WMS with advanced

visualization including 3D terrains. • Catalog Service - Web Profile (CS-W) is a catalog service

that will be built on a general Grid metadata service• Sensor Collection Service (SCS) fetches observations from

a sensor or group of sensors and will be integrated with research on Grid sensor services

• Sensor Planning Service (SPS) assists in 'collection feasibility plans' and to process collection requests for a sensor or group of sensors.

• Web Notification Service (WNS) will be replaced by standard Grid notification service

Page 21: SERVO Grid: Solid Earth Research Virtual Observatory Grid/Web Services and Portals Supporting Earthquake Science August 26 2004 Beijing China Geoffrey

http://rio.ucs.indiana.edu:8080/wmsClient/

QuakeTables+OGC Web Map Service QuakeTables+OGC Web Map Service DemoDemo

Page 22: SERVO Grid: Solid Earth Research Virtual Observatory Grid/Web Services and Portals Supporting Earthquake Science August 26 2004 Beijing China Geoffrey

Streams and Workflow NaradaBrokering can manage streams from

• Audio/Video conferences• Sensors• Inter-service communication in workflow

http://www.hpsearch.org/demo/ describes scripting managementinterface to NaradaBrokering

Grids involve streams as well as compute and data nodesWorkflow and dataflow like BPEL imply streams

Page 23: SERVO Grid: Solid Earth Research Virtual Observatory Grid/Web Services and Portals Supporting Earthquake Science August 26 2004 Beijing China Geoffrey

SERVOGrid OntologySERVOGrid Ontology SERVOGrid has many types of metadataSERVOGrid has many types of metadata We are designing RDFS descriptions for the We are designing RDFS descriptions for the

following components:following components:• Simulation codes, mesh generators, etc.Simulation codes, mesh generators, etc.• Visualization toolsVisualization tools• Data typesData types• Computing resourcesComputing resources• ……

These are easily expressed as These are easily expressed as RDFSRDFS (actually (actually DAML) “nuggets” of information.DAML) “nuggets” of information.• Create instances of theseCreate instances of these• Use properties to link instances.Use properties to link instances.

Page 24: SERVO Grid: Solid Earth Research Virtual Observatory Grid/Web Services and Portals Supporting Earthquake Science August 26 2004 Beijing China Geoffrey

Some Sample RelationshipsSome Sample Relationships

Danube

Disloc

Application

Computer

Fault

DataType

Stress Map

DataFormatUSC Fault DB

Data Storage

GMT

Viz Appl

installedOn

storedIn

usesInput

createsOutput

visualizedBy

installedOn

Page 25: SERVO Grid: Solid Earth Research Virtual Observatory Grid/Web Services and Portals Supporting Earthquake Science August 26 2004 Beijing China Geoffrey

More InformationMore Information SERVOGrid/QuakeSim:SERVOGrid/QuakeSim:

• http://quakesim.jpl.nasa.gov/http://quakesim.jpl.nasa.gov/ Full Portal Demo: Full Portal Demo:

• http://complexity.ucs.indiana.edu:8080http://complexity.ucs.indiana.edu:8080• Request an accountRequest an account• Downloads available in NovemberDownloads available in November

Fault databaseFault database• http://infogroup.usc.edu:8080http://infogroup.usc.edu:8080

GPS and Seismic Database Demo:GPS and Seismic Database Demo:• http://gf3.ucs.indiana.edu:6060/cce/sql/http://gf3.ucs.indiana.edu:6060/cce/sql/

Setting up your own GPS or Seismic databaseSetting up your own GPS or Seismic database• http://complexity.ucs.indiana.edu/~gaydin/cce/install/install.htmlhttp://complexity.ucs.indiana.edu/~gaydin/cce/install/install.html

Publications:Publications:• http://grids.ucs.indiana.edu/ptliupages/publications/http://grids.ucs.indiana.edu/ptliupages/publications/• http://grids.ucs.indiana.edu/ptliupages/presentations/http://grids.ucs.indiana.edu/ptliupages/presentations/

Page 26: SERVO Grid: Solid Earth Research Virtual Observatory Grid/Web Services and Portals Supporting Earthquake Science August 26 2004 Beijing China Geoffrey

Some Grid Controversies• 1) There are several proposals for the Web Service

extensions needed for Grids – why do we ignore?– OGSI (GT3)

– WSRF (GT4)

– WS-GAF (Newcastle)

– WS-I+ (Pure Web Services)

• We use WS-I+ approach – can later add extensions when consensus clear– This approach adopted by next phase of UK e-Science Program

• 2) Web Services are too slow as use HTTP with clumsy ASCII XML data (SOAP)?– Currently no problem but can use separate control channel from

data channel if need high performance

Page 27: SERVO Grid: Solid Earth Research Virtual Observatory Grid/Web Services and Portals Supporting Earthquake Science August 26 2004 Beijing China Geoffrey

iSERVO Strategy• Agree on what (type of) resources and capabilities need to put on the

ISERVO Grid– Computers, instruments, databases, visualization, maps, job

submittal ….• Agree on interfaces to resources from OGSA-DAI (databases) to

particular data structures (GML/OpenGIS) – specify in XML• Implement Resources and Capabilities as Services

– User Interface should be a portlet that can be integrated by the portal into web interface

• Make certain overarching Grid capabilities such as workflow, federation and metadata are sufficient

• SERVO Grid is a prototype of this strategy using several US sites rather than several countries– Can be naturally extended to iSERVO, education, emergency

response by extending resources• Web Service Architecture ensures continued interoperability and

extensibility

Page 28: SERVO Grid: Solid Earth Research Virtual Observatory Grid/Web Services and Portals Supporting Earthquake Science August 26 2004 Beijing China Geoffrey

Lessons LearnedLessons Learned Web service Web service performanceperformance is not an issue when is not an issue when

used to invoke services that take hours to used to invoke services that take hours to complete.complete.• Later real-time sensors will probe performanceLater real-time sensors will probe performance

ReliabilityReliability is a larger problem. is a larger problem.• Need monitoring/heartbeat services.Need monitoring/heartbeat services.

Information systemsInformation systems still have a long way to go. still have a long way to go.• UDDI UDDI is part of WS-I but has/had some well known is part of WS-I but has/had some well known

limitations.limitations.• WS-DiscoveryWS-Discovery has some interesting concepts but is too has some interesting concepts but is too

specialized to ad-hoc networks.specialized to ad-hoc networks.• Peer-to-peer Peer-to-peer systems provide many useful concepts like systems provide many useful concepts like

discovery and caching. discovery and caching. • Semantic WebSemantic Web provides powerful resource descriptions provides powerful resource descriptions

that could be exploited.that could be exploited.• XML DatabasesXML Databases slow slow

Page 29: SERVO Grid: Solid Earth Research Virtual Observatory Grid/Web Services and Portals Supporting Earthquake Science August 26 2004 Beijing China Geoffrey

Further iSERVO Challenges• Make everything a Service• Think about Data Curation

– Set up policies for observational data and criteria for inclusion in iSERVO data repositories

• Think about Data Provenance– Generate and maintain metadata describing ownership, origins

and transformations– Applies to both “experimental data” and results from

simulations (visualizations)

• Curation and Provenance change in research methodologies and requires funding!

• Education and Emergency Response/Planning interesting offshoots of iSERVO

Page 30: SERVO Grid: Solid Earth Research Virtual Observatory Grid/Web Services and Portals Supporting Earthquake Science August 26 2004 Beijing China Geoffrey

QuakeSim Portal for SERVOGridQuakeSim Portal for SERVOGrid

The services need user interfacesThe services need user interfaces• WSDL descriptions are all you need to create WSDL descriptions are all you need to create

client stubs (if not client applications).client stubs (if not client applications). The QuakeSim portal effort aggregates The QuakeSim portal effort aggregates

these service interfaces into a these service interfaces into a portalportal..• Customizable displays, access controls to Customizable displays, access controls to

services, etc.services, etc. QuakeSim is just one of many, many such QuakeSim is just one of many, many such

projects.projects. Challenge is to develop reusable portal Challenge is to develop reusable portal

componentscomponents

Page 31: SERVO Grid: Solid Earth Research Virtual Observatory Grid/Web Services and Portals Supporting Earthquake Science August 26 2004 Beijing China Geoffrey

OGCEOGCEConsortium

Individual portlet for the Proxy Manager

Use tabs or choose different portlets to navigate through interfaces to different services

2 Other Portlets

Each Servicehas its own portlet

Page 32: SERVO Grid: Solid Earth Research Virtual Observatory Grid/Web Services and Portals Supporting Earthquake Science August 26 2004 Beijing China Geoffrey

SERVOGrid Portal Screen SERVOGrid Portal Screen ShotsShots

Page 33: SERVO Grid: Solid Earth Research Virtual Observatory Grid/Web Services and Portals Supporting Earthquake Science August 26 2004 Beijing China Geoffrey

Computational Web Portal Computational Web Portal StackStack

Web service dream Web service dream is that core services, is that core services, service aggregation, service aggregation, and user interface and user interface development development decoupled.decoupled.

How do I manage all How do I manage all those user those user interfaces?interfaces?

Use portlets.Use portlets. Core Web Services

Portlet User InterfaceComponents

Application Web Servicesand Workflow

Aggregate Portals

Page 34: SERVO Grid: Solid Earth Research Virtual Observatory Grid/Web Services and Portals Supporting Earthquake Science August 26 2004 Beijing China Geoffrey

Portal ArchitecturePortal ArchitectureC

lient

s (P

ure

HT

ML,

Jav

a A

pple

t ..

)

Agg

rega

tion

and

Ren

derin

g

PortalInternalServices

Portlet Class

Portlet Class

Portlet Class

Portlet Class:WebForm

Gateway(IU)

Web/Gridservice

Web/Gridservice

Web/Gridservice

Computing

Data Stores

Instruments

GridPortetc.

(Java)COG Kit

Clients Portal Portlets Libraries Services Resources

LocalPortlets

Remoteor ProxyPortlets

Hierarchical arrangement

Page 35: SERVO Grid: Solid Earth Research Virtual Observatory Grid/Web Services and Portals Supporting Earthquake Science August 26 2004 Beijing China Geoffrey

Why Are Portlets a Good Idea?Why Are Portlets a Good Idea? You don’t have toYou don’t have to reinvent reinvent

everythingeverything• Makes it easy (but not effortless) to Makes it easy (but not effortless) to

share portal componentsshare portal components between between projects.projects.

• So you can pull in portlets from all the So you can pull in portlets from all the other earthquake grid projects.other earthquake grid projects.

You can easily You can easily combinecombine a wide range a wide range of of capabilitiescapabilities• Add document managers, collaboration Add document managers, collaboration

tools, RSS news lists, etc for your portal tools, RSS news lists, etc for your portal users.users.

Page 36: SERVO Grid: Solid Earth Research Virtual Observatory Grid/Web Services and Portals Supporting Earthquake Science August 26 2004 Beijing China Geoffrey

Lessons Learned: PortalsLessons Learned: Portals Developing good user interfaces is a lot of work.Developing good user interfaces is a lot of work.

• Effort doesn’t scale: how do you simplify this for Effort doesn’t scale: how do you simplify this for computational scientists to do it themselves without computational scientists to do it themselves without lots of background in XML, Java, portlets, etc?lots of background in XML, Java, portlets, etc?

Portal interfaces have advantages and disadvantages.Portal interfaces have advantages and disadvantages.• Everyone has a browser.Everyone has a browser.• But it has a limited widget set, a limited event But it has a limited widget set, a limited event

model, limited interactivity. model, limited interactivity. • You can of course overcome a lot of this with applets.You can of course overcome a lot of this with applets.

Following the service model, you can in principal use Following the service model, you can in principal use any number of GUIsany number of GUIs• Browsers are not the only possible clients.Browsers are not the only possible clients.• Web service interoperability means that Java Swing Web service interoperability means that Java Swing

apps, Python, Perl GUIs are all possible, but this has apps, Python, Perl GUIs are all possible, but this has not been fully exploited.not been fully exploited.

Page 37: SERVO Grid: Solid Earth Research Virtual Observatory Grid/Web Services and Portals Supporting Earthquake Science August 26 2004 Beijing China Geoffrey

Important Lessons/PrinciplesImportant Lessons/Principles Use Use OGCE Portal ArchitectureOGCE Portal Architecture and portal services and portal services Can expect GGF activities like OGSA to define/refine interfaces and Can expect GGF activities like OGSA to define/refine interfaces and

projects around the world to produce more powerful servicesprojects around the world to produce more powerful services• Obsolescing of implementationsObsolescing of implementations is a consequence of interoperability is a consequence of interoperability

Use Use Grids of Grids of ServicesGrids of Grids of Services Architecture Architecture• Interoperable Component Grids Built from interoperable servicesInteroperable Component Grids Built from interoperable services• Collaboration, Compute, Database, GIS, Sensor, VisualizationCollaboration, Compute, Database, GIS, Sensor, Visualization Grids Grids

Build a Build a GIS (Geographical Information Systems) GridGIS (Geographical Information Systems) Grid spanning spanning simulation/crisis management and different fields with simulation/crisis management and different fields with openGISopenGIS compliancecompliance• openGISopenGIS has defined Web Service Interfaces has defined Web Service Interfaces• VisualizationVisualization should build on these should build on these

Geoscience Education GridGeoscience Education Grid by transformations on research grid by transformations on research grid Emergency Response and Planning GridsEmergency Response and Planning Grids by adding real-time by adding real-time

control/collaboration and GIS toolscontrol/collaboration and GIS tools• These additions common to all crisesThese additions common to all crises

Collaboration between Collaboration between Beihang University Beihang University andand Indiana University Indiana University to to produce produce Web Service based audio/video conferencingWeb Service based audio/video conferencing