INFN Research Roadmap for

www.g-ows.orgEric Frizziero (INFN)Marco Verlato (INFN)Luigi Zangrando (INFN)

EGEE’09 Conference, Barcelona 21-25 September 2009EGEE’09 Conference, Barcelona 21-25 September 2009

A little of history…

• INFN started to work with geospatial web services in the context of CYCLOPS project– FP6 SSA of EGEE –> June 2006 - November 2008– Consortium:

• Civil Protection Agencies: ANPC (PT),DDSC (FR), DPC (IT), Pr. Chania (GR)• Scientific and Technological Centres: CNR (IT), EMA (FR), INFN (IT), TEI (GR),

UMINHO (PT)

– Main Outcomes:• Grid-enabled CP applications proof-of-concept• Research Strategies and Innovation Guidelines for a European CP e-

Infrastructure

Architectural framework

G-RISICO use-case

Application built on top of grid-enabled OWS:– OpenGIS WPS (Web Processing Service): standard description of GIS calculation (the process), here the wild fire risk assessment model– OpenGIS WCS (Web Coverage Service): standard access to geospatial information data

Grid-enabled WCS exampleGRID WCS

Traditional WCS

Grid-WCS

CYCLOPS VO support• Continued after the end of CYCLOPS project• 14 EGEE production grid sites in total (in France, Italy,

Portugal)• Potentially > 600 CPU-cores and > 10 TB of storage available• Central grid services

(WMS, LB, VOMS, LFC) at INFN

• G-OWS services (WCS and WPS) at INFN

• >14k test & demo jobs >100 CPU.days

CYCLOPS follow up• From CYCLOPS final review (Dec. 08)

• Draft work plan (from G-OWS charter)

VO-OWS package 1. OWS implementation on WMProxy2. Security aspects: basic security support

Site-OWS package1. OWS implementation on CREAM2. SOS implementation on Instrument Element

CREAM• CREAM (Computing Resource Execution And Management) service:

– general purpose framework for building grid services– functionalities/operations are pluggable

• for example, the functionality for accessing a database can be easily added and plugged to CREAM

• main functionalities provided: job management operations at the Computing Element (CE) level– allow the grid user to submit, cancel, monitor, … computational jobs – Computing Element: grid component acts as interface to

computational resources• single pc• cluster of pc handled by a LRMS (e.g. LSF, PBS/Torque, Condor)• supercomputer for High Performance Computing (HPC)

• CREAM describes and exposes its functionality through a Web Service interface

High level CREAM architecture• In the current architecture, it is possible to plug different interfaces to CREAM (e.g.

one for job management, one for database access, etc.). • The different commands (that is the operations defined on the WS interfaces) are

then managed by different command executors (pluggable).

CREAM coreCREAM core

OGSA-BES

Job Exec

BLAH

CREAMjob manag.

LRMS

??? WS

???Executor

Multiple Web service interfaces can coexistWeb service interfaces for job management

Possible extension of CREAM capabilities:OGC interfaces?

Job management functionalities

Job management functionalities implementation

Basic Execution Service (BES) an OGF specification

High level CREAM architecture• Persistency, fault tolerance• Support for both synchronous and asynchronous commands is provided• Asynchronous commands execution is implemented by a priority queue• A command can be also executed in a serial/parallel way

??? Executor

??? WS

priority cmd queue

Executor implementation

cmd

asynchronous commands

synchronous command

WS interface

cmdcmd

cmd

CREAM coreCREAM core

“Simplified” view of CREAM architecture

• Web service interface– WS-I compliance– WSDL 1.1– document/literal

• SOA (Service Oriented Architecture) paradigm adopted• Fully implemented in Java

– developed with Apache Axis (version 1.4) framework for Java• http://ws.apache.org/axis/

• CREAM runs as Java-Axis servlet on Tomcat 5.5 application server

Web service interfaceWeb service interface

AXISAXIS

Security layer X.509 + VOMSSecurity layer X.509 + VOMS

TomcatTomcat

To get access to the CREAM it is needed to cross the AuthN and AuthZ layers; The DN and VOMS attributes are extracted from the user's proxy certificate; The AuthZ is based on VOMS attributes and on the gridmap file;

SOAP engine (servlet)

servlet engine

“Generic” CREAM WS interface

• CREAM provides also a “generic” WS interface for executing commands

• The main operation is the execute() one which allows a client to execute synchronously/asynchronously the specified command implemented by CREAM through the associated command executor.– CommandResult execute(Command)

• Within the Command argument the client provides input parameters, execution category, specific executor name, etc

• the CommandResult returns the outputs produced (if executed synchronously) or the commandId (if executed asynchronously)

CREAM usage scenarios

• CREAM can be used:– through the gLite components (Web Services)– directly by the users

• they can build their own clients using a Web Service framework

Direct JobSubmission

Submissionthrough gLite WMS

CREAM CREAM CREAM

gLite WMS

G-RISICO and gLite

WMS

WPS

WCS

WCS

WCS

WCS

CE CE CE

gLitegLite

GUI

RISICO RISICO Business Logic

CREAM as coverage provider

WMS

WPS

WCS

WCS

WCS

WCS

CREAM CREAM CREAM

gLitegLite

GUI

RISICO

RISICO Business Logic

CREAM could become a coverage provider by adding the WCS interface

WCS JM WCS JM JM WCS

OGC-WPS vs CREAM interface• The WPS interface presents strong analogies with respect to the

CREAM “generic” one.– execute() operation

• This encourages a possible integration of WPS in CREAM• advantages:

– If (almost) all resources (e.g. geospatial data) are locally distributed and provided by a single site which must provided an adequate local cluster of pc handled by a LRMS • this avoid the intrinsic grid overhead

– Inheritance of the CREAM security level (X509, VOMS) which is gLite compliant

– Sophisticate asynchronous execution of commands• priority command queue• serial/parallel command execution

• Disadvantage: N/A

G-RISICO and CREAM

WCS

WCS

WCS

WCS

CREAM

GUI

WNWN WN WN

OGC-WPS

RISICOexec

Local WCS

Remote WCS

Local cluster

OGC WS interface

Security layer X.509 + VOMSSecurity layer X.509 + VOMS

RISICO Business Logic

WPS and WCS integrationWPS

WPS Exec(RISICO WF)

BLAH

Geospatial data

WCS

WCS Exec

BLAH

Priority cmd queue

cmd

asynchronous commands

synchronous command

cmdcmd

cmd

LRMS

WCS

WCS

Remote WCS

Direct access to the local WCS

cmd

cmdcmd

cmd

Conclusions

• A first preliminary analysis has shown that the CREAM framework could provide OWS services at site level

• A prototype could be developed in the context of future FP7 projects:– CYCLOPS-2 ?– Lifewatch related project?– …