page 1 laits laboratory for advanced information technology and standards 09/13/2005 the current...

LAITS

Laboratory for Advanced Information Technology and Standards

09/13/2005

The current status of GMU Geospatial Grid project

Laboratory for Advanced Information Technology and StandardsGeorge Mason University

[email protected]

September 13, 2005

Liping Di

LAITS


09/13/2005

Project Objectives

Making NASA EOSDIS data easily accessible to Earth science modeling and applications communities by combining the advantages of both OGC and Grid technology.

Making Grid technology geospatial enabled and OGC standard compliant and making OGC technology Grid enabled.

Allowing researchers to focus on science and not issues with computing, storage and bandwidth resources, and data receipt, data format and data set manipulation,

Achieving the integration in NASA EOSDIS and ESG (Earth System Grid).

LAITS


09/13/2005

Outline of Progress

Testbed Improvements - Hardware, Software, Data• Successfully build up a creditable, realistic VO as the project testbed

among members of project team and CEOS.• Testbed involves 7 machines.

Information Model -- CSW• Enhanced CSW information model for service type and data type

LAITS


09/13/2005

Outline of the Progress

OGC and Grid Software Development and Integration• OGC standard CSW development• WCS, WMS and CSW as Grid services• WCS Portal and WMS Portal to Grid• Intelligent Grid Service Mediator (iGSM)• Replica Optimization Service (ROS) and Data Transfer Service (DTS) • Tool development - automatically create CSW metadata catalog.• Tool development - transform netCDF data into HDF-EOS data.

Application integration• Integrated the GridWCS, GridWMS and GridCSW, iGSM, ROS, DTS

and other Grid Services for serving Geobrain data download.

LAITS


09/13/2005

Testbed Improvement - Hardware

GMU has set up a Apple cluster server:• 6 Apple G5 server nodes- 3 with dual 2.5GHz CPU and 3 with dual

2.0 GHz CPU with total of 12 GB RAM.• 22.6TB RAID storage.• 1GB network to Internet II and 100 MB to Internet I.• Hosted at ESDIS network lab of GSFC.

LLNL have purchased the Dell 2859 server:• Linux Server OS • 3.0GHz processors• 4 GB SDRAM• Mirrored 146GB disks for root partition • Dual power supplies• 24X IDE CD-RW/DVD ROM• SCSI card for external drives• 1.3 TB disk storage

LAITS


09/13/2005

Testbed Improvement - Data

Data in the RAID system– Finished the loading of Landsat data covering global for year 1975,

1990 and 2000. Total size of data 7TB.– Finished the loading of Shuttle DEM data covering Global for year

2000. The size of DEM is about 1TB– Loaded sample EOSDIS data into the server.

LLNL: Model data in netCDF are converted to HDF-EOS to be served by the Grid system.

Live access to data pools through searching ECHO enabled.

• The Grid CSW server as an operational client of ECHO 6.0

• Currently only able to access data at LPDAAC.

• Problem– the metadata (especially the data URL) is not current at ECHO.

LAITS


09/13/2005

Testbed Improvement - Grid Software

• Globus software

• DOE LLNL: GT3.2.1

– GMU: GT2.2 on a Linux machine, GT3.2 on Apple G5 and Sun Fire Solaris machines, GT3.9.2 on the Sun Fire Solaris machine, GT 4.0 on Apple G5.

– NASA Ames, installed gt3.2 on Linux machine

• CA– LAITS CA: LAITS certificates to Mac machine and Linux machine of GMU

LAITS.

– IPG CA: IPG certificates to all machines in the testbed.

– CEOS CA: CEOS certificates to Sun Fire Solaris machine at GMU LAITS.

– ESG CA: Certificate to the LLNL Linux machine.

LAITS


09/13/2005

Grid Security (GSI) and VO Setup

GMU (Solaris) (laits.gmu.edu)Globus 3.2, 3.9 with CEOS Certs.

GMU (Linux)(llinux.laits.gmu.edu)Globus 2.2 with Laits

Certs.

GMU CA center

Ames ipg05 (Linux)(ipg05.ipg.nasa.gov)Globus 3.2 with IPG

Certs.GMU LAITS VO NASA IPG VO

GMU (Linux)(data.laits.gmu.edu)

Globus 3.2, 3.9 with Laits Certs.

GMU (Mac)(geobrain.laits.gmu.edu)Globus 3.2, Globus 4.0

IPG CA center

NASA SGT (Linux)(arao2.sgt-inc.com)Globus 3.2 with CEOS

Certs.

NASA (Linux)(former.intl-interfaces.net)Globus 3.0 with CEOS Certs.

CEOS VO

Authentication among different VO

LLNL esg2 (Linux)(esg2.llnl.gov)

Globus 3.2 with ESG Certs.

LLNL ESG VO

ESG CA center

LAITS


09/13/2005

Catalog Information Model (IM) Improved

ISO 19115 Part one

ISO 19115 Part two(FGDC extension)

NASA ECS

ISO 19119ebRIM IM

HDF-EOS metadata IM(getCapabilities and describeCoverage)

GCMD Service Type IM

LAITS-defined Data Type IM

CSW Information Model

LAITS


09/13/2005

Software Development/Integration - GCSW

Improved the OGC Catalog Service for Web (CSW) server

• Provides a complete implementation of OGC CSW with enhanced database support, friendly user interface and friendly development guide for any OGC standards client and users.

Migrated the CSW server to the Grid environment to become a Grid enabled CSW (GCSW) service.

Deployed GCSW at GeoBrain (Mac), Laits (Solaris) and Data (Linux).

LAITS


09/13/2005

Software Development/Integration – GWCS&Portal

OGC Web Coverage Service (WCS) specification is the fundamental specification for geospatial coverage data access

WCS has been enhanced to process 4-D HDF-EOS data which is from LLNL netCDF modeling data.

Enhanced the Grid enabled WCS (GWCS).

Developed OGC standard compatible WCS portal which provides a single point of web entry to multiple GWCS within a Grid.

The GWCS has been deployed at all seven machines in the Grid, and the Portal has been deployed at GeoBrain.laits. gmu.edu (Mac) and Laits.gmu.edu (Solaris).

LAITS


09/13/2005

• The NWGISS WMS server has been migrated to Grid to become a Grid service.• It is called Grid-enabled Web Map Service (GWMS).• It works both as a web service and a Grid service.• Both OGC and Grid clients can work with the service.

• A GWMS portal is developed to provide a single point of web entry to all GWMS services in the Grid.

• Deployed GWMS on all 7 machines in the testbed and the portal on the GeoBrain and Laits machines.

Software Development/Integration – GWMS&Portal

LAITS


09/13/2005

(iGSM: intelligent Grid Service Mediator)

Support WCS portal and WMS portal to distribute their request to proper GWCS and GWMS.

Software Development/Integration - iGSM

WCS PortalWMS Portal

GCSWGWCS

GWMS

iGSM

ROS MDS

DTS

LAITS


09/13/2005

Functional Overview of iGSM

• Managing geodata access requests from OGC WCS portal and WMS portal and transfer those requests to GWCS (Grid-enabled Web Coverage Service) or GWMS (Grid-enabled Web Map Service)

– Accepts geodata requests from default WCS portal and WMS portal.

– Queries a ROS (Replica Optimization Service) for an optimized PFNInfo (Physical File Name Information) object

• Each PFNInfo contains a physical file name, a GridWCS service ID, and the host where the data file located

– When the received PFNInfo contains a valid service ID

• Requests a GridCSW (Grid-enabled Catalog Service for Web) for corresponding GridWCS/WMS URL to the service ID.

– When the received PFNInfo contains a null service ID

• Requests a GridCSW for available GridWCS(s) /WMS(s) among the Grid resources.

• Requests a ROS (Replica and Optimized Service) for selecting the best GridWCS/WMS among the resources returned from the GridCSW

• Requests a DTS (Data Transfer Service) for transferring the data to the selected system

– Querying the GridCSW deployed in the selected system for the geodata URI

LAITS


09/13/2005

Geospatial Grid with GCSW/GWCS/GWMS/iGSM/ROS/DTS

WCS PortalWMS Portal

GCSW GWMS

GWCS

iGSM

ROS

MDS

DTS

CSW Portal

User/Client Interface (Web Download & MPGC)

GeospatialCatalog

DB Replica DB

HDF-EOS Data

12 2

Laits (3)

Ames

LLNL

LAITS


09/13/2005

A Data Request Scenario based on the Integration

RLS

LAITS WCS/WMS Portal

CSW Portal

ESG Catalog

Client

1

2

3

Retrieval Manager

4

LAITSGridWCS

AmesGridWCS

LLNLGridWCS

+ default WCS portal IP

ROS

5

2

Logical data name

Physical data/service ID

MDS

6

Best server ID7

iGSM

Other WCS

LAITS GridCSW

HDF-EOS Data

Other Data

Ames DTS

9

9

9

8

LAITS


09/13/2005

Work Plan for the next year

• The main work in the next year will be concentrated on the on-demand virtual product generation based on geospatial processing models. – Upgrade from GT 3.2 to GT 4.0– Determine the requirements of metadata structures for a

materialized product catalogue; – Access to all data in data Pools through ECHO. Access to ESG

data through the ESG gateway at LLNL.– Use workflow language BPEL4WS for representing the execution of

the transformation prescriptions. Build the planner that converts from the prescriptions into a workflow language.

– Integrate workflow engine (BPELPower) into Grid, enabling OGC CSW to search and OGC WCS to retrieve virtual data the same way as non-virtual data;

– Demonstrate an overall virtual data scenario in Globus Grid environment.

LAITS


09/13/2005

Acknowledgement

• The project team includes Prof. Liping Di (PI, GMU), Dr. Piyush Mehrotra (Co-I, NASA Ames), Dr. Dean Williams (Co-I, DOE LLNL), Dr. Chaumin Hu (NASA Ames), Dr. Aijun Chen (implementation lead, GMU), Dr. Yuqi Bai (GMU), Mr. Yang Liu (GMU), Yaxing Wei (GMU).

• The project team sincerely thanks NASA ESTO for providing the financial support and technical guidance.

page 1 laits laboratory for advanced information technology and standards 09/13/2005 the current...

Documents