ames research centerdivision 1 information power grid (ipg) overview anthony lisotta...

Ames Research Center Division1

Information Power Grid (IPG)

Overview

Anthony [email protected]

Computer Sciences CorporationNASA Ames

May 2, 2003


Outline

• What are Grids?• Current State of Information Power Grid

(IPG)• Future Directions• Wrap-up


Grids• A Grid is a system that*:

1.coordinates resources that are not subject to centralized control …

2.… using standard, open, general-purpose protocols and interfaces …

3.… to deliver nontrivial qualities of service.

• Grid software is “middleware”

Resources Networks

This is a Grid Enabled Infrastructure

* - Ian Foster, July 2002


Normal Grid Characteristics

• An underlying security infrastructure such as the Grid Security Infrastructure (GSI), which is based on public key technology– Protection for at least authentication information as it flows

from resource to resource

• Readily accessible information about the resources on the Grid via a single mechanism, the Grid Information Service (GIS)

• Single sign-on • A seamless processing environment • An infrastructure that is scalable to a large number

of resources• The ability for the grid to cross administrative

boundaries


The Grid TodayCommon

Middleware

- Abstracts independent, hardware, software, user ids, into a service layer with defined APIs

- Provides comprehensive security,

- Allows for site autonomy

- Provides a common infrastructure based on middleware

Grid Middleware

User

Application

Site A Site B

Network

Infrastructure

1

Request info from the grid

1

2Get response2

3Make selection and submit job

3

The underlying infrastructure is abstracted into defined APIs thereby simplifying

developer and user access to resources, however, this layer is not intelligent.


Outline




IPG VisionTo make the practice of large-scale science

and engineering, as well as other widely distributed, data intensive NASA activities, much more effective than it is today.

Grid technology is the foundation to making this vision a success


IPG LocationsAmes Research Center

Jet Propulsion Labs

Glenn Research Center

Langley Research Center

Goddard Space Flight Center


LSF

Globus GRAM Grid FTP

MDS GRISPBS

Globus GRAMGrid FTP

MDS GRIS

MDS GIIS

GRAM Job

Job Manager Job:Files to pre-stageApplication to executeFiles to post-stage

IPG Job Manager Client

IPG Job Manager

Globus GRAM Client

3rd party copy

3rd party copy

Grid FTP Client

Current IPG Services

Launch Pad Web Portal

IPG Resource Broker

MDS Client

IPG Resource Broker Client

Input:•System requirements•How to use systemsOutput:•Suggested Job Manager Jobs

resource informatio

n


Outline




Future Directions• Focus on Managing Complex, Distributed

Data (MCDD)– Intelligently manage data in a grid

• IPG Advanced Services– easier to use– deliver value added services– easier to maintain


MCDD• Goal: Intelligently manage data in a grid• NASA data is inherently distributed e.g., various

Earth science archives.• Important focus of IPG• Access to files

– Initial use of grid-enabled Storage Resource Broker – Data staging and replica management building on grid

community research– Need grid support for file metadata

• NASA data can be on – Disk-resident data pools– Tertiary storage data archives

• Will build on considerable data grid work from the international grid community


MCDD - Objectives• Identify key data services abstractions that will

address a wide range of large-scale, federated, distributed NASA data problems– This potentially has considerable utility in many NASA areas

as it essentially seeks to abstract, regularize, and componentize the general operations that are performed on scientific data.

• Analyze these abstractions in the context of services oriented Grids and determine an implementation and prototyping strategy– Identify existing work in this area and design an

implementations approach that maximizes leverage from current work

• Demonstrate the prototype services within the context of some existing NASA applications


IPG Advanced Services• Goal: Location-independent computing

• Intelligent resource selection– User-specified preferences and requirements– Resource characteristics, access, allocations, cost, load,

performance predictions• Create system-dependent job from system-independent job

– Given a system (or systems) to use– Pick directories, set paths and environment, specify which

executables and libraries to move, specify which data files to move

• Reliable execution of system-dependent job– Application execution & file management

• Dynamic access and accounting– Run applications without a permanent local account– Charging resource use to remote accounting systems– Resource pricing

• Workflow management– Specify and execute workflows


Concept Architecture

Job Manager Job

User Program

Portability Manager

Portability Manager Client

Portability Manager

Job

Resource Selector Client

Resource Broker Client

Resource Broker

Resource Selector

resource requirements

and preferences

Job Manager Client

Resource Broker Job

resource sets

Dynamic Access Client

Allocation Manager

Performance

Prediction Clients

GIS ClientGIS

ClientData Grid

Client

PBS

Globus GRAM

Grid FTP

GRAM Job

IPG Job Manager

Globus GRAM Client

3rd party copy

Grid FTP Client

Testing and

Monitoring

resource info

allocations

resource and service status

Resource Pricing

prices

Access Policies

Allocation Client

Access Client

deduct from

allocation

access allowedmap user to userid

access allowed

Higher-level ServicesIn Development Now

Transitioned Services

Higher-level ServicesVersion 1 in use


Outline




SummaryS

tabi

lity

leve

l

2001 2002 200320001999

• Testbed• Learning Grids• Organizing Players

• Persistent Grid• Informal Organization• Lots of Teamwork

• Shift primary work to increase usability• Globus stabilized• Attempt inter-grid processing• Begin development of initial services

• Transition support of IPG to operations• Testbed in parallel using GT3?• Formalize NASA grid organizations• Continue to make the Grid easier to use• Research complex distributed

data management

2004

Testbed

Operations

Transition Period

ames research centerdivision 1 information power grid (ipg) overview anthony lisotta...

Documents