Leading the pervasive adoption of grid computing for research and industry

© 2005 Global Grid Forum The information contained herein is subject to change without notice

Standards, Industry, and the

Roadmap to Grid Adoption

Dr. David SnellingVice Chair of StandardsGlobal Grid Forum / Fujitsu Labs Europe

Motivation

• Need for Standards− Stability, Choice, Flexibility, Competition,

Collaboration, ...

• To Develop Standards we Need Clarity− Definitions of concepts− Organization of work through Architectural

Frameworks

• We also Need a Roadmap− Accelerate the development of the “right”

specifications− Track gaps and requirements− Demonstrate progress− Support planning in industry and research

Notions of Grid

• Collaboration Grids− Multiple institutions, secure, widely distributed, VOs− Service level agreements & commercial partnerships− Business model: Increase overall revenue

• Enterprise Grids− Virtualization of enterprise resources and applications − Aggregation and centralization of management− Business model: Reduce total cost of ownership

• Clusters− Networks of Workstations, Blades, etc.− Cycle scavenging, Homogeneous workload− Business model: Lower marginal costs

• Parallel Processing Systems− Parallel processing for single applications Incr

ea

sing

Co

mp

lexi

ty a

nd

Rev

enu

e

Parallel Processing and Cluster Grids

• Parallel Processing−Tightly coupled distributed systems−Standards:

• MPI and OpenMP

−Aimed at HPC−Code portability and performance!

• Cluster Grids−Loosely coupled distributed systems−Efficient scheduling of nodes for throughput−No standards, lots of players

• Queuing systems: LSF, PBS, LoadLeveler, ...• Specialist systems: CyberGRIP, gridMatrix, ...

Enterprise Grids Today

• Enterprise Grids are about− Virtualization: Uniform encapsulation of resources:

• Compute, data, applications, support, ...− Integration: Creation of a structured whole from the parts.− Automation: Most management tasks, mostly automatic.

• Examples− Fujitsu’s Triole Strategy− Oracle’s 10g Platform− Sun’s N1 Suite− HP’s Adaptive Enterprise− IBM’s “On Demand” Business

• Run your required services asefficiently as possible.

QuickTime™ and aTIFF (LZW) decompressor

are needed to see this picture.

Collaboration Grids Today

• Production First Generation Collaboration Grids− UK National Grid Service and TeraGrid

• Running Globus GT2− Team Shosholoza and others

• Running Unicore

• Web Service Collaboration Grids− Experimental Deployment

• Globus GT4, Unicore/GS− Barriers

• Confusion wrt Plain Web Services• Politics of the Standards Process

• Create new business opportunities throughcollaboration− Enterprise Grid technology as a basis.− Requirements beyond Enterprise Grids:

• Discovery, Security, Virtual Organizations (VOs),Decoupling, Composition ...

† http://www.forrester.com/go?docid=38314

Convergence: Enterprise & Collaboration Grids

• Technical Convergence− From Enterprise Grids

• Sophisticated virtualization• Management infrastructure• Automation

− From Collaboration Grids• Multi-domain security• Cyber partnerships (VOs)• Outsourcing

• The Need for Standards− Within the Enterprise

• Flexibility!− Between Enterprises

• Interoperability!

• Forrester’s − “Digital Business Networks”†

GGF and the Nature of Interoperability

• GGF is about− Enabling the pervasive adoption of grid computing for

research and industry by:• Defining grid specifications that lead to broadly adopted

standards and interoperable software• Fostering and broadening an international community for the

exchange of ideas, experiences, requirements, and best practices

• Implicit process:− Requirements Specifications Standards Interoperability− Note: Implementations are required do do the last three

steps well.

• Definitions:− Specifications: Normative document sufficient for

implementation− Standards: Specifications plus an open process.

Interoperability

• In a SOA context, this is very precise− Implementations interact “on the wire” between

different implementations, languages, and environments

• WS-SOA Offers Unprecedented QoS in this respect− Better than http, not quite as good as hardware

• Only possible by agreeing on a single specification− For GGF this specification is an Open Standard

Interoperation

• Adaptor Based Interaction Possible− A simple service wrapper for each client type

• e.g. JSDL to Unicore AJO to Globus JDL converters− Service composer frameworks possible

• e.g. NAREGI Grid composes Unicore, GT2, GT4, and WSs

• There is a Notion of “Abstract Service Equivalence”− OGSA V1.0 and V1.5 are instances of this− Greatly facilitates adaptor development and

deployment− Language specific standards help build better adaptors

• e.g. a Java API for the OGSA Base Profile or SAGA API.− If all clients (or services) implement adaptors for all

services (or clients) it creates a pleasant illusion of interoperability

Commercial Break

The GGF Roadmap Process

• End User&

TechnologyCommunity

StandardsGroups/Orgs

Vendorand

Open SourceCommunities

Use Casesand

Requirements

Architectures and Specifications

Solutionsand

Building Blocks

CreateValue

DeliverValue

Manage and steerstandards development

Communicate statusand progress

Input to implementation& deployment planning

Roadmap Organization

• Organized by Area, Group, and then Document

• Content for each Document− Document name and short description− GGF Document Type− Progress against key millstones

• Planned and completed dates for First Draft, Public Comment, and publication

− Key Words• Informs Grid Design, Defines Grid Architecture, OGSA,

Applications, Generic grid Component, Other, ...− Adoption Levels

• Unimplemented, Implemented, Interoperable, Community, Adopted, and Ubiquitous.

Adoption Level Definitions

• Unimplemented− Although the specification exists and may be viewed as

stable, no implementation exists. There may be prototypes under development within various organizations, which are not available outside that organization.

• Implemented− There exists at least one implementation that is generally

available for testing and/or deployment that according to the authors (or third parties) implement the specification.

• Interoperable− There exists at least two implementations, as defined above,

that interoperate. There must be a report detailing at least one interoperability workshop.

Adoption Level Definitions Continued

• Community− At least one of the interoperable implementations, as defined

above, is deployed and used on a regular basis by a specific community. This may be due to either a lack of acceptance of the specification by the community at large or due to the specialist nature of a specific specification.

• Adopted− There exists more than one interoperable implementation, as

defined above, and each implementation is used across several communities. Commercially supported implementations are available. This may be either as a product or support for an open source implementation. There may be some restriction on which platforms support the implementations or other aspects that restrict the availability of the implementations.

• Ubiquitous− Interoperable implementations exist for virtually all platforms.

Commercial support is available, but provided transparently as part of the supporting infrastructure.

Some Roadmap Statistics

• Roadmap Documents by Type− Recommendation Documents26− Informational Documents 30− Experimental Documents 3

• Roadmap Documents by Area− Applications 9− Architecture 6− Compute 9− Data 13− Infrastructure 6− Management 9− Security 7

Some More Statistics

• Published Documents− Compute/SRM 6− Data 10− Architecture 7− Applications/APME 7− Infrastructure/ISP/P2P 8− Security 10− Management 2− GFSG 5

• Published Draft-Recommendations Documents9

The Current Pipeline

• Statistics:− Published since GGF 15 9− In or after Public Comment 22− Others in the pipeline 5

• Publication Highlights− GFD.53: OGSA Roadmap− GFD.56: JSDL 1.0− GFD.58: Namespaces for XML Infosets− GFD.59: OGSA Profile Definition

• Progress Highlights− GWD.xx: WSRF OGSA Base Profile through Public Comment− GWD.xx: WS-Agreement through Public Comment

• Highlights from Public Comment− GWD.xx: ByteIO Suite - 2 specs− GWD.xx: DAI Suite - 3 specs

18Documents

in 12 Months

OGSA: Status November 2004

SYSTEMSMANAGEMENT

UTILITYCOMPUTING

GRIDCOMPUTING

Core Services

Base Profile WS-Addressing

Privacy

WS-BaseNotification

CIM/JSIM

WSRF-RAP

WSDM

WS-Security

Naming

OGSA-EMSOGSA Self Mgmt

GFD-C.16

GGF-UR

Data Model

HTTP(S)/SOAP

Discovery

SAML/XACML

WSDL

WSRF-RL

Trust

WS-DAI

VO Management

Information

Distributed query processing

ASP

Data CentreUse Cases &Applications Collaboration Multi MediaPersistent Archive

Data Transport

WSRF-RP

X.509

StandardEvolvingGapHole

Warning: Data may be inaccurate

OGSA: Status February 2006 (or soon)

SYSTEMSMANAGEMENT

UTILITYCOMPUTING

GRIDCOMPUTING

Core Services

Base Profile WS-Addressing

Privacy

WS-BaseNotification

CIM/JSIM

WSRF-RAP

WSDM

WS-Security

Naming

OGSA-EMSOGSA Self Mgmt

GFD-C.16

GGF-UR

Data Model

HTTP(S)/SOAP

Discovery

SAML/XACML

WSDL

WSRF-RL

Trust

WS-DAI

VO Management

Information

Distributed query processing

ASP

Data CentreUse Cases &Applications Collaboration Multi MediaPersistent Archive

Data Transport

WSRF-RP

X.509

StandardEvolvingGapHole

Warning: Data may be inaccurate

Implementations of GGF Specifications

• GFD.56: JSDL 6• GFD.62: PMA Charter 3• GFD.24: GSSAPI extensions 6• GFD.15: OGSI 5• GFD.20: GridFTP 5• GFD.52: GridRPC API 4• GFD.22: DRMAA 4

Implementations of GGF Drafts

• GWD.xx: SAML authorization callout 3• GWD.xx: VOMS attribute certificate format

4• GWD.xx: Daonity 1• GWD.xx: OGSA BES 2• GWD.xx: GGF Usage Record 4• GWD.xx: Usage Record Service 4• GWD.xx: WS-Agreement

6• GWD.xx: OGSA Byte IO 2• GWD.xx: WS-Naming 1• GWD.xx: SAGA 4

Implementations of GGF Drafts

• GWD.xx: CDDLM Smart Frog Language 1• GWD.xx: CDDLM Component Model 4• GWD.xx: CDDLM Deployment API 4• GWD.xx: CDDLM XML-CDL 4• GWD.xx: ACS 2• GWD.xx: WSRF OGSA Base Profile 3• GWD.xx: OGSA BSP Core 3• GWD.xx: OGSA BSP Secure Channel

3

Other Implementations

• GGF Derived Specifications− RFC3820 5− WSRF 5− WSN 5

• GFD.16 Certificate Policy Model40+

Summary

• 103 Implementations of GGF Specifications• The pipeline is still flowing

− Thanks Greg!

• More help is (always) needed

• Give yourselves a hand.

• Thank you