Grid Middleware Tutorial / Grid Technologies Intro Slide 1 /14

Grid Technologies Intro

Ivan Degtyarenko

ivan.degtyarenko dog csc dot fi

CSC – The Finnish IT Center for Science

June 3rd, 2008

Introduction to Grids and NorduGrid

Grid Middleware Tutorial / Grid Technologies Intro Slide 2 /14

Contents

● Part 1: Introduction to Grids and NorduGrid

What is the Grid? Role of Middleware Accessing the Grid Grid reality NorduGrid collaboration / M-grid as an example

● Part 2: NorduGrid middleware: ARC Submitting jobs in NorduGrid, writing job description

files Simple file transfers Monitoring jobs using the Grid Monitor graphical

interface

Grid Middleware Tutorial / Grid Technologies Intro Slide 3 /14

Hierarchy of the IT Resources

desktops

in-house resources

regional computer centers

grid infrastructures

international HPC centers

Grid Middleware Tutorial / Grid Technologies Intro Slide 4 /14

What is the Grid?

• no clear definition: Compute Grids, Science Grids, Data Grids, Cluster Grids etc.

• must be evaluated in terms of the applications, and scientific results that it delivers, not its architecture

• the key concept is the ability to negotiate resource-sharing arrangements among a set of participating parties and then to use the resulting resource pool

• a basic definition for grid might look like:   "a hardware and software infrastructure that provides dependable, consistent, pervasive and inexpensive access to high-end computational capabilities"

Grid Middleware Tutorial / Grid Technologies Intro Slide 5 /14

What is the Grid? (cont.)

Ian Foster, What is the Grid? A three point checklist, Grid Today, 1(6), 2002. A Grid is a system that:

• coordinates resources that are not subject to centralized control: a Grid integrates and coordinates

resources and users that live within different control domains

• using standard, open, general-purpose protocols and interfaces: a Grid is built from multi-purpose protocols

and interfaces that address such fundamental issues as authentication, authorization, resource discovery, and resource access

• deliver nontrivial qualities of service: the utility of the

combined system is significantly greater than that of the sum of its parts (response time, throughput, availability, security etc.)

Grid Middleware Tutorial / Grid Technologies Intro Slide 6 /14

Role of Grid Middleware

user applications

grid middleware

Gridresource

Gridresource

Gridresource

Gridresource

Grid Middleware Tutorial / Grid Technologies Intro Slide 7 /14

Grid fundamental services implemented in the middleware

• resource discovery and monitoring

• job submission and management

• brokering and data services

• information services

• resource management

Grid Middleware Tutorial / Grid Technologies Intro Slide 8 /14

Computational Grids: resource sharing principle

Goal is to minimize waste of resources: empty nodes are always available for use (dynamic sharing)

Grid Middleware Tutorial / Grid Technologies Intro Slide 9 /14

Accessing the Grid: Authentication

• one needs to be authenticated− every grid user holds a X.509 certificate issued by a

Certificate Authority

− being authenticated by a site means that the Issuer CA of your credential is trusted by the Grid resource

− certificates to be updated annually

• authentication and authorization are decoupled processes

user identification is called authorization

Grid Middleware Tutorial / Grid Technologies Intro Slide 10 /14

● users are grouped into Virtual Organizations (VO)

based on a common research area, country, funding agency, project, etc.

the same user can belong to several VOs

● resource providers grant access to VOs effective managing comparing to individual users implies trust towards the organization managing the VO for instance: e.g. in the M-grid the users of each site

form one VO, and we could combine all to a larger "M-grid VO" when negotiating with external parties

Accessing the Grid: Authorization

granting resource access is called authorization

Grid Middleware Tutorial / Grid Technologies Intro Slide 11 /14

Grid Security

WARNING! When working on the Grid,

you must accept that some information on your jobs and on your Grid identity is made public, i.e your name / affiliation

IP address of your client computer job names and duration runtime environment others

Grid Middleware Tutorial / Grid Technologies Intro Slide 13 /14

NorduGrid Collaboration

● past: Grid pilot project by Nordunet in 2001

implemented a production Grid system working non stop since May 2002

● present: a community around open source Grid middleware: NorduGrid ARC

national Grids (e.g. M-grid, SweGrid, NorGrid), users also outside the Nordic countries

real users, real applications funding for further

development (KnowARC, NDGF)

● open for anyone to participate

Grid Middleware Tutorial / Grid Technologies Intro Slide 14 /14

The Finnish Material Sciences Grid (M-grid)

● goal: throughput computing capacity mainly for the needs of physics and chemistry researchers

opened to all CSC customers in Nov 2005, not limited by the field of research

● joint project between seven Finnish universities, Helsinki Institute of Physics and CSC

partners mainly laboratories and departments, not university IT centers

● first large initiative to put Grid middleware into production use in Finland

uses NorduGrid ARC middleware

● platform: Linux based PC clusters