Bryan Lawrence on behalf of the NDG, BADC and BODC.

Ray Cramer, Marta Gutierrez, Kerstin Kleese, Siva Kondapalli, Sue Latham, Roy Lowry, Kevin O’Neill, Ag Stephens, Andrew Woolf

NERC DataGrid: Googling for Secure Data

NERC DataGrid: Googling for Secure Data

British Atmospheric Data Centre

http://badc.nerc.ac.uk

British Atmospheric Data Centrehttp://badc.nerc.ac.uk

Outline

• NDG Aims and Metadata Taxonomy • Demonstration of NDG in action• NDG Authorisation – the security bit!• Status

British Atmospheric Data Centrehttp://badc.nerc.ac.uk

Timelines & Bottom Line

• 2002: E-science arrives at NERC:– Legacy Systems with millions of files and terabytes of data and existing

access and authorisation systems that cannot easily be replaced.– Complex existing DISCOVERY metadata systems.– Discovery (where it exists) based on Z39.50– Utilisation based on file retrieval.

• 2004: NERC DataGrid ready to move forward– New metadata systems describe data as well as datasets.– OAI based harvesting supports scalable FAST data discovery.– Requirements capture for new authorisation systems complete, and coding

underway for implementation.– New communities involved, and international discovery very close to

operational reality.• 2005:

– Utilisation based on metadata, on demand server side behaviours, grid-based back end parallelisation etc

British Atmospheric Data Centrehttp://badc.nerc.ac.uk

http://ndg.nerc.ac.uk

British Atmospheric Data Centre

British Oceanographic Data Centre

Simulations

Assimilation

Complexity + Volume + Remote Access = Grid Challenge

British Atmospheric Data Centrehttp://badc.nerc.ac.uk

NDG Metadata Taxonomy

British Atmospheric Data Centrehttp://badc.nerc.ac.uk

NDG Metadata Architecture

Service based model:• clear separation between discovery and use• discovery service standards compliant and interoperable

British Atmospheric Data Centrehttp://badc.nerc.ac.uk

(D) - Discovery

OAI

OAI

Open Archives Initiative – Digital Library Protocol for harvesting metadata.

NDG Supports Multiple Discovery Services – “build your own”

DirectoryInterchange

Format

DublinCore

GEOProfile

(Z39.50)

IntermediateSchema

Document(s)(XML)

XSLTProcessor

XSLTProcessor

XSLTProcessor

ISO 19115?

CatalogueInteroperabiltiy

Protocol ?

NDG DiscoveryServiceElement

XSLT IngestTransformation

ExistingMetadata

Multiple Protocol Support will be built into the “NDG Vanilla Discovery Service”

British Atmospheric Data Centrehttp://badc.nerc.ac.uk

Wider InternetNERC Grid

taperobot

XML data-base

XML data-base

BADC NDG Wrapper

OnlineData

OnlineData

BODC NDGWrapper

OnlineData

XML data-base

Group NDGWrapper

Software Agent

Grid User

Satellite Supercomputer

Research Group DataSources

Internet Link

Internet User

Internet LinkESG (&other)Applications

Wider Internet

NDGWeb

Portal

XML data-base

British Atmospheric Data Centrehttp://badc.nerc.ac.uk

Discovery

British Atmospheric Data Centrehttp://badc.nerc.ac.uk

Can order responses by title or data centre (or default random)Choose to go to

A service or B service.

Look at DIFs in either HTML or XML

Flexible Information Return

British Atmospheric Data Centrehttp://badc.nerc.ac.uk

Current Interface

British Atmospheric Data Centrehttp://badc.nerc.ac.uk

British Atmospheric Data Centrehttp://badc.nerc.ac.uk

British Atmospheric Data Centrehttp://badc.nerc.ac.uk

British Atmospheric Data Centrehttp://badc.nerc.ac.uk

British Atmospheric Data Centrehttp://badc.nerc.ac.uk

British Atmospheric Data Centrehttp://badc.nerc.ac.uk

British Atmospheric Data Centrehttp://badc.nerc.ac.uk

British Atmospheric Data Centrehttp://badc.nerc.ac.uk

British Atmospheric Data Centrehttp://badc.nerc.ac.uk

British Atmospheric Data Centrehttp://badc.nerc.ac.uk

British Atmospheric Data Centrehttp://badc.nerc.ac.uk

British Atmospheric Data Centrehttp://badc.nerc.ac.uk

British Atmospheric Data Centrehttp://badc.nerc.ac.uk

British Atmospheric Data Centrehttp://badc.nerc.ac.uk

Background activity being parallelised with GODIVA/CCLRC e-science collaboration (spectral -> gridpoint + CDMS + visualisation tools)

Download either plot or the data that went into the plot.

British Atmospheric Data Centrehttp://badc.nerc.ac.uk

British Atmospheric Data Centrehttp://badc.nerc.ac.uk

International Dimension

British Atmospheric Data Centrehttp://badc.nerc.ac.uk

Southampton Oceanography Centre

British Atmospheric Data Centrehttp://badc.nerc.ac.uk

Authorisation• Role-based access:

<dataset> <host> badc.nerc.ac.uk </host><name>ukmo-obs </name><access-requires> researcher <access-requires><access-requires> ukmo-obs </access-requires><processing-requires> nerc </processing-requires></dataset>

• Key concept: Only hosts that trust each other share data, even within a larger virtual organisation: e.g. at BADC:

<trusted><bodc><host>ndg.bodc.nerc.ac.uk</host><attribute remotename=”nerc”> nerc </attribute><attribute remotename=”ashoe”> ashoe </attribute><attribute remotename=”staff”> nerc </attribute> <other> bodc </other></bodc></trusted>

Signed “conditions of use” form exists for this dataset

British Atmospheric Data Centrehttp://badc.nerc.ac.uk

NDG Security

Certificate based, pass encrypted credentials between user and gatekeeper.

British Atmospheric Data Centrehttp://badc.nerc.ac.uk

Where are we?• Migration to web services underway for some components, new A services

in design phase, implementation details not yet obvious (e.g. GT4 etc).• Major effort on defining feature types for observation types so we can build

an OGC/ISO compatible data extractor for observations and numerical data.

• Security Infrastructure Development– Collaboration with CCLRC e-science, ECOGrid

• Ongoing work on metadata definition and population:– Oceanographic data– Atmospheric Chemistry data

• Major issues with (un)controlled vocabularies– Numerical Modelling data

• DIF numerical definition (moving to ISO), BADC and UK Community• Katherine Bouton’s work at NCAS/CGAM (“B” MODEL METADATA)

– Remote Sensing Data• Collaboration with NEODC and PML

• Ongoing work on databases and interfaces, DIF to ISO and “B”