www.badc.rl.ac.uk bryan lawrence head, british atmospheric data centre rutherford appleton...
Post on 18-Dec-2015
216 views
TRANSCRIPT
www.badc.rl.ac.uk
Bryan LawrenceHead, British Atmospheric Data Centre
Rutherford Appleton [email protected]
(with thanks to Tony Hey, the Director of the UK e-Science Program who provided most of
the slides: [email protected] )
The UK e-ScienceProgram
CEOS Meeting, Frascati, May 2002
www.badc.rl.ac.uk
Outline
• The Grid, UK e-science and the context • The UK e-science initiative
– The core Programme• Support for e-science projects and international involvement
• The Grid Network Team
• Grid Middleware R&D.
– Projects: e-healthcare (MIAS), MyGrid, ClimatePrediction.com, the NERC DataGrid.
• Concluding statements
www.badc.rl.ac.uk
‘e-Science is about global collaboration in key areas of science, and the next generation of
infrastructure that will enable it.’
‘e-Science will change the dynamic of the way science is undertaken’
John TaylorDirector General of Research Councils
Office of Science and Technology
E-Science and the Grid
www.badc.rl.ac.uk
UK e-Science Initiative
• £120M Programme over 3 years• £75M is for Grid Applications in all
areas of science and engineering• £10M for Supercomputer upgrade• £35M ‘Core Program’ to encourage
development of generic ‘industrial strength’ Grid middleware Require £20M additional ‘matching’
funds from industry
www.badc.rl.ac.uk
Excerpt from e-Science Director’s job
objectives
‘Develop effective collaborative Core Programme projects between the science base, industry and national funding agencies, and ensure the application and outcomes from the projects.’
www.badc.rl.ac.uk
UK e-Science Grid (1)
Newcastle
Edinburgh
Oxford
Glasgow
Manchester
Cardiff
Southampton
London
Belfast
DL
RAL Hinxton
National Centre: Edinburgh; + regional centres
www.badc.rl.ac.uk
UK e-Science Grid (2)
• All e-Science Centres donating resources to form a UK ‘national’ Grid– Supercomputers, clusters, storage, facilities
• All Centres will run same Grid Software- Starting point is Globus, Storage Resource Broker and Condor
• Work with Global Grid Forum and major computing companies (IBM, Oracle, Microsoft, Sun,….)– Aim to ‘industry harden’ Grid software to
be capable of realizing secure VO vision
www.badc.rl.ac.uk
Support for e-Science Projects
• ‘Grid Starter Kit’; continually updated- maintain library of Open Source Grid m/w- http://www.gridsupport.ac.uk/gridcentre.shtml
• Grid Support Centre in operation- leads Grid Engineering Group; supports users
• Training Courses- first courses given
• National e-Science Institute Research Seminar Programme – see website http://www.nesc.ac.uk
www.badc.rl.ac.uk
Support for International Involvement
• ‘GridNet’ funding- supports participation in the GGF
• ‘Grid Fellowships’ in Geneva and US• Links with major US Centres
- San Diego Supercomputer Center and NCSA
• Joint UK-NSF ‘N+N’ Meeting on e-Science- held in San Fransisco last year
• Other international collaborations- China, Singapore, India, ….
www.badc.rl.ac.uk
Grid Network Team
• Expert group to identify end-to-end network bottlenecks and other network issues- e.g. problems with multicast for Access Grid
• Identify e-Science project requirements
• Funding £0.5M traffic engineering/QOS project with PPARC, UKERNA and CISCO
www.badc.rl.ac.uk
SuperJanet4, June 2002
Scotland via Glasgow
NNW
Northern Ireland
MidMAN
TVN
South WalesMAN
SWAN&BWEMAN
WorldComGlasgow
WorldComEdinburgh
WorldComManchester
WorldCom
Reading
WorldComLeeds
WorldComBristol
WorldCom
London
WorldComPortsmouth
Scotland via Edinburgh
YHMAN
NorMAN
EMMAN
EastNet
External Links
LMN
KentishMANLeNSE
10Gbps
622Mbps155Mbps
20Gbps
2.5Gbps
www.badc.rl.ac.uk
Grid Middleware R&D
• £16M funding available for industrial collaborative projects
• £11M allocated to Centres projects plus £5M for ‘Open Call’ projects
• Set up two Task Forces- Database Task Force (Chaired by Norman Paton from Manchester Centre)- Architecture Task Force (Chaired by Malcolm Atkinson, Director of NeSC)
www.badc.rl.ac.uk
Generic Grid Middleware R&D
• Reports on Globus, SRB/Databases and .NET middleware
limitations of present Grid Middleware• Developing UK ‘Road Map’ for evolution
of present Grid Middleware- Short term improvements (6-12 months)- Longer term plans
adaptive, intelligent infrastructure database interfaces allowing querying
and extended transactions
www.badc.rl.ac.uk
UK e-Science Projects
• £75M for e-Science application ‘pilots’- spans all sciences and engineering
• Particle Physics and Astronomy (PPARC)- £20M GridPP and £6M AstroGrid
• Engineering and Physical Sciences (EPSRC)- funding 6 projects at around £3M each
• Biology, Medical and Environmental Science– £20M fund supporting a number of projects
www.badc.rl.ac.uk
Core Funded Projects
Many projects,
Some core-funded, some joint + RC funding
www.badc.rl.ac.uk
e-Healthcare Grand e-Healthcare Grand ChallengeChallenge
• Interdisciplinary Research Centre, MIAS:“From Medical Images and Signals to Clinical
Information”
• Funding £2M Joint IRC projects with MIAS on e-Healthcare applicationExample: Breast cancer surgery – normalization of mammography and
ultrasound scans– FE modelling of breast tissue
Deliver useful clinical information to surgeon ensuring privacy and security
www.badc.rl.ac.uk
• GridPP– links to EU DataGrid, CERN LHC
Computing Project, US GriPhyN and PPDataGrid Projects, and iVDGL Global Grid Project
• AstroGrid– links to EU AVO and US NVO
projects
Particle Physics and Astronomy (PPARC)
www.badc.rl.ac.uk
EPSRC e-Science Projects
6 Projects: Comb-e-Chem, DAME, Reality Grid, MyGrid, GEODISE, Discovery Net,
Example: My Grid: Personalised Extensible Environments for Data Intensive in silico Experiments in Biology– Manchester, EBI, Southampton, Nottingham,
Newcastle, Sheffield, GSK, Astra-Zeneca, IBM, Sun
www.badc.rl.ac.uk
MyGrid e-Science Workbench
• Goal is to develop ‘workbench’ to support:– Experimental process of data accumulation– Use of community information– Scientific collaboration
• Provide facilities for resource selection, data management and process enactment
• Bioinformatics applications – Functional genomics, pattern database
annotation
www.badc.rl.ac.uk
ClimatePrediction.com
• NERC thematic, e-science and national CORE demonstrator funding!
• Partnership between
www.badc.rl.ac.uk
ClimatePrediction.Com
Estimating Climate Uncertainty
- current estimates based on a handful of models.
-need to consider predictions based on 1000’s of ensembles - harness the power of 10,000’s of PCs
by providing downloadable model experiments
www.badc.rl.ac.uk
Managing a petabyte-scale massively-distributed data
archive
Scientificinvestigators
Participants &policy-makers
Summarystatistics
100Tb of key output at 10-20 sites
1Pb total output on 1M participants’ PCs
ESG-II/NERC DataGridGridFTP
HTTP (DODS URL) Live Access Server
HTTP HTTP
Datamining Peer-to-peer visualisation
Conventional FTP/HTTP
Obs
www.badc.rl.ac.uk
The NERC DataGrid
TheThe NERCNERC DataGridDataGridTheThe NERCNERC DataGridDataGrid
Proposal to the Natural Environment Research Council:
• Collaboration between two professional data centres, the CLRC e-science centre, and PCMDI (+ESG).
• Aim to improve the ability to locate and use both observational and simulation data: build software clients.
• Eventual expansion to include all NERC disciplines including Earth Observation (NEODC early adopter)
www.badc.rl.ac.uk
XML Cat&ClientServer (s)
1
NDG expected evolution
Computation
At USER Institution
Data Repositories
DataFile
010010010
Other: e.g. PML/ESSC
NERC DDC
DataFile
010010010
2
Catalogue
Client
Computation
Graphics
Based on ESG
Satellite
Local Catalogue
CatalogueIngestor4
3
Python API
CatalogueClient
Computation
Evolving to web services
5Docs
6
www.badc.rl.ac.uk
Concluding Statements
• Wide variety of UK Application projects using clusters, supercomputers, data repositories, remote working tools etc.
• Emphasis on support for data federation and annotation as much as computation
• Metadata and ontologies key to higher level Grid services
• For commercial success Grid needs to have interface to DBMS