Large-Scale Distributed Computing in the Netherlands

Dutc hG rid

an overviewDavid Groep, NIKHEF

What is DutchGrid?

• collaboration of 12 organisations• established in 2000• functions as a platform,

getting manpower & hardware from partner orgs

• major contributors– WTCW Virtual Laboratory Project (ICES/KIS-II)– ASCI (5 combined Dutch CS Faculties)– Foundation NCF (NWO – natl research council)– NIKHEF (EDG)

DutchGrid Sites

KNMI

Nijmegen

Delft

Leiden

Amsterdam

VU

WCW

Utrecht

Telin

NIKHEFSARAUvAAMOLFCWI

DutchGrid site map 20021104David Groep, NIKHEF

ASTRON, JIVE

WCW Amsterdam

Resources

• ASCI DAS-2 (VU, UvA, Leiden, TUDelft, Utrecht)– 200 dual P-III 1GHz CPUs– homogeneous clusters, 5 locations

• NIKHEF DataGrid clusters– 75 dual P-III ~ 1GHz – 1Gb/s IPv4 + 1Gb/s IPv6

• NCF Gridnational computer facilities foundation from NWO– 66 node dual AMD-K7 Fabric Research Cluster (NIKHEF)– 32 node duals “production quality” cluster (SARA)*– 10Gb/s optical “lambda” test bed†

– …• BioASP – various smaller O(10 node) clusters

Resources (cont.)

SARA – National HPC Centre• Processing

– SGI 1024 processor MPP• Mass storage

– StorageTek NearLine tape robot– currently: 500 TByte– will grow “indefinitely” (with funds from NCF)

• User expertise centre

In TotalOperations/deployment effort ~ 8 FTE availableResearch effort O (40 FTE) over 12 organisations

Resource Usage

CR1-1 CR1-2

CR2-1 CR2-2BR2

AR5

BR1AMS

10GPOS

2.5GPOS

BR1CHI

622M PO Prot.Tyco/IEEAF

AMSTERDAM

2x2.5Gb

622Mb L2 VPN

2xGbE622Mb “rest”Global Xing

622Mb to “rest”

NY

CR1-1 CR1-2

CR2-1 CR2-2BR2

AR5

BR1AMS

10GPOS

2.5GPOS

BR1CHI

622M PO Prot.Tyco/IEEAF

AMSTERDAM

622Mb L2 VPN

NY

CR1-1 CR1-2

CR2-1 CR2-2BR2

AR5

BR1AMS

10GPOS

2.5GPOS

622M PO Prot.Tyco/IEEAF

622Mb L2 VPN

NY

Networks

IGRID6509

BR4

SP01

OC48cSR

StarLight6509

ON

S15

454

ON

S

StarLightT640

ABILENE NY

ARGONNE IL

SingARENTANet2

…

StarLightRTR

STARTAP

GbE

GbE

CERNRTR

ONS

10G POS Tyco

10G POS Level3

2xGbE

10GbE

2xGbE

2.5Gb lambda

OC

48c

lam

bda

7xGbE

GbE

GbE’s

CHICAGO, IL

CERN,CH

18xGbEUvANIKHEF“NCSA”“ANL”

NIKHEF network load

Monthly graph NIKHEF-SURFnet5 IPv4 interconnectOctober/November 2002

New Initiatives inside NL

NCFGridstrengthening Grid Infrastructure in NL• 4 partners (NIKHEF, ASCI, SARA, ASTRON)• 600 kEUR funding• hardware (3 clusters, 10Gb/s networking)

and people (1.5 FTE over 3 years)• fabric research + production-quality grids• new apps: bio informatics, astronomy• approved August 2002

First cluster assembled and being tested…

New Initiatives inside NL (2)

Virtual Laboratory for e-ScienceDutch “knowledge infrastructure” project

• Higher-level services to efficiently use global resources• Both “bottom-up” and “top-down” in same project

• 14 partners, 6 application domains• middleware research programme

HPC SecAAA Networks

IntVisual.

Inf. Mngt

Intg ExpEnvs UIs

D.I.Sc. FoodInf BioInf LargeEquip

Biodiversity …

Test BedScaling

&Validation

in planning stageO (30-50 MEUR)

What is happening where in NL?

DutchGrid Op Services

• Authentication Services (Certification Authority)– EDG CACG “proper” CA– Student and educational CA– EDG “Tutorial” CA

• Regular coordination and info meetings• Installation support

– Have been distributing EDG 1.2 till now

• User support

Principal Apps

• HEP: DØ, Atlas, LHCb, AliceMonte-Carlo and Analysis

• OPERA – ozone monitoring• WTCW Virtual Laboratory

visual analysis environment for (small-scale) experimental science– Chemo-physical surface analysis (PIXE,FT-IR,SIMS)– BioASP – correlating “omics” databases– Virtual Surgery (planning by-passes, artery repairs)

• EcoGridcorrelating bio-diversity information databases

• Radio AstronomyLOFAR, AVO, VLBI – semi-online and offline processing

Research Areas in NL

• High-bandwidth networkingSURFnet, UvA, SARA, NIKHEF

• Generic AAA and `security’UvA, NIKHEF

• Cluster, Storage Management & MonitoringNIKHEF, SARA

• Adaptive programming environmentsFree University Amsterdam

• Memory-processor co-allocation TU Delft

• Federated Information & Content ManagementUvA

• Interactive visualisationCWI, UvA

Summary

• DutchGrid Platform started informally in 2000• de-facto focus for Grid computing in NL• ops (CA, web, …) supported by partners

• mix of people from all ‘layers’networks, fabrics, security&AAA, prog. environments, HPC visualisation, semantic DBs, applications