vl-e poc: what it is and what it isn’t jan just keijser vl-e p4 scaling and validation team tu...
Post on 19-Dec-2015
221 views
TRANSCRIPT
VL-e PoC:What it is and what it isn’t
Jan Just KeijserVL-e P4 Scaling and Validation Team
TU Delft Grid Meeting, December 11th, 2008
What is the VL-e PoC Environment?(from http://poc.vl-e.nl)
The Proof-of-Concept Environment (PoC) is the shared, common environment for e-Science of the Virtual Laboratory for e-Science. In the PoC, the different tools and services used by and provided by the project are available, and bound together in a service oriented approach. The PoC covers three distinct areas:
• A software Distribution, to be installed by any-one in VL-e interested in participating in the PoC
• A PoC Environment, the ensemble of systems that run the current PoC Distribution
• The PoC Central Facilities, those systems running the PoC Distribution that are centrally managed by the P4 Scaling and Validation Programme on behalf of the Project
The VL-e PoC R3 Distribution
VL-e PoC R3 Contents:Scientific Linux 4 32bit
gLite 3.1
Sun Java JDK 1.5.0_16
Plusfsl 4.0 gat 1.8.2 globus-toolkit 4.0.7graphviz 2.18 ibis 1.4 itk 3.4javagat 1.7.1 kepler-1.0.0rc1 lam 7.1.2lucene 2.3.1 MatlabMPI 1.2 Mesa3D 6.4.2modules 3.2.3 mpitb 2.1.73 mricro 1.3.9octave 2.1.73 paraview 3.2.1 swi-prolog 5.6.53R 2.6.2 Rmpi-0.5 sesame-client 2.0.1SRB client 3.4.2 taverna 1.7.1 vlet/vbrowser 0.8.1vtk 4.4.2 & 5.0.4 weka 3.4.12
Contributed packagesmono (C#) 1.2.5
see http://poc.vl-e.nl/distribution/R3contents.html for full list
The VL-e PoC Environment
Application development
NL-Grid production clusterCentral mass-storage facilities
+SURFnetInitial compute
platform
Stable, reliable, testedCert. releases Grid MW & VL-
software
VL-e Proof of Concept EnvironmentVL-e Rapid Prototyping
Environment
DAS-2/3,local resources
VL-e Certification Environment
NL-Grid FabricResearch Cluster
Test & Cert.Grid MW & VL-software
Compatibility
Flexible, test environment
Environments
Usage
Characteristics
Virtual Lab.rapid prototyping
(interactive simulation)
Flexible, ‘unstable’
PoC Release n Release Candidaten+1
Developers environment
is a shared, common environment, where different tools and services are both used and provided by the VL-e community
PoC / BigGrid CTB RPE (DAS-3)
● Rapid Prototyping Environment (DAS-3) System load is usually low Users have a single userid + home directory Some people say it's not a grid, but a
Cluster of Clusters● Proof-of-Concept Environment
Typically high load (90+ %) Users get access using X509 certificates and are
assigned pool accounts. Pool accounts are unique to each cluster and do not
share home directories, not even within a cluster Group rights are handled using Virtual Organisations
and VOMS No direct user logins allowed No inbound connectivity (listeners) on worker nodes
Differences between RPE and PoC
● Scientific Linux/Centos 4 32bit● Migration to Centos 5 64bit in 2009● Runs gLite grid middleware● Usually Torque/PBS as the batch system● Usually not allowed to run software on the
cluster headnode● Runs PoC software and lots of non-PoC
software● Job allocation is based on 1 job per core● Typically 2GB of RAM per job ● Multicore/MPI jobs not well supported (yet)
Typical PoC cluster characteristics
1. VO Software Group area VO has full control Must be installed per cluster Not supported at all by the P4 team or site
administrators
2. VL-e 'contrib track'• Contributed packages are not part of the regular PoC
release cycle
• Certified only to pass some basic installation tests
• Not supported by the P4 team, but exclusively by the original contributor
• Most sites will install contributed packages, but no guarantees
• See http://poc.vl-e.nl/distribution/contrib/ for details
How to get software to run on the PoC
3. Part of the next PoC release Nirwana/Walhalla/Heaven? Must pass 'contrib track' first Must be stable, reliable and tested software PoC Releases are scheduled every ~9 months NO major updates possible in between (except critical
security fixes)
How to get software to run on the PoC
3. Part of the next PoC release Nirwana/Walhalla/Heaven? Must pass 'contrib track' first Must be stable, reliable and tested software PoC Releases are scheduled every ~9 months NO major updates possible in between (except critical
security fixes)
There are some good points: Eternal glory Will be installed at all VL-e PoC clusters Supported by VL-e P4 Scaling and Validation team
How to get software to run on the PoC
VL-e User Applications seen on thePoC Infrastructure
• Data Intensive Science: DANS (KNAW/KB) Sciamachy (KNMI) eNMR (UU)
• Food Informatics: Bitterbase web service (Unilever)
• Medical Diagnosis & Imaging: fMRI, Vbrowser, Moteur (AMC) JavaGAT (VUmc)
• Bio Informatics