09.04.2008 theory, grid and vo matthias steinmetz (aip)
TRANSCRIPT
09.04.2008
Theory, Grid and VO
Matthias Steinmetz (AIP)
09.04.2008
applicationData
analysis
Super Computer
Super computer
Telescopes
Data Archive
DataArchive
PC Cluster
PC Cluster Data analysis
User
MIDDLEWARE
Resources are "virtualized", i.e. can not be identified
individually
Characteristics of a Grid: network of IT-Ressourcen
3VODCA: Theory, Grid and VO09.04.2008
VO and Grid
What is the dividing line between VO and Grid? Not well defined Example UK:
AstroGrid covers VO and Grid aspects Example Germany
GAVO: application layerAstroGrid-D: middle ware
4VODCA: Theory, Grid and VO09.04.2008
Theory and Grid
Benefits of the Grid Logistics (resource monitoring, scheduler/broker,
virtual organizations, …) Virtual Surveys (Millenium simulation) Enterprise computing (access to supercomputers
via grids, e.g. DEISA) Cloud computing (“task farming”) Volunteer computing (@home model) Visualization
5VODCA: Theory, Grid and VO09.04.2008
StellarIS: resource monitoring
Grid-Ressource-Map
Basiert auf GoogleMap
6VODCA: Theory, Grid and VO09.04.2008
Stellaris: Job monitoring
Webinterface for simple job monitoring
Time table forsubmitted
Jobs
Minutes
hours
days
7VODCA: Theory, Grid and VO09.04.2008
Analysing Cosmological Simulations in the Virtual Observatory:Designing and Mining the Millennium Simulation Database
Gerard Lemson German Astrophysical Virtual
Observatory
ARI, Heidelberg
MPE, Garching bei München
8VODCA: Theory, Grid and VO09.04.2008
Time evolution: merger trees
9VODCA: Theory, Grid and VO09.04.2008
10VODCA: Theory, Grid and VO09.04.2008
Merger trees :select prog.* from galaxies des , galaxies prog where des.galaxyId = 0 and prog.galaxyId between des.galaxyId and des.lastProgenitorId
Branching points :select descendantId from galaxies des where descendantId != -1 group by descendantId having count(*) > 1
11VODCA: Theory, Grid and VO09.04.2008
12VODCA: Theory, Grid and VO09.04.2008
Usage statistics
Up since August 2006 (astro-ph/...) ~210 registered users > 4.4 million queries ~ 35 billion rows (since March 2007)
# queries/day # rows/day # secs/day
13VODCA: Theory, Grid and VO09.04.2008
Enterprise Computing:Mare Nostrum Simulations at HLRZ
WMAP3 parameters, w=0.8Testrun using the grid: 10243+10243 particles
14VODCA: Theory, Grid and VO09.04.2008
NBODY6++ UseCase
Computer simulation of dense stellar systems
Example: globular clusters
Gravitational Star-Star interactionComplexity N2 (N: number of stars)
15VODCA: Theory, Grid and VO09.04.2008
Enterprise ComputersEnterprise Computers
Recent Development: GPU – Graphics Cards
GeForce 8800 GTX (NVIDIA)Using CUDA LibrarySpecial Interfaces and API from GRACE project ported.
Berczik et al. 2008Spurzem et al. 2008
16VODCA: Theory, Grid and VO09.04.2008
Cloud Computing: UseCase Dynamo
Visualization of results
Rechner 1 Rechner 2
Rechner 3 Rechner 4
2D-Darstellungder
Magnetfeldstärke auf der
Sternoberfläche
Querschnittdurchden
Stern
17VODCA: Theory, Grid and VO09.04.2008
<jsdl:JobDefinition xmlns="http://www.gacg-grid.de/namespaces/job-mgmt/2006/08/jsdl" xmlns:jsdl="http://schemas.ggf.org/jsdl/2005/11/jsdl" xmlns:jsdl-posix="http://schemas.ggf.org/jsdl/2005/11/jsdl-posix" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<jsdl:JobDescription> <jsdl:JobIdentification> <jsdl:JobName> Sample Dynamo run </jsdl:JobName> <jsdl:Description>
Use Case Dynamo </jsdl:Description> <jsdl:JobProject> n/a </jsdl:JobProject> </jsdl:JobIdentification><jsdl:Resources> <jsdl:FileSystem name="HOME"> <jsdl:Description> User's home directory </jsdl:Description> </jsdl:FileSystem> </jsdl:Resources>
[…]
<?xml version="1.0" encoding="UTF-8"?>
<job> <executable>test.x</executable> <directory>/${GLOBUS_USER_HOME}/
dynamo</directory> <stdout>test.out</stdout> <maxWallTime>100</maxWallTime> <maxMemory>1</maxMemory> <fileStageIn> […]
Dynamo JDSL und RSL
18VODCA: Theory, Grid and VO09.04.2008
Volunteer Computing: GEO600 / LIGO
Laser Interferometer Gravitational Wave Observatory
19VODCA: Theory, Grid and VO09.04.2008
GEO600/LIGO
Network von 4 Detectors Hanford (2000m side length) USALivingston (4000m side length) USAGEO600 ( 600m side length) GermanyVirgo (3000m side length) Italy
Pathfinder for LISA, Satellite mission with 3 detectorsside length: 5*109 m!
20VODCA: Theory, Grid and VO09.04.2008
Gravitationional waves: Data analysis via the Grid
Data analysis via small data packages, “embarrassingly parallel”.
Einstein@Home is, like SETI@Home, suitable to exploit idle cycles on work stations.
Einstein@Home is an ideal, simple Grid application, supporting many operation system.
Checkpointing and Recovery allows a very accurate control of CPU-Requirements and walltime.
Automatic software deployment job submission and job management, a good scalability of grid application can be obtained
Current workload: 30000 CPU h per week
21VODCA: Theory, Grid and VO09.04.2008
GEO600 – Resource Integration
user friendly User-Management via VOMRS
Resource information via MDS und StellarIS
Grid Service Monitoring
automatic job submission on D-Grid resources
Job monitoring und job management using a
Laptop
data management on Astrogrid-D Storage
Cluster
22VODCA: Theory, Grid and VO09.04.2008
Visualization of a galaxy merger
Simulation: two galaxies on collision orbit Visualization: 2D-projections of 3D snapshots
ZIB AIP
Submit Execution
ProC SubmitWorkflow
Video Workflow A
stop
exit
exit
start
GT4 submit
GT4 submit
Video Workflow B
ARI ...
23VODCA: Theory, Grid and VO09.04.2008
Grid-Visualization
Submission Host: ZIB ProC + Master workflow
Submission of video workflows Display of videos
Execution Hosts: AIP + ZAH PiCo + Video workflow
Projection to 2D Color coding
Future: Graphics rendering at LRZ, graphics output on local host
24VODCA: Theory, Grid and VO09.04.2008
Theory and Grid
Benefits of the Grid Logistics (resource monitoring, scheduler/broker,
virtual organizations, …) Virtual Surveys (Millenium simulation) Enterprise computing (access to supercomputers
via grids, e.g. DEISA) Cloud computing (“task farming”) Volunteer computing (@home model) Visualization