Some Grid Science

California Institute of Technology

Roy WilliamsPaul Messina

Grids and Virtual Observatory

Grids andand LIGO

Virtual Sky: Image Federation

Xray (ROSAT) theme

Change scale

Change theme

http://virtualsky.org/fromCaltech CACRCaltech AstronomyMicrosoft Research

Optical (DPOSS)

Coma cluster

Virtual Sky has140,000,000 tiles

140 Gbyte

VO and Grid ComputingSupercomputer center support for

Data Pipelines (not just MPI)

12 hr 250 Gb

HPSS

HP Superdome

VS1 Science productHPSS system

VS2 Web productWin2000 database

Raw data

CPU Utilization

VirtualSky.org Ingestion

Copy raw data and filter (background subtraction)

Resampling

Write VS1

Build VS2

Write VS2

Today ~ 3 TB

1 TB

140 GBComputation also takes about 12 hrs.

VO and database JoinR

AD

ec

mag

Jm

ag

Hm

ag

K

glo

ng

lat

flu

x I

flu

x Q

flu

x U

Infraredcatalog

(indexed by RA)

Radiocatalog

(indexed by glon)

Crossmatchengine

SQL select

SQL select

Need same sorting for crossmatchWho does it?Or can we impose a standard?

order by glon?

order by RA?

sort here?

Sorting 10^9 objects is expensive

VO Interoperability• Directory white/yellow pages

– VO Schema repository

• How to publish, how to define content• action> Document schema for

– archive content– web service capability– table/image/spectrum

• How to publish algorithms• Plug and play data services

– SOAP, UDDI, WSDL, Jini?

• Semantic web– “Question and answer in the language of the client” – Topic maps

VO and Interoperability

Client application (e.g., OASIS)

asks for available catalogs

then for attributes

makes a query

then displays the result

What is the standard interactionwith a catalog service?

Client Service

Want to be dynamic, be able to

add catalogs and the applications still work

VO and International

•Of course there are many astronomy data collections in the UK, Europe, Japan, etc.

•We intend to collaborate/coordinate with international efforts

LaserInterferometricGravitational waveObservatory

Listening to Collisions ofBlack Holes andNeutron Stars

LIGO

LIGO Pictures

Grid LIGO Architecture

Clientseg Web, Script, Agent

Clientseg Web, Script, Agent

Text requestText request Request Manager

Request Manager

GriPhyN LDAS

Gatekeeper(GRAM)

Gatekeeper(GRAM)

Science AlgorithmsSoftware Collaboratory

Parallel ComputingGridFTPGridFTP

Local Disk

Data

HPSS

GridFTPGridFTP

Replica CatalogReplica ManagementTransformation CatalogVirtual Data Catalog

Virtual Data Request

Data Movement

Globus RPC

other LDASCondor jobs

Objective: Add security & wide-area data servicesLDAS is Ligo Data Analysis System

Grid LIGO Logic• VD Request Request Manager

– XML, Key-value, script?– Action> Build document schema for

this• Do we have it already in data caches

– if yes, get from caches, use simple transformations

– else build LDAS script•Move relevant data to LDAS local

disk•Queue job to LDAS, get jobID•Pass updates, then result pointer

back to user•Perhaps cache result

International GW Grid

50 kbyte/sec

GEO

International GW Grid• Coincidence in Ligo-Virgo data

– Not astrophysics data yet….– Try seismic, electromagnetic data for now

Caltech

VirgoElectromagneticSeismic

Coincidence events!

Currently using rsync

LLO

LHO

Merging 3 streams to 1

Correlate signals(LDAS)

~12 kbyte/sec each way