2003/11/27cvo meeting in beijing current status of japanese virtual observatory masatoshi ohishi...

2003/11/27 CVO Meeting in Beijing

Current Status of Japanese Virtual Observatory

Masatoshi OhishiNational Astronomical Observatory of Japan

　　　　　　 and Sokendai

[email protected]


CollaboratorsNAOJ Y. Mizumoto N. Yasuda Y. Shirasaki M. Tanaka S. Honda

Ochanomizu U. Y. Masunaga

Osaka U.

S. Shimojo & his team

Titech

S. Matsuoka & his team

Fujitsu Ltd. K. Monzen K. Kawarai Y. Ishihara H. Yanaka Y. Yamaguchi M. Ishida N. Saito K. Abe J. Tsutsumi H. Nakamoto Y. Morita

Ken Miura


Large Telescopes of NAOJ

Subaru (optical & Infrared) @ Hawaii Nobeyama (Radio)

single dish : 45-m

interferometer : 6 x 10-m

solar interferometer : 84 x 75-cm HALCA – in orbit 8-m


Subaru


Subaru Telescope


45m Radio Telescope


VSOP


Infrared Satellite by JAXA

Launch : 2005 ?

JAXA is going to join JVO !!


Data Productivity

Nobeyama Radio ： ~1TB/yr Subaru@Hawaii ： ~20TB/yr ALMA （ planned ）　： ~PB/yr

Flood of excellent data (survey data)

Digitized & Archived


10 Gbps


HPC + High Speed NETWORK

+GRID Technology

→ it is possible to federate multi-wavelength-DBs and computing facilities for data analyses

→ Virtual Observatory


JVO prototype : Ver. 1

Establish seamless access to federated DBs by using the Grid technology

Define JVO Query Language, and implement its parser

Test environment is in a closed subnet


Monitor Window: Refreshed automatically in every 15 seconds

Dynamical creation of a workflow


Sample Outputs


Java2SE 1.4

Apache Tomcat

soapuddi

JDBC

UDDI　Registory　/　Servlet　Engine

Java2SE 1.4

JVO Server

UDDI4J

JVO　Controller

Globus Toolkit (Server)

JVO Service(count)

SQLlibrary

Java2SE 1.4

Maintenance Tool

UDDI4J

UDDI　Maintenance　Tool

Hypertextcontents

Solaris8　(Possible at Linux)

Redhat　Linux

JVO　Service

ORACLE

DB　Service

Free Software

Commercial Software

In House Software

Function

ApacheＳＯＡＰ

① AppletDownLoad

Globus Toolkit (Client)

Redhat　Linux

PostgreSQL (DBMS)

DB　Service

The following free software is included by globus toolkit.• OpenSSL (Secure Socket Library)• OpenLDAP (LDAP Server)• wu-ftpd (ftp server)

JVO Prototype System Architecture

③ Find Service

Netscape Communicator

Java2 Plugin

JVO Client(applet)

JVO　Client

② RequestObservation

Monitoring

UDDIData

SUBARUSupCamz-band• get & parse GSDL

• pre condition check• execute Service• post condition check

• register• update• delete

GSDL(copy)

JVO Service(select)

SQLlibrary

JVO Service(image)

JVO Service(X match)

ImageData

GSDLGridFTP

ImageData(copy)

⑥ spawn Service

⑤ execute Service

Check status

SUBARUDataBase

JVOController

Globus Toolkit

AVO

Globus Toolkit

NVO

Internet

ImageData

SUBARUSupCami-band


JVO　Service

SQLlibrary

SQLlibrary

JVO Service(count)

JVO Service(select)

Copy imate data by GridFTP

Copy imate data by GridFTP

JVO Service (X match)

JVO Service (image)

③ get GSDL

④ Get GSDL by GridFTP

GSDL

⑦ issue SQL

④’ Get GSDL by GridFTP

⑦’ issue SQL

⑤’ execute Service

ImageData(copy)

SUBARUDataBase

Gb　Ether

⑥’ spawn Service


JVO　Service

SQLlibrary

JVO Service(select)

JVO Service (X match)

GSDL

Redhat　Linux 2MassDataBase

PostgreSQL

DB　Service

2Mass DB

ImageData(copy)

ImageData


JVO Query Languagecreate view myEROtable asselect s.Bmag,

s.Rmag,t.Hmag,t.Kmag,...,sr.BOX(POINT(s.ra,s.dec),w,h)

as Rimage,tk.BOX(POINT(s.ra,s.dec),w,h)

as Kimage,...

from SUBARU s,2MASS t,...,SUBARU.R sr,2MASS.K tk,...

where XMATCH(s,b,...) < 3 arcsecand(s.Rmag-t.Kmag) > 6 magandBOX(POINT(ra0,dec0), w0, h0)and...

Specify search area with the same syntax as cutout image specification.

Create view with the user specified name in JVO system.

Select attributes from each catalog server. Column names can be expressed in UCD.

Select the catalog server.

Select the image data server.

Cross-match distributed catalogs.

Query condition based on distributed catalog.

Select cutout images from each image data server.Image area can be specified by BOX or CIRCLE operand.

create view myEROtable asselect s.Bmag,

s.Rmag,t.Hmag,t.Kmag,...,sr.BOX(POINT(s.ra,s.dec),w,h)

as Rimage,tk.BOX(POINT(s.ra,s.dec),w,h)

as Kimage,...

from SUBARU s,2MASS t,...,SUBARU.R sr,2MASS.K tk,...

where XMATCH(s,b,...) < 3 arcsecand(s.Rmag-t.Kmag) > 6 magandBOX(POINT(ra0,dec0), w0, h0)and...

Specify search area with the same syntax as cutout image specification.

Create view with the user specified name in JVO system.

Select attributes from each catalog server. Column names can be expressed in UCD.

Select the catalog server.

Select the image data server.

Cross-match distributed catalogs.

Query condition based on distributed catalog.

Select cutout images from each image data server.Image area can be specified by BOX or CIRCLE operand.


SkyQL & ADQL

JVOQL has been adopted as one of the basis for SkyQL (standard query language in IVOA)

XML description of SkyQL = ADQL

Astronomical Data Query Language


Serious Issue

Q) GT2 has too large overhead before starting actual jobs !! They are several seconds.

A) For batch jobs, such as database accesses, the overhead could be negligible. However users can never wait for several second for interactive data analyses.

Request) Overheads must be < 0.1 seconds

excluding the network delays.


Partially solved !!

Introduction of NMI

-- authentication module : binary Analysis of the source codes of GTK2

-- polling interval : 30 sec -> 3 sec Elapsed time was halved ! But …, still

very slow.


Improvement of U.I.

Need to know about SQL to use JVOQL

→

An editor to create JVOQL from GUI

Edittable on screen

→ more flexibility for advanced users


Search for Gravitational Lensescaused (?) by Cosmic Strings

SXDS data observed by Subaru

Query results were obtained less than 5 min., displaying SEDs

It has been proven

that VO can

accelerate

researches.


Cosmic Strings?

Theoretical origin of the elementary particles which existed (?) at t ~ 10-35 s ，T ~ 1015 GeV

Not a POINT, has its SIZE Width < 10-22m, Length ~ size of the

Universe. Mass of 10km string ~ the Earth


Gravitational Lens A phenomenon that the space-time is distorted by

a huge mass object, and multiple images of another object behind the massive object are observed around it.


Development of Prototype2

For preparation of Operational System User registration : single sign-on User area over Firewall : GSI-SFS Adoption of OGSA Implementation of several analysis tools


Linux

クロスマッチ処理

データ検索処理

データ数 count

PostgreSQL

画像管理　　 DB

WSDL

データベースサーバ２

SUBARUSupCam　i-band

Globus Toolkit V3

OGSA-DAI

解析処理

検索依頼

Gb　Ether

Solaris8

Webブラウザ

利用者操作画面

Windows,Solaris8,　Linux

Globus Toolkit V3

作ｽｹｼﾞｭｰﾙ成　Ⅰ

JVO-QL翻訳ｻｰﾊﾞ

ポータルサーバ

Solaris8

クロスマッチ処理

分散情報管理OGSA-DAI

DAISGR

検索結果画面

条件指定画面分散処理制御

・ DAISGR 検索・ GWDSL 解釈・準備、実行、結果確認

Web Server

ｽｹｼﾞｭｰﾙ作成　Ⅱ

データ検索処理Globus Toolkit V3

データ数 count

PostgreSQL


Linux

データ検索処理

データ数 count

ORACLE


ｾｷｭﾘﾃｨ管理

資源管理

Globus Tk

ﾎﾟｰﾀﾙｻｰﾌﾞﾚｯﾄ

GWSDL

WSDL

データベースサーバ N

データベースサーバ１

SUBARUSupCam　z-band

2MASS

統合検索指示

登録情報検索機能

統合検索結果表示機能

検索状況モニター画面

統合検索状況モニター

Registry

利用者管理機能

状況表示

データ保存

Globus Toolkit V3

OGSA-DAI

Globus Toolkit V3

OGSA-DAI

解析処理

検索依頼GT3 利用者データ

結果サーバ

GSI-SFS

Linux

解析処理

解析サーバGb　Ether

解析指示

統

　

合

　

検

　

索

　

実

　

行

　

制

　

御

　

機

　

能

　

GT3 利用者データ

Local　disk

GSI-SFS

FW


Local　disk

GSI-SFS


Local　disk

GSI-SFS


Road Map

2003 Prototype 2

→ development of components for operation

2004 start to develop operational system

late 2004 ?? trial use 　2005-2006 trial use & upgrade

2007 prepare for partial operation of

ALMA


ALMA in Chile (sub-mm wave)

2003/11/27cvo meeting in beijing current status of japanese virtual observatory masatoshi ohishi...

Documents

beijingcvo meeting

beijingvsopcvo meeting

mcvo meeting

beijingsubarucvo meeting

earthcvo meeting

workflow cvo meeting

beijing10 gbpscvo meeting

advanced userscvo meeting