SXDS database and Japanese Virtual Observatory Yuji Shirasaki and JVO collaborationsNational Astronomical Observatory of Japan

What is Virtual Observatory ? Virtual Observatory (VO) is an collection of the astronomical DBs which are accessible with a standard protocol over the internet.

Real ObservatoryVirtual ObservatorySkyCatalog DB, Data ArchiveTelescopeInternetDetectorComputerData ReductionReduced data will be provided or auto reduction service Weather sensitiveany time OK (need network connection)Long observation timeShort observation timeSmall portion of SkyWhole Sky

Why do we need a Virtual Observatory ? Avalanche of the observational data Nobeyama~ 1TB/year Subaru~ 20TB/year ALMA~ 1PB/year XMM, Chandra, Astro-E2, Astro-F, ... The data growth rate (T2 < 1 year) is higher than the improvement of computational power (T2 ~ 18 months) and network bandwidth (T2 ~ 20 months) the newest informational technology, Data Grid and Computational Grid. A multiwavelength study is crucial for understanding the nature of astronomical objects, however... Calibration and analysis methods are highly non-uniform across archives manual data reduction is required. It is a physical and mental barrier to the multiwavelength study .

What is enabled in the VO era ?Easy and seamless access to a large number of databases. You dont need to access to tens or hundreds of web sites.Automated data reduction. You are free from calibration issues. Federation of databases and analysis tools. Efficient data analysis environment.Discovery of rare and exotic objects based on multiwavelength observations of billions of objects instead of hundreds.Discovery of transient phenomena exploring the time-domain.Discovery of ...VORequestresultfederation of data and analysis services by VO standardsDigital Universe is on your desktop !

DBDBDBRegistry: Data & Analysis Service Discovery Service.OpenSkyServer: VO Compliant Data Service.VO Query Language: Multi-Purposes Standard Query Language for VOVOTable: Tabular Data Transfer Format (XML)Key components realizing the VO

Resource Meta Data in RegistryResource metadata describes what data and computational facilities are available where, and once identified, how to use them.

IdentityTitleShortNameIdentifier

CurationPublisherPublisherIDCreaterContributorDateVersionContact

General ContentSubjectDescriptionSourceReferenceURL Type ContentLevelRelationshipRelationshipID

Collection and Service ContentFacilityInstrumentCoverageResolutionUCDFormatRights

InterfaceInterfaceURLBaseURLHTTPResultsMIMEType

CapabilitiesStandardURIStandardURLMaxSearchRadiusMaxReturnRecords

data qualityDataQualityUncertainty

VO Query LanguageUnified Query Language based on JVO Query LanguageVO Query Language defines a syntax for searching astronomical data.

VOTableVOTable is an XML format for exchanging tabular data

International Virtual Observatory Alliance (IVOA) http://www.ivoa.net/14 projects, ~$25 million in R&Dan alliance of existing and future national and international projects to define standards on access to any kind of astronomical resources (database, analysis tools and so on...)

Working Group in the IVOARegistry : standardization of meta data VOTable : define XML format for the exchange of tabular data VO Query Language : standard query language for astronomical data baseData Access Layer : define and formulate VO standards for remote data access.UCD : defining and standardizing meta data (Unified Content Descriptors) Data Modeling : the IVOA data modeling standardization Grid & Web Service : Use of Grid technologies and Web Services in the VO context

IVOA Interest Groups Application: intended to support developers and users of Virtual Observatory applications VO Theory: formed with the goal of ensuring that theoretical data and services are taken into account in the IVOA standards processMailing List discussions:http://www.ivoa.net/forum/

Interoperability Meeting2003-05Cambridge, UK 2003-10Strasbourg, France 2004-05Boston, USA 2004-09Pune, India 2005-05?Japanaimed at making significant progress in generating new standards powering the development of the world wide Virtual Observatory initiatives.

Japanese Virtual ObservatoryTakes the initiative of DB standardization in the Japanese astronomical community.Nobeyama, Subaru and ALMA DBDatabases managed by ISAS and other institutes.Contributes to standardization of VO query language.Provides a VO portal where one can seamlessly access to the VO compliant data and analysis services.

JVO CollaboratorsProject ScientistsNAOJMizumotoOhishiShirasakiTanakaHondaICRCYasudaOchanomizu U.

MasunagaSystem EngineersFujitsu Ltd.MonzenKawaraiIshiharaYanakaYamaguchiIshidaSaitoAbeTsutsumi

SEC Ltd.

MoritaNakamotoKobayashiYoshida

JVO PrototypeExperiment for DB federationFunctionality test of JVO Query LanguageGlobus Toolkits 3Remote processing and file transfer.Web based User InterfaceTomcat and Struts Distributed DatabasesDB1 SXDS Suprime-Cam (Catalog & Image)DB2 SXDS XMM (Catalog & Image)DB3 SDFDB4 SDSS SpectrumDB5 2MASS

Architecture of JVO Proto 2User inputCreate an observation procedureResolve data service location using registryExecute data search and/or data analysisSave results in the user DB CatalogDBGrid ServiceMetaDBGrid ServiceGrid ServiceMetaDBUser DB Data SearchJVOQL AnalysisParametersXPathXMLJVOQL(VOTable)JVOQL(VOTable)VOTable FITSVOTableFITSParameter ListXML etc.CatalogDBFITSFITSResource MetadataJVO ServerServer 1Server 2Server 3JVO Portal Server

Demo 1: Cross match & Image requestcross match resultCross match of the optical and X-ray catalogs of SXDS and image retrievals.Grid ServiceSearch Request(JVOQL)resolve data service URLRegistry Server(1)(3)(4)(7)(6)(5)(2)catalog searchimageimagexmatch requestresult(8)

Sample Queryselect opt.POS_EQ_RA_MAIN as ra, opt.POS_EQ_DEC_MAIN as dec, opt.N18APMAGB as mag_B, opt.N18APMAGR as mag_R, opt.N18APMAGi as mag_i, opt.N18APMAGz as mag_z, x.POS_EQ_RA_MAIN as ra_x, x.POS_EQ_DEC_MAIN as dec_x, x.flux0, x.flux1, x.flux2, x.flux3, x.flux4, img_opt.BOX(POINT(ra, dec), 20 arcsec, 20 arcsec) as image_opt, img_x.BOX(POINT(ra, dec), 20 arcsec, 20 arcsec) as image_x

from naoj.sxds.sxdsR1 opt, naoj.xmm.xmm_epic_sxds x, naoj.sxds.sxds_image img_opt, naoj.xmm.xmm_image img_x

where XMATCH(opt, x) < 5 arcsec and opt.N18APMAGR < 24 and BOX(POINT(34.5, -5.0), 0.1, 0.1)optical catalogX-ray catalogoptical imageX-ray imagePrecision for the cross identification of the optical and X-ray objects.region selection

Demo 2: Cosmic String SearchString searchData request to the SXDS optical catalog, GL candidate selection, String search by pattern recognition.Grid ServiceAnalysis Server Grid ServiceData requestGL candidate selectionString searchresolve data service URL in each requestRegistry Server(1)(4)(6)(2)(5)(3)(7)catalog searchresultGL cand. selectionresult(8)

Cosmic Stringhttp://www.damtp.cam.ac.uk/user/gr/public/cs_evol.html Prediction by Unified theory super heavy cosmic strings with linear mass density of 1022 g/cm in the early universe. The lens effect by a long cosmic string undistorted lensed image co-aligned in a direction of string network distributed in a very large scale. Because of its large scale nature, wide fied deep survey is crucial for its discovery. Data mining from Subaru Suprime-Cam image data

User Authentification

JVO QL EditorCreate SQLSearch

Search Status

Search Result

Cosmic String Search Demo using SXDS Data base Catalog Search GL Candidate SearchSelect pair objects of similar color. Cosmic String SearchPattern recognition

Cosmic String Search Result

Summary2004 Prototype 3development of components for operational systemSubaru Suprime-Cam reduced data DBISAS DARTS ?Test connection to the international VO2005 start to develop operational systemlate 2005 ?? trial use2006-2007 trial use & upgrade2007 prepare for partial operation of ALMARoad map of the JVO project

SubaruNobeyamaALMA

Astro-FAstro-EYOHKOHALCA

JVO Query Languageselect optCat.ra, optCat.dec, xCat.ra, xCat.dec, optImage.Image, xImage.Imagefrom optCat, xCat, optImage, xImagewhere ((ICRS, 270.0 deg, -1.5 deg), 0.2 deg) ~ (optCat.ra, optCat.dec)and Distance((optCat.ra, optCat.dec), (xCat.ra, xCat.dec)) < 5 arcsecand optImage.regionSky = ((optCat.ra, optCat.dec), 10 arcsec)and xImage.regionSky = ((optCat.ra, optCat.dec), 10 arcsec)and optImage.FORMAT = FITSand xImage.FORMAT = FITSSelect objects located in a circle region centered at ra=270 and dec=-1.5 with 0.2 deg radius from optical catalog. Identify X-ray counter part for each selected object with 5 arcsec precision. Get FITS images of 10 radius size from optical and X-ray image data service.

DB Search

I would like to talk on the Virtual Observatory which is now a world wide project for making standard of astronomical database system.and I also talk on the current status of Japanese Virtual Observatory and present a demos which are using the SXDS database.Why do we need a Virtual Observatory ? One of the reason is that due to a technological innovation, astronomer is suffered by an avalanche of the high quality observational data. For example, Subaru is generaing 20 TB of data in each year and ALMA will generate 1 PB in a year. and other spacemission like Astro-E2 and Astro-F will start soon. Astronomical data is now doubled in 1 year, To overcome this situation new The what is enabled by the VO ? First of all, easy and seamless access to a large number of databases becomes possible. You now dont need to access to tens or ...Second, automated data reduction is crucial for realizing the virtual observatory system. This is one of the issues to be accomplishedespecially in the large telescope project like ALMA.Federation of databases and analysis tools become possible by the VO standard interface, then data is transferred just among the databaseand analysis server.In this way, astronomical study become efficient and a lot of discoveries are expected to be made.Here, I show standard interfaces needed to realize the VO.The most important component is a Registry. Registry is a metadata database for finding data & analysis service in the VO.VO Query Language is a multi-purposes standard query language for VO.VOTable is s VO Standard of tabular data transfer format.The resource metadata stored in the registry are listed here. These describe what data and computational facilities are availablewhere, and once identified, how to use them. These metadata should be provided by each data provider.

It defines a syntax for searching the astronomical data.Currently there are three kinds of query languages are defined.One is a SQL based Astronomical Data Query Language, this is used for query to the catalog database.The other is URL based query, they are called Simple Image Access Protocol and Simple Spectrum Access Protocol.There are used for search observation data.We are now discussing to unify those query language to a one unified query languge based on the JVO Query Languagewhich is designed among our JVO group.It is expressed in XML format and it contains table metadata which describes the content of each column.Tabular data itself is described here in this way.

International Virtual Observatory Alliance (IVOA) is formed in last year, it is an alliance of existing and future national and international projects to define standrad on access to astrnomical database.Currently 14 projects are joined to this alliance.Seven working groups are formed in the IVOA. Each group are discussion standardization of each category.Recently new Interest Groups are formed in the IVOA. Application group is formed to support developers and users of virtual observatory applications.VO Theory group is formed with the goal of ensuring that theoretical data and services are taken into account in the IVOA standards process.Among those groups mailing list discussions are main part of the activity.The archive is stored at this URL you can also subscribe to the mailing list at this URL.

In addition to the mailing list discussions, we have interoperability meeting twice in a year.This is aimed at making a significant progress by a face-to-face discussions.Next meeting will be held at Pune, India on September and In the next year it will be held at Japan,so we welcome for any interested people of Japanese astronomical community to attend this meeting.

Japanese Virtual Observatory is a project that is aimed to takes the initiative ...We have constructed a JVO Prototype system for en experiment of the distributed database federation and forfunctionality test of JVO Query Language.Globus Toolkit 3 is used for build a framework of remote processing and data transfer system.Web based user interface is developed using tomcat and struts which is a free software for making a web application.Databases are constructed on the independent servers, each of which has data of SXDS Here is an architecture of JVO proto 2. First a user sends a data request message written in the JVO Query Languge. JVO portal server parse the query language and make an observation procedure. In that process data service location is resolved using the registry.Data request messages are sent to each data server. Data server return a search result in a VOTable format.Search results are saved in the user DB area, user can access to search result from there..

Here is a sample query to the SXDS database.This query is requesting the data of objects which are identified in both of the optical and X-ray band.The request data is described in the select clause, this part describe the data of optical catalog, and this part describes the data of X-ray catalog. These two lines are request of image around the object in optical and X-ray.Precision for the cross identification is specified here and search region is specified by a BOX predicate. I will show you several snapshots of JVO prototype user interface. This is a user login page, you need to enter username, password and observation name.

This is a JVO Query Language Editor. Query condition can be specified on this web form.

You can find database in the JVO