1Radio Astronomy in the LSST Era – NRAO, Charlottesville, VA – May 6-8th 2013.

LSST Survey Data Products

Mario Juric LSST Data Management Project Scientist

Radio Astronomy in the LSST Era, NRAOMay 6-8th, 2013

2Radio Astronomy in the LSST Era – NRAO, Charlottesville, VA – May 6-8th 2013.

LSST Survey Data Products

r<24.5 (<27.5@10yr) 18000+ deg2 10mas astrom.

Imaging the visible sky, once every 3 days, for 10 years (825 revisits)

Telescope Images Catalogs

3Radio Astronomy in the LSST Era – NRAO, Charlottesville, VA – May 6-8th 2013.

LSST: Data Volume

− One 6.4-gigabyte image every ~17 seconds− ~1000 visits (two back-to-back images), per night− 15 terabytes of raw scientific image data / night

− ~2 million real time events per night, for 10 years− Alerts transmitted within 60 seconds of end of observation

− A catalog of ~38 billion observed objects (24 B. galaxies, 14 B. stars)

− A catalog of ~5 trillion detections (“sources”) and ~32 trillion measurements (“forced sources”)

4Radio Astronomy in the LSST Era – NRAO, Charlottesville, VA – May 6-8th 2013.

LSST: Open Data, Open Source

− LSST data, including images and catalogs, will be available with no proprietary period to the astronomical community of the United States, Chile, and International Partners

− Transient alerts will be made available for world-wide distribution within 60 seconds, using standard VO protocols

− LSST data processing stack will be free software (licensed under the GPL, v3-or-later)

− All science will be done by the community (not the Project!), using LSST’s data products

5Radio Astronomy in the LSST Era – NRAO, Charlottesville, VA – May 6-8th 2013.

Levels of Data Products

Nightly

Annual

User-created

http://ls.st/srd

6Radio Astronomy in the LSST Era – NRAO, Charlottesville, VA – May 6-8th 2013.

Data Rights

Transients: EveryoneSolar System: US/Chile/Int’l Partners

US/Chile/Int’l Partners

Creation: US/Chile/International Partners, as granted by a TAC

Use: As granted by the owner

7Radio Astronomy in the LSST Era – NRAO, Charlottesville, VA – May 6-8th 2013.

Level 1:

TransientsMoving Objects

8Radio Astronomy in the LSST Era – NRAO, Charlottesville, VA – May 6-8th 2013.

Level 1: Transients

− Sized for: 2M/night (average), 2k/visit (average), 10k/visit (peak)− Issued in 60 seconds (broadcast world-wide as VOEvents)− All measurements performed on image differenced against a deep template

− Plan to include:– position– flux, size, and shape– light curves in all bands (up to a ~year; stretch: all)– variability characterization (eg., low-order light-curve moments,

probability the object is variable)– cut-outs centered on the object (template, difference image)

− Do not plan to include:– variability classification (eg., “this object is an RR Lyrae”)

9Radio Astronomy in the LSST Era – NRAO, Charlottesville, VA – May 6-8th 2013.

Level 1: Moving Objects

− Solar system objects are detected in difference images as sources with positive flux. These sources are then processed by the Moving Object Pipeline (MOPS), which links sources together into tracks of individual objects and determines orbits for them.

− Availability: within 24hrs of orbit determination

− Planning to include:– Object identification– Orbital elements– Photometric properties in each band– Data quality flags– No shape reconstruction– No taxonomical classification

10Radio Astronomy in the LSST Era – NRAO, Charlottesville, VA – May 6-8th 2013.

Level 2:

Annual Data Processing

11Radio Astronomy in the LSST Era – NRAO, Charlottesville, VA – May 6-8th 2013.

Level 2: Data Releases

− Data releases– Performed annually, except in Year 1

• Two DRs in the first year (6-mo)– No proprietary period (US/Chile/Partners)

− Complete reprocessing– Every DR will reprocess all data taken up to the beginning of that DR

− Major products– Images– Catalogs

− Accessing the catalogs– Database and SUI– Remote access APIs, VO protocols (e.g., Table Access Protocol)

12Radio Astronomy in the LSST Era – NRAO, Charlottesville, VA – May 6-8th 2013.

Level 2 Processing

LSST processing will begin with production of a series of coadds, where detection of objects will occur.

Detections on co-adds will be deblended and associated to form a master object list.

The objects will be simultaneously characterized in all observed epochs (MultiFit).

Time variability will be characterized by independent measurement of sources in individual epochs.

* * * draft * * *

13Radio Astronomy in the LSST Era – NRAO, Charlottesville, VA – May 6-8th 2013.

Level 2 Catalog Guiding Principles

− Defining the contents of Level 2 catalogs:

1. “Maximize science enabled by the catalogs”• The large majority of LSST science cases should be enabled by just

the catalog.• Reprocessing images is costly and difficult for the end-user.

2. “Provide simple but useful, commonly used, external or derived, quantities”• Example: E(B-V) values for each object.• Example: Photo-z using well known, published, algorithms.

14Radio Astronomy in the LSST Era – NRAO, Charlottesville, VA – May 6-8th 2013.

Example: Object Table

− Object characterization (models):– Moving Point Source model (11 parameters)– Double Sérsic model (bulge-disk decomposition; 18 parameters)

• Maximum likelihood peak• Samples of the posterior

− Object characterization (model-free):– Centroid: (α, δ), per band– Adaptive moments and ellipticity measures (per band)– Petrosian and Kron fluxes and radii (per band)

− Colors:– Seeing-independent measure of object color

− Variability statistics:– Period, low-order light-curve moments

− Metadata: Deblend status, flags, etc…

15Radio Astronomy in the LSST Era – NRAO, Charlottesville, VA – May 6-8th 2013.

LSST Data Products Definition Document

Coming soon (August 2013)

LSST Final Design Data Products Definition document

An overview of LSST data products LSST and associated software plans.

16Radio Astronomy in the LSST Era – NRAO, Charlottesville, VA – May 6-8th 2013.

Level 3:

User-created Data Products

17Radio Astronomy in the LSST Era – NRAO, Charlottesville, VA – May 6-8th 2013.

Level 3

− “Level 3” products will be created by the community using LSST’s hardware and software.

– For processing that would greatly benefit from being co-located with the LSST data• Likely weighted towards science cases involving the images• Envisioning a TAC

– For alternative catalogs (e.g., alternative photo-Z calculation)• Like “mydb” in SDSS

– Can migrate to Level 2, if found to be universally useful• LSST will provide support for this migration (e.g., help with the

necessary software engineering)

18Radio Astronomy in the LSST Era – NRAO, Charlottesville, VA – May 6-8th 2013.

Level 3

− Examples:– Searching for extremely faint Kuiper-Belt Objects using “shift-and-

stack”– Characterization of diffuse structures (eg., ISM)– Extremely crowded field photometry (eg., globular clusters)

− LSST has a mandate to provide 10% of its total storage and computing resources for Level 3– Example: DR2 Level 3 allocation for all users (~in year 2022) is:

• 20 TFLOPS compute• 300TB database (includes the “mydb” space), 1PB disk space

− LSST archive will be located in the National Petascale Computing Facility at NCSA. Significant additional supercomputing is expected to be available at the same site (e.g., NPCF currently hosts the Blue Waters supercomputer).

Algorithms: Co-adds w. background matchingExample: Background-matched co-add of SDSS Stripe 82 in the vicinity of M2,

Generated wit LSST pipeline prototypes.

20Radio Astronomy in the LSST Era – NRAO, Charlottesville, VA – May 6-8th 2013.

Following LSST Development

− General LSST news: @LSST− LSST Data Management news: @mjuric

− Public LSST website: http://lsst.org

− Software development: http://dev.lsstcorp.org

Thank you for your attention!

Questions?