Transiting Exoplanet Survey Satellite

Daryl Swade

Archive Peer Review

June 18, 2014

2

Agenda

TESS Ground System Architecture MAST Operations Overview Interfaces MAST design

Archive System Archive User Interface

Schedule Testing plan Deliverables

IT Security Document Status Forward Work

3

TESS Ground Segment Architecture

4

MAST Overview

300 TB – Current size archive holdings. 3 TB – Average Ingest Rate per month. 19 TB – average data distribution rate per month. 1,5000,000 – Average number of searches per month.

5

TESS Raw Data

Data types Kepler:

• long cadence target – 150,000 targets, 32 pixels/target, 30 minutes cadence

• short cadence target – 300 targets, 85 pixels/target, 1 minute cadence• full frame image – one per month

TESS: • Target – 25000 targets/month, 100 pixels/target, 1 minute cadence• Full Frame Image – 30 minute cadence

TESS raw data volume: 6600 GB/ year (uncompressed) Target data: 3500 GB/year Full Frame Image data: 3100 GB/year

Kepler raw data volume: 380 GB/year (uncompressed) Long cadence: 293 GB/year Short cadence: 84 GB/year FFI: 2.3 GB/year

6

Archive Operations

TESS monthly processing (Kepler quarterly) Transferred from SPOC via hard drive

TESS archive data volume – 50 TB/year (to be refined) Total Kepler archive data volume – 19 TB (Q0-Q17)

• Does not consider reprocessed data

No proprietary period on data Availability depends primarily on SPOC processing

time (<1 month) Primary TESS web site hosted by MIT

Links to MAST web site

7

MAST Interfaces

SPOC

TSO

MAST

Target Pixel Data

Full Frame Images

Collateral Data

Light Curves

Centroid Data

Cotrending Basis Vectors

Threshold Crossing Events

Data Validation Reports

TESS Objects of Interest Catalog

S/c Data (from POC)

Threshold Crossing Events

Data Validation Reports

8

TESS Archive Data Products from SPOC

Full Frame Image Target Pixel Files

All pixels for a single target Raw and calibrated values

Light Curves Output of photometric analysis and cotrending

Collateral Pixel Files Calibration data collected on-board

Ground calibration data Cotrending Basis Vectors

• Systematic trends present in the ensemble of flux data for each CCD

Pixel Response Function• Point spread function and pointing and electronic systematics

Flat Field• Pixel response non-uniformity map

9

TESS Data Products from TSO

TESS Input Catalog (TIC) Threshold Crossing Events (TCE) summary report

Identified from light curves Data Validation (DV) results

Characterize and validate TCEs TESS Objects of Interest (TOI) Catalog Follow-up observations

10

Archive System Overview

query results

queries

Ingest

Distribution

Archive User Interface

data files

data files

catalog updates

queriesquery results

data request

quer

ies

quer

y re

sults

Archive user

catalog updates

data

base

repl

icati

on

data files

Storage Broker

Data Receiptmanifest

Primary data store

Safe store

On-line cache

data files

data files

data request

science metadata

file informationdata files

Archive Catalog

Archive Catalog

11

Archive Functional Overview (1/3)

Data Receipt Receives data from the SPOC Receives data from the TESS Science Office Accepts data and generate an archive manifest

Ingest Stores TESS data in the Storage Broker Catalogs TESS metadata the Archive Database

Storage Broker Provides a storage service manager for the file I/O on the primary, secondary safestore, and public data stores

12

Archive Functional Overview (2/3)

Archive Catalog Provides an on-line science catalog of TESS

metadata for searches, data mining, and data retrieval

• Implemented on relational DBMS

• Contains metadata extracted from the science data keyword values

Stages TESS catalogs for archival use Data Distribution

Retrieves requested data from storage broker Distributes data to archive users via electronic

mechanisms or removable media Archive Operator Interface

Maintains an operator interface for monitoring and control of the archive system

13

Archive Functional Overview (3/3)

Archive User Interface Makes TESS data available to the astronomical community

• Web-based query/preview/retrieval interface

Hosts software tools required to search the TESS archive catalog

• This includes user access to target lists.

Hosts software tools required to access data

Provides data mining, visualization, and correlation with other major mission datasets

• International Virtual Observatory Alliance (IVOA) standards compliant

14

Archive Catalog

Three main types of database tables File tracking tables – What datasets and files

exist, the file/dataset status, and the relationship between them.

Instrument and data specific tables Common Archive Observation Model (CAOM)

Database

15

Common Archive Observation Model (CAOM)

Unified model for metadata about observation (position, time, wavelength) implemented in a database enabling easy uniform cross-mission searches

Extensible for mission-specific data May require additional static data not in FITS file

Enables easy creation of Virtual Observatory (VO) services VO allows s/w access to data holdings

Current version enabled as part of the MAST Data Discovery or Portal Interface

16

Data Distribution Options

Electronic mechanisms Streamed through browser Data delivered to MAST staging area for user down load via browser or FTP

Data delivered directly to user disk via SFTP Wget and cURL scripts created and delivered to user

Removable media: Hard drive, DVD Archive Operator has the ability to override specified

user preference.

17

Archive Distribution Performance

MAST has a requirement to be capable of distributing 2.5 TBytes of archived science data products externally and 9.4 TBytes internally per month.

MAST is implementing scalable hardware and networking technologies, with high-bandwidth ISP to the internet to handle increased loads.

Historic data distribution metric includes internal transfers and uncompressed volumes. Internal data transfers do not use ISP bandwidth. JWST and TESS estimates include internal transfers.

18

Archive User Interface: Overview

Archive User Interface (AUI) provides ability to: Search for data products by:

• Data file name• Target name, position or list of positions• Other observation parameters (instrument, waveband, program ID,

etc.)

Explore and discover data through the Data Portal Request retrieval of those products

19

Kepler Data Search and Retrieval Form

20

Results of Kepler Data Search

21

Data Retrieval Request

22

Kepler Target Search Form

23

Results of Target Search

24

MAST Portal: Internal Observations

25

MAST Portal: VO

26

Archive Development Schedule

Phase B (April 1, 2014 – October 1, 2014 ) Continue developing TESS Archive Operations Concept Continue developing TESS-MAST ICD

• Work with SPOC to develop SPOC-MAST ICD section• Work with TSO to develop TSO-MAST ICD section

Continue to participate in Ground Segment Monthly Technical Reviews

Participate in TESS Mission PDR (September 2014) Phase C (October 2, 2014 – November 29, 2016)

Develop TESS specific archive components based on Archive Operations Concept and TESS-MAST ICD

Participate in TESS Mission CDR Participate in Ground Segment testing

Phase D (November 30, 2016 – July 17, 2017) Develop TESS Archive User’s Guide

27

query results

queries

Ingest

Distribution

Archive User Interface

data files

data files

catalog updates

queriesquery results

data request

quer

ies

quer

y re

sults

Archive user

catalog updates

data

base

repl

icati

on

data files

Storage Broker

Data Receiptmanifest

Primary data store

Safe store

On-line cache

data files

data files

data request

science metadata

file information

data files

Archive Catalog

Archive Catalog

Archive System Enhancements for TESS

TESS mission specific s/w

28

Archive Development Effort

Staff Phase(s) Total FTE Avg. FTEProgram Manager B-D 2.0 0.050Project Manager B-D 1.4 0.035Systems Engineer B-D 5.4 0.14Archive Database Engineer C-D 3.6 0.12Archive Software Engineer C-D 3.6 0.12Archive Scientist D 1.2 0.11Archive System Admin. C 0.01 0.01

Phase B: April 1, 2014 – October 1, 2014 Phase C: October 2, 2014 – November 29, 2016Phase D: November 30, 2016 – July 17, 2017

29

Testing Plan

MAST will work with the SPOC and TSO to develop test data.

MAST will participate in Ground Segment end-to-end tests.

30

MAST Deliverables

Deliverable Responsibility Phase Date of DeliveryTESS Archive Operations Concept Systems Engineer B PDRTESS-Archive ICD

SPOC-ArchiveTSO-Archive

Systems Engineer B/C PDR/CDR

Archive Catalog Design Specification Systems Engineer B/C PDR/CDRArchive Architecture Systems Engineer B/C PDR/CDRArchive Software Design Systems Engineer C CDRTESS Archive User Interface Software Engineers C/D GS TestArchive Test Procedures Systems Engineer D GS TestArchive Operations Procedures Archive Operator D LaunchTESS Archive User’s Guide Archive Scientist E Launch

31

IT Security

MAST will protect all TESS data against unauthorized access per requirements established by NPR 2810.1, Security of Information Technology.

STScI is moving towards becoming a NASA external system STScI will have a single Moderate system Security

Plan Compliance with FISMA and NPR 2810.1A still

required

32

Document Status

TESS Archive Operations Concept Initial draft available dated June 12, 2014

TESS-MAST ICD SPOC-MAST ICD

• Initial draft available

TSO-MAST ICD• Initial draft available by PDR

33

Forward Work

Systems Engineering Complete TESS Archive Operations Concept Work with SPOC to complete SPOC-MAST section

of TESS-MAST ICD Work with TSO to complete TSO-MAST section of

TESS-MAST ICD Prepare for PDR

Archive Software Development Data Receipt and Ingest Archive Catalog Archive User Interface