L. NExScI Operational Readiness

Rafael Millan-GabetProject Scientist, NExScI

&Mihseh Kong, NExScI Software Engineer

David Imel, IPAC & NExScI ManagerChas Beichman, NExScI Director

NExScI Compliance

• This presentation discusses NexScI readiness to assume responsibility for the archive

• Relationship to ORR Success Criteria:Satisfies requirement for NExScI to perform processing and archiving of LBTI-produced data

• Concerns: None• Liens: None

L-2

NExScI Responsibilities (1/3) (PLRA 2014 Section 3.1.3)

NExScI shall be responsible for1. Developing common formats for data products and coordinating with UA to

archive data products:a. Interface with UA to receive NASA LBTI datab. Archive the NASA LBTI data at NExScIc. Provide interface for science community access to the archived datad. Monitor data product quality assessments conducted by UA as a service to observers

L-3

All part of the archive service: Developed FITS file formats and keywords for LBTI nulling data Automatic data transfers and ingestion Automatic implementation of periods of exclusive data use and public release Web interface to the archive as well as scripted data access HTML summary tables and browse plots

NExScI Responsibilities (2/3) (PLRA 2014 Section 3.1.3)

NExScI shall be responsible for2. Selecting HOSTS Science Team members:

a. Prepare call for proposalsb. Conduct peer reviewc. Select proposalsd. Manage science deliverables

L-4

Call for proposals issued in Feb 2012 and selections announced in June 2012 Followed standard peer-review and selection procedures LBTI Project PI (UA) and NASA Project Scientist at NExScI co-lead the HOSTS Science Team Defined a set of 6 deliverables for the Science Team Main ST activities to date:

o Monthly telecons + focused telecons (e.g., on the analysis pipeline)o Participation in observing runso Developed target list (Weinberger et al. ApJ, 2015)o Developed exozodi model (Kennedy et al. ApJ, 2015)o Developed quantitative mission success criteria (Mennesson)

Deliverable Due Date CommentPreliminary target list 9/2014 Published paper Weinberger et al., ApJ, 2015Final target list 3/2015 Tied to start of operationsReport on data calibration methods 5/2016 After 1 year of experience with the dataReport on interpretation and implications for exo-Earth imaging

9/20159/20169/2017

Incremental releases

Final report and plans for publications 3/2018

NExScI Responsibilities (3/3) (PLRA 2014 Section 3.1.3)

NExScI shall be responsible for3. Providing oversight and guidance to the ExEPO to protect the LBTI’s science

objectives and implement NASA HQ policies

L-5

NExScI shall be responsible for4. Providing management oversight of LBTI during the Operations Phase:

a. Generate monthly report of science and operations statusb. Provide point of contact to UA and ExEPO for resolution of issues and concerns

The Project Scientist at NExScI provides these functions for ExEP and HQ

NExScI is prepared to assume management oversight functions Plan is to transition at the end of the Science Validation Phase

Science Data Management (PLRA 2014 Section 3.4)

Addresses archival obligations and data rights for• Commissioning data• HOSTS• HOSTS/LEECH• Non-NASA users

L-6

Periods of exclusive data use are implemented in the archive, data goes public when they expire Note: HOSTS/LEECH periods of exclusive data use same as for HOSTS

Consistent with these obligations, the current version of the archive supports LBTI nulling data Small additional archive development needed to support LEECH data planned for FY16

o If LMIRCam data are needed to mitigate PWV, this task may be de-scoped, or replaced with modifications needed to support the new ancillary data for nulling, if needed

Data Rights Implementation

L-7

Project ID associated with each observation at the observatory via the Observe Web tool

Project ID keyword in header of each Level-0 image; then propagated to L1/L2 files by the pipeline

Data Requirements(PLRA 2014 Section 4.2)

Addresses data archiving and public distribution obligations described before, plus types of data and timeliness of transfers and archive ingestions:

All scientific data generated by the NASA HOSTS survey shall be provided from the LBTI observatory to NExScI for archiving and distribution. An instrument data management plan shall be developed by NExScI and UA to address the total activity associated with the flow of NASA science data, from planning through acquisition, processing, data product generation, and validation, to archiving and preservation. NExScI shall develop common formats and coordinate archiving of data with UA. NExScI shall provide an interface to UA to receive NASA LBTI data. NExScI shall provide a link to the archived data for distribution to the various observers for calibration and analysis.

Three types of data products shall be archived; the delivery timescales in each case are appropriate to the needs of the HOSTS survey:

1. Level 0: Merged raw images; delivered from the observatory to NExScI within 72 hrs of observation

2. Level 1: Internally calibrated science data, delivered from the observatory to NExScI within 20 work days of observation. Internally calibrated data consists of: cleaned Level 0 images (dark subtracted, bad pixels removed, flat fielded, background subtracted) and files containing derived quantities (null and stellar fluxes, associated errors, diagnostic quantities)

3. Level 2: Externally calibrated science data, delivered from the observatory to NExScI within 40 work days of observation. Externally calibrated data files contain fully calibrated nulls and associated errors (effects of instrument’s transfer function removed), and result from processing multiple Level 1 files obtained on the target star and its calibrator stars.

L-8

Data Management Plan: delivered and in signature cycle (NExScI, UA, JPL, HQ) The data pipeline (UA) automatically takes Level 0 data and outputs the Level-1 and -2

products as described above Detailed data formats documented in ICD and documentation for archive users Demonstrated automatic data transfers and archive ingestion for the three levels of data, and

under the required times, on several occasions during the commissioning phase

LBTI NExScI Archive Demo (1/4)

L-9

• Password-protected login for PIs• Data search by criteria, or from

lists of dates or objects• Search L0, L1, or L2 data

LBTI NExScI Archive Demo (2/4)Level-0 Images

L-10

• Dynamic HTML tables• Data download options

LBTI NExScI Archive Demo (3/4)L1 Nightly Summary, OB Summary, and Processed Image Cubes

L-11

LBTI NExScI Archive Demo (4/4)Level-1 and -2 Browse Plots

L-12

Holdings• The archive currently contains all commissioning and science verification

data taken in nulling mode• All data (science and engineering) are available to instrument and Science

Team members• Some of the science verification data taken during commissioning

(12-month period of exclusive data use) is of good quality and can be made public now

L-13

Plan

L-14

ORR

Apr 28 May 28

Archivepublic

FY 15 FY 16

Oct

• Archive ops(data ingestions)

• Development frozen

• Archive ops and maintenance (data ingestions)• Resume development:

– Bug fixes and user feedback– Expected evolution of data files during SVP– LEECH data

FY 17

Oct

• Archive ops & maint(data ingestions)

FY 18

Oct

•Ex

tens

ions

to o

ther

LB

TI m

odes

or L

BT

inst

rum

ents

pen

ding

G

O d

ecis

ion

Summary

L-15

PLRA ID Description Status Comment3.1.3.1 Data formats, interface to UA, NExScI

archive✔ • Archive will be made public following ORR

board report• Archive dev frozen after ORR until FY16 due

to funding invested instead in additional resources towards instrument commissioning success

• Data still expected to evolve during SVP• Level of funding supports only minor archive

modifications post public release3.1.3.2 HOSTS Science Team selection and

activities✔

3.1.3.3 Project Scientist functions ✔

3.1.3.4 Management oversight at NExScI ✔ • Deferred to end of SVP3.4 Data rights and data to be archived ✔ • LEECH work deferred to FY16

• Other modes pending GO program decision4.2 Types of data products and timeliness

of transfers. Data Management Plan✔