1

NICERDataIntroduc0on

2017-08-23CraigMarkwardt

GSFC

2

•  NICERprojectdeliveringfirst"unvalidated"datasetstoteamforevalua=onandvalida=on

•  Morethan1600datasetsinthearchivedelivered

•  Contentsofthefilearedescribed•  Caveats

3

•  HEASoGversion6.22hasbeenreleased.NICERsoGwareisincluded•  NICERcalibra=ondataisnowpartofHEASARCCALDB•  Changesto“Ini=alReleaseData”

–  Namingofdatasetsnolonghastargetnameindirectoryname–  The“UFA”eventfile,nominallyunfiltered,nowhasbeenlightly

filteredfordiskspaceandperformancereasons.(Undershootsandovershootsremoved)

–  Filterfile(auxil/ni*.mkf)hasmanynewcolumns.See‘[elpniprefilter’fordescrip=ons.Newtask‘nimake=me’canbeusedforstandard=mescreening.

–  A]tudeprocessingisnowimproved.Bothstartracker(ST)andinstrument(INST)a]tudesarepreserved,butSTa]tudeisnowusedforprocessing.

–  OrbitfilenowincludesSinglePointSolu=onGPSsolu=onforrarecaseswhenfilteredGEONSsolu=onisnotavailable.ThisextensionisORB

–  Metadatakeywordsarepopulated–  Processinglogfileiscomplete

4

•  ChangestoNICEReventdata– BothPIandPI_FASTpulseheightsarecalibrated– NeweventfilecolumnMPU_UNDER_COUNTusedforcalibra=onpurposes

– Merged-MPUeventfileGTIiscombinedwith“AND”logic(intersec=onofindividualMPUGTIs)

–  IndividualMPUGTIsarepreservedinthemergedfileasGTI_MPUnextensions

–  Incleanedeventfile,thereisaGTI_SELextensionwhichistheinput=meselec=onGTIusedforfinalscreening,forreproducibilityreasons

5

•  Each“observa=onsegment”willbeinaseparatedirectoryiden=fiedbyits10-digitobserva=onID–  Observa=onIDisuniquetoasegment–  Scienceopsmaysplitsegmentsat~dailyboundaries–  Anobserva=onsegmentmaycontainmorethanoneNICERorbit

•  Capturedineventfilegood=meintervals(GTIs)•  Directorylayout

–  NNNNNNNNNN/(observa=ondirectory)•  x=/ -X-rayTimingInstrumentdata

–  event_uf/ -unfilteredX-rayevents–  event_cl/ -cleaned&calibratedX-rayevents–  hk/ -XTIhousekeepingfiles

•  auxil/ “Mission-level”orbit,a]tude&HK•  log/ Processinglogfiles

6

•  FundamentalscienceproductisX-rayeventdata–  Level1uncalibratedunfilteredeventdatastoredin

x=/event_uf–  Level2calibratedandcalibrated+screeneddataarestoredin

x=/event_cl–  Level1&higherdataisdeliveredwitheveryobserva=on

•  Theeventfilenamingschemeis,–  niNNNNNNNNNN_0mpuN_UU.evt

•  NisMPUnumber0-6individually•  N=7indicatesallMPUscombined•  UUindicateslevel

–  uf=unfiltered–  ufa=calibratedunfiltered(“lightlyfiltered”toremoveoverandundershoots)–  cl=calibratedandfiltered(cleaned)

–  Example:ni1707030136_0mpu7_cl.evt•  Obsid1707030136,cleaneddata,allMPUscombined

7

•  Therearetwoextensionsofinterest– Extension1:EVENTS–X-rayeventlist– Extension2:GTI–GoodTimeIntervaltable

•  NICERfollowsstandardizedformatsforeventlistsandGTIs,soX-rayastronomersshouldbefamiliarwiththem.(Seenextpage)

•  Level1“uf”filesareoriginaldatastreamfromeachMPU– Therearesevenfiles,oneforeachMPU

•  Level2“ufa”and“cl”fileshaveall7MPUsmergedintoonefile,calibra=onsapplied

8

HDU 2 EVENTS BinTable 9 cols x 16744802 rows

Col Name Format[Units](Range) Comment 1 TIME 1D [s] Time of events 2 RAWX 1B [pixel] (0:7) Event X position RAW coordinates 3 RAWY 1B [pixel] (0:6) Event Y position RAW coordinates 4 PHA 1I [chan] (0:4095) Slow Pulse Height Analyzer 5 PHA_FAST 1I [chan] (0:4095) Fast Pulse Height Analyzer 6 DET_ID 1B Detector ID number - 10*MPU+FPM 7 DEADTIME 1B [s] Event dead time 8 EVENT_FLAGS 8X MPU Event Flags 9 TICK 1K MPU tick count of event 10 PI 1I Slow Pulse Invariant 11 PI_FAST 1I Fast Pulse Invariant 12 MPU_A_TEMP 1E MPU Analog Temperature 13 MPU_UNDER_COUNT 1J MPU undershoot rate 14 PI_RATIO 1E Ratio PI/PI_FAST

HDU 3 GTI BinTable 2 cols x 13 rows

Col Name Format[Units](Range) Comment 1 START 1D [s] GTI start time 2 STOP 1D [s] GTI stop time

HDU 4 PPS_TREND BinTable 3 cols x 11500 rows

Col Name Format[Units](Range) Comment 1 TIME 1D [s] GPS reported time at 1-PPS 2 TICK 1K MPU tick count at 1-PPS 3 PKT_TICK 1K MPU tick count at time of X-ray packet accumula

GREEN=Level1→2Addi=ons

9

•  Thereare56modulesarrangedasfollows(seefollowingpage):– sevenMPUslabeled0-6

– eightFPMsperMPU,labeled0-7•  ThemodulesarelabeledwithasingleintegerDET_IDintheeventlist:

– DET_ID=10xMPU+1xFPM– Example:27meansMPU2,FPM7

10

1-100

1-201

1-302

1-403

1-504

1-706

1-605

2-110

2-211

2-312

2-413

2-514

1-807

2-615

3-120

3-221

3-322

3-423

3-524

2-716

3-625

4-130

4-231

4-332

4-433

4-534

2-817

3-726

5-140

5-241

5-3 42

5-443

5-544

3-827

4-635

6-150

6-251

6-352

6-453

5-645

4-837

4-736

7-160

7-362

7-564

7-766

6-554

5-847

5-746

7-261

7-463

7-665

7-867

6-655

6-857

6-756

+Y

+X

1-7

06

MechanicalDrawingPosi=on

ScienceDetectorID

MPU(0-6)

FPM(0-7)

11

•  TimeisreportedintheTIMEcolumn•  TIMEiselapsedTTsecondssincetheepoch

2014-01-01T00:00:00UTC•  TSTARTandTSTOPreportthestartandstopofgood=me•  EXPOSUREistotallive=me•  ConversionofNICER=mestampstoabsolute=meinMJD

canbedonewiththefollowing:– MJD(TT)=MJDREFI+MJDREFF+(TIMEZERO+TIME)/86400

– MJD(UTC)=MJD(TT)+LEAP_INIT•  BUT:SeetheCaveatspagesforimportantlimita0ons!

CurrentlyLEAP_INITisnotproperlyfilledsoyouwillneedtouseLEAP_INIT=2manually.

12

•  UncalibratedLevel1incolumnsPHAandPHA_FAST–  EachMPUandFPMhasdifferentPHAscale!–  Thus,useofuncalibratedPHAisoflimitedusebyscien=sts

•  CalibratedLevel2pulseheightinPIandPI_FAST(“ufa”and“cl”files)–  Everymoduleisplacedonsameenergyscale!

•  PI=(ENERGY/10eV)–  Example:PI=110correspondsto1.10keV

•  NICERcalibrated“ufa”andcleaned“cl”fileshavethepulseheightcalibra=onappliedforbothPIandPI_FAST

•  NOTE:theNICERresponsebelow~250eVcanbecomplicated;thestandardscreeningremoveseventsbelow200eV.

13

•  TheNICERteamhasnowclearlyiden=fiedthedifferencebetweenPIandPI_FASTasadiscriminatorofbackgroundevents– nonX-raysthatinteractinouterregionsofthedetectorscreatedifferentchargecloudsthanconcentratedX-rays,whichresultindifferentpulseheightsinslowandfastchains

•  AcolumnnamedPI_RATIOisdefined PI_RATIO=PI/PI_FAST

•  TheNICERteamwillprovideimprovedguidancehowtousethiscolumnforbackgroundfiltering

14

•  Gaincalibra=onofbothchainsallowstocalculatePI_RATIO=PI/PI_FAST,whichisanindicatorofbackgroundevents

•  Blackcutabove,includedinnicermergeclean,willexcludebackgroundevents

PI_R

ATIO

PI=Energy/10eV

Background

RealX-rayEvents

15

•  ThereareseveralNON-X-rayeventtypes.TheseareindicatedintheEVENT_FLAGSbitcolumn…(‘x’meansdon’tcare)

–  EVENT_FLAGS==xxxxx1:“undershoot”reset–  EVENT_FLAGS==xxxx1x:“overshoot”reset–  EVENT_FLAGS==xxx1xx:soGwaresample

–  EVENT_FLAGS==xx1xxx:fastsignalchaintriggered–  EVENT_FLAGS==x1xxxx:slowsignalchaintriggered–  EVENT_FLAGS==1xxxxx:firsteventinMPUpacket

•  TherecanbeMANY,MANYmorenonX-raysthanX-rays!

16

•  ToretrievevalidX-rays,usethefollowingfilter:– EVENT_FLAGS=x1x000

(requireslowchain,don’tcarefastchain,rejectresets)

– Thisscreeningcriteriahasbeenappliedtogeneratethedeliveredcleaned“cl”file.

•  ToretrieveFAST+SLOWevents,usethefollowingfilter:– EVENT_FLAGS=x11000

(requirefast+slowchain,rejectresets)

•  Example:–  niextract-events ‘input.evt[EVENT_FLAGS=bx11000]’ output.evt gti=GTI

17

•  Wecurrentlyrecommendthatanalystsusethetoolnicermergecleantoscreendata.Thistaskwillautoma=callyremovenonX-rays.

•  Thetaskwillautoma=callyscreenoutdatabelow200eV.

•  WearecurrentlyrecommendingthatuserswillprobablyneedtorejectcountswithPI<250eV.

PSRJ0437Spectrum

TargetSpectrum

~250eV

18

•  “Image”columnsRAWX/RAWYcanbeusedtomakearudimentarydetectorplaneimage(use“fv”)

•  DEADTIMEisinstrumentdead=measrecordedbytheMPUforeachevent.– EachMPUoperatesindependentlyfordead-=mepurposes

•  TICKistheuncalibratedinstrument-recordedevent=meinunitsof40nanoseconds.– NOTEthateachMPU=cksatadifferentand=me-variablerate.UsetheTIMEcolumn.

•  ThePPS_TRENDextensionishousekeepingdata

19

•  AnysoGwarethatunderstandsbasicX-rayeventlistsshouldworkwithNICERdata.

•  Youwillprobablyhavetodosomescreeningoutsideofxselecttoremovenon-X-rays.

•  Youcanusexselectandextractor.–  Examplexselectsession:

> read events ./ni1707030136_0mpu7_cl.evt> set binsize 1> extract curve> plot curve> set phaname PI> extract spectrum> plot spectrum

•  Scien=ststypicallyusetheirown=mingsoGware!

20

•  Thesedataares=llconsideredunvalidated,andscien=stsshouldbeonthelook-outforoddoroff-nominalbehaviors.

•  Thenextpagesdescribecertainknownissuesthatdonotneedtobereported

21

•  Ithasbeenreportedthattheremaybea1secondabsolute=mingoffsetofNICER=mestamps

•  ThisoffsetwouldmanifestitselfwhencomparingNICERdatatootherobservatoriesanda1-seconddifference.

•  TheNICERteamisawareofthisissueandisinves=ga=ng,butwantedtogetdataouttotheteamwithoutdelay,forthemanyscien=stswhoarenotsensi=veto1second=mingoffsets.

•  Iftheissueisresolvedanda=mecorrec=onisneeded,theteamwillprovideasmalltoolneededtoadjustdatasetsin-place.

22

•  Duringpre-launchtes=ngandflight,somedetectorscanproduceasingleeventcountwithinafewmicrosecondsofthe1secondGPS=mingpulse.

•  ThisisknowntoMITas"PPSNoise,"althoughformallyitisnotnoise.

•  ThereareanumberofwaysthisPPSnoisemaymanifestitself–  falseX-raycountsatexactcorrelatedwith1-secondrollover

–  correlateddead-=meatexact1-secondrollover–  BEWAREofexact1-secondpulsarsorQPOs

•  Teamisinves=ga=nghowtoscreenthese,buttherewillalwaysatleastbedead=meat1-secondrolloverboundaries

23

•  MostofNICERanalysisisgearedtowardscreeningoutundesirableeventsforspectralor=mingprocessing.

•  Formostsensi=veanalysishowever,dead-=memaybeimportant.

•  Consideringdead=me,–  EachMPUtriggersindependently– AllMPUeventsshouldbeconsideredtoes=matedead=me,notjustthecleanedeventlist

–  EachMPUes=matesitsamountofdead=meonaper-eventbasis

•  Therefore,scien=stswishingtocontempla=ngdetaileddead=meanalysisshouldgobacktotheper-MPU"uf"filesandtallydead=mefromthoseevents.

24

•  Almostallofthemetadatakeywordsarefilledwithproperinforma=on,buttheLEAP_INITkeywordisnot.

•  Thiskeywordisintendedtoaidscien=ststoconvertbetweentheTT=mesystemandUTC(seepreviousslides)

•  CurrentlyLEAP_INITisfilledwithzero,whichiserroneous– ForallNICERobserva=onstodate,LEAP_INIT=2

•  Scien=stsshouldtakecareun=lthisisfixed.