monthly operations report · · 2015-07-14probav_l3_s1_toc_100m 653.35 4379.76 328.75 ......

MONTHLY OPERATIONS

REPORT

MOR#018

Reporting period from 16-May-2015 to 15-Jun-2015

Reference: PROBA-V_D5_MOR-018_2015-06_v1 Author(s): Erwin Wolters, Dennis Clarijs, Sindy Sterckx, Flip Boonen Version: 1.0 Date: 18/06/2015

Monthly Operations Report PROBA-V Operations Contract No. 4000111291/14/I-LG - 1310174

PROBA-V_D5_MOR-018_2015-06_v1 Monthly Operations Report of 15

DOCUMENT CONTROL

Signatures Author(s) Erwin Wolters, Dennis Clarijs, Sindy Sterckx, Flip Boonen Reviewer(s) Dennis Clarijs Approver(s) Dennis Clarijs Issuing authority

Change record

Release Date Pages Description Editor(s)/Reviewer(s)

1.0 18/06/2015 All Initial version



TABLE OF CONTENT

1. Summary .............................................................................................................................................. 4 2. System Infrastructure ........................................................................................................................... 4 3. Image Processing Services .................................................................................................................... 5 3.1. Ingested and archived products ........................................................................................................... 5 3.2. Generated and archived products ........................................................................................................ 5 3.3. Backup and archiving service ............................................................................................................... 6 3.4. Dissemination service .......................................................................................................................... 7 3.5. End-user activity .................................................................................................................................. 7 4. Image Calibration services ................................................................................................................... 10 4.1. Radiometric Calibration ...................................................................................................................... 10 4.2. Geometric Calibration ......................................................................................................................... 12 5. Anomalies ........................................................................................................................................... 13 5.1. System related issues .......................................................................................................................... 13 5.2. Image processing issues ...................................................................................................................... 14 6. Scheduled activities for the next period(s) .......................................................................................... 15 7. Operational remarks ........................................................................................................................... 15



1. Summary In the past month, 27 syntheses were incomplete, one shows a major gap due to a missing TFF, 8 a large gap, 12 a medium gap and 6 syntheses have negligible gaps. Only 4 syntheses are complete without any anomalies. Most of the missing data in the past month was caused by decompression errors of the data and ‘missing spectral bands’. An thorough investigation is ongoing and extra logging has been added to the processing chain to determine root causes of the increase of these types of issues. On the PDF platform, the GeoTiff option has been released. All users have access now to this format. However, the VITO image processing centre will first generate one year of GeoTiff data on disk before any public announcement will be made. This is done to prevent a high load on the PDF workers every time a GeoTiff format needs to be generated. In this period, the geometric calibration for PROBA-V switched to using Landsat GLS 2010 data instead of Landsat 2000 data. This was done to improve the geometric stability for PROBA-V, but will only be noticeable when a new set of ICP files will be implemented. There were no issues with the image quality during this reporting period, the image quality is well within geometric and radiometric requirements. Currently developments are ongoing to improve the cloud detection and to make the data INSPIRE compliant.

2. System Infrastructure

Category % Up Time % Down Time (*)

Switches 99.99 0.01

Database Servers 99.99 0.01

Mid Term File Servers 99.98 0.02

Short Term File Servers 99.99 0.01

Master Servers 99.98 0.02

Worker Nodes 99.98 0.02

PDF 99.98 0.02

Table 1: System Infrastructure availability for this reporting period

(*) - Pmaster 11 down due to a network connection problem - 2 PDF workers down due to a network connection problem - Worker nodes: pnode 4 server experienced downtime due to a power dip - PDF: Planned downtime to move ORA4-server



3. Image Processing Services

3.1. Ingested and archived products

Product Type Total Received Missing data, ingested by VITO

Archived

METEO 248 248 248

TFF 309 309 1(*) 309

Table 2: Ingested and archived products for this reporting period

(*) 1 PASS 8889 missing at USAK ground station

3.2. Generated and archived products Product Type Total Processed Error Archived

PROBAV_L1A - Calibration 321 321 321

PROBAV_L1A - Nominal 2613 2592 21(*) 2599

PROBAV_L1C 2592 2592 2592

PROBAV_L3_S1_TOA_100M 31 31 31

PROBAV_L3_S1_TOC_100M 31 31 31

PROBAV_L3_S1_TOC_NDVI_100M 31 31 31








PROBAV_L3_S1_TOA_1KM 31 31 31

PROBAV_L3_S1_TOC_1KM 31 31 31


PROBAV_L3_S10_TOC_NDVI_1KM 3 3 3

Table 3: Generated and archived products for this reporting period

(*) 17 x L1A error due to a decompression error, 4 x L1A error due to pixel shift



3.3. Backup and archiving service

Product type Total Files Total File Size (GB)

TFF 299 793.58

L1A 2837 1385.36

Database transaction logs 1030 63.61

Database incremental back-up 80 33.28

Database full back-up 40 565.77

Table 4: Back-up data volumes for this reporting period

Product type Total Files Total File Size (GB)

PROBAV_TRANSFERFRAMES 309 880.54

PROBAV_L1A 2918 1533.64

PROBAV_L1C 2592 2973.72

PROBAV_L3_S1_TOA_100M 32 799.06

PROBAV_L3_S1_TOC_100M 31 802.80

PROBAV_L3_S1_TOC_NDVI_100M 31 92.24

PROBAV_L3_S5_TOA_100M 6 570.52

PROBAV_L3_S5_TOC_100M 6 591.06


PROBAV_L3_S1_TOA_300M 31 343.88

PROBAV_L3_S1_TOC_300M 31 357.54

PROBAV_L3_S10_TOC_300M 3 68.43


PROBAV_L3_S1_TOA_1KM 31 50.99

PROBAV_L3_S1_TOC_1KM 31 52.52

PROBAV_L3_S10_TOC_1KM 3 9.46

PROBAV_L3_S10_TOC_NDVI_1KM 3 0.68

ICP_GEOMETRIC_CENTRE 0 0

ICP_ GEOMETRIC _LEFT 0 0

ICP_ GEOMETRIC _RIGHT 0 0

ICP_RADIOMETRIC_CENTRE 1 0.04

ICP_RADIOMETRIC_LEFT 1 0.04

ICP_RADIOMETRIC_RIGHT 1 0.04

METEO_ECMWF 248 0.31

METEO_METEOSERVICES 248 1.32

POLARMOTION 1 <0.01

Table 5: Archived data volumes for this reporting period



3.4. Dissemination service Product type Added to catalogue Ordered Delivered

PROBAV_L1C 2590 157 159

PROBAV_L3_S1_TOA_100M 32 878 467

PROBAV_L3_S1_TOC_100M 32 402 334



PROBAV_L3_S5_TOC_100M 7 554 569


PROBAV_L3_S1_TOA_300M 33 196 224

PROBAV_L3_S1_TOC_300M 33 896 1851

PROBAV_L3_S10_TOC_300M 4 435 439


PROBAV_L3_S1_TOA_1KM 33 119 159



PROBAV_L3_S10_TOC_NDVI_1KM 3 95 135

Table 6: Ordered and delivered products for this reporting period

3.5. End-user activity 24 new user(s) were registered in this reporting period. The total number of users registered for PROBA-V data and that have ordered data is 446 with 77 different nationalities representing 369 different companies/universities.

Product type Africa Asia Europe N-America Oceania S-America

PROBAV_L1C 3.51 362.80 0.71

PROBAV_L3_S1_TOA_100M 34.06 1178.02 0.10 0.70

PROBAV_L3_S1_TOC_100M 653.35 4379.76 328.75

PROBAV_L3_S1_TOC_NDVI_100M 3.56 0.13 0.52 0.08

PROBAV_L3_S5_TOA_100M 413.62 0.41 1.64

PROBAV_L3_S5_TOC_100M 14.46 90.49 96.99


PROBAV_L3_S1_TOA_300M 0.37 770.90 0.08

PROBAV_L3_S1_TOC_300M 0.12 5900.71

PROBAV_L3_S10_TOC_300M 2.50 116.16


PROBAV_L3_S1_TOA_1KM 0.00 112.28 0.04



PROBAV_L3_S1_TOC_1KM 822.73

PROBAV_L3_S10_TOC_1KM 0.59 25.82 8.75

PROBAV_L3_S10_TOC_NDVI_1KM 0.00 0.40 1.70 7.22

Table 7: Data download (GB) in total per Origin of the User for the reporting period

Product Type Global

L1C 367.01

PROBAV_L3_S1_TOA_100M 1212.89

PROBAV_L3_S1_TOC_100M 5361.86

PROBAV_L3_S1_TOC_NDVI_100M 4.28








PROBAV_L3_S1_TOA_1KM 112.32



PROBAV_L3_S10_TOC_NDVI_1KM 9.33

Table 8: Data download (GB) in total for the reporting period

Company # of downloaded

products

GEOSYS 1000

VITO 898

University of Leicester 417

Hanoi University of Mining dan Geology & TU Vienna 398

nsmc 282

rennes 2 211

Infoterra 179

Universite catholique de Louvain 178

Institute of Remote Sensing and Digital Earth 177

Charles Darwin University 159

Table 9: Top 10 user companies for the reporting period



Company # of registered users

BELGIUM 52

CHINA 32

ITALY 28

FRANCE 21

NETHERLANDS 19

UNITED STATES 19

UNITED KINGDOM 16

BRAZIL 15

ARGENTINA 15

INDIA 13

Table 10: Top 10 countries with most registered users List of issues raised by users:

ProbaV - PROBA-V stitching issue - Quotation for PROBA-V images - Cost 100m PROBA-V - Sample image over DomeC - Quotation for PROBA-V images - PROBA-V circulation reports - Cannot order S5_TOC_NDVI_20150216 - naming PROBAV S5 TOC NDVI 100 m - PROBA-V geotiff format PDF portal - S1_TOA_20150414_1KM xml size error - How can I download the subset area from global data - Forgot Password - PDF customization options - unable to register



4. Image Calibration services

4.1. Radiometric Calibration

Calibration request type Total Processed Not received Error

CLOUDS 17 17 0 0

DARK CURRENT 21 21 0 0

MOON 2 1 1 0

RAYLEIGH 58 57 0 1

SNOW 0 0 0 0

SUN_GLINT 20 19 1 0

Table 11: Calibration Image requests for this reporting period

Calibration image type Total Valid Invalid (*)

PROBA_V_L1A_CALIBRATION 1 1 0

PROBA-V_L1B_CALIBRATION 320 258 62

PROBA-V_L1B_INTERSECTION 688 476 212

PROBA-V_L1B_OVERLAPREGION 0 0 0

Table 12: Processed calibration images for this reporting period

(*) Due to insufficient overlap with the calibration region of interest, not enough pixels (e.g. clouds contamination), site not sufficiently uniform (illumination), etc. Radiometric calibration assessment The Rayleigh, Libya-4, DCC and Lunar calibration methods show stable results over the last months for the VNIR strips. A decreasing trend is observed for some of the SWIR strips (mainly for RIGHT SWIR2, the LEFT SWIR strips, and CENTER SWIR1 and SWIR2 strips). This trend is however not confirmed based on the lunar calibration results for CENTER SWIR2 (i.e. strip observing the moon). It is at this moment unclear if the observed trend in the Libya-4 desert results is caused by an overall response degradation of a SWIR strip ( which would need an update of the absolute calibration parameter) and/or if it is also due to changes in the SWIR vignetting/equalization profiles . In this latter case, an update of the different SWIR strips absolute calibration coefficient might cause unwanted inter-strip and inter-camera biases. Therefore a more in-depth analysis is deemed necessary to identify the causes of the observed changes and to define the proper action w.r.t the required ICP file parameter update. These analyses could include the application of an experimental vicarious vignetting method (as was done during the commissioning phase)and the application of the OSCAR desert approach to some more desert sites. Although the model absolute accuracy is lower for these sites, the relative accuracy is expected to be sufficient to detect distinct trends in FOV variations over time.



Similar as in previous reporting periods sudden jumps in the dark current values of a few SWIR pixels, distributed over the different strips and cameras, have been observed which explains the presence of a few stripes in the SWIR images. An update of the dark current values will therefore be done in the next days. Radiometric ICP file ICP dark values will be updated in the coming days, the current ICP files are

PROBAV_ICP_RADIOMETRIC#LEFT_20150527_V01

PROBAV_ICP_RADIOMETRIC#CENTER_ 20150527_V01

PROBAV_ICP_RADIOMETRIC#RIGHT_ 20150527_V01



4.2. Geometric Calibration

Calibration image type Total Processed Error

PROBA-V_L1C_INTERSECTION 14124 14124 0

Table 13: Processed calibration images for this reporting period

During the previous month, the average ALE was much more stable than the month before, due to the implementation of new ICP files. The average ALE was < 60 m (σ < 49 m). In general, daily values were within the range 45 – 60 m, with peak values occurring at 18/5 (68 m), 23/5 (70 m), 4/6 (70 m), and 5/6 (69 m). The geometric accuracy was within the requirement of < 300 m, with an average value of better than 99.7% for all cameras. Geometric ICP file

PROBAV_ICP_GEOMETRIC#LEFT_20150506_V01

PROBAV_ICP_GEOMETRIC#CENTER_20150506_V01

PROBAV_ICP_GEOMETRIC#RIGHT_20150506_V01

Figure 1 - Absolute Location Error for this reporting period



5. Anomalies

5.1. System related issues A detailed description of each issue is available in the issue tracking system http://jira.vgt.vito.be

Key Summary Status Created Component/s

PROBAVUS-7 Very small images fail to process Resolved 10/01/2014 General

PROBAVUS-31 new ICP file generation Resolved 17/04/2014 Geometry

PROBAVUS-54 Quicklook generation fails after release ProductCustomisation tools for GeoTiff generation

Resolved 02/06/2015 Software

PROBAVUS-55 PPT problem with clipping and stitching Resolved 02/06/2015 Software

2 new issues were logged during this reporting period 0 issue(s) was resolved and closed during this reporting period 2 issues are resolved but remain to be closed formally 2 issues are resolved but remain in the list logging purposes 0 issue(s) is open and remain to be solved

http://jira.vgt.vito.be/browse/PROBAVUS-39







5.2. Image processing issues A detailed description of each issue is available in the Weekly Report and the image processing tracking system https://juniper.vgt.vito.be/ciptools The below table gives an overview of the S1’s of this reporting period:

# S1 Dates

Major Gaps (> 21600 km² (missing TFF)) 1 09/06

Large Gaps (< 21600 km²) 8 26/05, 31/05, 12/06, 30/05, 21/05, 24/05, 20/05, 11/06

Medium Gaps (< 10000 km²) 12 22/05, 27/05, 10/06, 07/06, 01/06, 02/06, 05/06, 04/06, 15/06, 18/05,

06/06, 14/06

Minor Gaps (< 3600 km²) 4 28/05, 19/05, 16/05, 23/05

Negligible Gaps (< 1000 km²) 2 08/06, 03/06

Complete synthesis (no gaps) 4 17/05, 25/05, 13/06, 29/05

Table 14: Overview of S1 for this reporting period

https://juniper.vgt.vito.be/ciptools



6. Scheduled activities for the next period(s)

Software: o An update of the JAVA version for the DIF component will be done and tested to

support a FTP server switchover at the Kiruna DRS

Hardware: o Some servers in the PROBA-V processing chain are end-of-life. An assessment will be

done to determine which servers can and will be upgraded.

Development: o An improvement of the cloud detection algorithm is under investigation. o Developments to make the metadata INSPIRE compliant are ongoing.

No other activities scheduled.

7. Operational remarks

No operational remarks.

monthly operations report · · 2015-07-14probav_l3_s1_toc_100m 653.35 4379.76 328.75 ......

Documents