annual meeting, june 15-16 2015, istanbul, turkey wp2: data aggregation and products generation in...
TRANSCRIPT
![Page 1: Annual Meeting, June 15-16 2015, Istanbul, Turkey WP2: Data aggregation and products generation in the Atlantic Sea: results and criticalities EMODnet](https://reader035.vdocuments.net/reader035/viewer/2022070411/56649f515503460f94c73a6a/html5/thumbnails/1.jpg)
Annual Meeting, June 15-16 2015, Istanbul, Turkey
WP2: Data aggregation and products generation
in the Atlantic Sea: results and criticalities
EMODnet Chemistry
With IFREMER, IEO, IHPT and RIHMI‐WDC contributions
![Page 2: Annual Meeting, June 15-16 2015, Istanbul, Turkey WP2: Data aggregation and products generation in the Atlantic Sea: results and criticalities EMODnet](https://reader035.vdocuments.net/reader035/viewer/2022070411/56649f515503460f94c73a6a/html5/thumbnails/2.jpg)
Annual Meeting, June 15-16 2015, Istanbul, Turkey
Outline
• Data inventory of the second harvesting
• Data aggregation / validation phase : issues and solutions
• Regional data buffer : space and time distribution by parameter
• DIVA maps production
![Page 3: Annual Meeting, June 15-16 2015, Istanbul, Turkey WP2: Data aggregation and products generation in the Atlantic Sea: results and criticalities EMODnet](https://reader035.vdocuments.net/reader035/viewer/2022070411/56649f515503460f94c73a6a/html5/thumbnails/3.jpg)
3/15
Data collected by MARIS robot (28/10/2014) :
Data inventory (1)
CDI type Vertical profiles
Time Series
Total
New CDIS 13613 99 13712
Updated CDIs 2601 27 2628
Old CDIs 7040 231 7271
Total 23254 357 23611
Annual Meeting, June 15-16 2015, Istanbul, Turkey
![Page 4: Annual Meeting, June 15-16 2015, Istanbul, Turkey WP2: Data aggregation and products generation in the Atlantic Sea: results and criticalities EMODnet](https://reader035.vdocuments.net/reader035/viewer/2022070411/56649f515503460f94c73a6a/html5/thumbnails/4.jpg)
4/15
Data collected by MARIS robot (28/10/2014) :
Data inventory (1)
CDI type Vertical profiles
Time Series
Total
New CDIS 13613 99 13712
Updated CDIs 2601 27 2628
Old CDIs 7040 231 7271
Total 23254 357 23611
More than twice the first harvesting !
Annual Meeting, June 15-16 2015, Istanbul, Turkey
![Page 5: Annual Meeting, June 15-16 2015, Istanbul, Turkey WP2: Data aggregation and products generation in the Atlantic Sea: results and criticalities EMODnet](https://reader035.vdocuments.net/reader035/viewer/2022070411/56649f515503460f94c73a6a/html5/thumbnails/5.jpg)
5/15
First harvesting
Data inventory (2)
Second harvesting
Old Updated
New
Enlargeddata coverage
Annual Meeting, June 15-16 2015, Istanbul, Turkey
![Page 6: Annual Meeting, June 15-16 2015, Istanbul, Turkey WP2: Data aggregation and products generation in the Atlantic Sea: results and criticalities EMODnet](https://reader035.vdocuments.net/reader035/viewer/2022070411/56649f515503460f94c73a6a/html5/thumbnails/6.jpg)
6/15
Distribution by partners
Annual Meeting, June 15-16 2015, Istanbul, Turkey
CDI type
IEO Ifremer IHPTRIHMI-NODC
IMR UniHB
New CDIs
2396 40 0 1670 3 9
Updated CDIs
5242 0 3677 146 0 0
Old CDIs
7764 6465 876 2676 33 0
Total 15402 6505 4553 4492 36 9
Data inventory (3)
![Page 7: Annual Meeting, June 15-16 2015, Istanbul, Turkey WP2: Data aggregation and products generation in the Atlantic Sea: results and criticalities EMODnet](https://reader035.vdocuments.net/reader035/viewer/2022070411/56649f515503460f94c73a6a/html5/thumbnails/7.jpg)
7/15Annual Meeting, June 15-16 2015, Istanbul, Turkey
CDI type IEO Ifremer IHPTRIHMI-NODC
IMR UniHB
New CDIs 2396 40 0 1670 3 9
Updated CDIs 5242 0 3677 146 0 0
Old CDIs 7764 6465 876 2676 33 0
Total 15402 6505 4553 4492 36 9
Data inventory (3)
Distribution by partners
![Page 8: Annual Meeting, June 15-16 2015, Istanbul, Turkey WP2: Data aggregation and products generation in the Atlantic Sea: results and criticalities EMODnet](https://reader035.vdocuments.net/reader035/viewer/2022070411/56649f515503460f94c73a6a/html5/thumbnails/8.jpg)
8/15
2nd harvestingCDI type
Phos Amon Silicate Nox
New CDIs 2230 250 762 835
Updated CDIs 2551 1835 2241 2416
Old CDIs 6538 2916 4107 4221
Total 11319 5001 7110 7472
Annual Meeting, June 15-16 2015, Istanbul, Turkey
Distribution by parameters
Data inventory (4)
![Page 9: Annual Meeting, June 15-16 2015, Istanbul, Turkey WP2: Data aggregation and products generation in the Atlantic Sea: results and criticalities EMODnet](https://reader035.vdocuments.net/reader035/viewer/2022070411/56649f515503460f94c73a6a/html5/thumbnails/9.jpg)
9/15
• Use only data with Quality Flag = 0, 1, 2, 6 as initial collection to use only good data
• Unit conversion and aggregation (P35 vocabs): mol/l
• Deletion of empty lines in a station : some levels with no measurements for the parameter
• Deletion of lines with negative depth (incorrect missing value)
• Perform QC with ODV
Search for out of “broad range” values with Flag = 1
Set a QC for all measurements with Flag = 0
Set default values with Flag = 0, 1 to Flag = 9
Deal with Flag 6 and zero values
• Write a report on QC and send results to MARIS and data providers
Quality Checks - common rules
Annual Meeting, June 15-16 2015, Istanbul, Turkey
Data aggregation and QC
![Page 10: Annual Meeting, June 15-16 2015, Istanbul, Turkey WP2: Data aggregation and products generation in the Atlantic Sea: results and criticalities EMODnet](https://reader035.vdocuments.net/reader035/viewer/2022070411/56649f515503460f94c73a6a/html5/thumbnails/10.jpg)
10/15
Quality checks – problems faced
• Problem of unit conversion : ug/l to umol/l e.g. reload russian data and some spanish data
• Problem in CDI updating e.g.high spanish Phosphate to delete
• Some of the reported errors to data centres have not been corrected : we cannot just import the new collection in the
collection built up after the first harvesting. We have to re-make the whole QC work.
Annual Meeting, June 15-16 2015, Istanbul, Turkey
Data aggregation and QC
![Page 11: Annual Meeting, June 15-16 2015, Istanbul, Turkey WP2: Data aggregation and products generation in the Atlantic Sea: results and criticalities EMODnet](https://reader035.vdocuments.net/reader035/viewer/2022070411/56649f515503460f94c73a6a/html5/thumbnails/11.jpg)
11/15Annual Meeting, June 15-16 2015, Istanbul, Turkey
Data aggregation and QC
5 Aggregated Data Collection Qced produced and sent to MARIS :- Phosphate : Global_PHOS_collection.txt- Ammonium : Global_AMON_collection.txt- Nox : Global_NOX_collection.txt- Silicate : Global_SLCA_collection.txt- Timeseries of these 4 parameters : Global_nutrients_TS_CDIs.txt
![Page 12: Annual Meeting, June 15-16 2015, Istanbul, Turkey WP2: Data aggregation and products generation in the Atlantic Sea: results and criticalities EMODnet](https://reader035.vdocuments.net/reader035/viewer/2022070411/56649f515503460f94c73a6a/html5/thumbnails/12.jpg)
12/15
Perform QC with ODV Search for out of range values with Flag = 1 Set a QC for all measurements with Flag = 0 Set default values with Flag = 0, 1 to Flag = 9 Deal with Flag 6 and zero values
Annual Meeting, June 15-16 2015, Istanbul, Turkey
Data aggregation and QC
Annual Meeting, June 15-16 2015, Istanbul, Turkey
![Page 13: Annual Meeting, June 15-16 2015, Istanbul, Turkey WP2: Data aggregation and products generation in the Atlantic Sea: results and criticalities EMODnet](https://reader035.vdocuments.net/reader035/viewer/2022070411/56649f515503460f94c73a6a/html5/thumbnails/13.jpg)
13/15
Flag value
PHOS Before
QC
PHOS after QC
0 43 0
1 29994 29417
2 19 19
3 40 114
4 4 5
9 0 2
total 30100 29557
Annual Meeting, June 15-16 2015, Istanbul, Turkey
Data aggregation and QC
Perform QC on New Phosphate with ODV Search for out of range values with Flag = 1 Set a QC for all measurements with Flag = 0 Set default values with Flag = 0, 1 to Flag = 9 Deal with Flag 6 and zero values
![Page 14: Annual Meeting, June 15-16 2015, Istanbul, Turkey WP2: Data aggregation and products generation in the Atlantic Sea: results and criticalities EMODnet](https://reader035.vdocuments.net/reader035/viewer/2022070411/56649f515503460f94c73a6a/html5/thumbnails/14.jpg)
14/15
Regional data buffer
Annual Meeting, June 15-16 2015, Istanbul, Turkey
Phosphate data
a) Station map
d) Data bins e) X/Y distribution f) Season histogram
b) Vertical plot c) Time histogram
![Page 15: Annual Meeting, June 15-16 2015, Istanbul, Turkey WP2: Data aggregation and products generation in the Atlantic Sea: results and criticalities EMODnet](https://reader035.vdocuments.net/reader035/viewer/2022070411/56649f515503460f94c73a6a/html5/thumbnails/15.jpg)
15/15
Nox data
a) Station map
d) Data bins e) X/Y distribution f) Season histogram
b) Vertical plot c) Time histogram
Annual Meeting, June 15-16 2015, Istanbul, Turkey
Regional data buffer
![Page 16: Annual Meeting, June 15-16 2015, Istanbul, Turkey WP2: Data aggregation and products generation in the Atlantic Sea: results and criticalities EMODnet](https://reader035.vdocuments.net/reader035/viewer/2022070411/56649f515503460f94c73a6a/html5/thumbnails/16.jpg)
16/15Annual Meeting, June 15-16 2015, Istanbul, Turkey
Silicate data
a) Station map
d) Data bins e) X/Y distribution f) Season histogram
b) Vertical plot c) Time histogram
Regional data buffer
![Page 17: Annual Meeting, June 15-16 2015, Istanbul, Turkey WP2: Data aggregation and products generation in the Atlantic Sea: results and criticalities EMODnet](https://reader035.vdocuments.net/reader035/viewer/2022070411/56649f515503460f94c73a6a/html5/thumbnails/17.jpg)
17/15Annual Meeting, June 15-16 2015, Istanbul, Turkey
Regional data buffer
Ammonium data
a) Station map
d) Data bins e) X/Y distribution f) Season histogram
b) Vertical plot c) Time histogram
![Page 18: Annual Meeting, June 15-16 2015, Istanbul, Turkey WP2: Data aggregation and products generation in the Atlantic Sea: results and criticalities EMODnet](https://reader035.vdocuments.net/reader035/viewer/2022070411/56649f515503460f94c73a6a/html5/thumbnails/18.jpg)
18/15Annual Meeting, June 15-16 2015, Istanbul, Turkey
DIVA maps production
DIVA maps settings :
settings Atlantic area
DIVA version 4.6.9 + weighting patch
Time period 10-years running periods with yearly step for the 4 seasons
Time range 1962-2014
Seasons winter (0103), spring (0406), summer (0709), autumn (1012)
Vertical layers IODE standards (33 levels)
parameters PO4, SiO4, Nox
![Page 19: Annual Meeting, June 15-16 2015, Istanbul, Turkey WP2: Data aggregation and products generation in the Atlantic Sea: results and criticalities EMODnet](https://reader035.vdocuments.net/reader035/viewer/2022070411/56649f515503460f94c73a6a/html5/thumbnails/19.jpg)
19/15Annual Meeting, June 15-16 2015, Istanbul, Turkey
Problems faced :•Some difficulties in installing DIVA on a Window PC via cygwin : inconsistencies between netcdf and gfortran librairies
•Handle the different steps to produce an analyse
•Optimizing the parameter estimation: Do not run on the 46 periods – 4 seasons
chose a decade with high coverage Some SN profiles very noisy
test binning and weighting options
•Very long time-consuming runs reduce the space and time resolution
•Pb of generation of the big 4D netcdf file from the 3D files
DIVA maps production
![Page 20: Annual Meeting, June 15-16 2015, Istanbul, Turkey WP2: Data aggregation and products generation in the Atlantic Sea: results and criticalities EMODnet](https://reader035.vdocuments.net/reader035/viewer/2022070411/56649f515503460f94c73a6a/html5/thumbnails/20.jpg)
20/15Annual Meeting, June 15-16 2015, Istanbul, Turkey
DIVA maps production
Phosphate at surface (0m) – summer (0709)
DIVA – 2005/2014 WOA – 1955/2012
![Page 21: Annual Meeting, June 15-16 2015, Istanbul, Turkey WP2: Data aggregation and products generation in the Atlantic Sea: results and criticalities EMODnet](https://reader035.vdocuments.net/reader035/viewer/2022070411/56649f515503460f94c73a6a/html5/thumbnails/21.jpg)
21/15Annual Meeting, June 15-16 2015, Istanbul, Turkey
DIVA maps production
Phosphate at surface (0m) – summer (0709)
DIVA 2005/2014 Data bins
![Page 22: Annual Meeting, June 15-16 2015, Istanbul, Turkey WP2: Data aggregation and products generation in the Atlantic Sea: results and criticalities EMODnet](https://reader035.vdocuments.net/reader035/viewer/2022070411/56649f515503460f94c73a6a/html5/thumbnails/22.jpg)
22/15Annual Meeting, June 15-16 2015, Istanbul, Turkey
DIVA maps production
Silicate at surface (0m) – summer (0709)
DIVA 2002/2011 WOA – 1955/2012
![Page 23: Annual Meeting, June 15-16 2015, Istanbul, Turkey WP2: Data aggregation and products generation in the Atlantic Sea: results and criticalities EMODnet](https://reader035.vdocuments.net/reader035/viewer/2022070411/56649f515503460f94c73a6a/html5/thumbnails/23.jpg)
23/15Annual Meeting, June 15-16 2015, Istanbul, Turkey
DIVA maps production
Silicate at surface (0m) – summer (0709)
DIVA 2002/2011 Data bins
![Page 24: Annual Meeting, June 15-16 2015, Istanbul, Turkey WP2: Data aggregation and products generation in the Atlantic Sea: results and criticalities EMODnet](https://reader035.vdocuments.net/reader035/viewer/2022070411/56649f515503460f94c73a6a/html5/thumbnails/24.jpg)
Annual Meeting, June 15-16 2015, Istanbul, Turkey
Planning for the next weeks
Data extraction (and weighting)
Parameters optimization
Run analysis Internal expert validation of products
Phosphate
Silicate
Nox
DIVA maps production
4D Netcdf files ready
![Page 25: Annual Meeting, June 15-16 2015, Istanbul, Turkey WP2: Data aggregation and products generation in the Atlantic Sea: results and criticalities EMODnet](https://reader035.vdocuments.net/reader035/viewer/2022070411/56649f515503460f94c73a6a/html5/thumbnails/25.jpg)
Annual Meeting, June 15-16 2015, Istanbul, Turkey
Planning for the next weeks
Data extraction (and weighting)
Parameters optimization
Run analysis Internal expert validation of products
Phosphate
Silicate
Nox
If needed
DIVA maps production
4D Netcdf files ready
![Page 26: Annual Meeting, June 15-16 2015, Istanbul, Turkey WP2: Data aggregation and products generation in the Atlantic Sea: results and criticalities EMODnet](https://reader035.vdocuments.net/reader035/viewer/2022070411/56649f515503460f94c73a6a/html5/thumbnails/26.jpg)
Annual Meeting, June 15-16 2015, Istanbul, Turkey
Planning for the next weeks
Data extraction (and weighting)
Parameters optimization
Run analysis Internal expert validation of products
Phosphate
Silicate
Nox
If needed
DIVA maps production
Work with other parameters : oxygen, chlorophylle, …
4D Netcdf files ready
![Page 27: Annual Meeting, June 15-16 2015, Istanbul, Turkey WP2: Data aggregation and products generation in the Atlantic Sea: results and criticalities EMODnet](https://reader035.vdocuments.net/reader035/viewer/2022070411/56649f515503460f94c73a6a/html5/thumbnails/27.jpg)
Thanks!
Annual Meeting, June 15-16 2015, Istanbul, Turkey