mercator-ocean.eu/marine.copernicus.eu

Six years class4 intercomparison with

IV-TT dataset

OceanPredict’19 meeting - May 8, 2019

REGNIER Charly

DREVILLON, Marie, SZCZYPTA, Camille, HERNANDEZ, Fabrice and IV-TT members

Introduction

This Intercomparison is based on class4 files « model equivalents of the observations in

time and space (best estimate, forecast and persistence at different lead times and clim) »

Used to:

❖ Control and Monitor the quality of the systems

❖ Described the main differences in 2D and 3D space

❖ Measure the forecast accuracy

Aim of this presentation is to show you this three aspects but also new metrics like

contengency table approach and perspectives with a new CLASS4 UV dataset

Materials

• Since 2013 daily production of Class4

• Datasets used (available on crypted NOAA ftp server)

• SST dritings Buoys since 2013

• Temperature and salinity from Argo profiles since 2013

• Sea Ice concentration from AMS2 satellite since 2015

• L3 along track sla dataset from CMEMS since 2013

• UV drifters from CMEMS since April 2019

• ParticipantsGlobal systems• UK with Met Office (FOAM system ¼°)

• Canada with Environnement Canada (GIOPS system ¼°)

• US with NOAA (RTOFS system 1/12°)

• Australia with Bluelink (OMAPS system 1/10°)

• France with Mercator Ocean International (PSY3 and PSY4 systems)

Regional systems

• Arctic with NERSC system ¼° (2014 – 2018)

• Italia with INGV (MFS system)

• Brasil

• China with NMEFC

Monitoring the quality

Monitoring the performance of the systems

• Scores are available for all the GODAE regions with other scores (anomaly correlation, skill scores)• Need to go a step further to have a real time access to these scores

Oversampling in the ArcticBad representationof Pacific SummerWaters in november2018

Future evolutions

https://moniqua.mercator-ocean.fr

Number of observations

Misfit average for Temp (obs-mod)Oper 1/12°

Ran 1/12°

Frontend is ok, just need a short time to adapt the database to the statistics

Describing the quality in 2D and 3D

RMSD Salinity 0-500m in 2018

Best Analysis

BLK

GIOPS

FOAMPSY4

ENSEMBLE MEAN

HYCOM-RTOFS

GIOPS

RMSD Salinity 0-500m in 2018

Forecast 3D

BLK

FOAMPSY4

ENSEMBLE MEAN

HYCOM-RTOFS

Temperature error spread (Forecast 3D)

Period 2017-2018Thick blue line : mean diffDark blue shading : 75 % of distributionLight blue shading : 95% of distribution

Salinity error spread (Forecast 3 days)

Forecast Accuracy

Forecast accuracy in 2017

Evolution of the RMSD as a fonction of forecast lead timeThick line : percentile 75% distrThin Line : Percentile 95% distr

Forecast accuracy using skill scores

• Skill = 1 – (RMSDforecast /RMSDreference)

• 2 types of skill scores :

• Persistence Skill Score with persistence as a reference

• Climatology skill scores with Climatology WOA as a reference

PSS skill score 3D in 2018 (Ensemble mean)

Temperature (0-500m) Salinity (0-500m)

SSTSLA

New metrics and dataset

Derived quantities with Ice concentration

Contingency table approach

• Following the contingency table method described in Smith et al (2015)

• Theses statistics are computed on a stereopolar grid (with 0.2 threshold for ice/no ice)

• Possible to compute the following derived quantities▪ Proportion Correct Ice, PCI=a/(a+c)

▪ Proportion Correct Water, PCW=d/(b+d)

▪ Proportion Correct Total, PCT=(a+d)/n

▪ Frequency Bias, BIAS=(a+b)/(a+c)

n : number of points

PCT scores Forecast 3D

ARC

ANT

Update: add sea iceconcentration in DA

New dataset: Class4 UV drifters

OBS MODEL

• First set of 1 year already produced• Zonal currents are underestimated in the model• Need to work on filtered dataset without inertial oscillations to compare to daily

model outputs• Correct the data from errors (slippage and windage corrections)

Conclusion

▪ Class4 statistics are fundamental for operationnal verification

▪ Convergence in the scores for all the global systems

▪ Each system as not the same strenghts and weaknesses but are complementary

▪ The ensemble mean of all the global systems exhibits very good performances

▪ Forecast accuracy is improved over time, except for SST all the systems are close

▪ Still work to do on new metrics (threshold metrics, integrated values…)

▪ Work to do to put UV class4 in operationnal and to produce new statistics