multi-model calibrated probabilistic seasonal forecasts of ...may 09, 2018 · multi-model...
TRANSCRIPT
Multi-Model Calibrated Probabilistic
Seasonal Forecasts of Regional Arctic Sea
Ice Coverage2018 Polar Prediction Workshop
Arlan Dirkson, William Merryfield, Bertrand Denis
Thanks to Woosung Lee and Adam Monahan
Support: CanSISE and FRAMS
Departement des sciences de la Terre et de l’AtmosphereUniversite du Quebec a Montreal
May 8, 2018
1 / 13
Sea Ice Probability, SIO 2017
https://www.arcus.org/sipn/sea-ice-outlook
2 / 13
Sea Ice Probability, SIO 2017
https://www.arcus.org/sipn/sea-ice-outlook
2 / 13
Motivation
Sea ice forecasts on seasonal and sub-seasonal timescalesare uncertain → uncertainty should be quantified.
I Sea ice probability (SIP) metric for Sea Ice Outlook (Stroeve,2015; Wrigglesworth et al. 2017).
Forecast ensembles are (generally) small and caninadequately described the forecast probability distribution(Richardson, 2001).
I Estimate by fitting appropriate distribution to forecastensemble (Wilks, 2002).
Model errors can be large, and need to be corrected.I Multi-model averaging (cancellation of biases)I Calibration (based on past forecasts and observations)I Combination of both
3 / 13
Motivation
Sea ice forecasts on seasonal and sub-seasonal timescalesare uncertain → uncertainty should be quantified.
I Sea ice probability (SIP) metric for Sea Ice Outlook (Stroeve,2015; Wrigglesworth et al. 2017).
Forecast ensembles are (generally) small and caninadequately described the forecast probability distribution(Richardson, 2001).
I Estimate by fitting appropriate distribution to forecastensemble (Wilks, 2002).
Model errors can be large, and need to be corrected.I Multi-model averaging (cancellation of biases)I Calibration (based on past forecasts and observations)I Combination of both
3 / 13
Motivation
Sea ice forecasts on seasonal and sub-seasonal timescalesare uncertain → uncertainty should be quantified.
I Sea ice probability (SIP) metric for Sea Ice Outlook (Stroeve,2015; Wrigglesworth et al. 2017).
Forecast ensembles are (generally) small and caninadequately described the forecast probability distribution(Richardson, 2001).
I Estimate by fitting appropriate distribution to forecastensemble (Wilks, 2002).
Model errors can be large, and need to be corrected.I Multi-model averaging (cancellation of biases)I Calibration (based on past forecasts and observations)I Combination of both
3 / 13
Motivation
Sea ice forecasts on seasonal and sub-seasonal timescalesare uncertain → uncertainty should be quantified.
I Sea ice probability (SIP) metric for Sea Ice Outlook (Stroeve,2015; Wrigglesworth et al. 2017).
Forecast ensembles are (generally) small and caninadequately described the forecast probability distribution(Richardson, 2001).
I Estimate by fitting appropriate distribution to forecastensemble (Wilks, 2002).
Model errors can be large, and need to be corrected.I Multi-model averaging (cancellation of biases)I Calibration (based on past forecasts and observations)I Combination of both
3 / 13
Motivation
Sea ice forecasts on seasonal and sub-seasonal timescalesare uncertain → uncertainty should be quantified.
I Sea ice probability (SIP) metric for Sea Ice Outlook (Stroeve,2015; Wrigglesworth et al. 2017).
Forecast ensembles are (generally) small and caninadequately described the forecast probability distribution(Richardson, 2001).
I Estimate by fitting appropriate distribution to forecastensemble (Wilks, 2002).
Model errors can be large, and need to be corrected.
I Multi-model averaging (cancellation of biases)I Calibration (based on past forecasts and observations)I Combination of both
3 / 13
Motivation
Sea ice forecasts on seasonal and sub-seasonal timescalesare uncertain → uncertainty should be quantified.
I Sea ice probability (SIP) metric for Sea Ice Outlook (Stroeve,2015; Wrigglesworth et al. 2017).
Forecast ensembles are (generally) small and caninadequately described the forecast probability distribution(Richardson, 2001).
I Estimate by fitting appropriate distribution to forecastensemble (Wilks, 2002).
Model errors can be large, and need to be corrected.I Multi-model averaging (cancellation of biases)
I Calibration (based on past forecasts and observations)I Combination of both
3 / 13
Motivation
Sea ice forecasts on seasonal and sub-seasonal timescalesare uncertain → uncertainty should be quantified.
I Sea ice probability (SIP) metric for Sea Ice Outlook (Stroeve,2015; Wrigglesworth et al. 2017).
Forecast ensembles are (generally) small and caninadequately described the forecast probability distribution(Richardson, 2001).
I Estimate by fitting appropriate distribution to forecastensemble (Wilks, 2002).
Model errors can be large, and need to be corrected.I Multi-model averaging (cancellation of biases)I Calibration (based on past forecasts and observations)
I Combination of both
3 / 13
Motivation
Sea ice forecasts on seasonal and sub-seasonal timescalesare uncertain → uncertainty should be quantified.
I Sea ice probability (SIP) metric for Sea Ice Outlook (Stroeve,2015; Wrigglesworth et al. 2017).
Forecast ensembles are (generally) small and caninadequately described the forecast probability distribution(Richardson, 2001).
I Estimate by fitting appropriate distribution to forecastensemble (Wilks, 2002).
Model errors can be large, and need to be corrected.I Multi-model averaging (cancellation of biases)I Calibration (based on past forecasts and observations)I Combination of both
3 / 13
Multi-Model Forecast Calibration (in a nutshell)
Model 1
Model 2
Observations
4 / 13
Multi-Model Forecast Calibration (in a nutshell)
Model 2
Observations
Y = g(X1)
Y = h(X2)
Model 1
4 / 13
Multi-Model Forecast Calibration (in a nutshell)
Model 1
Model 2
Observations
Y = g(X1)
Y = h(X2)
4 / 13
Multi-Model Forecast Calibration (in a nutshell)
Model 1
Model 2
Observations
Y = g(X1)
Y = h(X2)
g
4 / 13
Multi-Model Forecast Calibration (in a nutshell)
Model 1
Model 2
Observations
Y = g(X1)
Y = h(X2)
g
h
4 / 13
Overview
Experiment Details
I Models: CanCM3 and CanCM4 (Canadian Seasonal to InterannualPrediction System; CanSIPS)
I Hindcasts: September, 2000-2017I Initialization Months: June, July, August
Multi-Model CalibrationI Performed on the sea ice concentration (SIC) variable per model
and per grid point.I Utilizes a parametric probability distribution suitable for SIC, the
zero- and one- inflated beta (BEINF) distribution (Ospina andFerrari, 2010).
I Trend-adjusted quantile mapping (TAQM) designed for the BEINFdistribution and accounts for trends (Dirkson et al., 2018, Jclim(submitted)).
5 / 13
Overview
Experiment DetailsI Models: CanCM3 and CanCM4 (Canadian Seasonal to Interannual
Prediction System; CanSIPS)
I Hindcasts: September, 2000-2017I Initialization Months: June, July, August
Multi-Model CalibrationI Performed on the sea ice concentration (SIC) variable per model
and per grid point.I Utilizes a parametric probability distribution suitable for SIC, the
zero- and one- inflated beta (BEINF) distribution (Ospina andFerrari, 2010).
I Trend-adjusted quantile mapping (TAQM) designed for the BEINFdistribution and accounts for trends (Dirkson et al., 2018, Jclim(submitted)).
5 / 13
Overview
Experiment DetailsI Models: CanCM3 and CanCM4 (Canadian Seasonal to Interannual
Prediction System; CanSIPS)I Hindcasts: September, 2000-2017
I Initialization Months: June, July, August
Multi-Model CalibrationI Performed on the sea ice concentration (SIC) variable per model
and per grid point.I Utilizes a parametric probability distribution suitable for SIC, the
zero- and one- inflated beta (BEINF) distribution (Ospina andFerrari, 2010).
I Trend-adjusted quantile mapping (TAQM) designed for the BEINFdistribution and accounts for trends (Dirkson et al., 2018, Jclim(submitted)).
5 / 13
Overview
Experiment DetailsI Models: CanCM3 and CanCM4 (Canadian Seasonal to Interannual
Prediction System; CanSIPS)I Hindcasts: September, 2000-2017I Initialization Months: June, July, August
Multi-Model CalibrationI Performed on the sea ice concentration (SIC) variable per model
and per grid point.I Utilizes a parametric probability distribution suitable for SIC, the
zero- and one- inflated beta (BEINF) distribution (Ospina andFerrari, 2010).
I Trend-adjusted quantile mapping (TAQM) designed for the BEINFdistribution and accounts for trends (Dirkson et al., 2018, Jclim(submitted)).
5 / 13
Overview
Experiment DetailsI Models: CanCM3 and CanCM4 (Canadian Seasonal to Interannual
Prediction System; CanSIPS)I Hindcasts: September, 2000-2017I Initialization Months: June, July, August
Multi-Model Calibration
I Performed on the sea ice concentration (SIC) variable per modeland per grid point.
I Utilizes a parametric probability distribution suitable for SIC, thezero- and one- inflated beta (BEINF) distribution (Ospina andFerrari, 2010).
I Trend-adjusted quantile mapping (TAQM) designed for the BEINFdistribution and accounts for trends (Dirkson et al., 2018, Jclim(submitted)).
5 / 13
Overview
Experiment DetailsI Models: CanCM3 and CanCM4 (Canadian Seasonal to Interannual
Prediction System; CanSIPS)I Hindcasts: September, 2000-2017I Initialization Months: June, July, August
Multi-Model CalibrationI Performed on the sea ice concentration (SIC) variable per model
and per grid point.
I Utilizes a parametric probability distribution suitable for SIC, thezero- and one- inflated beta (BEINF) distribution (Ospina andFerrari, 2010).
I Trend-adjusted quantile mapping (TAQM) designed for the BEINFdistribution and accounts for trends (Dirkson et al., 2018, Jclim(submitted)).
5 / 13
Overview
Experiment DetailsI Models: CanCM3 and CanCM4 (Canadian Seasonal to Interannual
Prediction System; CanSIPS)I Hindcasts: September, 2000-2017I Initialization Months: June, July, August
Multi-Model CalibrationI Performed on the sea ice concentration (SIC) variable per model
and per grid point.I Utilizes a parametric probability distribution suitable for SIC, the
zero- and one- inflated beta (BEINF) distribution (Ospina andFerrari, 2010).
I Trend-adjusted quantile mapping (TAQM) designed for the BEINFdistribution and accounts for trends (Dirkson et al., 2018, Jclim(submitted)).
5 / 13
Overview
Experiment DetailsI Models: CanCM3 and CanCM4 (Canadian Seasonal to Interannual
Prediction System; CanSIPS)I Hindcasts: September, 2000-2017I Initialization Months: June, July, August
Multi-Model CalibrationI Performed on the sea ice concentration (SIC) variable per model
and per grid point.I Utilizes a parametric probability distribution suitable for SIC, the
zero- and one- inflated beta (BEINF) distribution (Ospina andFerrari, 2010).
I Trend-adjusted quantile mapping (TAQM) designed for the BEINFdistribution and accounts for trends (Dirkson et al., 2018, Jclim(submitted)).
5 / 13
Multi-Model TAQM: September 2017, June-init
Example for grid cell in East-Siberian Sea
Step 1. Adjust Historical Data for Trend
1980 1985 1990 1995 2000 2005 2010 20150.0
0.2
0.4
0.6
0.8
1.0
Sea
Ice C
once
ntra
tion
t=2017
CanCM3
OriginalTrend-Adjusted
1980 1985 1990 1995 2000 2005 2010 20150.0
0.2
0.4
0.6
0.8
1.0
Sea
Ice C
once
ntra
tion
t=2017
CanCM4OriginalTrend-Adjusted
1980 1985 1990 1995 2000 2005 2010 2015Year
0.0
0.2
0.4
0.6
0.8
1.0
Sea
Ice C
once
ntra
tion
t=2017
Observations
OriginalTrend-Adjusted
6 / 13
Multi-Model TAQM: September 2017, June-init
Example for grid cell in East-Siberian Sea
Step 2. Fit Historical Data to BEINF Distribution
1980 1985 1990 1995 2000 2005 2010 20150.0
0.2
0.4
0.6
0.8
1.0
Sea
Ice C
once
ntra
tion
t=2017
CanCM3
OriginalTrend-Adjusted
0.0 0.2 0.4 0.6 0.8 1.00
1
2
3
4
Prob
abilit
y De
nsity
CanCM3OriginalTrend-Adjusted
1980 1985 1990 1995 2000 2005 2010 20150.0
0.2
0.4
0.6
0.8
1.0
Sea
Ice C
once
ntra
tion
t=2017
CanCM4OriginalTrend-Adjusted
0.0 0.2 0.4 0.6 0.8 1.00
1
2
3
4
Prob
abilit
y De
nsity
CanCM4OriginalTrend-Adjusted
1980 1985 1990 1995 2000 2005 2010 2015Year
0.0
0.2
0.4
0.6
0.8
1.0
Sea
Ice C
once
ntra
tion
t=2017
Observations
OriginalTrend-Adjusted
0.0 0.2 0.4 0.6 0.8 1.0Sea Ice Concentration
0
1
2
3
4
5
6
Prob
abilit
y De
nsity
ObservationsOriginalTrend-Adjusted
7 / 13
Multi-Model TAQM: September 2017, June-init
Example for grid cell in East-Siberian Sea
Step 3. Calibrate
– Quantile map fcst values 0 < xt < 1: xt = F−1o,beta[Fm,beta(xt)]
– Correct mean bias in P(xt = 0) and P(xt = 1)
0.0 0.2 0.4 0.6 0.8 1.00.0
0.2
0.4
0.6
0.8
1.0
F bet
a, Be
rnou
lli M
asse
s
CanCM3 Forecast: 2017
uncal, xt
taqm, xt
0.0 0.2 0.4 0.6 0.8 1.00.0
0.2
0.4
0.6
0.8
1.0
F bet
a, Be
rnou
lli M
asse
s
CanCM3 Historical: 1981-2016
CanCM3, xObs, y
0.0 0.2 0.4 0.6 0.8 1.0Sea Ice Concentration
0.0
0.2
0.4
0.6
0.8
1.0F b
eta,
Bern
oulli
Mas
ses
CanCM4 Forecast: 2017
uncal, xt
taqm, xt
0.0 0.2 0.4 0.6 0.8 1.0Sea Ice Concentration
0.0
0.2
0.4
0.6
0.8
1.0
F bet
a, Be
rnou
lli M
asse
s
CanCM4 Historical: 1981-2016
CanCM4, xObs, y
8 / 13
Multi-Model TAQM: September 2017, June-init
BS = 0.217
CanCM3 (raw)
BS = 0.129
CanCM4 (raw)
BS = 0.136
CanCM3+CanCM4 (raw)
BS = 0.096
CanCM3 (calibrated)
BS = 0.091
CanCM4 (calibrated)
BS = 0.087
CanCM3+CanCM4 (calibrated)
0.0
0.2
0.4
0.6
0.8
1.0
June-init 2017 September SIP P(SIC>0.15)
observed ice edge
9 / 13
Probabilistic Hindcast Skill: September 2000-2017
red=skill blue=no skill
CanCM3+CanCM4 (raw) CanCM3+CanCM4 (raw) CanCM3+CanCM4 (raw)
CanCM3+CanCM4 (calibrated) CanCM3+CanCM4 (calibrated) CanCM3+CanCM4 (calibrated)
-1.0
-0.75
-0.5
-0.25
0.0
0.25
0.5
June-init July-init August-init
2000-2017 September Hindcast Skill CRPSS = 1 CRPSfcst/CRPSclimo
10 / 13
Early Forecast Contribution: Route Access
0.0
0.2
0.4
0.6
0.8
1.0
May-init 2018 September Probability of OW Access
Low probability for access via both the NSR and NWP (< 20% for both)
11 / 13
Early-Consensus Forecast Contribution: Regional SIA
Extreme Low6%
Low 81%
High8%
Extreme High
5%
Arctic BasinIce Free
Baffin Bay/Labrador Sea
Extreme Low13%
Low 75%
High8%
Greenland Sea
Extreme Low
20%
Low
20%
High
60%
Barents Sea
Extreme Low
19%
Low
19%High7%
Extreme High
54%
Kara SeaExtreme Low
27%Low
22%
High
41%Extreme High
10%
Laptev Sea
Extreme Low
34%Low14%
High
39%Extreme High
14%
East Siberian SeaExtreme Low
29%Low29%
High12%
Extreme High
30%
Chuckchi Sea
Extreme Low
30%Low29%
High
38%
Beaufort SeaExtreme Low
30%Low 35%
High
32%
Canadian Archipelago
Arctic Basin: Low (veryconfident)
Canadian Archipelago: Low(somewhat uncertain)
Greenland Sea: Low (veryconfident)
Barents/Kara Sea: High orExtreme High (somewhatconfident)
Laptev/East-Siberian Sea:High (somewhat uncertain)
Beaufort/Chukchi Sea:Equal Prob (very uncertain)
Baffin Bay/Labrador Sea:Ice Free (very confident)
12 / 13
Methods are open source! :)https://adirkson.github.io/SIC-probability
Thank you for listening!
13 / 13