MJO Forecasting MJO Forecasting

Metrics/DiagnosticsMetrics/Diagnostics

Matthew Wheeler – Matthew Wheeler – Australian Bureau of Australian Bureau of Meteorology/CAWCRMeteorology/CAWCR

Klaus Weickmann – Klaus Weickmann – NOAA/Physical Sciences DivisionNOAA/Physical Sciences Division

on behalf of the U.S.-CLIVAR MJO Working Group on behalf of the U.S.-CLIVAR MJO Working Group (and others)(and others)

MJO WG Terms of Reference

(from http://www.usclivar.org/Organization/MJO_WG.html)

• Develop a set of diagnostics to be used for assessing MJO simulation fidelity and forecast skillforecast skill. 

• Develop and coordinate model simulation and prediction experimentsprediction experiments, in conjunction with model-data comparisons, which are designed to better understand the MJO and improve our model representations and forecastsforecasts of the MJO.

• Raise awareness of the potential utility of subseasonal and MJO forecastsMJO forecasts in the context of the seamless suite of predictions.

• Help to coordinate MJO-related activities between national and international agencies and associated programmatic activities.

• Provide guidance to US CLIVAR and Interagency Group (IAG) on where additional modeling, analysis or observational resources are needed. 

MJO forecasting/prediction forms an MJO forecasting/prediction forms an integral part of the Terms of Reference of integral part of the Terms of Reference of the MJO WG, thus there is a subgroup the MJO WG, thus there is a subgroup focussed on this activity.focussed on this activity.

Our first goal has been to develop a Our first goal has been to develop a diagnostic to measure the state of the MJO diagnostic to measure the state of the MJO in forecast models.in forecast models.

This talk summarizes the work of this This talk summarizes the work of this subgroup to date. Apologies for subgroup to date. Apologies for inaccuracies….inaccuracies….

Plan for this talkPlan for this talk

1. Overview of the adopted combined-EOF 1. Overview of the adopted combined-EOF metricmetric

2. Examples of current application at the 2. Examples of current application at the Operational Modelling CentresOperational Modelling Centres

3. Current issues (e.g. precip vs. OLR)3. Current issues (e.g. precip vs. OLR)

4. Verification and a statistical benchmark4. Verification and a statistical benchmark

5. A straw-man recipe5. A straw-man recipe

6. Future plans (e.g. a multi-model 6. Future plans (e.g. a multi-model ensemble)ensemble)

7. Further metrics?7. Further metrics?

1. Overview of the adopted 1. Overview of the adopted combined-EOF metriccombined-EOF metric

Discussions to date have led to the adoption of the Discussions to date have led to the adoption of the so-called Wheeler-Hendon combined EOF index.so-called Wheeler-Hendon combined EOF index.

The problem:The problem: Traditional methods of band-pass Traditional methods of band-pass time filtering introduce end effects, and influence is time filtering introduce end effects, and influence is spread across time.spread across time.

The solution:The solution: Extract the MJO signal by projecting Extract the MJO signal by projecting daily model/analysis/observed fields onto a pair of daily model/analysis/observed fields onto a pair of pre-defined multivariate EOF spatial structures. pre-defined multivariate EOF spatial structures.

MJO spatial structures defined using EOFs of the MJO spatial structures defined using EOFs of the combined fields of 15combined fields of 15S-15S-15N averaged OLR, u850, N averaged OLR, u850, and u200.and u200.

Prior removal of annual cycle and components of Prior removal of annual cycle and components of interannual variability (e.g. ENSO) was required, interannual variability (e.g. ENSO) was required, but still possible in real time.but still possible in real time.

Defined using all seasons of data.Defined using all seasons of data.

Wheeler and Hendon (Mon. Wea. Rev., 2004)Wheeler and Hendon (Mon. Wea. Rev., 2004)

Madden and Julian (1972)Madden and Julian (1972)

The EOFs describe the The EOFs describe the convectively-coupled vertically-convectively-coupled vertically-oriented circulation cells of the oriented circulation cells of the MJO that propagate eastward MJO that propagate eastward along the equator. along the equator.

Monitor the MJO life-cycle in phase Monitor the MJO life-cycle in phase spacespace

Projection on EOF1

Pro

ject

ion o

n E

OF2

An example from An example from observations observations (NCEP/BoM (NCEP/BoM analyses and analyses and satellite OLR)satellite OLR)

Define MJO Phases Define MJO Phases 1-8 for the 1-8 for the generation of generation of composites and composites and impacts studies.impacts studies.

Composites for different seasons demonstrate that Composites for different seasons demonstrate that the all-season index still captures the strong the all-season index still captures the strong seasonality exhibited by the MJO.seasonality exhibited by the MJO.

Dec-Feb CompositeDec-Feb Composite May-June CompositeMay-June Composite

……and impacts and impacts work has been work has been done based on done based on the RMM index.the RMM index.

TC TracksTC Tracks

Rain Event Rain Event ProbabilitiesProbabilities

Example impacts in North Americabased on RMM phases

Signal/Noise for 2 meter air temperatureEight MJO Phases, DJF 1979-2006

Max ~+0.5 sigma => 67% prob > 0 anomaly

2 3 4 5

6 7 8 1

2. Examples of current 2. Examples of current application at the Operational application at the Operational Modelling CentresModelling Centres

Obs/AnalysisObs/Analysis

UK Met UK Met OfficeOffice (Nick (Nick Savage)Savage)

14-day ensemble 14-day ensemble prediction systemprediction system

ForecastsForecasts

Uses same EOFs as WH04.Uses same EOFs as WH04.

For the “observed” trajectory, For the “observed” trajectory, uses their own model uses their own model analyses (incl. for OLR). analyses (incl. for OLR).

Climatologies are computed Climatologies are computed from the NCEP Reanalyses from the NCEP Reanalyses (same as WH04).(same as WH04).

NCEPNCEP (Gottschalck, (Gottschalck, Higgins, L’Heureux, Higgins, L’Heureux, Wang, Vintzileos..Wang, Vintzileos..….)….)

15-day forecasts 15-day forecasts with Global with Global Ensemble Forecast Ensemble Forecast System (20 System (20 members)members)Generated their own EOF Generated their own EOF

structures (but resemble structures (but resemble WH04 very closely).WH04 very closely).

Observed trajectory uses Observed trajectory uses combination of combination of operational analyses and operational analyses and Reanlayses, plus satellite Reanlayses, plus satellite OLR. OLR.

Climatologies from NCEP Climatologies from NCEP Reanalyses.Reanalyses.

Obs/AnalysisObs/Analysis

ForecastForecast

http://www.cdc.noaa.gov/MJO/Forecasts/index_phase.htmlhttp://www.cdc.noaa.gov/MJO/Forecasts/index_phase.html

Currently six operational centres are Currently six operational centres are contributing forecasts of the RMM index.contributing forecasts of the RMM index.

Each are updated daily on the Experimental Each are updated daily on the Experimental MJO Prediction web-site at NOAA/PSD.MJO Prediction web-site at NOAA/PSD.

3. Current issues3. Current issues Each centre currently uses a slightly Each centre currently uses a slightly different recipe.different recipe.

However, the results don’t appear overly However, the results don’t appear overly sensitive to sensitive to this, provided the EOF structures this, provided the EOF structures are the same or are the same or very similar.very similar.

Some centres (e.g. CMC) don’t save Some centres (e.g. CMC) don’t save OLR, and in some models the OLR, and in some models the relationship between OLR and precip is relationship between OLR and precip is not the same as that observed. not the same as that observed. Should Should we be using precip instead?we be using precip instead?

145E190E

90E127E

WH04 EOFs WH04 EOFs using OLR, using OLR, u850, and u850, and u200.u200.

Same, except Same, except using pentad using pentad CMAP rainfall CMAP rainfall instead of instead of daily OLR.daily OLR.

1979-2001 data. 1979-2001 data. No filtering except No filtering except some removal of some removal of low frequencies.low frequencies.

73E

Precip versus OLR issuePrecip versus OLR issue

The computed CMAP/u850/u200 EOFs are shifted The computed CMAP/u850/u200 EOFs are shifted longitudinally compared to the OLR/u850/u200 EOFs! longitudinally compared to the OLR/u850/u200 EOFs! This would result in a relative rotation in the index phase This would result in a relative rotation in the index phase space.space.

However, as the EOFs are a pair, any linear combination However, as the EOFs are a pair, any linear combination of them is equally applicable. So we may perform an of them is equally applicable. So we may perform an orthogonal rotationorthogonal rotation..

EOF1new = (1EOF1new = (1×EOF1 – 1.6×EOF2) / sqrt(1×EOF1 – 1.6×EOF2) / sqrt(122+(1.6)+(1.6)22))EOF2new = (EOF2new = (1.61.6×EOF1 + 1×EOF2) / ×EOF1 + 1×EOF2) / sqrt(1sqrt(122+(1.6)+(1.6)22))

These new CMAP EOFs still remain a pair in quadrature, These new CMAP EOFs still remain a pair in quadrature, but match the phasing of the original OLR EOFs much but match the phasing of the original OLR EOFs much more closely!more closely!

145E 145E

90E 90E

WH04 EOFs WH04 EOFs using OLRusing OLR

CMAP rotated:CMAP rotated:

EOF1-1.6EOF2 EOF1-1.6EOF2

1.6EOF1+EOF21.6EOF1+EOF2

/ / sqrt(1sqrt(122+(1.6)+(1.6)22))

rotated

rotated

So we now have available equivalent So we now have available equivalent precip/u850/u200 EOF structures, if precip/u850/u200 EOF structures, if desired.desired.

But a problem is that there is no real-time But a problem is that there is no real-time daily precipitation dataset available for daily precipitation dataset available for creation of the “observed” part of the phase-creation of the “observed” part of the phase-space trajectory.space trajectory.

Removal of low-frequency variability Removal of low-frequency variability (e.g. ENSO) is not straight-forward, (e.g. ENSO) is not straight-forward, especially for models that have only a especially for models that have only a short history.short history.

Different methods result in a translation in Different methods result in a translation in the RMM Phase space.the RMM Phase space.

……another issueanother issue

4. Verification and a 4. Verification and a statistical benchmarkstatistical benchmark

A statistical benchmark forecast of RMM1 and RMM2 A statistical benchmark forecast of RMM1 and RMM2 can be provided through a first-order vector can be provided through a first-order vector autoregressive model: autoregressive model: (Maharaj and Wheeler, (Maharaj and Wheeler, Int. J. Climatol.,Int. J. Climatol., 2005)2005)

Which provides a very similar forecast to lagged Which provides a very similar forecast to lagged linear regression linear regression (e.g., Jiang et al., (e.g., Jiang et al., Mon. Wea. Rev.,Mon. Wea. Rev., 2007) 2007)

Example Example statistical statistical benchmarbenchmark k forecastsforecasts

Spirals heading Spirals heading around the originaround the origin

The easiest forecast verification statistic The easiest forecast verification statistic to calculate is the correlation between to calculate is the correlation between the forecasted and ‘observed’ RMM1/2 the forecasted and ‘observed’ RMM1/2 values.values.

For the benchmark statistical scheme, the For the benchmark statistical scheme, the correlation skill is correlation skill is 0.520.52 for 15-day forecasts for 15-day forecasts (using forecasts from all seasons, and averaging correlations for (using forecasts from all seasons, and averaging correlations for RMM1/2; Maharaj and Wheeler 2005).RMM1/2; Maharaj and Wheeler 2005).

For the BMRC coupled seasonal forecast model For the BMRC coupled seasonal forecast model (POAMA), the correlation skill is (POAMA), the correlation skill is ~0.5~0.5 at 15 at 15 days days (computed from a comprehensive hindcast set for 1980-(computed from a comprehensive hindcast set for 1980-2005).2005).

PSD model also does worse than the statistical PSD model also does worse than the statistical benchmark - correlation skill for the model is benchmark - correlation skill for the model is ~~0.1 lower0.1 lower than the statistical benchmark at than the statistical benchmark at 14 days.14 days.

DJF Actual DJF Actual Forecast SkillForecast Skill

DJF Potential DJF Potential PredictabilityPredictability

Can also Can also stratify in stratify in many ways.many ways.e.g., from POAMA:e.g., from POAMA:

Can answer questions Can answer questions about “predictability about “predictability barriers” as well.barriers” as well.(e.g. Xianan Jiang has looked at (e.g. Xianan Jiang has looked at this)this)

Courtesy of Harun RashidCourtesy of Harun Rashid

Statistical Model

5. A straw-man recipe5. A straw-man recipeGood arguments exist for standardising the Good arguments exist for standardising the calculations between Centres (e.g., if we want calculations between Centres (e.g., if we want to create a multi-model ensemble).to create a multi-model ensemble).

Initially it has been very informative to see the Initially it has been very informative to see the different calculation and presentation different calculation and presentation strategies of the six Centres currently strategies of the six Centres currently involved, but it is now time to bring the best of involved, but it is now time to bring the best of those strategies together.those strategies together.

This makes all the more sense given our This makes all the more sense given our proposal to WGNE to ask all Operational proposal to WGNE to ask all Operational Centres to contribute.Centres to contribute.

Proposed recipeProposed recipe (either to be computed by each (either to be computed by each

Centre themselves, or by a single volunteering Centre):Centre themselves, or by a single volunteering Centre):

a)a) Use WH04 EOFs Use WH04 EOFs (or equivalent precip (or equivalent precip structures; some Centres may choose to try both)structures; some Centres may choose to try both)

b)b) WH04 normalization factors for each field WH04 normalization factors for each field (OLR=15.1 Wm(OLR=15.1 Wm-2-2, u850=1.81 ms, u850=1.81 ms-1-1, u200=4.81 ms, u200=4.81 ms-1-1))

c)c) All use the same climatology computed All use the same climatology computed from NCEP Reanalyses and observed from NCEP Reanalyses and observed OLR/precip.OLR/precip.

d)d) All use the same methodology for removal All use the same methodology for removal of ENSO and other low-frequency variability of ENSO and other low-frequency variability (i.e., the removal of variability linearly related (i.e., the removal of variability linearly related to an ENSO SST index and removal of mean of to an ENSO SST index and removal of mean of previous 120 days). previous 120 days).

Note that the use of the same Note that the use of the same climatology and estimate of low-climatology and estimate of low-frequency variability will generate a frequency variability will generate a bias in the models from the outset (i.e., bias in the models from the outset (i.e., the forecast initial condition “day 0” will the forecast initial condition “day 0” will not exactly match that from other not exactly match that from other models or the observations).models or the observations).

For verification purposes, this bias could For verification purposes, this bias could potentially be removed by correcting potentially be removed by correcting the model forecast RMM values until the the model forecast RMM values until the “day 0” values (i.e., from the model “day 0” values (i.e., from the model analysis/initial condition), exactly match analysis/initial condition), exactly match the “observed” RMM values on that day.the “observed” RMM values on that day.

Shift forecasts until the “day 0” points (circled in red) correspond exactly to the observations.

Anomaly correlations will then always begin at a value of 1.0 for day 0.

6. Future plans6. Future plans

1. WGNE has offered to disseminate a letter 1. WGNE has offered to disseminate a letter to all other Operational Centres asking them to all other Operational Centres asking them to calculate the proposed metric (hence the to calculate the proposed metric (hence the need for a standard recipe).need for a standard recipe).

2. Multi-model ensemble.2. Multi-model ensemble.

3. Acquisition and dissemination by who? 3. Acquisition and dissemination by who? Currently at NOAA-PSD, but will this move to Currently at NOAA-PSD, but will this move to NCEP? NCEP?

3. Journal article by whole group?3. Journal article by whole group?

7. Further metrics?7. Further metrics?

So far we have concentrated only on So far we have concentrated only on the canonical eastward-propagating the canonical eastward-propagating MJO. A metric designed specifically to MJO. A metric designed specifically to the northward propagation in the Asian the northward propagation in the Asian monsoon would also be desirable.monsoon would also be desirable.

THE ENDTHE END

Indian Ocean

MaritimeContinent

WesternPacific Ocean

WesternHemisphere

Africa

2

3

4

5

6

7

8

1

L

L

L

L

L

HH

H H

H

H

HH

H

L

L L

4-7 daysbetweenphases

Signals:0.2-0.40.4-0.8for > 1index

Matthews and Kiladis, 1999RWD into east Pac

Blade et. al., 2007nonlinear 3 vs. 7

H

MJO’s global teleconnection pattern250 mb , DJF 1979-05, 8 phases, ~27 cases/phase

Weickmann et. al, 1997tilts imply sources/sinks

The global wind oscillation (GWO) is lurking!

LH L

L

Rossby wavedispersion

“fm trough”

RMM2 >0 Phase 6-7 wPac; <0 phase 2-3 IORMM1 >0 Phase 4-5 Indo; <0 phase 8-1 WH

3 m/s

1 m/s

Regression of RMM1 and 2 at initial time Regression of RMM1 and 2 at initial time with week 2 verification, forecast or with week 2 verification, forecast or

forecast errorforecast error

Week 2 forecast Week 2 verification