WP5.5: Evaluation of seasonal-to-decadal scale impact-models forced with downscaled ERA-40, hindcasts and gridded observational datasets.

Andy Morse, University of LiverpoolA.P.Morse@liv.ac.uk

This presentation contains slides from RT5 WP5.5 partners

Andy Morse and Anne Jones, University of Liverpool, Mark Liniger and Christof Appenzeller, MeteoSwiss, Andrew Challinor, Tom Osborne, Julia Slingo and Tim Wheeler, University of Reading,Laurent DUBUS, Clarisse FIL-TARDIEU and Sylvie PAREY, EDFGiampiero Genovese, Fabio Micale, Pierre Cantelaube , Jean-Michel Terres, EU-JRCVittorio Marletto, ARPASimon Mason and Madeleine Thomson, IRI, Columbia University

Presentation Structure

1.Introduction

2. DoW 60 and 18 month

3. Information from Partners

WP6.3 vs. WP 5.5?

WP6.3 production of model runs, issues connected to downscaling and bias correction, use of impacts models

within ensemble prediction system.

WP5.5 is the validation of both the reference forecast data (e.g. ERA-40) and the EPS hindcasts and the validation of the impacts model against reference forecasts, forecasts from other gridded data and real observations e.g. crop yields

Multi-tier validation

Primary Objective

O5.e: Evaluation of the impacts models driven by downscaled reanalysis, gridded and probabilistic hindcasts over seasonal-to-decadal scales through the use of application specific verification data sets.

Scientific/Technical Questions

What’s the quality of impact models at seasonal time scales?

DoW review

WP5.5: Evaluation of seasonal-to-decadal scale impact-models forced with downscaled ERA-40, hindcasts and gridded observational datasets.

Leader: UNILIV (Morse). Participants: UREADMM (Wheeler/Slingo), ARPA-SIM (Marletto), JRC-IPSC (Genovese/Micale), METEOSWISS (Liniger/Appenzeller), LSE (Smith), IRI (Thomson/Mason), EDF (Dubus/ Fil-Tardieu/Parey), DWD (Biermann {Becker}).

WHO (Menne), FAO (Gommes), WINFORMATICS (Norton)

Key Objectives: assess skill-in-hand for a number of impact models

1. Run models appropriately downscaled ERA-40 data and gridded datasets developed in WP5.1

2. Run models with fully downscaled and bias corrected probabilistic ensemble hindcasts at seasonal to-decadal scales from WP6.3.

Models include agriculture models (crop yield models for Europe and the tropics, agri-environmental impact models), infectious tropical disease (malaria and meningitis) and energy demand.

WP5.5 Objectives 60 months

Major milestone 5.5 by month 60

Evaluation of forecast skill of seasonal-to-decadal scale impacts-models when driven with ENSEMBLES EPS.

Expected results and achievements: Assessment of the skill-in-hand for a number of impact models run with downscaled ERA-40 data, gridded observational datasets and fully downscaled and bias corrected probabilistic ensemble hindcasts at seasonal-to-decadal scales.

Expected deliverable: Evaluation report on the forecast skill of seasonal-to-decadal scale impacts-models.

from Morse, Doblas-Reyes, Hoshen, Hagedorn and Palmer (2005) Tellus (in press)

Tiers of Validation

Task 5.5.a: Assess ERA-40 reanalysis key impacts driver variables - parts of Europe, India and Africa. Gridded data from stations and satellites. Tier-0

Task 5.5.b: Evaluate forecast quality key variables probabilistic hindcasts - at appropriate impacts scales - links WP5.3. Key question - capture of seasonal cycles. Links to downscaling and bias correction work from other WPs – and links WP 6.3. Tier-1

Task 5.5.c: Skill of the impacts models driven by corrected (d/s b/c) probabilistic hincasts, compared with reference forecasts from

a. downscaled ERA-40, b. other gridded data sets. Tier 2

Task 5.5.d: Impact model verified against real observations driven by a. ERA-40 reanalysis, b. other gridded data sets and c. probabilistic seasonal hindcasts. Tier-3

WP5.5 Tasks 60 months

Management Issues

Attrition of partners – withdrawal three partners Two due to UN/EU contractual issues One (SME) due to commercial considerations

Use of 6 month report

Communication -monthly or bi-monthly newsletter?

18 Month Plan

Participant id (person-months):

UNILIV (2), UREADMM (9), ARPA-SIM (7), JRC-IPSC (2), METEOSWISS (1), LSE (2), IRI (1), EDF (1), DWD (1),

WHO (3), FAO (1), WINFORMATICS (0).

Objectives

Evaluation of the impacts models driven by downscaled reanalysis, gridded and probabilistic hindcasts over seasonal-to-decadal scales through the use of application

specific verification data sets.

Tasks 18 months

Headlines:

Limited runs used to develop validation systems

Health applications workshop

either evaluation of the state of the art or on setting the agenda for future research

Task 5.5.1: Seasonal application models tested ERA-40 data and selected models run DEMETER hindcasts development of validation systems.

Task 5.5.2: A workshop on use of seasonal probabilistic forecasting for health applications

Deliverables

D5.10: Workshop report on Lessons learned from seasonal forecasting: health protection (month 18)

Milestones and expected result : None in first 18 months

Information from Partners

LiverpoolARPAJRCReadingEDFMeteoswiss

other partners

Temperature

Malaria Model: Temperature dependence

Mosquito survival after Martens (1995)

At T = 25°C sporogonic cycle length = 15.9 days

2.9% survive to infectious stage

Analysis and diagram from Anne Jones

Malaria Model comparison new dynamic and existing rules based models

MARA

Slide 15 of 14

Prevalence = proportion of human population infected with malaria

Mapping Malaria Risk in Africa

Precipitation LT AM UT

Feb 2-4 (MAM) -0.094 -0.009 -0.020

Feb 4-6 (MJJ) -0.012 -0.039 -0.049

Temperature LT AM UT

Feb 2-4 (MAM) 0.080 0.148 0.230

Feb 4-6 (MJJ) 0.104 0.210 0.314

Prevalence LT AM UT

Feb 2-4 (MAM) 0.396 0.461 0.046

Feb 4-6 (MJJ) 0.167 0.289 0.178

Brier Skill Scores Feb 2-4 and 4-6

LT is the lower tercile event, AM the above the median event and UT the upper tercile event

63 ensemble members against a reference forecast made with ERA-40.

Liverpool – malaria model output

Data 1987 to 2002 grid points 17.5S 22.5E, 17.5S 25.0E, 17.5S 27.5E, 17.5S 30.0EAfter Morse et al. (2005) Tellus, in press

Wofost/CRITERIA

Demeter ds output (P, T)

Observations (P, T)

J

JMA

MJ

1

2

3

0

500

1000

1500

2000

2500

3000

3500

4000

4500

01/01/93 31/01/93 02/03/93 01/04/93 01/05/93 31/05/93

reference runs

4000

5000

6000

7000

8000

9000

1986 1988 1990

0.05

0.1

0.25

0.5

0.75

0.9

0.95

4000

5000

6000

7000

8000

9000

1986 1988 1990

0.05

0.1

0.25

0.5

0.75

0.9

0.95

4000

5000

6000

7000

8000

9000

1986 1988 1990

0.05

0.1

0.25

0.5

0.75

0.9

0.95

Demeter evaluation runs

General layout of the hindcast evaluation tests

N(y-1)………J0

Vittorio Marletto, ARPA

Figure xB. Box and whiskers distributions of predicted potential wheat yields (kg/ha) simulated using downscaled DEMETER daily hindcasts for the years 1977-1987 in a location near Modena. In the top graph the crop model was provided with observed weather data up to the 31st of March and supplemented with DEMETER downscaled hindcasts up to harvest date (end of June in Northern Italy). The above procedure was repeated but with observed data up to the 30th of April (centre) and to the 31st of May (bottom). DEMETER data refer to four models (CNRM, UKMO, SCWF, SMPI), 9 members and 2 downscaling replicates obtained by DMI using a weather generator. Potential wheat yields simulated by the Wofost/Criteria model with observed weather data only are also provided for comparison.

Vittorio Marletto, ARPA

JRC crop modelling results• Average (1995-1998) European weighted percentage error wheat yields hybrid JRC modelling system and DEMETER full growing season crop model estimation.

• JRC hybrid WOFOST based crop model run observed meteorological data (to date) with crop yield statistical estimation crop growth indicators.

• DEMETER ensemble driven WOFOST forecast February start date 180 days 63 members. Results Portugal excluded systematic bias. Results weighted on proportional contribution each member state.

MODEL RUN WEIGHTED YIELD ERROR (%)

± STANDARD ERROR

JRC February 7.1 ± 0.9

JRC April 7.7 ± 0.5

JRC June 7.0 ± 0.6

JRC August 5.4 ± 0.5

DEMETER (Feb. start) 6.0 ± 0.4

• Percentage error obtained DEMETER end of February lies between average error end June & August JRC operational system.

• Demonstrates ability DEMETER make forecast earlier in season than current methods.

After Cantelaube P., Terres J.M. (2005) Tellus submitted

Ensembles WP5.5. Simulation of the yield of groundnuts across India using the GLAM crop model and the ECMWF ERA40 reanalysis

from Challinor et al., in pressAndrew Challinor, Tom

Osborne,

Julia Slingo, Tim Wheeler

Correlations between observed yield (left), or modelled yield (right),

and ERA40 rainfall May-Nov. Significance is shown by dots

RT5 : EDF/R&D

Laurent DUBUSClarisse FIL-TARDIEU

Sylvie PAREY

EDF/R&D topics of interest•Weather forecasts (RT5 contribution)

Main aim : production facilities management at time scales

•Daily to weekly

•Monthly to annual

•Decadal

Needs and contribution

•Spatial and temporal downscaling methods

•Area of interest : France and Europe

•Tests on electricity demand forecasting

• Climate change (Interest for RT4 results, participation?)

Impact on mean climate

•Electricity production and consumption

Impact on meteorological extremes

•Dimensioning

•Crisis management

MeteoSwiss activities for WP5.5:Temperature based application models

Studer, S., Appenzeller, C., Defila, C, 2005:Clim. Change, in press.Liniger, Appenzeller 2005

Spr

ing

phas

e (r

ed/b

lue)

Example:Modeling swiss main variability in spring onset using growing degree days

days

daily TTGDD 0,max

DWD

Kai Biermann, DWD and René Jursa, Institute of Solar Energy Technology (ISET) e.V. Kassel

DEMETER ensemble members and wind power predictions linked by linear regression model

LSE - Lenny Smith

Other Partners

Key linkages

From WP 6.3 for

Timely and easy access to data – already in discussion RT1, RT2a, RT2b, RT3 and RT5

• Effective use of downscaling tools – already in discussion with RT2B, RT3 and RT2a

• RT5 for validation – with WP6.3 and WP5.5 directly linked

• Interest in RT4 findings

• RT7 economic impacts

WP 5.5 specific WP 5.1 gridded data Europe, WP 5.3 forecast quality and WP5.2 systematic errors, WP 5.4 extreme events, RT7 economic impacts.

Possible links WP 4.3 extreme weather conditions, WP 4.2 Regional Climate Change – based on known impacts model thresholds etc.

Questions?

Malaria Modelbackground to disease

• Malaria kills more than 2,000,000 people per

year

• 90% deaths sub-Saharan Africa

-mostly children

• Mechanisms of the disease known for over

100 years

• Anopheline mosquitoes and parasite

Plasmodium spp. with P. falciparum most

dangerous and cause of African epidemics

Malaria Model malaria life cycle

Slide 30 of 14

sporogonic cycle:

temperature dependent

biting/laying:

temperature dependent

larval stage:

rainfall dependent

Cost-loss Ratios and Potential Economic Value – malaria transmission simulation

Season MAM: Lead 2 to 4 months (Morse et al.2005 Tellus, in press)

Data 1987 to 2002 grid points 17.5S 22.5E, 17.5S 25.0E, 17.5S 27.5E, 17.5S 30.0E

Upper tercile prevalence

Infectious Disease and Epidemics

• Many infectious diseases, in the tropics, have a strong seasonal cycle related to the seasonal

climatic cycles

• Climatically anomalous years can lead to epidemics

• Time between trigger threshold to epidemic peak often too short to take effective

intervention – need for skilful and timely seasonal climate forecast

Epidemic Cycle

0

20

40

60

80

100

120

140

97

45

97

48

97

51

98

02

98

06

98

09

98

12

98

15

98

18

98

21

98

24

98

27

98

30

98

33

98

41

98

44

98

47

Reporting week

Nu

mb

er o

f ca

ses

Vaccine

Threshold

Effect

Selected Recent Papers

Molesworth, A.M., Cuevas, L.E., Connor, S.J., Morse A.P., Thomson, M.C. (2003).Environmental risk and meningitis epidemics in Africa, Emerging Infectious Diseases, 9 (10), 1287-1293.

Hoshen, M.B., Morse, A.P. (2004) A weather-driven model of malaria transmission, Malaria Journal, 3:32 (6th September 2004)  doi:10.1186/1475-2875-3-32 (14 pages)

Morse, A.P., Doblas-Reyes, F., Hoshen, M.B., Hagedorn, R.and Palmer, T.N. (2005) A forecast quality assessment of an end-to-end probabilistic multi-model seasonal forecast system using a malaria model, Tellus A, ( in press)

Sensitivity TestingInput Data

• Varied driving data to investigate sensitivity of model to magnitude and timing of seasons

• Interesting results were obtained by progressively lagging the temperature time series with respect to the rainfall time series

• For southern Africa very sensitive to temperature

• Relative timing is importantRainfall and temperature climatology for S28 grid point

a) peak prevalence and b) mosquito population as a function of lag for grid point S28

Temperature (degC)

10.0015.0020.0025.0030.0035.0040.00

1 61 121

181

241

301

361

421

481

541

601

661

721

781

841

901

961

1021

1081

1141

1201

1261

1321

1381

Dekadal Rainfall (mm)

0.0020.0040.0060.0080.00

100.00

1 61 121

181

241

301

361

421

481

541

601

661

721

781

841

901

961

1021

1081

1141

1201

1261

1321

1381

0.0000

0.1000

0.2000

0.3000

0.4000

0.5000

0.6000

0.7000

0 30 60 90 120 150 180 210 240 270 300 330 360

Lag (days)

Pea

k P

reva

len

ceP

ea

k P

reva

len

ce

Lag (days)a)

25000

27000

29000

31000

33000

35000

37000

39000

41000

0 30 60 90 120 150 180 210 240 270 300 330 360

Lag (days)

Max

MP

ea

k M

osq

uito

Po

pu

latio

n

Lag (days)b)

Analysis and diagrams from Anne Jones

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

5 10 15 20 25 30 35 40

Temperature (deg C)

Su

rviv

al P

rob

abil

ity

P

40% 60% 80% 100% Martens Lindsay-Birley

Processing and diagram from Anne Jones

Data source Bayoh, M. N., 2001 unpublished Ph.D.

Adult mosquito daily survival as function of temperature

The LagExample – W12 grid point climatology

Prevalence

0.0000

0.2000

0.4000

0.6000

0.8000

1.0000

361 421 481 541 601 661 721

No. Infectious Mosquitoes

0.0

0.5

1.0

1.5

2.0

361 421 481 541 601 661 721

Hu

nd

red

s

Population ratio M/N

0

1

2

3

4

5

361 421 481 541 601 661 721

Th

ou

sa

nd

s

Dekadal Rainfall (mm)

0

20

40

60

80

100

361 421 481 541 601 661 721

Daily Incidence

0

0.05

0.1

361 421 481 541 601 661 721

Daily Incidence

0

0.05

0.1

361 421 481 541 601 661 721

Mosquito infection probability=1

(Unlimited human reservoir)-> immunity /chronic infection component in model (infectious until next rainy season)

???

West Africa – malaria is rainfall driven

• mosquito population takes a long time to get infected from the small human reservoir

• limited secondary infections

Analysis and diagram from Anne Jones

Precipitation LT AM UT

Feb 2-4 (MAM) -0.094 -0.009 -0.020

Feb 4-6 (MJJ) -0.012 -0.039 -0.049

Temperature LT AM UT

Feb 2-4 (MAM) 0.080 0.148 0.230

Feb 4-6 (MJJ) 0.104 0.210 0.314

Prevalence LT AM UT

Feb 2-4 (MAM) 0.396 0.461 0.046

Feb 4-6 (MJJ) 0.167 0.289 0.178

Brier Skill Scores Feb 2-4 and 4-6 LT is the lower tercile event, AM the above the median event and UT the upper tercile event

After Morse et al. 2005

DEMETER Seasonal Forecasting System

• EU FP5 DEMETER – multi-model ensemble system www.ecmwf.int/research/demeter

• Seven modelling groups running AOGCMs in full forecast mode, 4 start dates per year running out to 6 months, hindcasts 1959 to 2000

• Data available from data.ecmwf.int/data/

• EU FP6 ENSEMBLES www.ensembles-eu.org DEMETER - hindcast biases

WP5.5 Evaluation of seasonal-to-decadal scale impact-models forced with downscaled ERA-40, hindcasts and gridded observational datasets. UNILIV (Morse), WHO (Menne), UREADMM (Slingo), ARPA-SIM (Marletto), JRC-IPSC (Genovese), METEOSWISS (Appenzeller), LSE (Smith), FAO (Gommes), IRI (Thomson), WINFORMATICS (Norton), EDF (Dubus), DWD (Becker).

First 18 months:Seasonal application models will be tested with ERA-40 data and (selected models) with DEMETER forecasts to commence development of validation systems (requires downscaled ERA-40 and DEMETER data and bias corrected DEMETER data) working on Tier-2 (ERA-40 reference forecast) and Tier-3 (full validation) validation systems.

Workshop on the use of seasonal probabilistic forecasting for health applications either 1. evaluation of the state of the art or 2. on setting the agenda for future research

Beyond:For fields of interest at temporal and spatial scales of interest to impacts modellers- the validation of ERA-40 data against other gridded data as available, Tier-1 validation of DEMETER (downscaled) variables ERA-40 and other gridded data sets, impacts models driven with ENSEMBLES seasonal-to-decadal forecasts on Tier-2 (reference forecast) validation and Tier-3 (real observations –e.g. crop yield) validations

ARPA crop modelling results

• Wheat yields 1977-1987, Modena, Italy.

• 72 ensembles (4 models (x9) x2) downscaling replicates

• WOFOST based crop model observed data to 31st March and onwards with DEMETER hindcasts to harvest date (end June)

• Box (IQR) whiskers (10th & 90th percentiles)

• Observed weather simulation (control) solid triangle

• Climatology based run hollow circle.After Marletto et al. (2005) Tellus submitted