sreps priority project cosmo general meeting cracov 2008 sreps priority project: final report c....

SREPS Priority Project COSMO General Meeting Cracov 2008

SREPS Priority Project:final report

C. Marsigli, A. Montani, T. Paccagnella

ARPA-SIM - HydroMeteorological Service of Emilia-Romagna, Bologna, Italy

F. Gofa, P. Louka

HNMS – Hellenic National Meteorological Service, Athens, Greece

http://www.hnms.gr/hnms/greek/index_html


This presentation is complemented by the posters:

On the impact of varying the value of a set of COSMO model parameters for short-range predictions. C. Marsigli, A. Montani, and T. Paccagnella

COSMO SREPS verification over Greece P. louka, F. Gofa and T Charantonis


Outline

Brief recall about the COSMO-SREPS methodology

& system set-up

Analysis of the results on MAP D-PHASE OP Multi Model ICs/BCs vs different Physics Parameter perturbations

Relationship between error and spread

Conclusions


COSMO-SREPS methodology

i.c. and b.c. perturbations -> INM multi-model multi-boundary ensemble (SREPS)

LAM perturbations -> physics parameter perturbations

COSMO model perturbations - smaller scale errors

driving model perturbations (ics and bcs) - larger scale errors


INM

SREPS

25 km

COSMO

SREPS

10 km

COSMO

DE EPS

2.8 km


COSMO at 25 km on IFS

IFS – ECMWF globalb

y I

NM

S

pain

COSMO at 25 km on GME

GME – DWD global

COSMO at 25 km on UM

UM – UKMO global

COSMO at 25 km on GFS

GFS – NCEP global


System set-up

16 COSMO runs 10 km

hor. res.40 vertical

levels

COSMO at 25 km on IFS

IFS – ECMWF globalb

y I

NM

S

pain

COSMO at 25 km on GME

GME – DWD global

COSMO at 25 km on UM

UM – UKMO global

COSMO at 25 km on GFS

GFS – NCEP global

P1: control (ope)

P2: conv. scheme (KF)

P3: tur_len=1000

P4: pat_len=10000

00 UTCJJA=53 SON=54


run nr.parameter name parameter description range default used1 ctrl ope2 lconv convection scheme T or KF T KF3 tur_len maximal turbulent length scale [100,1000] m 500 1504 tur_len maximal turbulent length scale [100,1000] m 500 10005 pat_len length scale of thermal surface patterns [0,10000] m 500 100006 rat_sea ratio of laminar scaling factors for heat over [1,100] 20 17 rat_sea ratio of laminar scaling factors for heat over [1,100] 20 608 qc0 cloud water threshold for autoconversion [0.,0.001] 0 0.0019101112

13 c_lnd

surface area density of the roughness elements over land [1,10] 2 1

14 c_lnd

surface area density of the roughness elements over land [1,10] 2 10

15 rlam_heat scaling factor of the laminar layer depth [0.1,10] 1 0.116 rlam_heat scaling factor of the laminar layer depth [0.1,10] 1 10

crsmincrsminc_soilc_soil

Minimal stomata resistance 15015011 2

0

50200Minimal stomata resistance

Surface area index of the evaporating soilSurface area index of the evaporating soil

[50,200] s/m[50,200] s/m

]0,c_lnd[

]0,c_lnd[


testing period

COSMO-SREPS was running regularly during the MAP D-PHASE DOP, at 00 UTC


HNMS

Verification(06-12/2007)Domain: Greece

Data used: SYNOP data covering Greece

Parameters verified:

o 2m temperatureo Mean Sea Level

Pressure (MSLP)o Precipitation

• Deterministic indeces

• Probabilistic indeces

Verification

JJA 2007

SON 2007

Domain: Alpine region

Data used:COSMO data

Synop data

COSMO I7 analyses

Analysis of:Multi Model ICs/BCs vs different Physics Parameter perturbations

•Z500/ t850/ T2m

•Precipitation

•Deterministic indeces

•Probabilistic indeces

Relationship between error and spread

ARPA-SIM


Obs. Network used by ARPA-SIM

SON=54 runsJJA=53 runs

COSMO observations218 synop stations on the

Alpine area


Greek SYNOP stations


Some remarks….

Alpine Region and Greece ! Two regions very different but both complex!

Different climatology

few and less intense precip. events in Greece (this is why Dec has been included for Greece)

summer 2007 very very hot in Greece


intra-group distance

Z500

CO

SM

O a

naly

sis

JJA 2007 - 50 days SON 2007 - 49 days

Same driving model

Different model parameters

Same model parameters

Different driving model

MultiModel ICs/BCs vs different Physics Parameter perturbations



Z500

CO

SM

O a

naly

sis

JJA 2007 - 50 days SON 2007 - 49 days

Same driving model

Different model parameters



MultiModel ICs/BCs vs different Physics Parameter perturbationsT8

50 C

OSM

O a

naly

sis

C

Different driving models play the major role



2mT Northern Italy

JJA 2007 - 50 days

SON 2007 - 49 days





As regards 2mT no big impact of physics parameter perturbations over Greece



SYNOP over D-PHASE area - Nearest grid point

JJA07 SON07

2m T - relationship between error and spread

COSMO I7 analysis

error vs spread Underdispersive


spread/skill relationship

2mT Alpine area (synop stations)

+12h +24h +36h

+48h +60h +72h

Reasonable even if Underdispersive


spread/skill relationship

t850 Alpine area (COSMO analyses)

Improvement with the time range

+12h +24h +36h

+48h +60h +72h


-0.80

-0.40

0.00

0.40

0.80

1.20

1.60

2.00

2.40

0 3 6 9 12 15 18 21 24 27 30 33 36 39 42 45 48 51 54 57 60 63 66 69 72

forecast range (h)

bia

s (K

)

m1

m2

m3

m4

m5

m6

m7

m8

m9

m10

m11

m12

m13

m14

m15

m16

2m temperature - BIASJJA07

Northern Italy

ecmwfgmencepukmo

T2m: BIAS

-2.5-2

-1.5-1

-0.50

0.51

1.52

2.5

6 12 18 24 30 36 42 48 54 60 66 72Forecast time

12345678

910111213141516avg

IFS GME NCEP UKMO

All periodGreece

The two regions are very

different!!

p4


2m Temperature by months

• Underestimation of summer temperatures June 2007 was exceptionally warm

with strong heat waves

The maximum temperature in Athens reached 46.2 ºC !

• October to December RMSE statistically acceptable.

• Different pattern between summer and autumn/winter months

-6

-5

-4

-3

-2

-1

0

1

2

6 12 18 24 30 36 42 48 54 60 66 72

Forecast time

BIA

S

Jun Jul Aug Sep Oct Nov Dec

0

1

2

3

4

5

6

6 12 18 24 30 36 42 48 54 60 66 72

Forecast time

RM

SE

Jun Jul Aug Sep Oct Nov Dec

Strong dependency on the season !


2m Temperature By Member

• Underestimation of Temperature.

• It seems that GME driven members have better skill than the others.

T2m: RMSE

1

1.5

2

2.5

3

3.5

4

4.5

6 12 18 24 30 36 42 48 54 60 66 72

Forecast time

12345678910111213141516avg

IFS GME NCEP UKMO

T2m: BIAS

-2.5-2

-1.5-1

-0.50

0.51

1.52

2.5

6 12 18 24 30 36 42 48 54 60 66 72Forecast time

12345678

910111213141516avg


GME seems to give a better performance regarding t2m (near surface parameter)


-0.80

-0.40

0.00

0.40

0.80

1.20

1.60

2.00

2.40

0 3 6 9 12 15 18 21 24 27 30 33 36 39 42 45 48 51 54 57 60 63 66 69 72

forecast range (h)

bia

s (K

)

m1

m2

m3

m4

m5

m6

m7

m8

m9

m10

m11

m12

m13

m14

m15

m16

2m temperature - BIAS

Synop observations over Alpine area

Nearest grid point – lsm + altitude correction

JJA07

ecmwfgmencepukmo

p4


2m temperature - MAE



JJA07

2.60

3.00

3.40

3.80

4.20

0 3 6 9 12 15 18 21 24 27 30 33 36 39 42 45 48 51 54 57 60 63 66 69 72

forecast range (h)

mae

(K

)

m1

m2

m3

m4

m5

m6

m7

m8

m9

m10

m11

m12

m13

m14

m15

m16

ecmwfgmencepukmo

p4


2m temperature - BIAS



SON07

-3.20

-2.80

-2.40

-2.00

-1.60

-1.20

-0.80

-0.40

0.00

0.40

0 3 6 9 12 15 18 21 24 27 30 33 36 39 42 45 48 51 54 57 60 63 66 69 72

forecast range (h)

bia

s (K

)

m1

m2

m3

m4

m5

m6

m7

m8

m9

m10

m11

m12

m13

m14

m15

m16ecmwfgmencepukmo

p4


2m temperature - MAE



SON07

2.60

3.00

3.40

3.80

4.20

0 3 6 9 12 15 18 21 24 27 30 33 36 39 42 45 48 51 54 57 60 63 66 69 72

forecast range (h)

mae

(K

)

m1

m2

m3

m4

m5

m6

m7

m8

m9

m10

m11

m12

m13

m14

m15

m16

ecmwfgmencepukmo

p4


TP 24h – ave 0.5x0.5 JJA07

+30h

noss 1400 700 400 150 50

IT + CH

+54h

Same driving models

Same driving models

Same perturbation

Big impact of multi model BCs!!!!!!!

Looking at the right column it is evident that even with few members the skill does not decrease too much when the driving models are different.

Same perturbation


TP 24h – ave 0.5x0.5

noss 800 500 300 100 50

SON07

+30h

IT + CH

+54h

Same driving model

Same driving model

Same perturbation

Same perturbation


June-December 200748hr pred

0.5

0.6

0.7

0.8

0.9

1

0 10 20 30 40

Threshold (mm/24hr)

RO

C

All mbs

Tdtk1

KF1

Tdtk2

Tdtk3


0.5

0.6

0.7

0.8

0.9

1

0 10 20 30 40

Threshold (mm/24hr)

RO

C

All mbs

IFS

GME

NCEP

UKMO

Same driving model Same Phys. Pert.


-0.8

-0.6

-0.4

-0.2

0

0.2

0.4

0.6

0 10 20 30 40

Threshold (mm/24hr)

BS

S

All mbs

IFS

GME

NCEP

UKMO


-0.5

-0.4

-0.3

-0.2

-0.1

0

0.1

0.2

0.3

0.4

0.5

0 10 20 30 40

Threshold (mm/24hr)

BS

S

All mbs

Tdtk1

KF1

Tdtk2

Tdtk3

Greece

KF better but!!


POD & FAR plots (6hr acc prec- Greece)Same driving model Same Phys. Pert.

POD: >0.1mm

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

6 12 18 24 30 36 42 48 54 60 66 72

Forecast time

12345678910111213141516

POD: >0.1mm

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

6 12 18 24 30 36 42 48 54 60 66 72

Forecast time

12345678910111213141516

FAR: >0.1mm

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

6 12 18 24 30 36 42 48 54 60 66 72

Forecast time

12345678910111213141516

FAR: >0.1mm

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

6 12 18 24 30 36 42 48 54 60 66 72

Forecast time

12345678910111213141516

IFS GME NCEP UKMO

Tiedtke1Kain-Fritsch1Tiedtke2Tiedtke3

Greece


Deterministic scores – ave 0.5 x 0.5 IT

pert

1mm/24h

5mm/24h

10mm/24h

SREPS Priority Project COSMO General Meeting Cracov 2008 JJA07 +30

h

IT

IT + CH

Same driving model

Same perturbation


Test of more parameter perturbations (same father)

16 LM runs at 10 km

P1: control (ope)

P2: conv. scheme (KF)

P3: parameter 1

P4: parameter 2

P5: …

IFS – ECMWF global

SON 07

On the impact of varying the value of a set of COSMO model

parameters for short-range predictions.

C. Marsigli, A. Montani, and T. Paccagnella


T2m deterministic scores – npo IT

-2.40

-2.00

-1.60

-1.20

-0.80

-0.40

0.00

0.40

0 3 6 9 12 15 18 21 24

forecast range (h)

bia

s (K

)

-2.40

-2.00

-1.60

-1.20

-0.80

-0.40

0.00

0.40

0 3 6 9 12 15 18 21 24

m1

m2

m3

m4

m5

m6

m7

m8

m9

m10

m11

m12

m13

m14

m15

m16

ctrl

KFtur_len=150tur_len=1000pat_len=10000rat_sea=1rat_sea=60qc0=0.001crsmin=50crsmin=200c_soil=0c_soil=2c_lnd=1c_lnd=10rlam_heat=0.1rlam_heat=10


2.00

2.10

2.20

2.30

2.40

2.50

2.60

2.70

2.80

0 3 6 9 12 15 18 21 24

forecast range (h)

mae

(K

)

T2m deterministic scores – npo IT

-2.40

-2.00

-1.60

-1.20

-0.80

-0.40

0.00

0.40

0 3 6 9 12 15 18 21 24

m1

m2

m3

m4

m5

m6

m7

m8

m9

m10

m11

m12

m13

m14

m15

m16

ctrl

KFtur_len=150tur_len=1000pat_len=10000rat_sea=1rat_sea=60qc0=0.001crsmin=50crsmin=200-2.40

-2.00

-1.60

-1.20

-0.80

-0.40

0.00

0.40

0 3 6 9 12 15 18 21 24

m1

m2

m3

m4

m5

m6

m7

m8

m9

m10

m11

m12

m13

m14

m15

m16

c_soil=0c_soil=2c_lnd=1c_lnd=10rlam_heat=0.1rlam_heat=10


Conclusions

Alpine Region and Greece ! Two regions very different but both complex!

Different climatology


Conclusions (Temperature and geopotential)

Perturbations -Multi Model ICs/BCs & Perturbations on Ph. Params:

the perturbing of model parameters produce a detectable spread among members

but

the use of different driving models seems to dominate with respect to physics parameter perturbations as regards the contribution to the spread; these contributions are different in the two seasons (2mT)

As regards the performance of the different driving models there is a strong dependence on forecast range, season, verification area


Perturbations -Multi Model ICs/BCs & Perturbations on Ph. Params:

the perturbing of model parameters produce a detectable spread among members

but

the use of different driving models seems to dominate with respect to physics parameter perturbations as regards the contribution to the spread

As regards the performance of the different driving models there is a strong dependence on forecast range, season, verification area



spread-skill relationship:

a correlation between error and spread exists, but the

system is under-dispersive -> a more comprehensive representation of model error will improve spread



Conclusions (precipitation)

•the use of different driving models seems to dominate with respect to physics parameter perturbations as regards the contribution to the spread

•It is not evident a systematic relationship between the quality of the precipitation forecast and ICs/BCs provided by a specific global model

•Perturbations of the convective schemes are more important than the perturbations of the particular parameters for turbulent and length scales used

•It seems that the members with Kain-Fritsch convective scheme have better resolution they overestimate precipitation more than Tiedtke scheme

•Precipitation amount is generally overestimated


On-going activities and future plans

continue the analysis over the DOP MAP D-PHASE :

statistical analysis of the system

comparison with the other available mesoscale ensemble systems (D-PHASE activity)

verification carried out by HNMS

introduce the new parameter perturbations (after an analysis of their impact also in summer)

analyse the impact of adding soil perturbations

consens


Last FTEs of Chiara were used for this

TASK

ROMEO

sreps priority project cosmo general meeting cracov 2008 sreps priority project: final report c....

Documents

perturbations inm

set of cosmo model parameters

lndtesting period cosmosreps

cosmosreps methodology

lndsurface area density

laminar layer depth0

gmegme dwd globalcosmo

umum ukmo globalcosmo