inm, madrid, december 2003 single column experiments at ecmwf, status of work, and plans for 2003...
DESCRIPTION
INM, Madrid, December 2003 Introduction Assimilation (OI vs. KF and synthetic mw Tb) Assimilation of observed mw Tb Additional experiments Development of production system Remaining work (Scientific) conclusions LayoutTRANSCRIPT
INM, Madrid, December 2003
Single column experiments at ECMWF, status of work, and plans for 2003
Pedro Viterbo and Gisela SeuffertEuropean Centre for Medium-Range Weather Forecasts
ELDAS 2nd Progress MeetingINM, Madrid, 10-11 December 2003
Thanks to: J.F. Mahfouf, H. Wilker, M. Drusch, and J.-C. Calvet
INM, Madrid, December 2003
• Introduction• Assimilation (OI vs. KF and synthetic mw Tb)• Assimilation of observed mw Tb• Additional experiments• Development of production system• Remaining work• (Scientific) conclusions
Layout
INM, Madrid, December 2003
• Introduction• Assimilation (OI vs. KF and synthetic mw Tb)• Assimilation of observed mw Tb• Additional experiments• Development of production system• Remaining work• (Scientific) conclusions
Layout
INM, Madrid, December 2003
Goals
• Build a soil moisture analysis system assimilating – 2m temperature and relative humidity– Thermal IR heating rates– MW brightness temperature
• Forced by observation based estimates of– Precipitation and radiation
• Study its properties and compare to OI in a controlled environment
• Build the production system
INM, Madrid, December 2003
Plans ( ELDAS 1st progress meeting)Assimilation aspects:• Minimize the combined errors in prediction of soil moisture, latent heat flux and
screen level observations
• Further mw-Tb assimilation experiments (viewing angle, times)
• Assimilation of heating rates
Reports:• Paper(s) focusing on the
- new features of assimilation method - assimilation of mw-Tb
- (assimilation of heating rates)
Technical aspects:• Summer 2003: Build production system for the annual data base
• End of 2003: Start production
Action: Completed using SCM
Action: Building …
Action: SCM test runs
Action: 2 papers-published in GRL (T,RH,Tb)-Accepted at JHM (OI, EKF)
Action: Still pending
INM, Madrid, December 2003
Soil moisture analysis systems Optimal Interpolation:• Used in the operational ECMWF-
forecast since 1999 (Douville et al., 2000)
• Fixed statistically derived forecast errors
• Criteria for the applicability of the method- atmospheric and soil exceptions- corrections when T and RH error are negatively correlated
Extended Kalman Filter:• Used in the operational DWD-
forecast since 2000 (Hess, 2001) *
• Updated forecast errors
• Criteria for the applicability of the method- no ‘direct’ atmospheric exceptions- soil exceptions still to be tested
* Changes:- Assimilation of 2m- T and RH, mw-Tb- Model forecast operator accounts for water
transfer between soil layers- Test adaptive EKF
INM, Madrid, December 2003
SCM experiment DesignAtm. initial conditions +dynamics forcing from
ECMWF reanalysis (ERA40)
Single-column model of theECMWF NWP model
+ microwave emissivity model
First guess: T2m,RH2m,HR(?)
Soil moisture analysis schemeOI or Extended Kalman Filter
Soil moisture Background error
Increments (daily)
Observations: T2m,RH2m,HR
Observation of precipitation + radiation
INM, Madrid, December 2003
ObservationsMurex:• 1.6 – 9.10.1997 (1995- 1998)• Forcing:
SW , (unbiased) LW , precipitation• Validation:
Soil Moisture, Rnet, H, G, LE=Rnet-H-G, Ts• Assimilation/Validation:
T2m, RH2m, synthetic mw-Tb
SGP 97:- 15.6 – 19.7.1997 - Little Washita site (2) (Central Facility site(3))- Forcing: SW , (simulated) LW , precipitation- Validation: Soil Moisture, Rnet, H, G, LE, Ts- Assimilation/Validation: T2m, RH2m, mw-Tb
INM, Madrid, December 2003
• Introduction• Assimilation (OI vs. KF and synthetic mw Tb)• Assimilation of observed mw Tb• Additional experiments• Development of production system• Remaining work• (Scientific) conclusions
Layout
INM, Madrid, December 2003
OI vs KF: Gain matrix (FIFE)
3
1
6
1
2)]([l m
lmbba kFNxHyKxx
160 180 200 220 240 260 280julian day
0.0000
0.0005
0.0010
0.0015
0.0020
Frob
eniu
s no
rm
OIEKF•Both systems distinguish between periods off strong and weak
influence of soil moisture on screen-level variables.•OI does that thanks to carefully selected thresholds; EKF has built-in dynamic dependency•In clear-sky, the FN of EKF is slightly larger than that of OI: EKF has a slight preference for the obs, rather than the bg.
INM, Madrid, December 2003
OI vs KF: Average increments (FIFE)
0.00
0.05
0.10
0.15
0.20
0.25
0.30
aver
aged
soi
l moi
stur
e in
crem
ents
[%]
soil layer 1 soil layer 2 soil layer 3
OI
EKF
INM, Madrid, December 2003
OI vs KF: Time series of increments (FIFE)
160 180 200 220 240 260 280julian day
-1.5-1.0
-0.5
0.0
0.5
1.01.5
soil
moi
stur
e in
crem
ents
[%]
soil layer 1soil layer 2soil layer 3
a)
160 180 200 220 240 260 280julian day
-1.5-1.0
-0.5
0.0
0.5
1.01.5
soil
moi
stur
e in
crem
ents
[%]
soil layer 1soil layer 2soil layer 3
b)
•EKF increments are at the same order of magnitude of OI, and come at the same time•OI has soil moisture increments similar across the 3 layers.•EKF puts more weight on the deeper layers
INM, Madrid, December 2003
Soilmoisture
Sensible Heat flux
(Synthetic) microwave Tb assimilation (MUREX)
INM, Madrid, December 2003
Daytime fit to observations (MUREX)
Corr. Bias RMS
Control 0.92 -1.80 1.81
KTRB 0.92 -1.52 1.82
KTR 0.92 -1.28 1.82
KB 0.92 1.15 1.88
2 metre temperature
Corr. Bias RMS
Control 0.79 0.44 11.86
KTRB 0.79 -1.98 11.75
KTR 0.82 -4.01 11.34
KB 0.80 -4.94 12.04
2 metre relative humidity
•The control simulation has a cold and wet bias, but hardly any bias on sensible heat flux, (and a wet bias in root zone moisture).•The assimilation of screen-level parameters tends to reduce the cold/wet bias, reducing soil moisture (moving away from observations), and giving too much sensible heat.•EKF tends to follow mainly 2T information, in detriment of 2RH.•The assimilation of mw Tb moves the root zone moisture closer to the (ground-based) observations
INM, Madrid, December 2003
Surface soil moisture
Microwave Tb
Soil moisture, Teff, mw-Tb at 6 LT
•Assimilation of mw Tb (on its own or combined with screen level parameters) brings the simulated Tb and the top soil moisture closer to the observations.
INM, Madrid, December 2003
• Introduction• Assimilation (OI vs. KF and synthetic mw Tb)• Assimilation of observed mw Tb• Additional experiments• Development of production system• Remaining work• (Scientific) conclusions
Layout
INM, Madrid, December 2003
Assimilation of mw Tb: Performance of 2T/RH
170 175 180 185 190 195 200julian day
290
295
300
305
310da
ily m
ean
2m-te
mpe
ratu
re [K
]a)OBS
CTRLKTRKTRBKB
Corr Bias RMS0.88 2.37 1.430.90 1.53 1.090.90 1.44 1.080.87 1.99 1.39
170 175 180 185 190 195 200julian day
30405060708090
100
daily
mea
n 2m
-rel
ativ
e hu
mid
ity [%
]
b)
OBSCTRLKTRKTRBKB
Corr Bias RMS0.57 -10.70 8.930.57 -5.67 7.180.58 -4.71 7.030.56 -7.41 8.37
2T
2RH
SGP97
INM, Madrid, December 2003
Surface soil moisture and Tb (SGP97)
170 175 180 185 190 195 200julian day
210220230240250260270280
brig
htne
ss te
mpe
ratu
re [K
]a)
Estar Obs
CTRL
KTRKTRB
KB
170 175 180 185 190 195 200julian day
10
20
30
40
vol
umet
ric s
oil m
oist
ure
[%]
b)Grav. Obs (5cm) with error barsObs 10cmderived from Estar
Tb
Top
soil
mo i
s tu r
e
•The control simulation (indeed, all simulations) are too warm and too dry. Model day-to-day variability of humidity exceeds observations.•Top soil moisture in the control simulation compares well with observations.•Assimilation of screen-level parameters decrease the warm/dry bias by 30-40%, but deteriorate the fit to top soil moisture.•Assimilation of mw Tb, on top of screen-level parameters, improves again the top soil moisture but deteriorates the fit to screen-level observations.
INM, Madrid, December 2003
Root zone soil moisture (SGP97)
170 175 180 185 190 195 200julian day
1012141618202224
root
zon
e so
il m
oist
ure
[%]
OBSCTRLKTRBKTRKB
Corr. Bias RMS 0.97 0.01 0.31-0.20 3.82 1.91-0.22 3.44 2.01 0.89 0.87 0.51
170 175 180 185 190 195 200julian day
1012141618202224
root
zon
e so
il m
oist
ure
[%]
a) OBSCTRLKTRBKTRKB
No precipitation simulation
•Results for root zone soil moisture are similar to those of the top layer.•The assimilation scheme responds correctly to a (very large) imposed error in the precipitation.
INM, Madrid, December 2003
Evaporative fraction (SGP97)
170 175 180 185 190 195julian day
0.2
0.4
0.6
0.8
1.0
evap
orat
ive
fract
ion OBS
CTRLKFTRKFTRBKB
•Evaporative fraction [EF=LE/(H+LE)], the relevant quantity for the surface impact on the atmosphere, is underestimated by the control simulation (cf. dry/warm bias).•EF is clearly improved when screen-level parameters are used.•And deteriorated again when mw Tb is added …
INM, Madrid, December 2003
EKF assimilation of microwave Tb• Assimilation of mw Tb:
– Transports surface soil moisture signal from 1st layer to deeper root zone
– Improves simulated soil moisture, surface energy fluxes, T,RH– Best results for atmosphere, when T,RH and Tb are assimilated– Assimilation of Tb needs better background:
• Different soil types• More soil layers• Removal of soil temperature bias necessary (results not shown here)
INM, Madrid, December 2003
• Introduction• Assimilation (OI vs. KF and synthetic mw Tb)• Assimilation of observed mw Tb• Additional experiments• Development of production system• Remaining work• (Scientific) conclusions
Layout
INM, Madrid, December 2003
Assimilation of satellite heating rate
160 180 200 220 240 260 280julian day
15
20
25
30
daily
mea
n ro
ot z
one
soil
moi
stur
e [%
]
ObsCtrlEKF assim. T,RH,SHREKF assim. T,RHEKF assim. SHR
Corr. Bias RMS0.94 3.46 2.000.81 0.89 2.440.79 0.97 2.530.94 2.95 1.58
Soilmoisture
Days when SHR is available (50% data missing, 25% cloudy)
•Variable SHR observation error depends on cloud fraction flag (how many hours arecloud free):
•Cloud fraction flag of neighbouring pixels•Cloud fraction flag of pixel
•Assimilating SHR:–Low data coverage does not allow for real conclusions
•When T, RH are available no extra information–SHR error difficult to define
INM, Madrid, December 2003
Winter simulations
Soilmoisture
200 300 400 500julian day
15
20
25
30
35da
ily m
ean
root
zon
e so
il m
oist
ure
[%]
ObsCtrl
KF assim. T,RHKF assim. T,RH flags
1.10.97 1.4.9897/98
•Without any additional flags EKF-system computes rather large soil moisture increments in winter•Flags necessary:
a) Low radiation (zenith angle) b) Freezing
INM, Madrid, December 2003
Soil temperature analysis
• Soil temperature analysis– 2m-T is assimilated at 3 and 6 LT (zenith angle dependent)– Soil temperature increments of similar magnitude to OI
• Combining soil moisture (SMA) and soil temperature (STA) analysis– Tests about the order of the SMA + STA analysis + avoidance of
SCM-runs• Cannot be based on the same background run• Almost no difference when SMA or STA is performed first• Better performance when final trajectory is calculated (expensive)
INM, Madrid, December 2003
• Introduction• Assimilation (OI vs. KF and synthetic mw Tb)• Assimilation of observed mw Tb• Additional experiments• Development of production system• Remaining work• (Scientific) conclusions
Layout
INM, Madrid, December 2003
Production system for ELDAS• Requirements:
– Annual database (1.10.1999–31.12.2000) of soil moisture for Europe – 0.2 x 0.2 regular lat/lon grid (15W-38E,35N-72N) – Computer time (cost efficiency)
• Starting point:– Experiments based on Single Column version of the ECMWF’s
NWP model (SCM) • Solutions:
– Add 1: Run n x n SCMs over Europe (each SCM runs independently)
– Add 2:• Run SCMs only for land points (about 25 000 SCMs)• I/O consideration• Open MP
– Add 3: Supervisor Monitor Scheduler (SMS)
INM, Madrid, December 2003
Progress of work
• Changes to the SCM +SMA source code– SCM structure has been changed to run n x n SCMs in one run– I/O netcdf I/O grib – OpenMPI parallelization (up to 8 processes on one thread)
• Forcing data– Composition of forcing data changed from one point to n x n points– Output netcdf Output grib
• Control Structure– First SMS layout
1) Soil moisture analysis
1) Get forcing data from Mars archive2) Prepare data for SCM INPUT
1) Background run
1) Get forcing data from Mars archive2) Prepare data for SCM INPUT
1) Soil moisture perturbation
1) Final (soil moisture) trajectory 2) Check success of SMA (Costfunctions)
1) Forecast run
1) Final (soil temperature) trajectory2) Check success of STA (costfunctions)
1) Soil temperature analysis
1) Soil temperature perturbation
INM, Madrid, December 2003
• Introduction• Assimilation (OI vs. KF and synthetic mw Tb)• Assimilation of observed mw Tb• Additional experiments• Development of production system• Remaining work• (Scientific) conclusions
Layout
INM, Madrid, December 2003
Future work (1)• Run validation points first in SCM mode
– Satisfies the validation group needs– Early warning on system performance– Learning exercise for Janneke– But entails:
• Delays on start of production• Extra work on software development (e.g., era40 forcing, validation diagnostics)
• Production system– What is still missing?
• Interpolation from gaussian grid to reg. 0.2 x 0.2 lat/lon grid• Incorporation of ELDAS maps (e.g. land cover)• Incorporation of ELDAS forcing data (precipitation, radiation)• Archiving of output in MARS • Observation (Re-analysis) data of 2mT and 2mRH for SMA +STA• Post-processing routines for parameters especially asked for by ELDAS
validation• ECMWF orography problems (LW)
– Final tests
INM, Madrid, December 2003
Production system• Estimated Production Time
– Analysis for one day• One SCM run for 1000 pixels needs 5 min on 8 nodes ~ 2 hours for 25000
pixels• 5 x SCMs are needed 10 hours for 25000 pixels
– Approx. 5-6 months for annual database• Further parallelization needed
– Splitting Europe into boxes– MPI, distributed memory
• Run the system at t511 (resolution 39 km)
• Expected Start of production
• Under normal circumstances– 6 weeks required to include missing bits and pieces
– 2 weeks final tests
?
INM, Madrid, December 2003
• Introduction• Assimilation (OI vs. KF and synthetic mw Tb)• Assimilation of observed mw Tb• Additional experiments• Development of production system• Remaining work• (Scientific) conclusions
Layout
INM, Madrid, December 2003
Conclusions• An E(xtended)KF was developed for land data assimilation, for the
assimilation of observations of screen-level T/RH, mw Tb, and SHR, a flexible introduction of new observation types, and usage of observed radiation and precipitation.
• The properties of such a system were systematically explored in a controlled environment (the atmosphere acts as a buffer, but the system does not feed back to the atmosphere), and confronted to the OI system operational at ECMWF, using the Single Column Model.
• “The devil is in the details”: The ratios sigma_o/sigma_b for the different observation types determines the response of the assimilations system.
• Screen-level parameters and mw Tb contain independent, and often contradictory, information on soil moisture, with possible contradictory impacts on surface fluxes. NWP centres tend to tune the assimilation to fit the evaporative fraction, since that is the quantity impacting on the atmosphere.
• The assimilation system will face biases (in both model and observations), mismatches of soil/vegetation parameters between model and real world, …