experimental real-time multi-model ensemble (mme

Experimental real-time multi-model ensemble (MME)prediction of rainfall during monsoon 2008:

Large-scale medium-range aspects

A K Mitra1,*, G R Iyengar

1, V R Durai2, J Sanjay

3,T N Krishnamurti

4, A Mishra4, D R Sikka

5

1National Centre for Medium Range Weather Forecasting (NCMRWF),Ministry of Earth Sciences, Noida 201 307, India.

2India Meteorological Department, New Delhi 110 003, India.3Indian Institute of Tropical Meteorology, Pune 411 008, India.

4Department of Meteorology, Florida State University, Tallahassee, USA.540 Mausam Vihar, New Delhi 110 051, India.

∗e-mail: [email protected]

Realistic simulation/prediction of the Asian summer monsoon rainfall on various space–time scalesis a challenging scientific task. Compared to mid-latitudes, a proportional skill improvement inthe prediction of monsoon rainfall in the medium range has not happened in recent years. Globalmodels and data assimilation techniques are being improved for monsoon/tropics. However, multi-model ensemble (MME) forecasting is gaining popularity, as it has the potential to provide moreinformation for practical forecasting in terms of making a consensus forecast and handling modeluncertainties. As major centers are exchanging model output in near real-time, MME is a viableinexpensive way of enhancing the forecasting skill and information content. During monsoon 2008,on an experimental basis, an MME forecasting of large-scale monsoon precipitation in the mediumrange was carried out in real-time at National Centre for Medium Range Weather Forecasting(NCMRWF), India. Simple ensemble mean (EMN) giving equal weight to member models, bias-corrected ensemble mean (BCEMn) and MME forecast, where different weights are given to membermodels, are the products of the algorithm tested here. In general, the aforementioned products fromthe multi-model ensemble forecast system have a higher skill than individual model forecasts. Theskill score for the Indian domain and other sub-regions indicates that the BCEMn produces thebest result, compared to EMN and MME. Giving weights to different models to obtain an MMEproduct helps to improve individual member models only marginally. It is noted that for higherrainfall values, the skill of the global model rainfall forecast decreases rapidly beyond day-3, andhence for day-4 and day-5, the MME products could not bring much improvement over membermodels. However, up to day-3, the MME products were always better than individual membermodels.

1. Introduction

The Asian monsoon is one of the major com-ponents of the earth climate system. Realisticmodelling, simulation and prediction of monsoon

are challenging scientific tasks for the Earth Sys-tem Science Community. For India, the monsoonrain is of enormous importance as it gives shapeto its agriculture, economy and rhythms of life.The science pertaining to monsoon has progressed

Keywords. India monsoon; rainfall forecast; global-models; multi-model ensemble (MME).

J. Earth Syst. Sci. 120, No. 1, February 2011, pp. 27–52© Indian Academy of Sciences 27

28 A K Mitra et al

significantly in the last two decades because of theincreased wealth of new data from satellite obser-vations, improved understanding of the processesand enhanced computing power. Numerical mod-els and data assimilation algorithms have been fur-ther improved at all major international centersacross the globe. The accuracy of weather forecastshas improved steadily in last three decades, andthe systematic error with forecast length in themedium range has reduced. However, compared tomid-latitudes, the skill for tropics is still lower, andis particularly of concern for rainfall forecast forthe Indian monsoon region (Webster et al 1998;Gadgil 2003; Krishnamurti et al 2006a, b; Woods2006). The errors even in large-scale rainfall pat-tern forecast still remains. These errors are due touncertainties in the assimilation–forecast system.These uncertainties could be due to the errors inthe prescribed initial states, which may arise fromobservational instruments, satellite estimates andthe data assimilation method. The forecast skillis also dependent on the synoptic situation, flowregime region and known (or unknown) tele-connections. The accuracy of a model also variesdepending on the formulation, horizontal–verticalresolutions and the parameterization schemes rep-resenting the small-scale processes in the model.The process of improving forecast skill of an indi-vidual modelling system through research anddevelopment (R&D) in modelling and data assim-ilation is rather a slow process. For example, atEuropean Centre for Medium-Range WeatherForecasting (ECMWF), the R&D of approximatelyten years resulted in skill improvement of one day.Therefore, from a practical forecasting aspect, wehave to learn to live with these uncertainties andthe current available skill of the models. The pointis how to make the best use of the forecasts fromany (or many) centers, when each center is havingdata from many member models.

In the context of different model data from dif-ferent centers and each center producing manyensemble members, the use of ensemble methodin short, medium range and even for short-termclimate prediction has become popular in recentyears. Because of the aforementioned uncertaintiesin the modelling system, the forecast error increaseas the forecast length increases, until it becomes nolonger useful. If we have an ensemble of forecasts(many forecasts), we can have a greater reliabilityof the forecasts. Forecast can be represented in aprobabilistic sense also. Clustering (or tubing) ofseveral similar forecasts is also useful. From manyensemble forecasts, we can obtain clues for possibleextreme/severe events. One single forecast (fromone single model) may fail to catch the likelihoodof an extreme event, but the ensemble might give

some extra clue on the extreme (or anomalous)episode. The current practices at major centers are:

(i) to perturb the initial conditions (scientificallybased) and make many member runs withthese different initial conditions,

(ii) make many runs by altering the model formu-lations, physical parameterizations, and

(iii) MME members using (i) and (ii).

As many centers are exchanging or providing modeldata in real-time, in principle, the MME can beattempted with very little computing resources.This MME is also popularly known as the poorman’s ensemble, as it uses meager amount ofresources. One assumption for MME is that themember models do provide some signal, and thenoise (and error) is less compared to signal. Whiledealing with the monsoon, we are aware of thelimitations of the models to capture the change inweather patterns, from dry to wet, or from wet todry. In monsoon, the formation and movement oflows and depressions have some limitations. Cor-rect simulation of the monsoon trough’s intensityis also an outstanding issue in modelling. If thegross errors are higher in intensity, track, positionand timings of the aforementioned monsoon sys-tems, then the MME may not be able to improvemuch above the crude member model’s skill. Evenif the skill score improves in a statistical sense,the forecast may not be of much use, as the sys-tem in MME output may be away from the realsystem with a different intensity. World Meteo-rological Organization’s (WMO) ‘The ObservingSystem Research and Predictability Experiment’(THORPEX) is planning to provide eight globalmodel’s data at a coarser resolution under‘THORPEX Interactive Grand Global Ensemble’(TIGGE) and each model is having around 15ensemble members, making it in total around 120members. The forecasting community has start-ed experimenting with the multi-model data prod-ucts, to examine the benefits in terms of skillenhancement and preparing probabilistic fore-casts, particularly for extreme events. In India, theusefulness of the MME forecast was identified andit was planned during 2007 that on an experimentalbasis, the monsoon rainfall MME forecast will beattempted from monsoon 2008 season, using what-ever model data available. A small team compris-ing of scientists from National Centre for MediumRange Weather Forecasting (NCMRWF), IndiaMeteorological Department (IMD) and IndianInstitute of Tropical Meteorology (IITM) was con-stituted by the Ministry of Earth Sciences (MoES)to start work on the experimental MME forecast.

At major weather/climate forecast centers,calibration of model outputs with respect to

MME prediction of rainfall during monsoon 2008 29

observations have been done to obtain an improvedskill. Different statistical post-processing tech-niques have been applied to model output para-meters for the scale, region and phenomenon ofinterest. These post-processing methods have en-abled the forecasters to obtain enhanced skilland value from models. MME is another post-processing (calibration) technique having thepotential to enhance the skill of rainfall prediction.In this report, the performance of the experimentalMME forecast of rainfall during monsoon 2008is studied. As the first attempt, only the large-scale aspects of monsoon rain from member mod-els and the multi-model products are documented.We are aware of the difficulties in producing rain-fall forecasts for smaller regions by the state-of-the-art global models. Significant errors are obviouslyexpected if one decides to come down to smallermeso-scales below the ‘large scale organized con-vective rainfall’ associated with monsoon. There-fore, as a first attempt, we study here the oneby one degree latitude/longitude grid rainfall datafrom four member models and the associated multi-model products in the medium range. It is wellknown that the simple average made from manymodels (simple ensemble mean, giving equal weightto each member model) generally produces higherskill score. Our interest here is to document theimprovement of the MME output (by giving dif-ferent weights to different models) over the simpleensemble mean.

2. Data and MME methodology

In this study, we use the daily medium range(day-1 to day-5) rainfall prediction data from fourstate-of-the-art operational global models, namelyGlobal Forecast System of National Centers forEnvironmental Prediction, USA (NCEP/GFS),United Kingdom Meteorological Office, UK(UKMO), Japan Meteorological Agency, Japan(JMA) and NCMRWF (India) for monsoon 2008season (June–September). These model data wereavailable up to day-7, but we restricted to testthe MME only up to day-5. NCMRWF globalassimilation–forecast system is an adapted versionof the NCEP/GFS system implemented in theyear 2007. Therefore, we expect, the basic char-acteristics of rainfall data from NCEP/GFS andNCMRWF to be broadly similar. The model dataused was at 1◦×1◦ uniform latitude/longitude res-olution, to represent the large scale aspect of themonsoon rainfall. These models were being runat their respective centers (countries) at a higherhorizontal and vertical resolution. NCEP/GFSmodel data was from runs made at 35 km

horizontal grid and 64 vertical layers. UKMOmodel data was from runs made at 40 km horizon-tal grid and 50 vertical layers. JMA model datawas from runs made at 20 km horizontal grid and60 vertical layers. NCMRWF/GFS model data wasfrom runs made at 50 km horizontal grid and 64vertical layers. Therefore, what it simulated wasof higher (finer) spatial scales than the results wehave analyzed at 1◦×1◦ resolution. As the purposeof this study is to note the skill enhancement com-ing from the multi-model algorithm, we thought itcan be experimented at even this 1◦×1◦ latitude/longitude resolution at which the data was pro-vided from the respective centers, and dependingupon the benefits, the algorithm can again betested on much higher resolution data. Some recentstudies on monsoon have shown the benefits ofinterpolating the coarser resolution data to a finergrid (sort of downscaling) and then do the MMEon the finer grids. We also plan to test that step inour future work. Therefore, in this study, we usethe original 1◦×1◦ data we received from the oper-ational centers via ftp. The corresponding dailyobserved gridded rainfall data at same 1◦×1◦ reso-lution was prepared by merging rain-gauge valueswith the satellite estimates (Mitra et al 2003).The gridded rainfall analysis data used in modelcalibration (training) has to be of good quality.Otherwise, it might degrade the MME results.

Early works by Krishnamurti et al (1999) showedthat it is possible to obtain skill improvements bothin weather and climate scales by the use of themulti-model technique named as ‘Super Ensemble’forecasting. Later it was extended for tropical pre-cipitation by incorporating multi-analysis conceptalong with the use of multi-models (Krishnamurtiet al 2000). Later Mishra and Krishnamurti (2007)applied the super-ensemble algorithm for Indianmonsoon and showed the skill enhancement usingseven global models data. A multi-model multi-analysis ensemble system was reported to evalu-ate the deterministic forecasts from UKMO andECMWF ensemble data (Evans et al 2000), andthey showed the superiority of the multi-modelsystem over the individual model data. Richard-son (2001) used multi-model and multi-analysisdata to produce both deterministic and proba-bilistic ensemble forecasts using four global mod-els from UKMO, ‘German Met Service’ (DWD),Meteo France and NCEP and showed that sim-ple ensemble mean and simple bias correctionproduce useful products. The probability ofprecipitation and rainfall distribution by usingmulti-model data from seven global models forAustralia region was studied by Ebert (2001).Multi-model multi-analysis data was also triedby using ECMWF and UKMO ensemble outputs

30 A K Mitra et al

for quasi-operational medium range forecasting(Mylne et al 2002) in both probabilistic and deter-ministic sense. They noted that the MME is morebeneficial than a single model Ensemble PredictionSystem (EPS) and the skill of winter was highercompared to summer season. Operational consen-sus forecast by including several models is seento outperform ‘direct model output’ (DMO) and‘model output statistics’ MOS forecasts (Wood-cock and Engel 2005). Multi-model product pre-pared by the use of NCEP/GFS and ECMWFdata also shows improvements in week-2 forecasts(Whitaker et al 2006), improving over the MOSforecast of individual models. Most of these stud-ies (algorithms) used simple ensemble mean ora mean of calibrated data from member models.However, by analyzing the past performance ofmodel for a region, it might be interesting to exam-ine if model dependent weights help further tocompute the final multi-model forecast. In a veryrecent study (Johnson and Swinbank 2009), theECMWF, UKMO and GFS global model data wereused to prepare multi-model ensemble products inmedium-range prediction. Here forecasts from bias-correction, model-dependent weights, and varianceadjustments were studied. It was found that themulti-model ensemble gives an improvement incomparison to calibrated single model ensemble.They also noted that only small improvementsfrom the use of model dependent weights and vari-ance adjustments were achieved.

Many studies mentioned above have reportedresults of MME from wind, MSLP, Z500 and t2 m,etc. Reports on MME of rainfall particularly fortropical regions are relatively less. Hamill et al(2008) have shown that precipitation is a differ-ent type of weather parameter and has to bedealt differently in calibration and statistical post-processing. It is highly variable in space and time.It is non-continuous, has non-negative values andnon-Gaussian in nature. They also discussed thetraining period (sample size) is important and par-ticularly so for very heavy rainfall amounts. It wasalso shown that the MME combination gives upto a two-day increase in predictability at day-7 formid-latitudes. MME for meso-scale weather sys-tems and associated summer heavy precipitationwere attempted for south-east US using high reso-lution six meso-scale model data (Cartwright andKrishnamurti 2007), which showed improvementswith the super-ensemble algorithm. Roy Bhowmikand Durai (2008, 2010) used the linear regressionmethod to find weights for member model and thenmake a multi-model ensemble product. However,the benefits of giving weights to member modelsover a simple ensemble mean were not documentedthere. In a recent study, the high resolution pre-cipitation MME forecast for Indian monsoon was

reported by Krishnamurti et al (2009). By down-scaling the data to a higher resolution and use ofsuper-ensemble technique, they have reported skillenhancement of precipitation forecasts in mediumrange for the Indian monsoon systems.

For the experimental rainfall MME during mon-soon 2008, the scheme used here is very similar tothat of Krishnamurti et al (2000) and Mishra andKrishnamurti (2007). It has a training phase and aforecast phase. During training phase, the membermodel forecasts were regressed (compared) againstobservation/analysis to obtain different weights. Amultiple linear regression is performed in this phaseto estimate the relative performance of membermodels. These weights are then passed on to theforecast phase to create the MME forecasts. Theweights for the models are now based on their pastperformance. In this process, it combines a set ofmulti-model forecasts to construct a single consen-sus forecast. The weights of the MME vary geo-graphically and with forecast lead time. The MMEforecast is constructed as:

MME = O +N∑

i=1

ai

(Fi − Fi

),

where Fi is the ith model forecast, Fi is the meanof the ith model forecast over the training period,O is the observed mean of the training period, ai isthe regression coefficient obtained by a minimiza-tion procedure during the training period, and N isthe number of forecast models involved. The coeffi-cient ai is derived from estimating the minimum offunction to satisfy the mean square error criteria:

G =Ntr∑

i=1

(MME

′

t − O′

t

)2

.

Ensemble mean bias removed is defined as:

BCE = O +N∑

i=1

1N

(Fi − Fi

).

In addition to removing the bias, MME scales theindividual model forecasts contributions accordingto their relative performance in the training periodin a way that is equivalent to weighing them math-ematically. In the initial days of June 2008 dur-ing monsoon 2008 season MME forecasting daily inreal time, we used the daily day-1 to day-5 rainfallforecasts data from same models for monsoon 2007.For training, the observed monsoon 2007 mergedsatellite gauge data was used. A running 92 daystraining period was always maintained with marchof days during the monsoon 2008 season. Asthe days progressed in 2008 season, we startedincluding recent past model and observation datato the 92 days training period. For example, on 20


Table 1. Dates considered as training period during monsoon 2008 (real time) MME forecast.

1 June JJA 2007 – 92 days4 June Start including data from June 2008, 1 June 2008

Remove days from August 2007, 31 August 20071 July 29–30 June 2007, JA07, 1–28 June 20084 July JA07, 2–30 June 2008, 1 July 200831 July JA07, 29–30 June 2008, 1–28 July 20084 August 3–31 July 2007, A07, July 2008, 1 August 200831 August 30–31 July 2007, A07, July 2008, 1–28 August 20081 September 13–30 June 2007, September 2007, June 2008, 15–28 August 20087 September 19–30 June 2007, September 2007, June 2008, 15 August–3 September 200830 September September 2007, 12–30 June 2008, 15 August–26 September 2008

June 2008, the latest data from 1–17 June 2008,were included in training. Similarly on 15 August2008, in addition to July, August data of 2007, weincluded July and August (up to 12th) data from2008 itself in training. However, the training periodwas maintained to 92 days. Table 1 shows the train-ing period dates for reference.

3. Results from monsoon 2008

As described in the previous section, during themonsoon 2008 (1 June– 30 September) the dailyreal time global model data from four centerswere collected and used in this multi-model ensem-ble forecast preparation on an experimental basis.Forecasts of rainfall were made from day-1 throughday-5. All member global model forecast and themulti-model forecast data for the forecast lengthof day-1 through day-5 were used to documentthe skill of rainfall forecast during the monsoon2008 season for the Indian region. The performanceof rainfall forecasts from member models and themulti-model algorithm are evaluated in terms oferror statistics and threshold statistics. In this sec-tion, we describe the error and threshold statistics.In the discussions, the skill for day-1, day-3 andday-5 will be shown and discussed for brevity.Figure 1(a, b and c) shows the member-modeland multi-model produced total rainfall during the2008 season made up from day-1, day-3 and day-5forecasts, respectively. In the figure, the observedrainfall is produced by merging rain-gauge andsatellite estimates from METEOSAT IR data. Atfirst glance, all the four member models seemto reproduce the observed monsoon rain closely.However, when the details are compared, we seeeach member model differs from the observationsin different ways, and at different regions. Forexample, in the day-1 forecasts (figure 1a) theNCMRWF and NCEP model produces too muchrain in the Arakan coast region to the east of Bay ofBengal. In NCEP model on the west coast of India

the north–south rainfall band extends too much tothe south and in northern plains (monsoon troughregion) the model produces more rain than obser-vations. In the UKMO model, it is seen to rainmore on the west coast, Himalayan foothills andnorthern Bangladesh. In contrast to all the mod-els, the JMA model produces the least rainfall.The multi-model products in the upper row whencompared with observations, look closer and morerealistic. The BCEMn and MME look superior tosimple ensemble mean (EMN). The EMN is supe-rior to member models, but still maintains the sig-nature of the strong biases of the member model.For example, the heavy rain along Arakan coast isseen in the EMN. The BCEMn and MME are ableto reduce the biases and look closer to observations.The MME products look superior to member mod-els and much closer to observations.

Similar to day-1 (figure 1a), the day-3 and day-5seasonal total rainfall from observations, membermodels and multi-model ensembles are shown infigure 1(b and c). The systematic biases of mem-ber models continue to remain on day-3 forecastswith changing magnitudes. However, the patternof biases continues to be of similar nature. NCM-RWF and NCEP models almost maintain the sim-ilar biases with slight enhancement. In UKMOmodel, the west coast rain gradually becomes closerto observations. However, over the foothills andnorthern Bangladesh, the rainfall positive biasesare seen to increase with time (from day-1 to day-3). JMA model shows gradual decreasing rainfallover all the regions around India. In contrast, inboth day-3 and day-5, the multi-model productslook closer to observations. Both the MME andBCEMn look better than simple EMN. Even inday-5 forecasts, the relatively lower biases in multi-model products are good signs of encouragementto continue R & D in MME forecasting algorithms.

To examine more clearly the rainfall biases ofthe four member models from the observations, thedifference of observed amounts from the predictedamounts are computed for day-1, day-3 and day-5

32 A K Mitra et al


Fig

ure

1.

Tota

lra

infa

llduri

ng

monso

on

2008

(JJA

S)

from

obse

rvati

ons,

mem

ber

model

sand

mult

i-m

odel

pro

duct

s:(a

)day

-1,(b

)day

-3,and

(c)

day

-5.

34 A K Mitra et al

Fig

ure

1.

(Continued

).


Fig

ure

2.

Diff

eren

ceofobse

rvati

ons

from

fore

cast

softo

talra

infa

ll(m

ean

erro

r)duri

ng

monso

on

2008

(JJA

S)

from

mem

ber

model

sand

mult

i-m

odel

pro

duct

s:(a

)day

-1,

(b)

day

-3,and

(c)

day

-5.

36 A K Mitra et al


Fig

ure

2.

(Continued

).

38 A K Mitra et al

forecasts and are shown in figure 2(a), 2(b) and2(c), respectively. And for reference, the seasonalrainfall in grey-scale is also given at the top leftcorner of the plot. The difference plot is interpretedas mean error during the monsoon 2008 season.In these difference (mean error) plots, the positivebiases (model is wetter) are shown in shades of blueand the negative biases (model is drier) are shownin shades of orange. Regions of dark blue shadesshow that the model produces more rain comparedto observations. These difference (forecast minusobservation) plots clearly bring out the regions ofmean biases for day-1, day-3 and day-5 forecasts.For all the days, the biases from the multi-modelproducts like EMN, BCEMn and MME (upperrow) show the least biases compared to membermodels. Among the multi-model products, the leastbiases are seen in both the MME and BCEMn ascompared with EMN for day-1, day-3 and day-5forecasts.

The above described mean error in a sense showsthe systematic error of the model. However, whilesumming for the season, errors of opposite signmight be getting cancelled in some regions andthe bias representation may not be fully informa-tive. Hence, the mean error (bias) plots have tobe examined in conjunction with the root meansquare error (RMSE) values for the aforementionedregions also. The RMSE of the member modeland the multi-model forecast products in compar-ison to the observations are given in figure 3(a),3(b) and 3(c) for day-1, day-3 and day-5, respec-tively. Again for reference, the observed plot isshown at top left corner of the diagram. For day-1(figure 3a) among member models NCMRWF andNCEP have higher RMSEs. UKMO RMSE is lesserthan NCEP and NCMRWF. Another feature of theRMSE is that the errors are more in models wherethe rainfall amounts are also more. For example,the west coast of India, monsoon trough region andthe Arakan coast region have the higher RMSEvalues. The forecasts from the multi-model prod-ucts have less RMSE compared to member mod-els. Again, among them the BCEMn and MMEhave lesser RMSE compared to EMN. When weexamine the day-3 (figure 3b) and day-5 (figure 3c)RMSEs we see for all member models the values aremore compared to day-1. The errors grow graduallyfrom day-1 to day-5. However, again in both day-3and day-5, the multi-model forecast products haveless RMSEs compared to their respective mem-ber models. On day-5, the BCEMn has the leastRMSE.

During monsoon, in medium range forecasts, itis important and challenging to predict the day-to-day rainfall associated with the passing tran-sient weather systems or the fluctuating strength

of the monsoon. Therefore, the rainfall anomalyin forecasts and observations has to be examinedin terms of their similarity. The anomalies of theobservation and forecasts are computed from theirrespective seasonal means during 2008 monsoon forday-1, day-3 and day-5 forecasts and are shown infigure 4(a), 4(b) and 4(c), respectively. The anom-aly correlation coefficients (ACC) are plotted in ascale of 0 to 1 in figure 4. Shades of yellow, orangeor red shows gradual increase of ACC from 0.5to 0.9, and ACC below 0.5 are shown in shadesof blue. For day-1 forecasts we see in general theACCs are higher for all member models. But themulti-model products like the EMN and BCEMnhave higher scores of ACC compared to membermodels. Going from day-1 (figure 4a) to day-3(figure 4b), immediately we see a drop in skill inACC for the member models. Only UKMO modelshows some skill. However, for day-3 forecasts theEMN and BCEMn have higher skills than mem-ber models and even the MME. For day-5 forecaststhe skill of member models all come below 0.2 interms of ACC. UKMO again has higher skill thanother member models. The multi-model productsfrom day-5 of EMN and BCEMn have higher skillcompared to all member models and the MME.From this it can be concluded that for monsoonrainfall forecasts in medium range, the currentstate-of-the-art models have some skill till day-3, which is slightly enhanced by the multi-modeltechnique. But for day-5, all the model and multi-model skills are very low, and may not be havingany forecast value. The modelling community haveto focus research on how to improve the skill ofrainfall associated with transient weather systemsbeyond day-3. In the seamless concept this maylead to improved monthly and seasonal forecast ofmonsoon rainfall.

All the above results discussed give some gen-eral idea of the quality of rainfall forecasts in termsof error statistics for monsoon for the memberglobal models and the products from the multi-model algorithm. But we are aware of the limi-tations of the numerical models in simulating thefinal products in model, i.e., the rainfall quantityat the right regions. Therefore, it is relevant toexamine and document the skill of rainfall forecastsin terms of rainfall amounts in different categories(different threshold amounts of rainfall) in termsof threshold statistics. Standard statistical parame-ters like equitable threat score (ETS), hit rate (HR)and bias score are computed for the comparisonsin different categories of rainfall amounts. A briefdescription of these categorical statistics is givenin Ebert et al (2007). ETS is commonly used asan overall efficiency measure for inter-comparisonof precipitation products. ETS gives the fraction


Fig

ure

3.

Root

mea

nsq

uare

erro

r(R

MSE

)ra

infa

llduri

ng

monso

on

2008

(JJA

S)

for

mem

ber

model

sand

mult

i-m

odel

pro

duct

s:(a

)day

-1,(b

)day

-3,and

(c)

day

-5.

40 A K Mitra et al


Fig

ure

3.

(Continued

).

42 A K Mitra et al


Fig

ure

4.

Anom

aly

corr

elati

on

coeffi

cien

t(A

CC

)ra

infa

llduri

ng

monso

on

2008

(JJA

S)

for

mem

ber

model

sand

mult

i-m

odel

pro

duct

s:(a

)day

-1,(b

)day

-3,and

(c)

day

-5.

44 A K Mitra et al

Fig

ure

4.

(Continued

).


of observed and/or detected rain that was cor-rectly detected and is adjusted for the number ofhits that could be expected purely due to ran-dom chance. HR (success rate) is the ratio of thenumber of correctly forecast points above a thresh-old to that of the number of forecast points abovethe corresponding threshold. The bias score is theratio of the estimated to predicted rain areas. Forfour different regions, namely, all of India, centralIndia, peninsular India, and the west coast wereselected to compute these categorical skill scores.These scores are obtained from the data coveringthe daily values for the entire 2008 monsoon seasonof 122 days (JJAS). The reason for selecting thesedomains is related to the known important syn-optic circulation features of the large-scale Indianmonsoon system and the associated rainfall overthe land regions. All of India covers all the landgrid points between 67◦E to 100◦E longitude, and7◦N to 37◦N latitude. The central India domaincovers the monsoon trough region, extending from73◦E to 90◦E longitude and 22◦N to 28◦N lati-tude. The west coast domain of India extends from70◦E to 78◦E longitude and 10◦N to 20◦N latitude.The peninsular India domain extends from 74◦E to85◦E longitude and 7◦N to 21◦N latitude. The per-formance measures used are in terms of differentthreshold statistics. Here, six thresholds from 1 to6 cm per day (interval of 1 cm per day, on x-axis)are considered. For all four domains, for the 2008monsoon season the skills are shown in figures 5–8, for different length of forecasts like day-1, day-3and day-5.

The left panel in figure 5(a, b, c) shows theETS for thresholds 1 to 6 cm per day, for day-1, day-3 and day-5 forecasts for the aforemen-tioned all India region. As well known, for anygiven day (forecast length) the ETS generally grad-ually decreases with increasing threshold. Again,with increasing length of forecast period (day-1to day-5) for each threshold rainfall category, theETS skill score decreases gradually. The ETS forall member models look similar for all forecastlength and thresholds. However, in general, theETS of UKMO is slightly higher than other mem-ber models for higher thresholds even with increas-ing forecast lengths and seen to have even higher(or comparable) skills than the multi-model prod-ucts at higher thresholds. The multi-model prod-ucts like the EMN, BCEMn and MME show higherskill in ETS compared to member models for dif-ferent length of forecasts and thresholds. In gen-eral, among the multi-model products the BCEMnshows higher skill than EMN and MME. There-fore, in terms of multi-model products for rainfall

in terms of ETS skill, we can conclude that theuse of multi-model has some benefits compared tousing single independent models for issuing rain-fall forecasts. The middle panel in figure 5(a, b, c)shows the bias scores for different threshold andforecast lengths for the aforementioned all Indiaregion box under consideration. If the bias value isgreater than 1, then the model has a positive bias(tendency to rain at more points compared toobserved points/regions for a particular thresholdunder consideration). It is interesting to see inday-1 forecasts that the member models are doingvery good for lower thresholds (up to 3 cm perday), and the multi-model products are not able toimprove upon them. And for higher thresholds, themulti-model is bringing the biases to the negativeside. The same is true for the bias score roughlyfor day-3 and day-5 forecasts. The right panel infigure 5(a, b, c) shows the hit rate of rainfall fore-cast for different threshold and forecast lengths.For the hit rate, the multi-model products aremostly seen to perform much better than the fourmember models. For all days, the hit rate fromMME and BCEMn are superior to simple EMNand the member models for all thresholds. Thisalso shows the power of the multi-model algorithmto be able to produce rainfall at right grids for dif-ferent thresholds. It is able to correct the rainfallamounts for different thresholds. MME is produc-ing the best skill for all days and thresholds for theall India region box.

After examining the all India region box, wenow study three more different and typical regionsof the monsoon. The central India box consideredis the most crucial for the monsoon forecastingpoint, as it is closely related to the monsoon trough(heart of the Indian monsoon system) position andintensity forecast. For state-of-the-art global mod-els, even in medium range, correctly simulatingthe monsoon trough position and intensity (alongwith embedded monsoon lows and depressions)is a very challenging task. The manifested rain-fall forecast is even more tough for the monsoontrough region (central India box here). Figure 6(a,b, c) shows the ETS, bias score and the hit ratefor different days of forecast length and differentthreshold rainfall values. We notice an immediatedrop in ETS skill in central India region as com-pared to the earlier discussed all India region skills.Beyond 3 cm/day threshold the ETS skill is aslow as 0.05 or even lower for all days. Only forthreshold of 1 and 2 cm, we find some skill fordifferent days. Here, again the multi-model prod-ucts are able to enhance the skill over the mem-ber models. Almost for all threshold and days, the

46 A K Mitra et al

JJA

S 2

008

All

Ind

ia [

67

– 10

0 E

, 7 –

37

N ]

Day

1

ET

S:

All

Ind

ia [

67-

100E

, 7-3

7N ]

Day

1

0

0.050.1

0.150.2

0.250.3

12

34

56

Rai

nfa

ll T

hre

sho

ld (

cm

)

ETS

NC

M

UK

M

GF

S

JMA

EM

N

BC

E

MM

E

Bia

s: A

ll In

dia

[ 6

7-10

0E, 7

-37N

] D

ay 1

0

0.51

1.52

2.53

3.5

12

34

56

Rai

nfa

ll T

hre

sho

ld (

cm

)

Bias

NC

M

UK

M

GF

S

JMA

EM

N

BC

E

MM

E

Hit

Rat

e: A

ll In

dia

[ 6

7-10

0E, 7

-37N

] D

ay 1

0

0.1

0.2

0.3

0.4

0.5

0.6

12

34

56

Rai

nfa

ll T

hre

sho

ld (

cm

)

Hit Rate

NC

M

UK

M

GF

S

JMA

EM

N

BC

E

MM

E

(a)

(b)

JJA

S 2

008

All

Ind

ia [

67

– 10

0 E

, 7 –

37

N ]

Day

3

ET

S:

All

Ind

ia [

67-

100E

, 7-3

7N ]

Day

3

0

0.050.1

0.150.2

0.250.3

12

34

56

Rai

nfa

ll T

hre

sho

ld (

cm

)

ETS

NC

M

UK

M

GF

S

JMA

EM

N

BC

E

MM

E

Bia

s: A

ll In

dia

[ 6

7-10

0E, 7

-37N

] D

ay 3

0

0.51

1.52

2.53

3.5

12

34

56

Rai

nfa

ll T

hre

sho

ld (

cm )

Bias

NC

M

UK

M

GF

S

JMA

EM

N

BC

E

MM

E

Hit

Rat

e: A

ll In

dia

[ 6

7-10

0E, 7

-37N

] D

ay 3

0

0.1

0.2

0.3

0.4

0.5

0.6

12

34

56

Rai

nfa

ll T

hre

sho

ld (

cm

)

Hit Rate

NC

M

UK

M

GF

S

JMA

EM

N

BC

E

MM

E

JJA

S 2

008

All

Ind

ia [

67

– 10

0 E

, 7 –

37

N ]

Day

5

Bia

s: A

ll In

dia

[ 6

7-10

0E, 7

-37N

] D

ay 5

0

0.51

1.52

2.53

3.5

12

34

56

Rai

nfa

ll T

hre

sho

ld (

cm

)

Bias

NC

MU

KM

GF

SJM

AE

MN

BC

EM

ME

Hit

Rat

e: A

ll In

dia

[ 6

7-10

0E, 7

-37N

] D

ay 5

0

0.1

0.2

0.3

0.4

0.5

0.6

12

34

56

Rai

nfa

ll T

hre

sho

ld (

cm

)

Hit Rate

NC

MU

KM

GF

SJM

A

EM

NB

CE

MM

E

ET

S:

All

Ind

ia [

67-

100E

, 7-3

7N ]

Day

5

0

0.050.1

0.150.2

0.250.3

12

34

56

Rai

nfa

ll T

hre

sho

ld (

cm

)

ETS

NC

MU

KM

GF

SJM

AE

MN

BC

EM

ME

(c)

Fig

ure

5.

Cate

gori

calra

infa

llfo

reca

stsk

illsc

ore

(ET

S,bia

s,hit

rate

)duri

ng

monso

on

2008

(JJA

S)

for

mem

ber

model

sand

mult

i-m

odel

pro

duct

s,fo

rth

eA

llIn

dia

regio

n:

(a)

day

-1,(b

)day

-3,and

(c)

day

-5.


ET

S:

Cen

. In

dia

[ 7

3-90

E,2

2-28

N ]

Day

1

0

0.050.1

0.150.2

0.250.3

12

34

56

Rai

nfa

ll T

hre

sho

ld (

cm

)

ETS

NC

M

UK

M

GF

S

JMA

EM

N

BC

E

MM

E

Hit

Rat

e: C

en. I

nd

ia [

73-

90E

,22-

28N

] D

ay 1

0

0.1

0.2

0.3

0.4

0.5

0.6

12

34

56

Rai

nfa

ll T

hre

sho

ld (

cm

)

Hit Rate

NC

M

UK

M

GF

S

JMA

EM

N

BC

E

MM

E

Bia

s: C

en. I

nd

ia [

73-

90E

,22-

28N

] D

ay 1

0

0.51

1.52

2.53

3.5

12

34

56

Rai

nfa

ll T

hre

sho

ld (

cm

)

Bias

NC

M

UK

M

GF

S

JMA

EM

N

BC

E

MM

E

JJA

S 2

008

C

entr

al In

dia

[ 7

3 –

90 E

, 22

– 2

8 N

] D

ay 1

(a)

ET

S:

Cen

. In

dia

[ 7

3-90

E,2

2-28

N ]

Day

3

0

0.050.1

0.150.2

0.250.3

12

34

56

Rai

nfa

ll T

hre

sho

ld (

cm

)

ETS

NC

M

UK

M

GF

S

JMA

EM

N

BC

E

MM

E

Hit

Rat

e: C

en. I

nd

ia [

73-

90E

,22-

28N

] D

ay 3

0

0.1

0.2

0.3

0.4

0.5

0.6

12

34

56

Rai

nfa

ll T

hre

sho

ld (

cm

)

Hit Rate

NC

M

UK

M

GF

S

JMA

EM

N

BC

E

MM

E

Bia

s: C

en. I

nd

ia [

73-

90E

,22-

28N

] D

ay 3

0

0.51

1.52

2.53

3.5

12

34

56

Th

resh

old

BiasN

CM

UK

M

GF

S

JMA

EM

N

BC

E

MM

E

JJA

S 2

008

Cen

tral

Ind

ia [

73

– 90

E ,

22 –

28

N ]

Day

3

(b)

ET

S:

Cen

. In

dia

[ 7

3-90

E,2

2-28

N ]

Day

5

0

0.050.1

0.150.2

0.250.3

12

34

56

Rai

nfa

ll T

hre

sho

ld (

cm

)

ETS

NC

M

UK

M

GF

S

JMA

EM

N

BC

E

MM

E

Hit

Rat

e: C

en. I

nd

ia [

73-

90E

,22-

28N

] D

ay 5

0

0.1

0.2

0.3

0.4

0.5

0.6

12

34

56

Rai

nfa

ll T

hre

sho

ld (

cm

)

Hit Rate

NC

M

UK

M

GF

S

JMA

EM

N

BC

E

MM

E

Bia

s: C

en. I

nd

ia[

73-9

0E,2

2-28

N ]

Day

5

0

0.51

1.52

2.53

3.5

12

34

56

Th

resh

old

Bias

NC

M

UK

M

GF

S

JMA

EM

N

BC

E

MM

E

JJA

S 2

008

Cen

tral

Ind

ia [

73

– 90

E ,

22 –

28

N ]

Day

5

(c)

Fig

ure

6.

Sam

eas

figure

5,ex

cept

for

centr

alIn

dia

regio

n.

48 A K Mitra et al

JJA

S 2

008

Wes

t C

oas

t [

70

– 78

E ,

10 –

20

N ]

Day

1

ET

S:

Wes

t C

oas

t [

70-7

8E, 1

0-20

N ]

Day

1

0

0.050.1

0.150.2

0.25

12

34

56

Rai

nfa

ll T

hre

sho

ld (

cm

)

ETS

NC

M

UK

M

GF

S

JMA

EM

N

BC

E

MM

E

Bia

s: W

est

Co

ast[

70-

78E

, 10-

20N

] D

ay 1

0246810

12

34

56

Rai

nfa

ll T

hre

sho

ld (

cm

)

Bias

NC

M

UK

M

GF

S

JMA

EM

N

BC

E

MM

E

Hit

Rat

e: W

est

Co

ast

[ 70

-78E

, 10-

20N

] D

ay 1

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

12

34

56

Rai

nfa

ll T

hre

sho

ld (

cm

)

HR

NC

M

UK

M

GF

S

JMA

EM

N

BC

E

MM

E

JJA

S 2

008

Wes

t C

oas

t [

70

– 78

E ,

10 –

20

N ]

Day

5

ET

S:

Wes

t C

oas

t [

70-7

8E, 1

0-20

N ]

Day

5

0

0.050.1

0.150.2

0.25

12

34

56

Rai

nfa

ll T

hre

sho

ld (

cm

)

ETS

NC

M

UK

M

GF

S

JMA

EM

N

BC

E

MM

E

Hit

Rat

e: W

est

Co

ast

[ 70

-78E

, 10-

20N

] D

ay 5

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

12

34

56

Rai

nfa

ll T

hre

sho

ld (

cm

)

HR

NC

M

UK

M

GF

S

JMA

EM

N

BC

E

MM

E

Bia

s: W

est

Co

ast

[ 70

-78E

, 10-

20N

] D

ay 5

-11357911

12

34

56

Rai

nfa

ll T

hre

sho

ld (

cm

)

Bias

NC

M

UK

M

GF

S

JMA

EM

N

BC

E

MM

E

JJA

S 2

008

Wes

t C

oas

t [

70

– 78

E ,

10 –

20

N ]

Day

3

ET

S:

Wes

t C

oas

t [

70-7

8E, 1

0-20

N ]

Day

3

0

0.050.1

0.150.2

0.25

12

34

56

Rai

nfa

ll T

hre

sho

ld (

cm

)

ETS

NC

M

UK

M

GF

S

JMA

EM

N

BC

E

MM

E

Hit

Rat

e: W

est

Co

ast

[ 70

-78E

, 10-

20N

] D

ay 3

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

12

34

56

Rai

nfa

ll T

hre

sho

ld (

cm

)

HR

NC

M

UK

M

GF

S

JMA

EM

N

BC

E

MM

E

Bia

s: W

est

Co

ast

[ 70

-78E

, 10-

20N

] D

ay 3

0246810

12

34

56

Rai

nfa

ll T

hre

sho

ld (

cm

)

Bias

NC

M

UK

M

GF

S

JMA

EM

N

BC

E

MM

E

(a)

(c)

(b)

Fig

ure

7.

Sam

eas

figure

5,ex

cept

for

wes

tco

ast

regio

n.


JJA

S 2

008

Pen

insu

lar

Ind

ia [

74

– 85

E ,

7 –

21 N

] D

ay 1

ET

S:

Pen

insu

la In

dia

[ 7

4- 8

5 E

, 7-2

1 N

] D

ay 1

0

0.050.1

0.150.2

0.250.3

12

34

56

Rai

nfa

ll T

hre

sho

ld (

cm

)

ETS

NC

M

UK

M

GF

S

JMA

EM

N

BC

E

MM

E

Hit

Rat

e: P

enin

sula

Ind

ia[

74-

85 E

, 7-2

1 N

] D

ay 1

0

0.1

0.2

0.3

0.4

0.5

0.6

12

34

56

Rai

nfa

ll T

hre

sho

ld (

cm

)

HR

NC

M

UK

M

GF

S

JMA

EM

N

BC

E

MM

E

Bia

s: P

enin

sula

Ind

ia[

74-

85 E

, 7-2

1 N

] D

ay 1

0246810

12

34

56

Rai

nfa

ll T

hre

sho

ld (

cm

)

Bias

NC

M

UK

M

GF

S

JMA

EM

N

BC

E

MM

E

JJA

S 2

008

Pen

insu

lar

Ind

ia [

74

– 85

E ,

7 –

21 N

] D

ay 3

ET

S:

Pen

insu

la In

dia

[ 7

4- 8

5 E

, 7-2

1 N

] D

ay 3

0

0.050.1

0.150.2

0.250.3

12

34

56

Rai

nfa

ll T

hre

sho

ld (

cm

)

ETS

NC

M

UK

M

GF

S

JMA

EM

N

BC

E

MM

E

Hit

Rat

e: P

enin

sula

Ind

ia [

74-

85

E, 7

-21

N ]

Day

3

0

0.1

0.2

0.3

0.4

0.5

0.6

12

34

56

Rai

nfa

ll T

hre

sho

ld (

cm

)

HR

NC

M

UK

M

GF

S

JMA

EM

N

BC

E

MM

E

Bia

s: P

enin

sula

Ind

ia [

74-

85

E, 7

-21

N ]

Day

3

0246810

12

34

56

Rai

nfa

ll T

hre

sho

ld (

cm

)

Bias

NC

M

UK

M

GF

S

JMA

EM

N

BC

E

MM

E

JJA

S 2

008

Pen

insu

lar

Ind

ia [

74

– 85

E ,

7 –

21 N

] D

ay 5

ET

S:

Pen

insu

la In

dia

[ 7

4- 8

5 E

, 7-2

1 N

] D

ay 5

0

0.050.1

0.150.2

0.250.3

(a) (b)

(c)

12

34

56

Rai

nfa

ll T

hre

sho

ld (

cm

)

ETS

NC

M

UK

M

GF

S

JMA

EM

N

BC

E

MM

E

Hit

Rat

e: P

enin

sula

Ind

ia [

74-

85

E, 7

-21

N ]

Day

5

0

0.1

0.2

0.3

0.4

0.5

0.6

12

34

56

Rai

nfa

ll T

hre

sho

ld (

cm

)

HR

NC

M

UK

M

GF

S

JMA

EM

N

BC

E

MM

E

Bia

s: P

enin

sula

Ind

ia [

74-

85

E, 7

-21

N ]

Day

5

0246810

12

34

56

Rai

nfa

ll T

hre

sho

ld (

cm

)

Bias

NC

M

UK

M

GF

S

JMA

EM

N

BC

E

MM

E

Fig

ure

8.

Sam

eas

figure

5,ex

cept

for

pen

insu

lar

regio

n.

50 A K Mitra et al

multi-model ETS skill show higher value than themember models. Among multi-model products, interms of ETS, the EMN and BCEMn show higherskill than MME. However, when the skill of themodels are only up to the threshold of 2 cm perday, we can not expect to use the model fore-cast for higher rainfall amount events. The multi-model algorithm improved slightly above the mem-ber models. The basic information content has tocome from member models only. For central Indiaregion, the bias score indicates a higher bias (rain-ing at more points) for almost all thresholds anddays by the UKMO and GFS models. With increas-ing thresholds (higher rainfall amount events) thebias of UKMO and GFS are seen to graduallyincrease. The bias of the NCMRWF model lookmodest compared to other member models. Forlower thresholds (up to 3 cm) for all days the multi-model products do a good job, by reducing the biasof the models. BCEMn is superior to EMN andMME. But for higher thresholds (above 3 cm) themulti-model products bring the bias to the negativeside, and again BCEMn is best among multi-modelproducts in terms of bias score for central Indiaregion. For the hit rate again (for central India) thescores fall rapidly with increasing thresholds. Themulti-model products have higher skill of hit ratecompared to member models. MME and BCEMnhave high skill compared to simple ensemble meanEMN. For all thresholds and days, the hit scorefrom multi-model products are higher compared tofour member global models.

West coast of India having the Western Ghatsmountains is another important region for themonsoon studies. The higher orography of theWestern Ghats interacts with the lower tro-posphere moist winds of the Arabian Sea (low-level westerly jet of the monsoon) to produce heavyrainfall amounts. The strength of the large-scalemonsoon flow is manifested in the low-level flowand the associated rainfall over the west coastregion of India. The ETS, bias score and the hitrate score for the west-coast region box are shownin figure 7(a, b, c) for different days and rain-fall thresholds. For the west-coast region, the ETSscore show rapid fall in skill with increasing thresh-old for all forecast lengths. The ETS skills arevery low beyond the threshold of 3 cm. For higherrainfall amounts the model ETS skill are very lowfor all days. For threshold up to 3 cm, the multi-model products are able to improve upon the mem-ber models. The EMN and the BCEMn are bet-ter than MME. As the skill of member modelsbeyond 3 cm are very low, what the multi-modelalgorithm enhances are unimportant from forecast-ing point of view. Basically, the model simulationfor the higher rainfall amounts has to be improvedfurther to reap any benefit from the multi-model

algorithm. The bias score for the west coast regionlook better than the central India and the all Indiascores. For higher rainfall thresholds (beyond 5 cm)particularly the NCMRWF and GFS producehigher positive biases, and the multi-model prod-ucts seem to improve the bias score there forall days. Here, the simple EMN look better thanBCEMn and the MME. In the hit rate scores, againfor all thresholds and days, generally the multi-model products are better than the member mod-els. For hit rate MME and BCEMn are better thanthe simple EMN.

The last region where the skill of rainfall forecastfrom member models and the multi-model prod-ucts are examined is the peninsular India region(figure 8a, b, c). Generally, this region rains lesswhen the monsoon trough is active. PeninsularIndia gets rainfall if some monsoon lows or depres-sions encroach into this area, or sometimes duringthe passage of rainfall bands from south to north.Therefore, this is also a sensitive region to moni-tor the model rainfall skill during monsoon. Simi-lar to other regions discussed earlier, here also, theETS skill score decreases with increasing thresh-old and increasing length of forecast days. Here,the member model skills are very low beyond 3 cmthreshold for day-3 and day-5. At lower thresholds(3 cm and below) the multi-model products namelythe BCEMn and the EMN show higher skill com-pared to member models. Except day-5, for higherthresholds (4 cm and above) also we see marginalimprovements in ETS from the multi-model prod-ucts, however, all the member model and the multi-model skills are so low that it can not be used forforecasting. For peninsular India, the bias score forhigher rainfall thresholds (above 4 cm) the mem-ber models show high positive biases. The BCEMnis able to reduce the biases for all days. The hitrate score for the peninsular region from the multi-model products (MME and BCEMn) also showimprovement up to 3 cm threshold for all days.Beyond 3 cm threshold the scores are very low forthe hit rate for all days, and the multi-model isunable to enhance the scores much above the mem-ber models.

4. Summary and future work

During monsoon 2008 on an experimental basisMME forecasting of large-scale monsoon precipita-tion in medium range was carried out in real-timeat NCMRWF/MoES, India. Apart from simpleensemble mean and bias corrected ensemble mean,linear regression based weights were given to fourglobal models to obtain the multi-model ensem-ble forecasts. In general, the multi-model ensemble


forecast products have higher skill than individualmodel product. The skill score for Indian domainand other sub-regions indicate that the bias cor-rected ensemble mean produces the best skill. Giv-ing weights to different model to get an MMEproduct helps to improve over individual membermodels marginally. The basic skills of the globalmodel rainfall decrease rapidly beyond day-3. Toget real benefits of the MME algorithm the basicsingle model skill of rainfall has to be improvedbeyond day-3. It was the first step in the learningexperiences in MME. It has worked like a feasibil-ity study of the MME experiment for rainfall fore-cast. While looking for enhanced skill in MME, asa by-product, the member models also get verifiedand we are able to keep track of the state-of-the-art model’s performance. This feedback is usefulfor continuous model development and other mod-elling related research. From poor man’s MME, weare progressing towards great grand MME underWMO/TIGGE. In coming years we plan to includeother newer model data into this MME system.In this study we use rather simple techniques andstandard scores to assess the usefulness and bene-fits of the MME forecast against member models.However, with increasing number of member mod-els and ensemble members from each model, to beable to understand and document the full potentialand usefulness of the MME products both in deter-ministic and probabilistic sense various skill scoreshave to be used (Cusack and Arribas 2008). Proba-bilistic ensemble forecasting has to be taken up fortropics. Bowler et al (2008) have shown the useful-ness of a short-range ensemble prediction systemwhich will be made operational at UKMO. Theyshow that the regional ensemble is more skillfulthan the global ensemble, and compares favourablyto the ECMWF ensemble for many variables. InIndia also a regional ensemble system for proba-bilistic forecasts has to be experimented.

Recent approach used by Krishnamurti et al(2009) on downscaling the global model data to aregional scale and then applying MME algorithmwill be experimented in a future study.

Acknowledgements

This report is the outcome of the MoES MMEproject. Steering committee members of theproject had provided useful suggestions during thecourse of this study. Thanks are due to Head,NCMRWF for his constant support and encourage-ment. Data used in this study were obtained fromNCEP (via ftp), UKMO (through NCMRWF) andJMA, Japan (through IMD). Daily METEOSAT-IR data available through ftp was used to preparethe observed rainfall estimates.

References

Bowler N E, Arribas A, Mylne K R, Robertson K B andBeare S E 2008 The MOGREPS short range ensembleprediction system; Quart. J. Roy. Meteor. Soc. 134 703–722.

Cartwright T J and Krishnamurti T N 2007 Warm seasonmesoscale superensemble precipitation forecasts in thesoutheastern United States; Weather Forecast. 22 873–886.

Cusack S and Arribas A 2008 Assessing the usefulness ofprobabilistic forecasts; Mon. Weather Rev. 136 1492–1504.

Ebert E E 2001 Ability of a poor man’s ensemble to predictthe probability and distribution of precipitation; Mon.Weather Rev. 129 2461–2480.

Ebert E E, Janowiak J and Kidd C 2007 Comparison of nearreal time precipitation estimates from satellite observa-tions and numerical models; Bull. Am. Meteorol. Soc. 8847–64.

Evans R E, Harrison M S J, Graham R J and Mylne K R2000 Joint medium-range ensembles from the Met. Officeand ECMWF system; Mon. Weather Rev. 128 3104–3127.

Gadgil S 2003 The Indian monsoon and its variability;Annu. Rev. Earth Planet. Sci. 31 429–467.

Hamill T M, Hagedorn R and Whitaker J S 2008 Probabilis-tic forecast calibration using ECMWF and GFS ensembleforecasts, part II: precipitation; Mon. Weather Rev. 1362620–2632.

Johnson C and Swinbank R 2009 Medium-range multimodelensemble combination and calibration; Quart. J. Roy.Meteor. Soc. 135 777–794.

Krishnamurti T N, Kishtawal C M, LaRow T E, BachiochiD R, Zhang Z, Williford C E, Gadgil S and SurendranS 1999 Improved weather and seasonal climate forecastsfrom multi-model super-ensemble; Science 285 1548–1550.

Krishnamurti T N, Kishtawal C M, Shin D W and WillifordC E 2000 Improving tropical precipitation forecasts froma multi-analysis super-ensemble; J. Climate 13 4217–4227.

Krishnamurti T N, Vijaya Kumar T S, Yun W T,Chakraborty A and Stefanova L 2006a Weather andseasonal climate forecasts using the superensembleapproach; In: Predictability of Weather and Climate (eds)Tim Palmer and Renate Hagedorn (ECMWF), Cam-bridge University Press, pp. 532–560.

Krishnamurti T N, Vijay Kumar T S and Mitra A K2006b Seasonal climate prediction of Indian summer mon-soon; In: The Asian Monsoons (ed.) Bin Wang, pp. 553–582.

Krishnamurti T N, Mishra A K, Chakraborty A andRajeevan M 2009 Improving Global Model Precipita-tion Forecasts over India using downscaling and the FSUsuperensemble. Part I: 1–5-Day forecasts; Mon. WeatherRev. 137(9) 2713–2735.

Mishra A K and Krishnamurti T N 2007 Current status ofmulti-model super-ensemble operational NWP forecast ofthe Indian summer monsoon; J. Earth Syst. Sci. 116(5)1–16.

Mitra A K, Dasgupta M, Singh S V and Krishnamurti T N2003 Daily rainfall for Indian monsoon region frommerged satellite and raingauge values: Large-scaleanalysis from real-time data; J. Hydrometeorol. (AMS)4(5) 769–781.

Mylne K R, Evans R E and Clark R T 2002 Multi-model multi-analysis ensembles in quasi-operational

52 A K Mitra et al

medium-range forecasting; Quart. J. Roy. Meteor. Soc.128 261–384.

Richardson D S 2001 Ensembles using multiple models andanalyses; Quart. J. Roy. Meteor. Soc. 127 1847–1864.

Roy Bhowmik S K and Durai V R 2008 Multi-model ensem-ble forecasting of rainfall over Indian monsoon region;Atmosfera 21(3) 225–239.

Roy Bhowmik S K and Durai V R 2010 Application of multi-model ensemble techniques for real time district levelrainfall forecasts in short range time scale over Indianregion; MAP 106(1–2) 19–35.

Webster P J, Magana V O, Palmer T N, Shukla J, TomasR A, Yanai M and Yasunari T 1998 Monsoons: Processes,predictibility and the prospects for prediction; J. Geo-phys. Res. 103C 14,451–14,510.

Whitaker J S, Wei X and Vitart F 2006 Improving week-2 forecasts with multimodel reforecast ensembles; Mon.Weather Rev. 134 2279–2284.

Woodcock F and Engel C 2005 Operational consensus fore-casts; Weather Forecast. 20 101–111.

Woods A 2006 Medium range weather prediction (USA:Springer Publications) 261 pp.

MS received 10 March 2010; revised 15 September 2010; accepted 21 September 2010

experimental real-time multi-model ensemble (mme

Documents