Plans and Progress

Data Preparation

CFS Reanalysis

Left to Right: Chi-Fan Shih, Will Spangler, Joey Comeaux, Bob Dattore, Steven Worley,

Dave Stepaniak, Hua Ji, Doug Schuster, Gregg Walters, Cecilia Banner

NCAR Data Support Section (and Roy Jenne) supplied high qualityNCAR Data Support Section (and Roy Jenne) supplied high qualityobservations for R1, R2, ERA40, JRA25, NARR, MERRA, & CFSRRobservations for R1, R2, ERA40, JRA25, NARR, MERRA, & CFSRR

TOVS, SSM/IGOES-IR

FROM CLASS

COADS

A number of NCDC datasets provided via NCAR

CFSRR Input Datasets Available from NCDC Archives

•TEMP PILOT AIREP PIREP ACARS SATOB SYNOP MARINE BOGUS

•R1/R2 merged data sources from NCAR, NCDC, NCEP, JMA, and ECMWF

•R1/R2 merged qc’d datasets were packaged and delivered to the JMA-25 project

•R1/R2 datasets were reprocessed and enhanced by NCEP/NCAR for ERA-40

•NCEP obtained the ECMWF MARS archive of conventional data, 1979-1994

•NCAR improved versions of key datasets, ie, COADS, ON29/RAOBS, ON124

•R1/R2 and new datasets were merged for the North American Regional Reanalysis

•The latest versions of all source datasets were merged for the MERRA project

•Much effort was made by MERRA people to find data gaps and other problems

•CFSRR will use conventional observations prepared for MERRA, 1978-2000

•CFSRR will use NCEP/NCO operational archives to overlay and complete 1997-2007

The History of Conventional Data Preparation for Reanalyses at NCEP, 1994-2007

1920 1930 1940 1950 1960 1970 1980 1990 2000--------------------|----|---------|---------|---------|---------|---------|---------|---------|---------|basua-----|1948-2005|----|----|----|----|----|----|--*****************************************************cards-----|1946-1966|----|----|----|----|----|----|*********************---|----|----|----|----|----|----|china-----|1954-1962|----|----|----|----|----|----|----|---*********--|----|----|----|----|----|----|----|countries-|1946-1993|----|----|----|----|----|----|************************************************-|----|france----|1948-1979|----|----|----|----|----|----|--********************************|----|----|----|----|gate------|1974-1974|----|----|----|----|----|----|----|----|----|----|----|---*|----|----|----|----|----|jma-------|1978-1994|----|----|----|----|----|----|----|----|----|----|----|----|--*****************|----|lie-------|1967-1967|----|----|----|----|----|----|----|----|----|----|-*--|----|----|----|----|----|----|mars------|1979-1994|----|----|----|----|----|----|----|----|----|----|----|----|---****************|----|mit-------|1958-1963|----|----|----|----|----|----|----|----|--******-|----|----|----|----|----|----|----|navyship--|1985-1996|----|----|----|----|----|----|----|----|----|----|----|----|----|----************---|navyua----|1966-1973|----|----|----|----|----|----|----|----|----|----|********-|----|----|----|----|----|on20------|1962-1972|----|----|----|----|----|----|----|----|----|-***********--|----|----|----|----|----|on29------|1973-2000|----|----|----|----|----|----|----|----|----|----|----|--****************************permships-|1973-1980|----|----|----|----|----|----|----|----|----|----|----|--********----|----|----|----|ptarmigan-|1950-1961|----|----|----|----|----|----|----************---|----|----|----|----|----|----|----|ruships---|1947-1978|----|----|----|----|----|----|-********************************-|----|----|----|----|rusice----|1954-1978|----|----|----|----|----|----|----|---*************************-|----|----|----|----|rusnp-----|1950-1991|----|----|----|----|----|----|----******************************************---|----|rusua-----|1960-1990|----|----|----|----|----|----|----|----|----*******************************----|----|safrica---|1961-1967|----|----|----|----|----|----|----|----|----|*******--|----|----|----|----|----|----|scherhag--|1954-1962|----|----|----|----|----|----|----|---*********--|----|----|----|----|----|----|----|td52------|1922-1971|----|-**************************************************---|----|----|----|----|----|td53------|1919-1969|---***************************************************|----|----|----|----|----|----|td54------|1943-1974|----|----|----|----|----|--********************************|----|----|----|----|----|td90------|1943-1962|----|----|----|----|----|--********************--|----|----|----|----|----|----|----|twerle----|1975-1976|----|----|----|----|----|----|----|----|----|----|----|----**---|----|----|----|----|usafua----|1973-1980|----|----|----|----|----|----|----|----|----|----|----|--********----|----|----|----|usctrl----|1946-1969|----|----|----|----|----|----|************************|----|----|----|----|----|----|--------------------|----|---------|---------|---------|---------|---------|---------|---------|---------| 1920 1930 1940 1950 1960 1970 1980 1990 2000

List of Radiosonde/Pibal Datasets Collected at NCEP for Reanalysis

A quantitative estimate of duplication in source archivesA quantitative estimate of duplication in source archives

Conventional observation counts. R3 is current collection of source archivesConventional observation counts. R3 is current collection of source archives

TOVSTOVS

TEMP/PILOTTEMP/PILOT

SYNOPSYNOP

AIRCFTAIRCFT

COADSCOADS

SATOBSATOB

PAOBPAOB

Except for TOVS, these datasets form the basis of the CFSRR conventional obs 1979-1997Except for TOVS, these datasets form the basis of the CFSRR conventional obs 1979-1997

After 1997 the NCEP CDAS and OPS archives are used as conventional data sourcesAfter 1997 the NCEP CDAS and OPS archives are used as conventional data sources

VTPRVTPR

The diagram illustrates how processing information such The diagram illustrates how processing information such as QC results, and observation modifications are recorded as QC results, and observation modifications are recorded in the BUFR files during the execution of the reanalysis. in the BUFR files during the execution of the reanalysis. One temperature level is shown as an example. One temperature level is shown as an example.

A BUFR data structure was designed for R1 to record feedback information A BUFR data structure was designed for R1 to record feedback information in the observations which is still used in operations and the CFSRR as wellin the observations which is still used in operations and the CFSRR as well

11stst guess guess

ob errorob error

analysisanalysis

““event” stackevent” stack

TOVS, ATOVS, AQUA Archives for CFSRR

TOVS HIRS/2 and MSU 1-B orbital files were obtained from NESDIS/SAA spanning all satellites for the years 1978-1997 for the NARR project. Synoptic 1-C BUFR brightness temperature files were created from the 1-B archive.

The NESDIS/NCDC CLASS server has since become operational and is a great resource for satellite data. Available data is constantly being expanded. Some data not available from CLASS may be obtained directly from NCDC.

For CFSRR, the NARR HIRS/2 and MSU 1-B archives are enhanced, and new holdings extended through to the present. The complete set of SSU orbit files were added to the archives, as well as AMSU-A, AMSU-B, HIRS/3, HIRS/4, MHS, AIRS, and AMSR-E data. Data sets were obtained through CLASS and NCEP archives.

New codes to convert raw orbit files to calibrated synoptic BUFR BT files are created for MSU, and SSU. Improvements include new calibrations for MSU NOAA-10-14 (Cheng-Zhi Zou), and more use of the imbedded QC data found in the orbital files.

1980 1990 2000 2010

AIRS

AMSR-E

AMSU-A

AMSU-B

HIRS/2

HIRS/3

HIRS/4

MHS

MSU

SSU

CLASS NCEP

CFSRR TOVS-ATOVS-AQUA Satellite Data Timelines

Satellite Start Date End Date

F-10 07/12/93 11/14/97

F-11 07/12/93 05/16/00

F-13 05/03/95 Present

F-14 05/07/97 Present

F-15 02/23/00 Present

F-16 11/20/05 Present

SDR orbitsssmit synoptic

BUFR filesspssmi ocean surface wind speed BUFR files

Defense Met Sat ProgramSpecial Sensor Microwave Imager

SSM/I Sensor Data Records

Availability of GOES data in CLASS is shown in the following table. 

Satellite Start Date End Date  Location

SMS-1 01/26/79 04/19/79 East

SMS-2+ 04/19/79 08/05/81 East

GOES-1 11/29/82 >05/31/83  Variable

GOES-2 02/20/78 01/26/79 East

GOES-3 11/20/78 03/05/81 West

GOES-4 03/05/81 11/26/82 West

GOES-5 08/05/81 07/30/84 East

GOES-6 06/01/83 01/21/89 Variable

GOES-7 03/25/87 01/09/96 Variable

GOES-8 09/01/94 04/01/03 West

GOES-9 01/09/96 07/21/98 GMS-5

GOES-9++ 04/23/03 07/13/04 West

GOES-10 07/21/98 06/21/06 West

GOES-11 06/21/06 Present West

GOES-12 04/01/03 Present East

High density SNDR and IMGR data is available in CLASS starting with GOES-8 in 1994. With some help from NESDIS/NCDC with cloud processing, we could hope to go at

least that far back. There was an IR sounder (VAS) on GOES-4-7 satellites, but whether anything can be done with them for reanalysis is currently undetermined.

V8 is the most recent version of the backscattered ultraviolet (BUV) algorithms that have undergone three decades of progressive refinement. Its predecessors, V7 for total ozone, developed about in 1998, and V6 for profile ozone, developed in about 1990, have been used to produce the acclaimed TOMS total ozone and SBUV(/2) ozone profile time series. V8 will correct several small errors in V7 and V6 that were discovered by extensive error studies using radiative transfer models and by comparison with ground-based instruments. The V8T uses only two wavelengths (318 nm and 331 nm) to derive total ozone. Other wavelengths are used for diagnostics and error correction. Experience with TOMS and SBUV/2 suggests that the algorithm is capable of producing total ozone with rms error of about 2%, though these errors are not necessarily randomly distributed over the globe. The errors typically increase with solar zenith angle and in presence of heavy aerosol loading.

NESDIS V8 SBUV TOTAL AND PROFILE OZONE

RETRIEVALS, 1978-2007

SBUV and SBUV/2 Satellite Time Periods for CFSRR

0

1

2

3

4

5

6

7

8

9

10

1978 1980 1982 1984 1986 1988 1990 1992 1994 1996 1998 2000 2002 2004 2006 2008

N7 N9a N11a N9d N14 N11d N16 N17 N18

Subset of the SBUV ozone retrievals selected for CFSRRSubset of the SBUV ozone retrievals selected for CFSRR

a – ascending node d – descending nodea – ascending node d – descending node

1980 1990 2000 2010

This shows when data types enter the CFSRR stream

TEMP/PILOTSYNOPAIRCRAFTMARINESATOBBOGUS

HIRSMSUSSU

SSMIWND

VADWNDPROFILRMETARERSWND

ACARSQKSWND

MSONET

HIRS3AMSUAAMSUB

GOESND 5X5AIRS

AMSREHIRS4MHS

GPSIPW

GOESND 1X1

GEOIMR

V8 OZONE

Tropical Cyclone Synthetic Wind Data

1978-2002

CSFRR will use Mike Fiorino’s TC Bogus wind dataset as an aid to locating storm vortices in the GSI analysis. The synthetic winds consist of a central location surrounded by four wind profiles.

SEA ICE for CFSRRSEA ICE for CFSRR

NASA GSFC recomputed (for instrument drifts) from Oct 1978 to approx 1996, when it will turn over to the NCEP ice.

The NCEP ice will be recomputed with present analysis methods, land masks, algorithms, etc. from 1995/10/01 to 2000/02/25 from prior grids of brightness temperatures.  From 2/25/2000 to

present using archived SSM/I passes.

The GLERL Great Lakes analysis will be used for the historic coverage of those lakes.  NIC Great Lakes will continue this towards the present.

GSFC analysis does not cover all ice covered areas of our interest. Those (great lakes, peripheral (to the arctic) lakes and seas) will be filled by similar method to NARR -- identifying proxy points and filling from there.  Proxies will be found from the 2/25/2000 to present period, where there is

full data available and the procedure can be continued into the future.

Half degree ice will be produced consistently throughout the period (vs. Reanalysis 1 switching from Nomura 1 degree to NCEP 1 degree to NCEP 0.5 degree).  Starting with 2/25/2000, a 5' lat-

long grid will also be available.

Where there are temporal gaps, they'll be filled by linear interpolation in time.  This is particularly a problem for the Great Lakes.  It is not currently a concern, with 3 SSM/I instruments flying, and

AMSR-E also providing comparable data for any area which can be analyzed by automated means.

Ingest of Snow Analyses for CFSRR• Current Ops T382 GDAS ingests 2 daily snow analyses

– global Air Force analysis (Air Force non-grib format, one file per hemisphere)– N. Hemisphere NESDIS 4-km analysis (known as IMS) in grib format– ops job-step (known as SNOW2MDL) by George Gayno of EMC merges the two

snow analyses above and maps merged anal toT382 GFS grid (or any GFS grid)

• CFS RR will assimilate both Air Force and IMS (as in Ops) for those years that both are available (1997-present) & only Air Force analysis before 1997 – Using SNOW2MDL job step, modified to account for data availability below

• 23-km resolution until about 2005 (date TBD), then 4-km resolution thereafter• IMS 23 km analysis available from Feb 1997 through the present • IMS 4 km analysis available from Feb 2004 through the present

– George Gayno has modified his job SNOW2MDL to be able to use only Air Force analysis prior to the 1997 start of NESDIS IMS analysis

– Air Force analysis available for all years needed• 1979 to 1998 in Grib format from COLA

» M. Fennessy of COLA obtained Air Force analyses for above period from NCDC/ETAC and then gribbed it with help from EMC

• 2002 to present in Air Force non-grib format used in present ops– (via EMC archive of Jesse Meng

• 1999-2001 (to be extracted from NCO run history files)• Jan-Oct 1999 period may need special attention due to CRAY fire impact on archives

– Option 1: Use archived GDAS “snogrb” file– Option 2: Obtain 1999 Air Force analysis from NCDC/ETAC archive and have COLA grib it

Some SST Options for CFSRR

• OI 1/4 Degree Daily Analysis from NCDC– AVHRR only satellite source thru May 2002 – Option of including AMSR June 2002 onward, greatly improving

satellite coverage in cloudy regions (especially important for daily analysis)

– May not be available in time:• Currently 1985-present, undergoing revision (perhaps available by

October 2007, per Dick Reynolds)• Nov 1981-Dec 1984, would need to first compute daily AVHRR SST

retrievals. (Pathfinder Project if funded, or Reynolds himself)

• OIv2 1-degree Weekly Analysis– Lower resolution and no AMSR satellite data– More consistent– Available Nov 1981 – present (AVHRR period only)

• If necessary, consider spinning backwards to required start date, using satellite quality field as initial first guess to retain some gradients

Some data sets we still hope to get for CFSRR

Reprocessed GMS winds used in JRA-25 (NCAR DSS)

ERS-1 and ERS-2 winds prior to mid-1997 (NCAR DSS)

Reprocessed GPS Occultation data from COSMIC (NCAR)

GOES SOUNDER 1x1 radiances (CLASS)