Integrating Community RT Components into JCSDA CRTM

Yong Han, Paul van Delst, Quanhua Liu, Fuzhong Weng,

Thomas J. Kleespies, Larry M. McMillin

Outline

Part I• Project objective• Approach• CRTM components• CRTM implementation status• Plans• Issues

Part II

CRTM framework (Paul van Delst)

JCSDA 3rd Workshop on Satellite Data Assimilation,20-21 April 2005

Project Objective

Fast and accurate community radiative transfer model to enable assimilation of satellite radiances under all weather conditions

Approach

• Integrate community RT components

• Provide CRTM framework to the community to minimize efforts in integrating RT components into the CRTM

• Interact with the community research groups during the integration process: assisting implementation and modifying the framework to accommodate their needs.

CRTM Components

Forward CRTM

SurfaceEmissivity/Reflectivity

Model(s)

AerosolAbsorption/Scattering

Model

GaseousAbsorption

Model

CloudAbsorption/Scattering

Model

RT Solution Source Functions

public interfaces

CRTM Initialization CRTM DestructionJacobian CRTM

CRTM Framework

• By Nov. 2004, the framework for both forward and Jacobian models was completed and distributed together with the documents.

• The framework details user and developer interfaces, data structures and program layouts

• The community is now using the framework as a vehicle to integrate RT components into the CRTM

Gaseous Absorption Module

• Function: provide gaseous (water vapor, Ozone, dry gases, etc.) optical depth profiles

• Models: OPTRAN and OSS (AER)

• Integration status:

– OPTRAN forward, Tangent-linear and Adjoint models have been integrated with the CRTM framework and tested.

– OSS forward model has been preliminarily integrated with the CRTM framework;

• OSS- and OPTRAN-based CRTMs

Channel Loop

Gaseous Optical depth(OPTRAN)

RT Solution

Channel loop done?

yes

no

channel i

R_chi

{R_ch1 , R_ch2, …, R_chn}

OPTRAN-based CRTM flowchart

CRTM Initialization

Cloud optical parameters

Aerosol optical parameters

Surface emiss. & reflect.

OPTRAN transmittance coefficients

Cloud optical parameter lookup tables

Aerosol optical parameterdatabase

Surface emissivity and reflectivity database

Computermemory

Node Loop

Gaseous Optical depth(OSS)

RT Solution

Loop over those channels engaged with node i

Channel loop done?

R_chk = R_chk + wkRi

Node loop done?

yes

yes

no

no

Node i

{R_ch1 , R_ch2, …, R_chn}

Cloud optical parameters

Aerosol optical parameters

Surface emiss. & reflect.

CRTM Initialization

OSS OD lookup table

Cloud optical parameter lookup tables

Aerosol optical parameterdatabase

Surface emissivity and reflectivity database

Computermemory

OSS weights & node-channel map

Computermemory

OSS-based CRTM flowchart

Surface Emissivity & Reflectivity Models

Microwave: Land – LandEM (Weng et al., 2001) Snow and sea ice (Yan & Weng, 2003) Ocean – wind vector dependent (Liu and Weng, 2003); wind speed dependent (English, 1998)

Infrared: Ocean – IRSSE (van Delst, 2003; Wu-Smith, 1997) Land – measurement database for 24 surface types in visible and infrared (NPOESS, Net Heat Flux ATBD, 2001) - regression method

Integration into CRTM will be completed in June, 2005

Cloud optical parameter module

NESDIS/ORA lookup table (Liu et al., 2005): mass extinction coefficient, single scattering albedo, asymmetric factor and Legendre phase coefficients

– IR: spherical particles for liquid water and ice cloud (Simmer, 1994); non-spherical ice cloud (Liou and Yang, 1995; Macke, Mishenko et al.; Baum et al., 2001).

– MW: spherical particles for rain drops and ice cloud (Simmer, 1994).

Integration with CRTM will be completed in June

Aerosol optical parameter module

• The initial version includes only dust aerosol absorption (no scattering) - aerosol optical depth profile (NASA GSFC).

• Integration into the pCRTM (current operational RTM) is completed; integration with CRTM is underway.

RT Solution Module

• Four RT solvers being integrated into CRTM

• Solve RT equations for a plane-parallel, multiple-layer atmosphere

RT Solution Module

• UW Successive Order of Interaction (SOI) – Truncated doubling technique to compute layer transmission,

reflection and source functions; SOS (successive orders of scatterings) to integrate emission and scattering events from surface to the top of atmosphere (Heidinger et al., 2005), IR and MW.

– Forward, tangent-linear and adjoint models.

– The three models have been preliminarily integrated with the CRTM framework.

RT Solution Module

• NOAA/ETL Discrete-ordinate tangent linear radiative transfer model (DOTLRT)

– Matrix operator method to compute layer transmission, reflection and source function, adding method to combine layers and surface (Voronovich et al., 2004), IR and MW.

– Forward and Jacobian models and HG phase function lookup table

– Codes were received in February with the DOTLRT integrated with an earlier version of the CRTM framework (forward interface only). Now ETL is revising the codes.

RT Solutions (cont.)

• UCLA vector -4 stream model

– Delta-4 stream algorithm to compute layer transmission, reflection and source function analytically; adding method to combine layers and surface (Liou et al., 2005), IR and MW.

– Forward and Jacobian models.

– Forward model is being integrated into CRTM.

RT Solutions (cont.)

• NESDIS/ORA Vector DIScrete-Ordinate Radiative Transfer (VDISORT)– Solve for full polarimetric vector, multiple stream radiative

transfer equation with polarization from surface and atmosphere as well as their interaction (Weng and Liu, 2003), VIS, IR and MW.

– Forward and Jacobian models.

– Forward model integration will be completed in June

– Will be used as a benchmark and research tool

Plans

• By the end of June, 2005, a beta version CRTM will be completed with the following components:

– Gaseous absorption modules: OPTRAN and OSS if completed

– Cloud optical parameter databases: ORA and ETL lookup tables

– Surface emissivity and reflectivity module with LandEM, MW SeaIce/Snow emissivity model, MW Ocean emissivity model, IRSSE, and IR land emissivity database.

– RT solution modules: VDISORT and the following modules or programs if completed: UW SOI, ETL RT Solver and UCLA Vector Delta-4 Stream.

Plans (cont.) CRTM test and assessment

• Before passing the CRTMs to the data assimilation system for impact evaluation, we will work with the community to test and assess the CRTMs for

(1) software reliability, stability and maintainability

(2) model accuracy

(3) computation efficiency

(4) memory use

Note that we assume the developers will fix software bugs and any other deficiencies in their codes.

• To test the software and models, we will soon provide a set of model inputs including surface data for ocean, land, snow, and ice, and profiles of temperature, water vapor, ozone, water, ice and aerosol parameters.

• We will also provide theoretical results for comparisons. Data may be created by LBLRTM and VDISORT, or other models such as Doubling-Adding method, Monte Carlo methods.

• Sensors: AIRS, AMSU, HIRS, and WINDSAT

Plans (cont.)

• Testing of the beta version CRTM will be completed at the end of September and the tested code will be provided to JCSDA.

• Continue to work with the community to integrate RT components.

• Conduct comparisons between CRTM calculations and observations (CloudSat CALIPSO, ARM, etc.)

Issues

• Layer to level profile conversion

• OPTRAN vs. OSS

Layer to level profile conversion

• The NWP system produces layer temperature profiles, but some RT components require level temperature profiles

• Possible solutions:

(1) Assuming Tlayer(i) = 0.5*(Tlevel(i-1) + Tlevel(i)), with known Ts and

{Tlayer(i), i=1, n}, solve the equation for {Tlevel(i), i=0, n}

(2) Predict {Tlevel(i), i=0, n} from Ts and {Tlayer(i), i=1, n} using regression

technique: y = Ax

(3) Interpolation

Examples of layer to level temperature conversion

Original level profileA layer profile is constructed from it:T_lay(i) = 0.5*(T_lev(i)+T_lev(i+1))

The difference between the original level profile and that retrieved from the layer profile by solving the equations. 0.5 k error is added to the surface air temperature.

The difference between the original level profile and that by interpolating the layer profile on the level grids. 0.5 k error is added to the surface air temperature.

Comparison between OPTRAN and OSSComparison between OPTRAN and OSS

Yong Han, Larry McMillin and Xiaozhen XiongYong Han, Larry McMillin and Xiaozhen Xiong

NOAA/NESDIS/ORANOAA/NESDIS/ORA

Jean-Luc Moncet, Gennadi Uymin and Sid BoukabaraJean-Luc Moncet, Gennadi Uymin and Sid Boukabara

AER, Inc AER, Inc

Data sets for the comparisonsData sets for the comparisons

• UMBC 101 level 48 profile setUMBC 101 level 48 profile set• ECMWF 101 level 52 profile setECMWF 101 level 52 profile set• For each set the following data are prepared:For each set the following data are prepared:

– LBLRTM SRF-averaged gaseous transmittances for training LBLRTM SRF-averaged gaseous transmittances for training OPTRANOPTRAN

– LBLRTM Monochromatic radiances for training OSSLBLRTM Monochromatic radiances for training OSS

– Ground-truth channel radiances obtained by convolving Ground-truth channel radiances obtained by convolving LBLRTM monochromatic radiances with the SRFsLBLRTM monochromatic radiances with the SRFs

• Settings for the independent data set: Settings for the independent data set: – Specular surface is assumed: IR emissivity = 0.98; MW Specular surface is assumed: IR emissivity = 0.98; MW

emissivity = 0.6emissivity = 0.6

– Surface pressures are varied among different profilesSurface pressures are varied among different profiles

• Data are prepared (by AER, Inc) for the following sensors:Data are prepared (by AER, Inc) for the following sensors:AIRS_aqua, HIRS3_n17, AMSU_n17, SSMIS_f16AIRS_aqua, HIRS3_n17, AMSU_n17, SSMIS_f16

But results shown here only for AIRS, HIRS, AMSU and SSMISBut results shown here only for AIRS, HIRS, AMSU and SSMIS

Problem in choosing a common training data setProblem in choosing a common training data set

Initially we want to train and test OPTRAN and OSS Initially we want to train and test OPTRAN and OSS with the same data sets, but unfortunately OPTRAN with the same data sets, but unfortunately OPTRAN and OSS are sensitive to different issues and and OSS are sensitive to different issues and therefore have different requirements for the therefore have different requirements for the training data. OPTRAN is better trained with the training data. OPTRAN is better trained with the UMBC set and OSS is better trained with five UMBC set and OSS is better trained with five perturbations of the ECMWF set.perturbations of the ECMWF set.

OSS independent test (UMBC48 set)

-0.05

0

0.05

0.1

0.15

0.2

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

AMSU_n17 channel number

Tb

dif

fere

nc

e b

etw

ee

n O

SS

an

d L

BL

OPTRAN independent tes t (UM BC48 set)

-0.05

0

0.05

0.1

0.15

0.2

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

AMSU_n17 channel number

Tb d

iffer

ence

bet

wee

n O

PTR

AN

and

LB

L (K

)

RMS difference

Mean difference

OPTRAN vs. OSS at AMSU channels

OSSTrained with ECMWF setTested with UMBC set

OPTRANTrained with ECMWF setTested with UMBC set

0

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

1 201 401 601 801 1001 1201 1401 1601 1801 2001 2201

AIRS channel num ber

RM

S d

iffe

ren

ce

(K

)

0

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

1 201 401 601 801 1001 1201 1401 1601 1801 2001 2201

AIRS channel number

rms

(K)

OPTRAN-V7 vs. OSS at AIRS channels

OSSTrained with ECMWF setTested with UMBC set

OPTRANTrained with UMBC setTested with ECMWF set

Water vapor Jacobians at strong water vapor channels

Water vapor Jacobians at weak water vapor channels

Computation & Memory Efficiency Computation & Memory Efficiency

OPTRAN-V7Forward, Jacobian+Forward

OPTRAN-compForward, Jacobian+Forward

OSSJacobian+Forward

AIRS 7m20s, 22m36s 10m33s, 35m12 3m10s

HIRS 4s, 13s 5s, 17s 9s

Time needed to process 48 profiles with 7 observation angles

OPTRAN-V7OPTRAN-V7

single, doublesingle, doubleOPTRAN-compOPTRAN-comp

double precisiondouble precisionOSSOSS

Single precisionSingle precision

AIRSAIRS 33, 6633, 66 55 9797

HIRSHIRS 0.26, 0.50.26, 0.5 0.040.04 44

Memory resource required (Megabytes)

SummarySummary

• Radiance accuracy:Radiance accuracy:– Trained with the ECMWF data set (for a nominal accuracy = 0.05K) and tested Trained with the ECMWF data set (for a nominal accuracy = 0.05K) and tested

with the UMBC set, OSS has an overall accuracy better than 0.05 K; trained with with the UMBC set, OSS has an overall accuracy better than 0.05 K; trained with the UMBC data set and tested with the ECMWF data set, OPTRAN has an overall the UMBC data set and tested with the ECMWF data set, OPTRAN has an overall accuracy better than 0.1 Kaccuracy better than 0.1 K

– A good OSS feature is that its radiance accuracy can always be improved by A good OSS feature is that its radiance accuracy can always be improved by increasing the number of nodes. However, there is a trade-off between the increasing the number of nodes. However, there is a trade-off between the accuracy and the computation and memory efficiencies.accuracy and the computation and memory efficiencies.

• Jacobian accuracy:Jacobian accuracy:– Both OPTRAN and OSS provide accurate temperature Jacobians and Jacobians for Both OPTRAN and OSS provide accurate temperature Jacobians and Jacobians for

strong absorbersstrong absorbers

– The OSS Jacobian model may perform poorly for weak absorbers due to the fact The OSS Jacobian model may perform poorly for weak absorbers due to the fact that OSS is trained in radiance space and the weak absorbers are weighted low that OSS is trained in radiance space and the weak absorbers are weighted low under the training thresholds; OPTRAN can provide reasonable Jacobians for under the training thresholds; OPTRAN can provide reasonable Jacobians for weak absorbers because OPTRAN is trained in transmittance space and errors for weak absorbers because OPTRAN is trained in transmittance space and errors for each gaseous components are minimized.each gaseous components are minimized.

• Computation efficiency:Computation efficiency:OSS is significantly faster than OPTRANOSS is significantly faster than OPTRAN

• Memory requirement:Memory requirement:– The amount of memory taken by OSS depends not only on the number of The amount of memory taken by OSS depends not only on the number of

channels, but also on the degree of node overlap. For the sensors considered channels, but also on the degree of node overlap. For the sensors considered here, OSS takes significantly more memory than OPTRAN. here, OSS takes significantly more memory than OPTRAN.

– Compact OPTRAN is superior in memory use, taking only a small fraction of the Compact OPTRAN is superior in memory use, taking only a small fraction of the amount of memory required by OSS and OPTRAN-V7. amount of memory required by OSS and OPTRAN-V7.