Next-generation OMI SO2 Retrieval Algorithm based on Principal

Component AnalysisCan Li1,2, Joanna Joiner2, Nick Krotkov2, P. K. Bhartia2

1ESSIC, University of Maryland College Park2NASA GSFC

18th OMI Science Team MeetingKNMI, De Bilt, The Netherlands

March 13, 2014

Email: Can.Li@nasa.gov

Outline• Background and Motivation• Methodology (Framework)• Application to OMI– Results (Planetary Boundary Layer SO2)

– Results (Volcanic SO2)

• Data Continuity: Comparison of OMI and OMPS

• Next Steps and Conclusions

Background and Motivation

•Motivation: Band Residual Difference (BRD) algorithm fast and sensitive, but large noise and artifacts (only 3 pairs of wavelengths)•Objective: develop an innovative approach to utilize the full spectral content from OMI while maintaining computational efficiency

Operational OMI SO2 (Sept. 2004 – Feb. 2008)

Basis – Spectral fitting algorithms

First look at the DOAS Equation:Measured sun-normalized radiances

Various gas absorbers (O3, SO2 etc.)

Rayleigh and Mie scattering, surface reflectance etc.

The Ring effect

Plus additional measurement artifacts terms (e.g., wavelength shift, stray light, etc.) and/or radiance data

correction schemes

Utilization of the full spectral content, but some terms are difficult to model (e.g., RRS)

Methodology (Framework): PCAInstead of explicit modeling of ozone, RRS, and other

instrumental features, we use a data-driven approach based on principal component analysis (PCA) with spectral fitting

Measured N-value spectrum

PCs from SO2-free regions, (O3 absorption, surface reflectance, RRS, measurement artifacts etc.) other than SO2 absorption

Pre-calculated SO2 Jacobians (assuming O3

profiles, albedo, etc.)

SO2 column amount

Fitting of the right hand side to the spectrum on the left hand side -> SO2 column amount and coefficients of PCs

(See Guanter et al., 2012; Joiner et al., 2013; Li et al., 2013)

Application to OMI•Spectral window: 310.5-340 nm – avoid stray light at shorter wavelengths

•Each row (scan position) processed individually – different characteristics between different rows of the 2-D CCD

•Each swath processed individually – account for orbit-varying dark current

•# of PCs determined dynamically – exclude SO2-related PCs and avoid overfitting by checking the correlation between PCs and SO2 Jacobians

•1st step: Simple Jacobians similar to those used in operational BRD algorithm for straight-forward comparison

Step 1 Ps

See Li et al., [GRL, 2013] for details

Principle Components and Residuals

(a-c) First few PCsBlue line: scaled reference Ring spectrum

(d) Least squares fitting residuals for a pixel near Hawaii

(Var.% 99.8492)

(Var.% 0.1264)

Example PCs from entire row # 11, Orbit 10990

(Var.% 0.0217)

(Var.% 5.32E-5)

(Var.% 4.79E-5)

PC #1: Mean spectrum

PC #2: O3 absorptionPC #3: Surface reflectance (also Ring signature)

PCs #4 and #5: likely measurement artifacts, noise (>99.99% variance explained)

Smaller residuals with SO2 Jacobians fitted

Results: noise and artifact reduction

•PCA algorithm reduces retrieval noise by a factor of two as compared with the BRD algorithm•SO2 Jacobians for PCA algorithm calculated with the same assumptions as in the BRD algorithm

August, 2006

OMI operational BRD

Results: Boundary layer pollution SO2

eastern U.S., August 2006

PCA Operational BRD

PCA algorithm reveals major SO2 point sources (circles), with much reduced noise and artifacts.

Ex. Sudbury, Canada (~220 kt in 2006) Ex. analyzed with pixel averaging (super sampling) reveals

details of emission sources [e.g., Fioletov et al., 2011]

PCA, 2006 BRD, 2006 only BRD, 2004-2012

•One year’s worth of PCA retrievals yield results similar to that from 3-5 years worth of BRD data. •Global survey shows that PCA SO2 removes most artifacts in BRD data without significantly altering signals from real sources (Fioletov and McLinden, personal communication)

Largely hidden by artifacts

Volcanic SO2: Kasatochi eruption August 7-8, 2008

For volcanic SO2, nonlinearity due to saturation at shorter wavelengths• Iteration of SO2 Jacobians (pre-calculated assuming loadings of 1 - 500 DU)• Shift of spectral fitting window to longer wavelengths

PCA closest to estimated released SO2 mass of ~2200 kt based on observed decay of SO2 [Krotkov et al., 2010]

Transport of the plume

Ln(SO2)

August 10, 2008

August 11, 2008

August 12, 2008

•Our algorithm eliminates the need for explicit instrument-specific radiance correction schemes

•Test on OMPS: minimal changes to algorithm –biggest is the use of OMPS slit function for Jacobians–spectral window, etc., same as in OMI

•Reduces the chance of introducing artifacts/biases between different instruments

Comparison with OMPS

OMI and OMPS comparison

OMPS and OMI PCA SO2 retrievals show good agreement despite somewhat different sampling

October, 2013

OMI and OMPS comparison

Both OMI and OMPS PCA SO2 retrievals show enhanced SO2 loading over northern China in January 2013, when severe pollution attracted media and public attention.

OMPS, Jan. 2013 OMI, Jan. 2013

OMI and OMPS comparison

OMI and OMPS PCA SO2 data show similar seasonal patterns and SO2 signals over eastern India (several coal-fired power plants built in recent years) [Lu et al., 2013].

Next Steps• Expanded table for SO2 Jacobians to more accurately account for

measurement conditions (e.g., O3 amount, reflectivity, geometries)

• Addition of scattering weight to output to allow convenient adjustment of SO2 column amount based on user-provided profile

• Inclusion of error estimates

• Operational implementation, public release – >1 year processed and currently under evaluation– Initial release for boundary layer pollution this year– Improved Jacobians and volcanic data to follow soon– On 12 CPUs, 1-2 days to process a year of OMI data

Conclusions• Significant improvements in retrieval quality – PCA algorithm

uses full spectral content from OMI and similar instruments offering increased temporal resolution and source detection

• Computation efficiency – over an order of magnitude faster than comparable spectral fitting algorithms; increasingly important given the greater data volumes expected from future missions (e.g., TROPOMI, TEMPO)

• Maximal data continuity between instruments – no need to develop instrument-specific radiance data correction schemes

• Flexibility – fitting window can be easily adjusted to optimize sensitivity under different conditions

Backups

Results: Daily boundary layer SO2August 13, 2006 August 14, 2006

August 15, 2006 August 16, 2006