About uncertainties of non-radioactiveatmospheric pollution modelling

Fernando MartínHead of the Atmospheric Pollution Division

Environment DepartmentCIEMAT

TERRITORIES WorkshopMadrid, Spain, 13-14 June 2018

Index

• Introduction• What is model uncertainty?• How to measure the model uncertainty?• Type of errors.• Sources of error.• How to reduce uncertainty?

Air Quality Introduction• Main pollutants:

– Primary: NOx, Particles (PM10, PM2.5), CO,SO2.

– Secondary, formed from primary: Ozone (fromVOC+NOx), secondary particles (PM10, PM2.5).

• Main sources. Most are area or linesources:

– Traffic. Urban areas.– Residential or domestic combustion. Urban

areas– Power generation and industries. Urban or

suburban areas.– Waste treatment and disposal. Urban or

suburban areas.– Agriculture and livestock farming. Rural areas.

• Effects of air pollutants on:– Health– Vegetation (ecosystems and crops)– Buildings

• How air quality is controled:– Air quality monitoring networks– Air quality modelling

Air Quality Modelling

• Main processes:– Emission– Transport and difussion.– Chemical reactions (ozone and acid rain)– Deposition (dry and wet)

• Type of models:– Gaussian models. Few used except for point sources.– Lagrangian models.– Eulerian models. Very used.

• Several scales: From planetary or hemispheric up to urban/street scale.

• Applications:– Air quality assessment. What is the air quality in a region?– Air quality impact. What is the impact of pollutant source?– Air quality forecast. What will the air quality be?– Air quality improvement. What is the impact of strategies for

improving air quality?

What is model uncertainty?

• Model uncertainty:– How the models represents the real world?– How well the models results fits the observations?

• EU Directive 2008/50/EC (on ambient air quality and cleaner airfor Europe) definition:– “The accuracy for modelling … is defined as the maximum deviation of the

measured and calculated concentration levels, over the period considered bythe limit value, without taking account the timing of the events”.

– Focused for models used for air quality assessment (no timing is relevant).It is important to estimate well the exceedance of air quality standards in anarea, but not exactly when it happens

– Not valid for forecasting (timing is relevant). Need to predict well the airpollutant concentrations in a place in the correct time.

Modelling quality objectives in the EuropeanDirectives

What is model uncertainty?

How to measure the modeluncertainty? Comparison of modeloutputs and observations fromexperimental campaigns and/orrecorded data by airquality/meteorological stations.– Graphical techniques– Statistical techniques.

• Very important to predict well thehigh concentrations.

• Need to compare model outputswith measured data which arerepresentative of an area similar tothe model resolution.

0 2 4 6 8 10 12 14 16 18Predicción

0

2

4

6

8

10

12

14

16

18

Obs

erva

ción

Scatter PlotObs vs PredObs = 0.61 * pred + 1.79

R=1R=2

R=0.5

))C-C()C-C((

))C-C)(C-C((=r

1/22pp

2oo

ppoo

2)( po CCRMS

)C+C0.5(

C-C=FB

po

po

Some metrics

• US Environmental Protection Agency (US EPA, 1991; 2005) providesguidances for model validation where monitoring stations data are denseenough and for pollutant concentrations above a threshold (120 g/m3 forozone)– Mean Normalized Bias (NMBE)– Mean Normalized Gross Error (NMGE)– Unpaired Peak Prediction Accuracy (UPPA). Refered to maxima hourly

concentrations in the modelled domain for every day of the simulated period

• Model quality objectives. Criteria for good performance:– MNBE less than 5, 15%;– MNGE less than +30, +35%;– UPA less than 15, 20%.

100)'t,'x(C

)'t,'x(C)t,x(CA

maxo

maxomaxpu

Some metrics• Very important to predict well the exceedances of air

quality standards• Statistics for category forecast: Accuracy (A), Bias (B),

False Alarm Rate (FAR), Critical Success Index (SCI),Probability of Detection (POD), Skill Score (SS)

Some metrics• Mathematical formulation of the AQ Directive

quality objectives (Denby et al, 2011, Guidanceon the use of models for the European AirQuality Directive)

• Relative Directive Error (RDE) definedmathematically at a single station as follows:

where OLV is the closest observedconcentration to the Limit Value concentration(LV) and MLV correspondingly ranked modelledconcentration.

• The maximum of this value found at 90% ofthe available stations is then the MaximumRelative Directive Error (MRDE).

Some tools

• OpenAir tool (UK NaturalEnvironment Research Council(NERC)

• Openair is an R package developedfor the purpose of analyzing airquality data

• Many functions for air quality modelevaluation using the flexiblemethods to easily evaluate modelsby season, hour of the day etc.

• These include key model statistics,Taylor Diagram, ConditionalQuantile plots.

Model Quality Objectives:FAIRMODE Approach – Delta tool

When comparing model outputs with observations, uncertaintyof observations U0 has to be taken into account.

Model Quality Objectives:FAIRMODE Approach – Delta tool

What value for U0?• Uncertainty varies according to

concentration (estimates basedon monitoring)

• Uncertainty data provided bythe experimentalist communityor measurement inter-comparison exercises.

Model Quality Objectives:FAIRMODE Approach – Delta tool

ConcentrationU

0

Delta Tool Software(FAIRMODE – Thunis et al, 2012, 2017,…)

The Target diagram (Pederzoli, Thunis et al, 2012)

2 = 2 + 2+ 2 − 22

X - Y

leftSDondominatesR1

rightRondominatesSD1

Right - Left

=

Radius = ≤ 1

Type of errorsCo= Coa + C0' + C0 Cp= Cpa + Cp' + Cp

Perfect observation (no errors)

observed stochastic (random) variability

data error in Co (i.e., instrument)

Predicted ensemble average

Predicted (random) variability

Input data error

− = − + + + ∆ + ∆Total modeluncertainty

Model errorPhysics, chemistry,numerical

Stochasticuncertainty

Data errors(measurementsand model inputs)

Better model design,less model error,But more model inputsrequired

Better model inputs,less data error

(Hanna and Drivas, 1987)

Example of model uncertainty

• EURODELTA III project.• Simulating air pollutant concentrations and deposition in Europe

with six models

20061 JUN-30 JUN

Maps of wetdeposition of NOx

Sources of modelling uncertainty

MODELLING ERRORS• Model formulation and parameterization. Missing processes and

approximations within the model that do not take into account all the realprocesses and effects:

– Chemical schemes, including rate constants and unaccounted reactions and processdescriptions in both gas and aerosol phases

– Boundary layer parameterization especially turbulence closure or parametrization.– Transport and dispersion (e.g. boundary layer description and vertical exchange)– Surface/air interaction and deposition rates– Sub-grid effects, higher order chemical processes associated with non-homogenous

concentration distributions• Finite numerical scheme.

– Numerical errors (aliasing, numerical diffusion, truncation, etc).– Approximations associated with grid sizes (especially mean grid concentrations) and

time steps. Need of choosing a suitable grid cell and time steps for the processes tosimulate.

Effect of no using chemical reactions of NOx

Madrid24 Octubre 2017

TECNAIRE. Reunión Comité Científico-Técnico

20

12 UTC 12 UTC

NO2R (ppb) (NO2R − NO2T)

NOx-O3 photostationarystate mechanismCFD _street canyonmodelling

Meteorological stationSonic anemometersAir Quality Station

NO2 ExperimentalNO2T non-reactiveNO2R reactive

Sources of modelling uncertaintyINPUT DATA ERRORS:• Emissions data and inventories. Probably the most important source of uncertainty:

– Missing sources– Emission rates, some times wrong or based on outdated emission factors.– Emission timing, very difficult to assign time profiles of emissions for a huge number of sources– Spatial disaggregation. Horizontal and vertical position of emissions, including stack heights and

plume rise models– VOC speciation (many compounds, natural and anthropogenic sources), fraction NO2/NOx– Size distribution of primary PM (depends of the source type)

• Meteorological input data,– Sometimes comes from meteorogical stations but generally meteorological models are used.

• Boundary and initial conditions. Nested domains from large scales to smaller ones.• Geographical data. Need of a suitable spatial resolution:

– Land use data. Some data can be outdated. It can be important in regions with importantchanges (fast growing urban areas)

– Topography. Generally the data are good except in the case of using low resolution data whenhigh resolution is required

Uncertainty in emission inventories

How to reduce the model uncertainty?

• Improving the model, especially in chemicalprocesses, boundary layer parametrization,turbulence schemes,…

• Improving the input data, especially for emissioninventories

• Post processing of the model outputs:– Data assimilation or fusion to reduce model bias using

observed concentrations in air quality stations– Data assimilation to improve spatial resolution