data quality report card: just how good are we? · 2015. 1. 8. · data quality report results for...
TRANSCRIPT
Data Quality Report Card: Just How Good Are We?
January 15, 2015 Aman Bains
1
Presentation Outline
Background on Quality Measurements and Objectives
Data Quality Report Results for 2013 ARB’s PQAO Challenges Looking Ahead
2
3
Measurement Quality Objectives
Measurements are made in order to assess the quality of the data produced One-point QC checks Flow rate verifications Performance audits “Objectives” are acceptance criteria for quality attributes, usually dictated in CFR, for rendering data for record to be of “good quality.”
Precision, Accuracy and Bias
Precision Good Accuracy Good
Precision Good Accuracy Bad
Precision Bad Accuracy Good
4 Bias distortion in one direction
Tools for Assessing Precision
5
Pollutant Precision
1-pt QC checks
Collocated Measurements
Gaseous CO, NO2, O3, SO2
Continuous PM2.5 PM10
Manual PM2.5
PM10 (high-vol) PM10 (low-vol)
Tools for Assessing Accuracy/Bias
6
Pollutant Accuracy/Bias
1-pt QC checks
Flow Rate Verifications (required in
AQS)
Flow Rate Verifications
(not required in AQS)
Performance Audit
Gaseous CO, NO2, O3, SO2
Continuous PM2.5 PM10
Manual PM2.5
PM10 (high-vol) PM10 (low-vol)
Measurement Quality Objectives (Criteria)
7
Pollutant Precision % (CV) Bias % Accuracy via
Audits %
Gaseous 1-pt QC Checks Through- the-Probe
CO 10 ± 10 ± 15 NO2 15 ± 15 ± 15 O3 7 ± 7 ± 10
SO2 10 ± 10 ± 15
Particulates Collocated Samples Flow Rate Flow Checks
PM10 10 ± 4 ± 10 PM2.5 10 ± 4 ± 4
Data Quality Report Provides data producers and users an assessment
of data quality Identify where improvements are needed Ambient data capture rates Precision and accuracy of criteria pollutants Compared against measurement quality
objectives (federal criteria) Gases (CO, O3, NO2, SO2) Particulates (PM10, PM2.5)
8
AMP 430 “Ambient” Data Capture Data producers should review data quarterly to ensure
that data are uploaded in a timely manner.
Goal is to have at least 75% of the possible ambient data uploaded to AQS.
9
AMP 256 Data Quality Indicator Report
10
2013 Ambient Data Reported
11
*
*Includes ARB and 21 other air monitoring districts reporting under ARB’s PQAO
*
2013 Ambient Data Reported
12
* *
*Includes ARB and 21 other air monitoring districts reporting under ARB’s PQAO
2013 Data Quality Results – Gases
Percentage of sites uploading 100% 1-pt QC Precision Criteria
Met?
Accuracy/ Bias
Criteria Met?
Pollutant ARB Bay Area
San Diego
South Coast
CO 90 100 67 86 NO2 87 100 88 69 O3 93 100 89 90
SO2 80 100 100 63 13
2013 Ambient Data Reported
14
*Includes ARB and 21 other air monitoring districts reporting under ARB’s PQAO
* *
2013 Data Quality Results – Particulates
Pollutant ARB* Bay Area
South Coast
San Diego
Precision Criteria
Met?
Accuracy / Bias
Criteria Met?
PM10 100 100 100 100
PM2.5 99 100 100† 100 x
15 *Collocation requirement (15%) not met in ARB’s PQAO
†South Coast met criteria for one method for the year 2013
Three-Year Average at ARB’s PQAO Level
16
Pollutant 2011-2013 Result (%) Criteria (%)
Gaseous Precision Bias Precision Bias
CO 4.70 ±3.61 10 ± 10
NO2 5.35 ±3.89 15 ± 15
O3 3.75 ±2.80 7 ± 7
SO2 4.18 ±3.22 10 ± 10
Particulates Precision Bias Precision Bias
PM10 5.19 ±1.28 10 ± 4
PM2.5 20.56 ±0.50 10 ± 4
Percent of Sites Meeting the Precision Criteria for Gases – 2013
Pollutant US ARB Bay Area
South Coast
San Diego
CO 98 97 100 100 100 NO2 99 98 100 100 100 O3 98 98 100 100 100
SO2 98 100 100 100 100
17 This assessment is performed at the site level.
Percent of Sites Meeting the Precision Criteria for PM – 2013
Pollutant US Region 9* ARB Bay
Area South Coast
San Diego
PM10 69 50 100 100 100 100
PM2.5 55 14 18 0 67 0
18 Percentage is based on the number of sites with collocated samplers.
* Includes Arizona, Nevada, and Hawaii. Excludes California.
Why PM2.5 Precision Criteria More Difficult to Meet?
Lower concentrations lead to higher imprecision Lower cut-off limits for evaluation: PM2.5: 3 µg/m3
PM10: 15 µg/m3
Higher cut-off limits lead to better precision
Improve precision through operational practices Consistency in practice should help
19
Summary: Precision and Accuracy
20
Precision Accuracy
Gases
Particulates
o Less than required collocated sites
o Annual precision criteria not met for PM2.5, but improving
*
* PM flow rate verification data should be uploaded
Challenges Air Quality System (AQS) Accurate reflection of network Uploading data Correct coding (e.g., collocated samplers) Important to get accurate info in AQS
Other Investigate PM2.5 precision
21
Looking Ahead AQS training and corrections Adjust PM collocated monitors Improve PM2.5 precision Upload PM flow rate verification data Develop additional assessment tools (graphics) Provide timely data quality reviews Continue using 3-year average to track trends
and identify possible data quality issues
22
Questions? Aman Bains, (916) 323-0032, [email protected] Patrick Rainey, Manager (916) 327-4756, [email protected] Reports available at ARB’s website: www.arb.ca.gov/aaqm/qa/dqreports/dqreports.htm
23