Quality Control/Quality Assurance

Annabelle Allison

ITEP/TAMS Center

2

Quality Assurance

PLAN

DO

CHECK

FIX

KEEP DOING IT!

3

Quality Assurance Definitions

Common sense written down

“Integrated system of management activities involving planning, implementation, assessment, reporting, and quality improvement to ensure that a process, item, or service is type and quality needed and expected by client”

4

Quality System

Structured, documented management system

Describes policies, principles, organizational authority for ensuring quality needed in product

Provides framework for planning, doing, documenting, & assessing work and for carrying out QA/QC activities

5

Quality Control (QC)

Technical activities Measures performance of the process Compares against defined standards“Customer” sets standards & requirements

QC is part of QA

6

Quality Control

Check (Measure… compare with limits… graph it… document and report)

Fix

7

Quality

System: management (common sense)

Assurance: policies (serve our community to produce needed result)

Control: assessments (we check up on our work)

8

American National Standard ANSI/ASQC E4-1994

Specifications and Guidelines for Quality Systems for

Environmental Data Collection and Environmental Technology Programs

National consensus standard authorized by American National Standards Institute (ANSI)Developed by American Society for Quality Control (ASQC)Consistent with ISO standardsEPA Guidance for Quality Assurance Project Plans: EPA QA/G-5 based on E4

9

Why is a QAPP Important?

Saves moneySaves timeHelps communicationNo later misunderstandingsEverything contained in or referenced in one documentPublic document in case of litigation

10

Using SOPs

Easier to write than QAPPs

Describe what is done

Once SOPs are written, QAPP is much easier

Reference and attach SOPs to QAPP

11

Using Existing Materials

Plagiarism when writing QAPPs and SOPs is okay!Guidance for the Preparation of Standard Operating Procedures (SOPs) for Quality-related Documents: EPA QA/G-6 Use template QAPPs for BAM and TEOM in TAMS diskette

12

6 Elements of Data Management

(1) Data processing and transmission (WHAT)

(2) Data end use and integrity protection (WHY)

(3) Data access (WHO)

13

6 Elements of Data Management (cont.)

(4) Data dissemination (WHERE)

(5) Data storage and retrieval (HOW)

(6) Data disposal (WHEN)

Document everything!

14

Error

…the difference between your answer and the “truth”

15

Continuous PM QC

Instrument stability checks (EPA calls this general QA in their 1995 memo; see Attachment A in the TAMS template QAPPs)

Precision

Bias

16

Instrument Stability

Internal diagnostics conducted with each measurement (no error codes)TEOM uses standard filter kit available from dealer BAM uses standard foil or internal sourceUse these stability test kits more frequently at beginning of program, thereafter as necessary and at least twice a year

17

Results of Stability Checks

Use dealer’s guidelinesWithin 2.5% for TEOM standard filterWithin 7% for BAM foilPlot results on control chart (see Excel table and linked chart as figure 14-1 in TAMS template QAPPs)

18

Precision Error

Unavoidable

Sometimes up, sometimes down–“random”

Estimated by measuring the same thing several times

Minimized by carefully following procedures

19

Precision

Collocated analyzers is too expensive, so use repeated measurements of the same thing to estimate precision

Flow rate is the most important parameter for PM

Flow rate changes with time—how much it varies is estimate of precision error

20

Two Ways To Estimate Precision of Flow Rate

At beginning of program, use external flow rate transfer standard in field and compare flow rate transfer standard’s result with that shown (or set) on analyzer (one-point check)

Conduct this check at least every 2 weeks

21

During the First Months

Ensure external flow rate transfer standard is checked against another flow rate transfer standard at least once (every six months if you want to use second, easier way to assess precision)Record every check of analyzer’s flow rate, verify it’s within 4% of external flow rate transfer standard’s flow rate for 3 consecutive biweekly checks

22

Easier Way To Estimate Precision after Several Months

Use “alternative procedure:” Record analyzer’s set point flow rate, assume same flow rate if measured by external flow rate transfer standardRecord analyzer’s indicated flow rateCalculate relative percent difference, use value as precision error of flow rate (use relative percent difference calculation; see cells B5 – G28 in the Tribal Data Analysis spreadsheet, pg “PM10 Automated”)

23

Bias Error

Minimized by calibrating equipment against a standard

Make sure standard is certified against NIST-traceable standard (keep certificate!)Compare transfer standard to field equipment

Measures any bias in your equipment

24

Definitions

EPA calls “accuracy” what others call “total error”

Accuracy, or total error, combines both precision and bias errors

25

Continuous PM Accuracy

Can estimate only accuracy of flow rate Most important parameter, so that’s okay

Use external flow rate transfer standard to compare indicated (or set) flow rate of analyzer with that shown by transfer standard

26

Accuracy (cont.)

Flow rate transfer standard must not be same used to calibrate analyzer

Audit may be conducted by tribal staff, but not routine site operator

Conduct flow rate audit at least once per year for each analyzer

27

Accuracy Calculations

Relative percent difference between audit flow rate transfer standard and your analyzer’s flow rate is estimated accuracy of your flow rate

No other (inexpensive) way to estimate accuracy, so this is your estimated accuracy, or total error