1

How to Choose a Laboratory that Will Meet Your Data Assessment and Testing Needs?2013 NEMCLegal Defensibility of Data

Prepared by Kim Watson, RQAP-GLPDeborah Gaynor, Ph.D, Phoenix Chemistry Services

2

Where We’ve Been – 1970’s

Selecting a Laboratory:

Location, Price, Size

Preference, Performance

Law of Environmental Services--

Purchase any two:

■ QUALITY

■ COST

■ TURNAROUND TIME

3

Where We Are Now - 2013

Selecting a Laboratory:

Location, Price, Size

Preference, Performance

Law of Environmental Services--

Purchase any two:

■ QUALITY

■ COST

■ TURNAROUND TIME

■ MULTIPLE ACCREDITATIONS

4

Are We Evaluating Laboratories Effectively?

■ What is the cost of accurate, defensible, analytical measurements??

■ If it isn’t worth it, don’t buy it.

■ If it isn’t written down, it didn’t happen.

■ Why do sampling and analysis if resulting data isn’t traceable, defensible, and good science.

5

Are We Evaluating Laboratories Effectively?

■SCIENCE – WHAT IS IT ALL ABOUT?

-Lord KelvinQuote.pdf

6

Where We Need To Be

Extended List of Criteria Should Include the Following■ QAPP – Value of the UFP QAPP, Standard Formats and Consistency

■ SOPs – Read them – Don’t Just Collect them

■ Data Quality Objectives

Measurement Performance Criteria

Accuracy and Precision

■ DOCs – Performed with modifications

■ Calibrations – How are they performed? per method.

■ Full Data Packages – Organization as per CLP with Bookmarks

7

Where We Need To Be – (con’t)■ Internal Data Review - Policy and procedures for data package

review and manual integration review.

■ EDDs – What’s your favorite format?

■ Chromatography (Signal/Noise), Interpretation of TICs

■ Reporting Error Rate

Including Sample Login

■ Non Analytical Services

Shipping Services

Bottle Orders

■ PE Performance – Policy for suspending an analyte

8

Where We Need To Be-Examples

■ SOPs: Read them – Don’t Just Collect them

Examples: Hexavalent Chromium and Method 8151A Extraction Modification

■ Data Quality Objectives: Setting Appropriate Data Quality Objectives

Measurement Performance Criteria

Accuracy and Precision

■ Calibrations – How are they performed

■ Non Analytical Services

Shipping Services

Bottle Orders

9

Ask for and Examine SOPsAsk for the laboratory’s SOPs for the methods they will run for you. Include

the sample preparation SOPs as well as the analytical SOPs.

■ Example 1:

The laboratory will be analyzing hexavalent chromium in soils. At the validator’s request,

they provided the SOPs for not just analysis of hexavalent chromium, but also for

measurement of pH and oxidation-reduction potential (ORP).

Guess what: the ORP SOP did not list a platinum (or other noble metal) electrode in the

materials section, and the procedure did not include instructions for using an indicator

electrode as well as a reference electrode. This opens up the question whether or not

the laboratory actually is measuring ORP, or just obtaining erroneous readings from their

pH meter.

10

Ask for and Examine SOPsAsk for the laboratory’s SOPs for the methods they will run for you. Include

the sample preparation SOPs as well as the analytical SOPs.

■ Example 2: The laboratory provided their preparation and analysis SOPs for Method 8151A

(chlorinated acid herbicides).

Here’s what the validator found upon comparing the soil extraction SOP to the method:

Method 8151A example

■ Example 3: A welcomed way to reference the differences between a Method to the Lab SOP.

■ MADEP EPH/PAH

Pages from 3-OL-2025 EPH_rev1_1_SOP.pdf

Method 8151A--Extraction Procedures for Solids

Method 8151A (reference method per laboratory SOP)

1. Add 50 g soil to Erlenmeyer; adjust

pH to 2 with concentrated HCl. Monitor pH (<2) for 15 minutes to ensure stability.

2. Add 20 mL acetone and shake 20

minutes. Add 80 mL diethyl ether and shake 20 minutes.

3. Decant extract and measure volume recovered.

4. Repeat acetone followed by ether extraction twice more, shaking 10 minutes with each solvent and decanting each time. Check extraction efficiency (volume extract recovered should be ≥ 75% solvent added); repeat extraction as needed. Check pH; re-acidify and re-extract if not <2.

Note: sonication may be used, and will produce a methylene chloride/acetone extract.

5. Hydrolyze in aqueous KOH (reflux in a water bath for 1-2 hours).

6. Transfer to a separatory funnel. Extract three times with 100 mL aliquots of methylene chloride (discard solvent extracts).

7. Acidify with H2SO4. Extract first with 40 mL diethyl ether and twice more with 20 mL diethyl ether; combine extracts.

8. Dry the ether extract, and perform clean-up if needed.

9. Concentrate to 0.5 mL. Add 0.1 mL methanol. Adjust volume to 1.0 mL with diethyl ether.

10. Esterify, then allow ether to evaporate until volume is about 2 mL; dissolve into 5 mL hexane.

Laboratory SOP

1. Add 5 g sample to a 40 mL VOA vial; add 20 mL distilled water and 1 dropper-full of H2SO4. Cap and shake thoroughly. Monitor pH (<2) for 15 minutes to ensure stability.

2. Add 20 mL diethyl ether, shake 2 minutes.

3. Volumetrically transfer a 5 mL

aliquot of the extract to a scintillation vial containing approximately 2 g anhydrous acidified sodium sulfate. Let rest at least 1 hour.

4. Quantitatively transfer to a 10 mL vessel containing 5 mL methyl tert-butyl ether (MTBE).

5. Hydrolyze in aqueous KOH (reflux in a water bath for 1-2 hours).

6. Transfer to a separatory funnel, and extract three times with 100 mL aliquots of methylene chloride (discard solvent extracts).

7. Acidify with H2SO4. Extract first with 40 mL diethyl ether and twice more with 20 mL diethyl ether; combine extracts.

8. Dry the ether extract, and perform clean-up if needed.

9. Concentrate extract to 2 mL. Add 250 μL methanol, and adjust to 3mL final volume using MTBE.

10. Esterify, then adjust volume to 5 mL using MTBE over 0.2 g silicic acid.

AMRO SOP OL-2025 Rev. 0 11/7/2005 Review date: 02/08/10

____________________________________________________________________________Extractable Petroleum Hydrocarbons Revision 1.1 MADEP-EPH-04 Page 25 May 2004

9.10.2.4 Evaluation of DDT breakdown, and Pentachlorophenol and Benzidine tailing is not required(section 7.3.1.1)

9.10.2.5 All Target PAH Analytes described in Table 2 must meet the initial and continuing calibrationrequirements for the SW-846 Method 8270C described in WSC-CAM-II B unless specificallyexcepted in this section.

9.10.2.6 Range Calibration Factors must be based on all the individual aliphatic or aromatic calibrationstandards described in Tables 1 and 2, that are included within the specified range as defined by theEPH marker compounds described in Table 5. Range Calibration Factors are determined bydividing the summation of the peak areas (Total Ion Current ) for all individual calibration standardcomponents that elute within a specified range (i.e., C9 – C18 Aliphatic Hydrocarbons, 6components) by the total concentration injected.

9.10.2.7 Evaluation of the System Performance Check Compounds (SPCC) and Calibration CheckCompounds (CCC) alone, as described in Sections 7.3.4 and 7.3.5, respectively, are insufficient toverify calibration. All target analytes must be evaluated in the ICAL and CCV and meet theperformance standards described in Table 7 below.

9.10.2.8 Evaluation of Continuing Calibration Standard (equivalent to the CCV described in SW-846Method 8270C ) standards is required at the beginning and end of each analytical sequence. Notefour compounds may exhibit percent differences or percent drifts greater than 25% but less than 40%in the terminal CCVs per Sec. 10.4.2. Use the macro SV2EPH to correct the ranges for the surrogateconcentrations, then print the Continuing Calibration Report to screen and to the printer. When asequence is complete use the macro CCVSUMSV to calculate and report each CCV initial-terminalpair..

9.10.2.9 The analytical batch for EPH analyses may include the analysis of up to 20 samples completedwithin 12 hours of the batch’s tune.

9.10.2.10 The performance standards for the EPH Aliphatic and Aromatic Ranges and comparableperformance standards for the Target PAH Analytes are presented below in Table 7. In additionto these performance standards, the performance standards for the Target PAH Analytes must alsomeet the requirements of SW-846 Method 8270C as described in WSC-CAM-II B, Table II B-1.

Table 7. Modified SW-846 Method 8270C Analytical QC Requirements and Performance Standards forTarget PAH Analyte and EPH Aliphatic and Aromatic Range Analyses

PERFORMANCE STANDARDQC ELEMENT Target PAH Data EPH Range and Alkane DataInitial Calibration (% RSD) ≤ 15 ≤ 25Opening CCV (%drift) ≤ 20 ≤ 25Closing CCV (%drift) ≤ 20 ≤ 25 Method Blanks < RL < RL

Internal Standard (IS)Area Count of IS must be within 50and 200% of associated OpeningCalibration*

Area Count of IS must be within50 and 200% of associatedOpening Calibration*

Surrogate Recovery 40 – 140% 40 – 140%Fractionation Surrogate Recovery Not Required 40 – 140%Laboratory Control Sample (LCS) 40 –140% 40 –140%LCS Duplicate (RPD) <20 for water, <30% for soil/sediment < 25Matrix Spike (MS)/MS Duplicate1 40 –140%; RPD < 50 40 –140%; RPD < 50LCS/LCSD Naphthalene or2-Methylnaphthalene Breakthrough

≤ 5 % for either constituent in EPHaliphatic fraction2

≤ 5 % for either constituent inEPH aliphatic fraction

1. At discretion of data user. * Use the macro SISSY3 to generate the internal standard summary report.2. Naphthalene and 2-Methylnaphthalene must be measured in EPH aliphatic fraction of each sample for GC/MS

analysis. Sample must be re-fractionated if concentration of either compound >5% in aliphatic fraction

9.10.3 If the Aliphatic range concentrations are quantified by GC/MS, Naphthalene and , 2-Methylnaphthalene must beidentified and quantified in the aliphatic hydrocarbon fraction using SW-846 Method 8270C, using an internal

PDF created with FinePrint pdfFactory trial version http://www.pdffactory.com

11

Set Appropriate Data Quality Objectives

■ MDLs (LODs) –

Are they realistic?

REALISTIC - Laboratory runs a sensitivity check with each Initial Calibration

■ Measurement Performance Criteria (MPCs)

Accuracy and Bias:

Surrogate and LCS recovery limits for all matrices (uncertainty) eXAMPLE CLimits WIDE.pdf

Anything <10% is NOT an acceptable lower control limit!

Quality Control Results

Job Number: 480-39446-1Client: WaterMark Environmental, Inc.

Water

1.0

Lab Control Sample - Batch: 480-122386

Client Matrix:

Lab Sample ID:

Dilution:

Instrument ID:

Lab File ID:

Initial Weight/Volume:

Final Weight/Volume:

R1092.D

5 mL

5 mLUnits: ug/L

Method: 624

Preparation: N/A

HP5973RLCS 480-122386/4

Analysis Date: 06/05/2013 2238

Analysis Batch:

Prep Batch:

Leach Batch:

480-122386

N/A

N/A

Prep Date:

Leach Date:

N/A

N/A

Analyte QualLimit% Rec.ResultSpike Amount

20.0 20.9 104 52 - 1621,1,1-Trichloroethane

20.0 18.0 90 46 - 1571,1,2,2-Tetrachloroethane

20.0 19.8 99 52 - 1501,1,2-Trichloroethane

20.0 20.0 100 59 - 1551,1-Dichloroethane

20.0 17.6 88 1 - 2341,1-Dichloroethene

20.0 20.2 101 49 - 1551,2-Dichloroethane

20.0 20.3 101 1 - 2101,2-Dichloropropane

20.0 18.8 94 18 - 1901,4-Dichlorobenzene

100 105 105 1 - 3052-Chloroethyl vinyl ether

20.0 21.2 106 37 - 151Benzene

20.0 15.3 77 45 - 169Bromoform

20.0 20.1 100 1 - 242Bromomethane

20.0 19.6 98 70 - 140Carbon tetrachloride

20.0 20.7 103 37 - 160Chlorobenzene

20.0 17.4 87 53 - 149Chlorodibromomethane

20.0 20.2 101 14 - 230Chloroethane

20.0 20.9 104 51 - 138Chloroform

20.0 17.3 86 1 - 273Chloromethane

20.0 18.5 92 1 - 227cis-1,3-Dichloropropene

20.0 18.4 92 35 - 155Dichlorobromomethane

20.0 21.0 105 37 - 162Ethylbenzene

20.0 20.4 102 1 - 221Methylene Chloride

20.0 20.0 100 64 - 148Tetrachloroethene

20.0 20.2 101 47 - 150Toluene

20.0 20.3 101 54 - 156trans-1,2-Dichloroethene

20.0 17.7 89 17 - 183trans-1,3-Dichloropropene

20.0 20.5 102 71 - 157Trichloroethene

20.0 22.3 111 17 - 181Trichlorofluoromethane

20.0 19.8 99 1 - 251Vinyl chloride

Surrogate % Rec Acceptance Limits

1,2-Dichloroethane-d4 (Surr) 95 72 - 130

4-Bromofluorobenzene (Surr) 98 69 - 121

Toluene-d8 (Surr) 94 70 - 123

TestAmerica Buffalo Page 45 of 865

12

Set Appropriate Data Quality Objectives

■ Measurement Performance Criteria (MPCs)

Precision:

-What is realistic for the matrix? Normal soils may not produce the tight precision risk assessors want to see!

“One must not expect more precision than the subject matter allows”

Aristotle, The Nichomachean Ethics

13

Are Calibrations Performed Appropriately?

■ SW-846 Methods set the guidance, but the TNI standard must set the boundaries.

Method 8000C for example..

BFB Tune Acceptance Criteria

Single scan BFB.pdf

Multi-day calibrations are being performed by many laboratories, and ABs do not have a consensus on what is acceptable.

8260_Rawdata-2.pdf

524VOC_CalibrationSummaryCorrection.pdf .

dhg

Callout

improperly evaluated per SW846

14

Are Calibrations Performed Appropriately? (con’t)

■ SW-846 Methods set the guidance, but the TNI standard must set the boundaries. (con’t)

Laboratories must push the boundaries to compete.

8000C discussion

Multi-point calibrations are being performed that cannot be verified, yet the laboratory remains NELAC-accredited.

multi-point IC example

Laboratory Response to validator’s challenge to multi-day Initial Calibration:

“The SW846 reference in 8000C, pg.51 states "It is recommended that if a reanalysis is to be performed it should commence within the time frame of the original initial calibration event or 8 hours from the original analysis if such a time frame is undefined by the method." This is recommendation and a general rule that is typically followed but it is a recommendation. Since SW846 is a guidance document and not a prescriptive method, this was allowed as acceptable by the Volatiles department.”

file:///C|/Phoenix/WEM/OMYA%20Proctor/102476_103224/RE_%...oin%20regarding%20SDG%20Nos_%20102476%20and%20103224.txt

From: "Kathleen Noonan-Kelley" <kathn@eailabs.com>To: "Deborah Gaynor" <dgaynor@phoenixchemistryservices.com>Cc: "Pam Gagnon" <pamg@eailabs.com>; "Miles E. Waite" <mwaite@waiteenv.com>Subject: RE: questoin regarding SDG Nos. 102476 and 103224Date: Monday, October 31, 2011 8:57 AM

Hello Deb-

The software limits the number of points which may be viewed/printed on theForm 6. The response factors on the Form 6 do represent all of the pointsused in the calibration. The analysts reviews each compound electronicallywhen determining the acceptability of the initial calibration. The softwareallows the analyst to view all of the points for each compound and asses theresponse factors or R values for method compliance.

I apologize for the omission of the 5035 log. Please see attached. This logindicates the weights and volumes associated with the vials/samples for102476. A unique ID, 5mLs of methanol and a "prep weight" is recorded foreach vial before it leaves the laboratory in a bottle order. A "finalweight" is taken when the vials are received back at the lab. The resulting"net weight" reflects the mass of the sample used in calculation. Shouldany sample require additional volumes of methanol it would be noted in thesoil prep logbook.

EAI's 8260B SOP is also attached.

Any additional questions regarding data may be forwarded directly to me.

Thank you for your time.

file:///C|/Phoenix/WEM/OMYA%20Proctor/1024...g%20SDG%20Nos_%20102476%20and%20103224.txt (1 of 6) [11/8/2011 2:21:53 PM]

15

Expect the Best Non-Analytical Services

■ Shipping Services –

Bottle orders, packing materials included, field packs, courier service.

■ Sample Receiving –

Immediate log-in.

COCs emailed back to sampler or field project manager within 48 hours of receipt.

Sample Acceptance criteria are scientifically defensible.

SampleTemperatureAcceptancePolicy.pdf

16

Expect the Best Non-Analytical Services

■ Data Deliverables

Full Data Package as described in the CLP SOW (for full validation) or or find out what “CLP like” looks like…

Choice of EDD formats—done correctly!

■ Responsiveness

Laboratory Project Manager is always available by phone or email

Corrective Actions are performed if needed

Revisions are received promptly

17

What Can the Accreditation Sector Do?

■ What can the Laboratory Accreditation Industry do to increase the value of laboratory services?

■ Is it appropriate to expect more from an accredited laboratory?

What are the differences between accredited and non-accredited laboratories?

Does consistency of performance matter to purchasers of laboratory services?

Does accreditation contribute to marketing success?

18

What Can the Accreditation Sector Do?

■ What is the role of cost in laboratory selection?

If results are rejected due to laboratory deficiencies, what is the cost to the data user?

If data results end up in litigation, are they robustly defensible?

Is there a cost differential (and how big is it?) between accredited and non-accredited laboratories?

■ Are expectations set appropriately?

Impact on protection of human health.

Impact on protection of ecosystem health and viability.

19

Value Added by Accreditation

■PE Performance Tracking Notification of failures

and suspicions

Passing 2 out of 3 consecutive PEs

■Hands and Eyes on Laboratory’s Quality System and Technical Performance!

20

Value Added by Accreditation

■Accreditation by Matrix, Technology, Analyte

■Accreditation by Matrix & Technology (NEFAP)

21

Value Added by Accreditation

■Third Party Assessors and Assessments Technically Sound Science

Strong understanding of analytical systems and software

Strong understanding of management systems

22

ACKNOWLEDGMENTS

Stone Environmental Inc.and

Phoenix Chemistry Services

Would like to thank all the laboratories that provided examples!

23

Questions?