development of chemistry indicators scientific steering committee meeting july 26, 2005 sediment...

78
Development of Development of Chemistry Indicators Chemistry Indicators Scientific Steering Committee Meeting July 26, 2005 Sediment Quality Objectives For California Enclosed Bays and Estuaries

Post on 18-Dec-2015

213 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Development of Chemistry Indicators Scientific Steering Committee Meeting July 26, 2005 Sediment Quality Objectives For California Enclosed Bays and Estuaries

Development of Development of Chemistry IndicatorsChemistry Indicators

Scientific Steering Committee Meeting

July 26, 2005

Sediment Quality Objectives

For California Enclosed Bays and Estuaries

Page 2: Development of Chemistry Indicators Scientific Steering Committee Meeting July 26, 2005 Sediment Quality Objectives For California Enclosed Bays and Estuaries

Objectives

Data preparation

SQG calibration and development

Validation

Conclusions

Next steps

Presentation OverviewPresentation Overview

Data screening & processing

Strata

Calibration & validation subsets

Existing national SQGs

Calibration of national SQGs

New approaches

Categorical classification

Correlation

Predictive ability

Page 3: Development of Chemistry Indicators Scientific Steering Committee Meeting July 26, 2005 Sediment Quality Objectives For California Enclosed Bays and Estuaries

Presentation OverviewPresentation Overview Objectives

Data preparation

SQG calibration and development

Validation

Conclusions

Next steps

Page 4: Development of Chemistry Indicators Scientific Steering Committee Meeting July 26, 2005 Sediment Quality Objectives For California Enclosed Bays and Estuaries

Chemistry IndicatorsChemistry Indicators

Several challenges to effective use

– Bioavailability

– Unmeasured chemicals

– Mixtures

Page 5: Development of Chemistry Indicators Scientific Steering Committee Meeting July 26, 2005 Sediment Quality Objectives For California Enclosed Bays and Estuaries

ObjectivesObjectives

Identify important geographic, geochemical, or other factors that affect relationship between chemistry and biological effects

Develop indicator(s) that reflect relevant biological effects caused by contaminant exposure

Develop thresholds and guidance for use in MLOE framework

Page 6: Development of Chemistry Indicators Scientific Steering Committee Meeting July 26, 2005 Sediment Quality Objectives For California Enclosed Bays and Estuaries

ApproachApproach

Use CA sediment quality data in developing and validating indicators

– Address concerns and uncertainty regarding influence of regional factors

– Document performance for realistic applications

Investigate multiple approaches

– Both mechanistic and empirical methods

– Existing methods used by other programs

– Existing methods calibrated to California

– New approaches

Page 7: Development of Chemistry Indicators Scientific Steering Committee Meeting July 26, 2005 Sediment Quality Objectives For California Enclosed Bays and Estuaries

ApproachApproach

Evaluate SQG performance

– Use CA data

– Use quantitative and consistent approach

– Select methods with best performance for expected applications

Describe response levels (thresholds)

– Consistent with needs of MLOE framework

– Based on observed relationships with biological effects

Page 8: Development of Chemistry Indicators Scientific Steering Committee Meeting July 26, 2005 Sediment Quality Objectives For California Enclosed Bays and Estuaries

Objectives

Data preparation

SQG calibration and development

Validation

Conclusions

Next steps

Presentation OverviewPresentation Overview

Data screening & processing

Strata

Calibration & validation subsets

Page 9: Development of Chemistry Indicators Scientific Steering Committee Meeting July 26, 2005 Sediment Quality Objectives For California Enclosed Bays and Estuaries

Data ScreeningData Screening

Appropriate habitat and geographic range

– Subtidal, embayment, surface sediment samples

Chemistry data screening

– Valid data (from qualifier information)

– Nondetect values (estimated)

– Completeness (metals and PAHs)• Minimum of 10 chemicals: metals and organics

– Habitat type (surface, embayment, subtidal)

Standardized sums:DDTs, PCBs, PAHs, Chlordanes

Page 10: Development of Chemistry Indicators Scientific Steering Committee Meeting July 26, 2005 Sediment Quality Objectives For California Enclosed Bays and Estuaries

Data ScreeningData Screening

Toxicity data screening

– Valid data

– Selection of candidate acute and chronic toxicity test

– Lack of ammonia interference• EPA toxicity test thresholds

– Acceptable control performance

– Matched data (toxicity and chemistry)• Same station, same sampling event

– Test method: amphipod mortality only• Eohaustorius or Rhepoxynius

Amphipod Mortality in Sediment (Paired Samples) from California

Amphipod (EE or RA) Mortality (%)

0 20 40 60 80 100

Am

pe

lisca

mo

rta

lity

(%)

0

20

40

60

80

100

Eohaustorius estuarius Rhepoxynius abronius

Page 11: Development of Chemistry Indicators Scientific Steering Committee Meeting July 26, 2005 Sediment Quality Objectives For California Enclosed Bays and Estuaries

Data ScreeningData Screening

Analyses Dataset Screening Steps

No. Samples Retained

Chemistry Total Number of Samples 6955

Valid data 6934

Nondetect values 6934

Completeness 5962

Habitat type 2768

Toxicity Total Number of Samples 3349

Valid data 3308

Selection of desire chronic and acute 3242

Lack of ammonia interference 3221

Acceptable control performance 3200

Matched data 1988

Amphipod survival data 1741

Page 12: Development of Chemistry Indicators Scientific Steering Committee Meeting July 26, 2005 Sediment Quality Objectives For California Enclosed Bays and Estuaries

Objectives

Data preparation

SQG calibration and development

Validation

Conclusions

Next steps

Presentation OverviewPresentation Overview

Data screening & processing

Strata

Calibration & validation subsets

Page 13: Development of Chemistry Indicators Scientific Steering Committee Meeting July 26, 2005 Sediment Quality Objectives For California Enclosed Bays and Estuaries

StrataStrataAre there differences in contamination among

regions of CA that are likely to affect the development of a chemical indicator?

Geographic Strata

– North (North of Pt. Conception)

– South (South of Pt Conception

Habitat Strata

– Ports, Marinas, Shallow

Magnitude of contamination

Relationship between contamination and toxicity

Page 14: Development of Chemistry Indicators Scientific Steering Committee Meeting July 26, 2005 Sediment Quality Objectives For California Enclosed Bays and Estuaries

StrataStrata

Northern CA Southern CA

Chemical Name Units N 50th

Percentile 90th

Percentile N 50th

Percentile 90th

Percentile

Cadmium mg/kg 847 0.21 0.47 1099 0.40 1.34 Copper mg/kg 824 42.10 67.98 1115 90.10 299.00 Mercury mg/kg 886 0.26 0.45 1112 0.28 1.02 Lead mg/kg 833 21.96 39.70 1115 46.30 122.60 Nickel mg/kg 817 86.72 116.03 1099 20.70 36.00 Tributyltin mg/kg 340 1.44 30.00 690 59.35 448.20 Zinc mg/kg 831 114.00 166.00 1115 198.00 388.60 Chlordane ug/kg 866 0.87 7.42 1169 8.84 68.63 Chlorpyrifos ug/kg 2 44.12 66.50 11 0.50 1500.00 DDT ug/kg 866 4.61 24.00 1169 25.84 115.97 Dieldrin ug/kg 879 0.24 2.03 926 0.82 2.80 Mirex ug/kg 667 0.12 1.12 631 0.25 1.00 ppDDE ug/kg 875 1.50 7.44 1162 12.72 68.97 PAHs ug/kg 845 1077.60 2736.68 1097 898.48 8575.66 PCBs ug/kg 759 8.19 35.94 957 31.40 219.33 Eohaustorius estuarius % 479 82.50 96.80 465 87.60 99.00 Rhepoxynius abronius % 105 86.30 100.00 505 80.90 95.80

Page 15: Development of Chemistry Indicators Scientific Steering Committee Meeting July 26, 2005 Sediment Quality Objectives For California Enclosed Bays and Estuaries

StrataStrata

Northern and Southern California Eohaustorius estuarius Mortality in Sediment vs. mSQGQ1q

mSQGQ1q

0.01 0.1 1 10 100

Mor

talit

y (%

)

0

20

40

60

80

100

Southern EE mSQGQ1q Northern EE mSQGQ1q

Page 16: Development of Chemistry Indicators Scientific Steering Committee Meeting July 26, 2005 Sediment Quality Objectives For California Enclosed Bays and Estuaries

Strata DecisionsStrata Decisions

Treat North and South as separate strata

– Different contamination levels and sources

– May be different empirical relationships with effects

– Adequate data for statistical analyses

Do not distinguish among habitat regions

– Limited data for some habitats

– Added complexity of application

Page 17: Development of Chemistry Indicators Scientific Steering Committee Meeting July 26, 2005 Sediment Quality Objectives For California Enclosed Bays and Estuaries

Objectives

Data preparation

SQG calibration and development

Validation

Conclusions

Next steps

Presentation OverviewPresentation Overview

Data screening & processing

Strata

Calibration & validation subsets

Page 18: Development of Chemistry Indicators Scientific Steering Committee Meeting July 26, 2005 Sediment Quality Objectives For California Enclosed Bays and Estuaries

Calibration and Validation DatasetsCalibration and Validation Datasets

Calibration/development dataset

– Screened data minus withheld validation data

– Calibration of SQGs

– Development of new SQGs

– Comparison of performance

Validation dataset

– Confirm performance of candidate SQGs

Page 19: Development of Chemistry Indicators Scientific Steering Committee Meeting July 26, 2005 Sediment Quality Objectives For California Enclosed Bays and Estuaries

Validation DatasetValidation Dataset

Independent subset of SQO database plus new studies

Approximately 30% of data, selected randomly to represent contamination gradient

North and South data are proportional between the calibration/development and validation datasets

Page 20: Development of Chemistry Indicators Scientific Steering Committee Meeting July 26, 2005 Sediment Quality Objectives For California Enclosed Bays and Estuaries

Bay/Estuary Samples inBay/Estuary Samples inDatabase After ScreeningDatabase After Screening

Number of Samples

(matched chemistry & toxicity)

Stratum

Calibration/Development

Validation

North 504 298

South 800 328

Page 21: Development of Chemistry Indicators Scientific Steering Committee Meeting July 26, 2005 Sediment Quality Objectives For California Enclosed Bays and Estuaries

Objectives

Data preparation

SQG calibration and development

Validation

Conclusions

Next steps

Presentation OverviewPresentation Overview

Existing national SQGs

Calibration of national SQGs

New approaches

Page 22: Development of Chemistry Indicators Scientific Steering Committee Meeting July 26, 2005 Sediment Quality Objectives For California Enclosed Bays and Estuaries

National SQGsNational SQGs

Two main types of approaches

– Empirical and Mechanistic

Empirical

– Intended to aid in prediction of potential for adverse impacts

– Derived from analysis of extensive field datasets

– Various approaches for development of chemical values

– Little explicit consideration of bioavailability

– Incorporate a wide range of chemicals

– Work best when applied to mixture of contaminants in a sediment

Page 23: Development of Chemistry Indicators Scientific Steering Committee Meeting July 26, 2005 Sediment Quality Objectives For California Enclosed Bays and Estuaries

Empirical SQGsEmpirical SQGs

SQG Metric SourceERM

Effects Range Median

Analysis of diverse studies and effects values

Mean Quotient for Chemical Mixture

Long et al.

Consensus MEC

Mid-range effect concentration

Geometric mean of similar guidelines

Mean Quotient for Chemical Mixture

MacDonald et al, Swartz, SCCWRP

SQGQ-1

Mid-range effect concentration

Subset of chemical guidelines from various sources

Mean Quotient for Chemical Mixture

Fairey et al.

Logistic Regression

Regression model for each chemical

Probability of Toxicity (Pmax) for Chemical Mixture

Field et al.

Page 24: Development of Chemistry Indicators Scientific Steering Committee Meeting July 26, 2005 Sediment Quality Objectives For California Enclosed Bays and Estuaries

National SQGsNational SQGs

Mechanistic

– Intended to assess potential for impacts due to specific chemical groups, not predict overall effects

– Derived using equilibrium partitioning and toxicological dose-response information

– Incorporate water quality objectives

– Explicit consideration of bioavailability

– Applicable to a restricted range of chemicals

– Work best when applied to specific contaminants

Page 25: Development of Chemistry Indicators Scientific Steering Committee Meeting July 26, 2005 Sediment Quality Objectives For California Enclosed Bays and Estuaries

Mechanistic SQGsMechanistic SQGs

SQG Metric Source

EqP Organics

Acute and chronic effects

Organic Carbon Normalized

Sum of Toxic Units (TU)

EPA + CA Toxics Rule

EqP Metals

Acid Volatile and Organic Carbon Normalized

Difference Between metal concentration and strong binding capability

EPA

Page 26: Development of Chemistry Indicators Scientific Steering Committee Meeting July 26, 2005 Sediment Quality Objectives For California Enclosed Bays and Estuaries

National SQGsNational SQGs

SQGs

Chemicals ERM SQGQ1 Consensus LRM EqP

Organics EqP

Metals Cadmium + + + + + Copper + + + + + Lead + + + + + Mercury + + + Nickel + + + + Tributyltin Zinc + + + + + Chlordane + + Chlorpyrifos DDTs + + + + Dieldrin + + + + + Mirex p’p’ DDE + + PAHs + + + + + PCBs + + + +

Other Metals 3 1 3 4 0 1 Other Organics 0 0 0 0 33 0

Page 27: Development of Chemistry Indicators Scientific Steering Committee Meeting July 26, 2005 Sediment Quality Objectives For California Enclosed Bays and Estuaries

Objectives

Data preparation

SQG calibration and development

Validation

Conclusions

Next steps

Presentation OverviewPresentation Overview

Existing national SQGs

Calibration of national SQGs

New approaches

Page 28: Development of Chemistry Indicators Scientific Steering Committee Meeting July 26, 2005 Sediment Quality Objectives For California Enclosed Bays and Estuaries

Calibration of National SQGsCalibration of National SQGs

Objective: Improve empirical relationship between chemistry and effects by modifying national SQGs to address potential sources of uncertainty

Variation in bioavailability of organics

Variation in natural background concentration of metals

CA-Specific variations in chemical mixtures

Differences in organic carbon content of sediment influences exposure

Metal content of sediment matrix varies according to particle type and source material

Relative proportions of contaminants within regions of State may differ from national average

Page 29: Development of Chemistry Indicators Scientific Steering Committee Meeting July 26, 2005 Sediment Quality Objectives For California Enclosed Bays and Estuaries

Organics Bioavailability CalibrationOrganics Bioavailability Calibration

TOC normalization to represent changes in bioavailability

– Conc./TOC

Evaluate whether predictive relationship for chemical classes is improved after normalization

– Correlation analysis

Use normalized values as basis for SQG calibration if there is evidence of improved predictive relationship

Page 30: Development of Chemistry Indicators Scientific Steering Committee Meeting July 26, 2005 Sediment Quality Objectives For California Enclosed Bays and Estuaries

TOC NormalizationTOC Normalization

Relationship to sediment toxicity is not improved by TOC normalization of organics

South

Spe

arm

an C

orre

latio

n (r

)-0.1

0.0

0.1

0.2

0.3

0.4

0.5NonNormCorrelation NormCorrelation

North

ChlordaneDDTs

LPAHHPAH

TPAHPCBs

Spe

arm

an C

orr

ela

tion

(r)

-0.1

0.0

0.1

0.2

0.3

0.4

0.5NonNormCorrelationNormCorrelation

Page 31: Development of Chemistry Indicators Scientific Steering Committee Meeting July 26, 2005 Sediment Quality Objectives For California Enclosed Bays and Estuaries

Metal Background CalibrationMetal Background Calibration

Metals occur naturally in the environment

– Silts and clays have higher metal content

– Source of uncertainty in identifying anthropogenic impact

– Background varies due to sediment type and regional differences in geology

Need to differentiate between natural background levels and anthropogenic input

– Investigate utility for empirical guideline development

– Potential use for establishing regional background levels

Page 32: Development of Chemistry Indicators Scientific Steering Committee Meeting July 26, 2005 Sediment Quality Objectives For California Enclosed Bays and Estuaries

Reference Element NormalizationReference Element Normalization

Established methodology applied by geologists and environmental scientists

Reference element covaries with natural sediment metals and is insensitive to anthropogenic inputs

– Regression between reference element and metal developed using a dataset of uncontaminated samples

– Regression line indicates natural background metal concentration for different sediment particle size composition

Use of iron as reference element validated for southern California

– 1994 and 1998 Bight regional surveys

Page 33: Development of Chemistry Indicators Scientific Steering Committee Meeting July 26, 2005 Sediment Quality Objectives For California Enclosed Bays and Estuaries

Iron Normalization ApproachIron Normalization Approach

Log transformed data

Selected subset of “reference” stations from SQO database

– Least potential for anthropogenic metal enrichment

– Nontoxic stations in lowest 30th percentile of DDT, PCB, and PAH concentrations

– Reviewed selected stations using GIS to eliminate redundant and likely impacted sites

Calculated regressions

Used residuals from regression as normalized values

– Compared relationship of normalized/non -normalized data to toxicity

Page 34: Development of Chemistry Indicators Scientific Steering Committee Meeting July 26, 2005 Sediment Quality Objectives For California Enclosed Bays and Estuaries

Southern California ResultsSouthern California Results

Significant regressions obtained for metals of interests in all strata

SoCal

bl ack=r obust , r ed=98, bl ue=94, gr een=KWNor m

Zi nc

0

100

200

300

i r on2

0 1 2 3 4 5 6 7

Zinc

Iron (%)

Page 35: Development of Chemistry Indicators Scientific Steering Committee Meeting July 26, 2005 Sediment Quality Objectives For California Enclosed Bays and Estuaries

Residual CalculationResidual CalculationSoCal

bl ack=r obust , r ed=98, bl ue=94, gr een=KWNor m

Zi nc

0

100

200

300

i r on2

0 1 2 3 4 5 6 7

Zinc

Iron (%)

Residual = actual-predicted concentration

Residual = relative metal enrichment

Used for correlation analysis with amphipod mortality

Page 36: Development of Chemistry Indicators Scientific Steering Committee Meeting July 26, 2005 Sediment Quality Objectives For California Enclosed Bays and Estuaries

Iron NormalizationIron Normalization

Relationship to sediment toxicity is not improved by iron normalization of metals

South

ArsenicCadmium

ChromiumCopper

LeadMercury

NickelSilver Zinc

Sp

ea

rma

n C

orr

ela

tion

(r)

0.0

0.1

0.2

0.3

0.4

0.5 NonNormCorrelationNormCorrelation

North

ArsenicCadmium

ChromiumCopper

LeadMercury

NickelSilver Zinc

Spe

arm

an C

orr

ela

tion

(r)

-0.2

-0.1

0.0

0.1

0.2

0.3

0.4

0.5NonNormCorrelationNormCorrelation

Page 37: Development of Chemistry Indicators Scientific Steering Committee Meeting July 26, 2005 Sediment Quality Objectives For California Enclosed Bays and Estuaries

Normalization SummaryNormalization Summary

TOC and iron normalization are apparently not effective for improving relationships between chemistry and toxicity

Have not pursued use of normalized data in calibrating/developing SQGs

Iron normalization may be useful for establishing background metal levels

Page 38: Development of Chemistry Indicators Scientific Steering Committee Meeting July 26, 2005 Sediment Quality Objectives For California Enclosed Bays and Estuaries

Calibration of SQGsCalibration of SQGs

Adjustment of models or chemical specific values based on California data

Logistic Regression Model (Pmax)

– Excluded individual chemical models with poor fit• Antimony, Arsenic, Chromium, Nickel

– Adjusted Pmax model to fit CA data (N, S, All)

ERM

– Derived CA-specific values using modified method of Ingersoll et al.

– Sample-based analysis

Page 39: Development of Chemistry Indicators Scientific Steering Committee Meeting July 26, 2005 Sediment Quality Objectives For California Enclosed Bays and Estuaries

CA ERM CalculationCA ERM Calculation

Select paired chemistry and amphipod toxicity data by stratum

– Log transform all chemistry data

Classify samples as toxic/nontoxic based on 20% mortality threshold

Calculate median concentration of the nontoxic samples

Select only those toxic samples where concentration of individual chemicals > 2x nontoxic median

CA ERM = median concentration from screened toxic samples

– At least 10 toxic samples required for ERM calculation

Page 40: Development of Chemistry Indicators Scientific Steering Committee Meeting July 26, 2005 Sediment Quality Objectives For California Enclosed Bays and Estuaries

Substantial differences in some ERM values derived

for California datasets compared to

nationally derived values

Chemical Name NOAA ERM

Southern CA ERM

Northern CA ERM

2-Methylnaphthalene 670.0 23.6 20.2 4,4'-DDE NA 38.3 3.8 Acenaphthene 500.0 24.5 19.0 Acenaphthylene 640.0 47.0 19.8 Anthracene 1100.0 215.5 60.8 Arsenic 70.0 19.1 NA Benz(a)anthracene 1600.0 540.0 169.5 Benzo(a)pyrene 1600.0 630.0 225.3 Cadmium 9.6 1.2 0.6 Chlordane_Z 6.0 23.1 4.0 Chromium 370.0 110.0 291.0 Chrysene 2800.0 739.9 239.0 Copper 270.0 208.0 91.2 DDT_Z 46.1 60.0 13.1

Dibenz(a,h)anthracene 260.0 130.0 23.4 Dieldrin 8.0 2.0 0.8 Fluoranthene 5100.0 723.0 410.9 Fluorene 540.0 46.2 NA Lead 218.0 94.5 56.4 Mercury 0.7 0.8 0.7 Naphthalene 2100.0 33.4 42.5 Nickel 51.6 42.0 NA PCB_Z 180.0 125.4 21.3 Phenanthrene 1500.0 275.9 310.6 Pyrene 2600.0 1000.0 480.0 Silver 3.7 1.1 0.4 Tributyltin NA 308.0 30.0 Zinc 410.0 406.9 214.5

Page 41: Development of Chemistry Indicators Scientific Steering Committee Meeting July 26, 2005 Sediment Quality Objectives For California Enclosed Bays and Estuaries

Objectives

Data preparation

SQG calibration and development

Validation

Conclusions

Next steps

Presentation OverviewPresentation Overview

Existing national SQGs

Calibration of national SQGs

New approaches

Page 42: Development of Chemistry Indicators Scientific Steering Committee Meeting July 26, 2005 Sediment Quality Objectives For California Enclosed Bays and Estuaries

New SQG CharacteristicsNew SQG Characteristics

Compatible with multiple line of evidence assessment framework

Capability to include/adapt to new contaminants of concern

Adaptable to different application objectives

Able to use toxicity and benthic community impact data in development

Result reflects uncertainty of empirical relationship

– Categorical classification and multiple thresholds

– Based on individual chemical models or values

– Thresholds can be adjusted

– Accept continuous and categorical data

– Some type of weighting based on strength of relationship

Page 43: Development of Chemistry Indicators Scientific Steering Committee Meeting July 26, 2005 Sediment Quality Objectives For California Enclosed Bays and Estuaries

Kappa StatisticKappa Statistic

Developed in 1960-70’s

– Peer-reviewed literature describes derivation and interpretation

Used in medicine, epidemiology, & psychology to evaluate observer agreement/reliability

– Similar problem to SQG development and assessment

– Accommodates multiple categories of classification

– Multiple thresholds can be adjusted by user

– Categorical or ordinal data

– Result reflects magnitude of disagreement (can be used to weight values)

Sediment quality assessment is a new application

Page 44: Development of Chemistry Indicators Scientific Steering Committee Meeting July 26, 2005 Sediment Quality Objectives For California Enclosed Bays and Estuaries

KappaKappaEvaluates agreement between 2 methods of classification

– Chemical SQG result

– Toxicity test result

– Magnitude of error affects score

 Toxicity Result

 SQG Result

High Moderate Marginal Reference

High

Moderate

Low

Reference

T1T3 T2

Page 45: Development of Chemistry Indicators Scientific Steering Committee Meeting July 26, 2005 Sediment Quality Objectives For California Enclosed Bays and Estuaries

 Toxicity

Kappa = 0.48 SQG Category

High Moderate Marginal Reference

High 60 30 20 1

Moderate 33 50 25 0

Low 10 14 65 6

Reference 3 7 20 25 

  

Chemical 1Chemical 1Good Association Between Concentration and EffectGood Association Between Concentration and Effect

(most of errors in cells adjacent to diagonal) (most of errors in cells adjacent to diagonal)

Page 46: Development of Chemistry Indicators Scientific Steering Committee Meeting July 26, 2005 Sediment Quality Objectives For California Enclosed Bays and Estuaries

Chemical 2 Chemical 2 Poor Association Between Concentration and EffectPoor Association Between Concentration and Effect

(more errors in categories distant from diagonal) (more errors in categories distant from diagonal)

 Toxicity

Kappa = 0.27 SQG Category

High Moderate Marginal Reference

High 60 1 20 30

Moderate 33 50 0 25

Low 14 10 65 6

Reference 20 7 3 25 

Page 47: Development of Chemistry Indicators Scientific Steering Committee Meeting July 26, 2005 Sediment Quality Objectives For California Enclosed Bays and Estuaries

Kappa Analysis OutputKappa Analysis Output

Kappa (k)

– Similar to correlation coefficient

– Confidence intervals

Multiple thresholds

– Optimized for correspondence to effect levels

– Applied to other data to predict effect category (cat)• E.g., Category 1, 2, 3, or 4

Page 48: Development of Chemistry Indicators Scientific Steering Committee Meeting July 26, 2005 Sediment Quality Objectives For California Enclosed Bays and Estuaries

New Kappa SQGsNew Kappa SQGs

Derived Kappa and thresholds for target chemicals using amphipod mortality data

– As, Cd, Cr, Cu, Pb, Hg, Ni, Ag, Zn , t chlordane, t DDT, t PAH, t PCB

Calculated Kappa score for each chemical in sample

– k x cat

Mean weighted Kappa score

– Average of k x cat

– Each constituent contributes to final classification in a manner proportional to reliability of relationship

– Mixture joint effects model

Maximum Kappa

– Highest Kappa score for any individual chemical

– Independent mixture effects model

Page 49: Development of Chemistry Indicators Scientific Steering Committee Meeting July 26, 2005 Sediment Quality Objectives For California Enclosed Bays and Estuaries

Objectives

Data preparation

SQG calibration and development

Validation

Conclusions

Next steps

Presentation OverviewPresentation Overview

Categorical classification

Correlation

Predictive ability

Page 50: Development of Chemistry Indicators Scientific Steering Committee Meeting July 26, 2005 Sediment Quality Objectives For California Enclosed Bays and Estuaries

Evaluation ProcessEvaluation Process Compare performance of candidate SQG approaches in a

manner relevant to desired application

– Ability to accurately classify presence and magnitude of biological effects based on chemistry

– California marine embayment data

Use statistical measures to identify short list of best performing approaches

– Categorical classification

– Correlation

Validate performance results

– Validation dataset

Rank candidate approaches

Examine significance of differences

– Predictive ability

Page 51: Development of Chemistry Indicators Scientific Steering Committee Meeting July 26, 2005 Sediment Quality Objectives For California Enclosed Bays and Estuaries

Evaluation of SQGsEvaluation of SQGs

Categorical (ability to classify each station into one of four toxicity response categories)

– Kappa value

– Level 1=<10% mortality, Level 2=10-20%, Level 3=20-40%, Level 4=>40%

– SQG thresholds optimized for best score

Spearman’s correlation coefficient

– Nonparametric measure of association

– Independent of Kappa calculation

Validation

– Used same thresholds selected for calibration dataset

Page 52: Development of Chemistry Indicators Scientific Steering Committee Meeting July 26, 2005 Sediment Quality Objectives For California Enclosed Bays and Estuaries

SQG Evaluation: NorthSQG Evaluation: North

SQGSpearmanCorrelation Kappa

Mean Weighted Kappa 0.54 0.36Northern CA mERMq 0.37 0.29Northern CA Pmax 0.35 0.29Max Weighted Kappa 0.40 0.26mConsensusq 0.29 0.24mERMq 0.37 0.24mSQGQ1q 0.28 0.22National Pmax 0.27 0.19Chronic EqP TU -0.08 0.08Acute EqP TU -0.09 0.08

Page 53: Development of Chemistry Indicators Scientific Steering Committee Meeting July 26, 2005 Sediment Quality Objectives For California Enclosed Bays and Estuaries

SQG Evaluation:SouthSQG Evaluation:South

SQGSpearmanCorrelation Kappa

Mean Weighted Kappa 0.46 0.31Max Weighted Kappa 0.43 0.27

Southern CA Pmax 0.32 0.21mERMq 0.29 0.18Southern CA mERMq 0.28 0.18National Pmax 0.22 0.16mSQGQ1q 0.25 0.16mConsensusq 0.22 0.13Chronic EqP TU -0.06 0.04Acute EqP TU -0.08 0.03

Page 54: Development of Chemistry Indicators Scientific Steering Committee Meeting July 26, 2005 Sediment Quality Objectives For California Enclosed Bays and Estuaries

SQG Validation:NorthSQG Validation:North

All top ranked SQGs validate

SQGSpearmanCorrelation Kappa

Mean Weighted Kappa 0.47 0.31Northern CA mERMq 0.38 0.26Max Weighted Kappa 0.36 0.22Northern CA Pmax 0.31 0.21mSQGQ1q 0.38 0.21National Pmax 0.30 0.17mERMq 0.31 0.17mConsensusq 0.14 0.12

Page 55: Development of Chemistry Indicators Scientific Steering Committee Meeting July 26, 2005 Sediment Quality Objectives For California Enclosed Bays and Estuaries

SQG Validation:SouthSQG Validation:South

All top ranked SQGs validate

SQGSpearmanCorrelation Kappa

National Pmax 0.34 0.24Southern CA Pmax 0.34 0.21mSQGQ1q 0.39 0.21Mean Weighted Kappa 0.36 0.20mERMq 0.29 0.18Max Weighted Kappa 0.34 0.18Southern CA mERMq 0.22 0.15mConsensusq 0.21 0.07

Page 56: Development of Chemistry Indicators Scientific Steering Committee Meeting July 26, 2005 Sediment Quality Objectives For California Enclosed Bays and Estuaries

Significance of DifferencesSignificance of Differences

Are the differences in performance significant to the user?

– Do differences in SQG ranking correspond to greater accuracy, applicability, or utility of the SQG?

– Better predictive ability (efficiency)?

– Better sensitivity or specificity?

Need to look at the data

Page 57: Development of Chemistry Indicators Scientific Steering Committee Meeting July 26, 2005 Sediment Quality Objectives For California Enclosed Bays and Estuaries

Mean Weighted Kappa NOAA ERM

EqP Acute So CA ERM

SQGs Applied to So CA DataSQGs Applied to So CA Data

Page 58: Development of Chemistry Indicators Scientific Steering Committee Meeting July 26, 2005 Sediment Quality Objectives For California Enclosed Bays and Estuaries

Predictive AbilityPredictive Ability

Guideline Value

0 20 40 60 80 100 120

Freq

uenc

y

True Positive(Hit/Toxic)

Toxic Sample Distribution

A

BFalse Negative(No Hit/Toxic)

AB

Threshold

Nontoxic Sample Distribution

True Negative(No Hit/Nontoxic)

DC

False Positive(Hit/Nontoxic)

CD

Negative Predictive Value =C/(C+A) x 100(percent of no hits that are nontoxic)=Nontoxic Efficiency

Specificity=C/(C+D) x 100(percent of all nontoxic samples that are classified as a no hit)

Positive Predictive Value =B/(B+D) x 100(percent of hits that are toxic)=Toxic Efficiency

Sensitivity=B/(B+A) x 100(percent of all toxic samples that are classified as a hit)

Page 59: Development of Chemistry Indicators Scientific Steering Committee Meeting July 26, 2005 Sediment Quality Objectives For California Enclosed Bays and Estuaries

South: mERMqSouth: mERMq

SQG performance is threshold dependent

Inverse relationship between efficiency (toxic or nontoxic) and specificity or sensitivity

Improved SQG accuracy when greater efficiency obtained

Improved SQG utility when greater sensitivity or specificity obtained without sacrificing efficiency

Page 60: Development of Chemistry Indicators Scientific Steering Committee Meeting July 26, 2005 Sediment Quality Objectives For California Enclosed Bays and Estuaries

South: mERMqSouth: mERMq

Plots of efficiency vs. specificity or sensitivity illustrate tradeoffs in SQG performance at different thresholds

Page 61: Development of Chemistry Indicators Scientific Steering Committee Meeting July 26, 2005 Sediment Quality Objectives For California Enclosed Bays and Estuaries

South: Candidate SQGsSouth: Candidate SQGs

Mean weighted Kappa shows improved overall utility for distinguishing both nontoxic and toxic samples

Nontoxic Efficiency vs Specificity

Nontoxic Efficiency

60 70 80 90 100

Spe

cific

ity

0

20

40

60

80

100

NOAA mERMq SouthSo Cal mERMqKappa SouthPMax South

Toxic Efficiency vs Sensitivity

Toxic Efficiency

40 50 60 70 80 90 100

Sen

sitiv

ity

0

20

40

60

80

100

NOAA mERMq SouthSo Cal mERMqKappa SouthPMax South

Page 62: Development of Chemistry Indicators Scientific Steering Committee Meeting July 26, 2005 Sediment Quality Objectives For California Enclosed Bays and Estuaries

North: Candidate SQGsNorth: Candidate SQGs

Mean weighted Kappa shows improved specificity and toxic efficiency

Nontoxic Efficiency vs Specificity

Nontoxic Efficiency

60 70 80 90 100

Spe

cific

ity

0

20

40

60

80

100

NOAA mERMq NorthNor Cal mERMqKappa NorthPMax North

Toxic Efficiency vs Sensitivity

Toxic Efficiency

40 50 60 70 80 90 100

Se

nsi

tivity

0

20

40

60

80

100

NOAA mERMq NorthNor Cal mERMqKappa NorthPMax North

Page 63: Development of Chemistry Indicators Scientific Steering Committee Meeting July 26, 2005 Sediment Quality Objectives For California Enclosed Bays and Estuaries

Evaluation and Validation SummaryEvaluation and Validation Summary

North

– Mean weighted Kappa has highest performance

– Northern California ERM and Northern California Pmax also perform better than others

South

– Mean weighted Kappa has highest performance

– Max Kappa also performs better than others

Validation results consistent with evaluation

– The approaches are robust

Page 64: Development of Chemistry Indicators Scientific Steering Committee Meeting July 26, 2005 Sediment Quality Objectives For California Enclosed Bays and Estuaries

Presentation OverviewPresentation Overview

Objectives

Data preparation

SQG calibration and development

Validation

Conclusions

Next steps

Page 65: Development of Chemistry Indicators Scientific Steering Committee Meeting July 26, 2005 Sediment Quality Objectives For California Enclosed Bays and Estuaries

ConclusionsConclusions

Pursue mean weighted Kappa as component of chemistry LOE

– Best relationship with toxicity

– Easily adaptable to new chemicals or different datasets

– Provides information on strength of relationship

Use EqP benchmarks as component of stressor identification, not chemical LOE score

– Predictive value not strong enough

– Provide guidance on calculation and interpretation

Page 66: Development of Chemistry Indicators Scientific Steering Committee Meeting July 26, 2005 Sediment Quality Objectives For California Enclosed Bays and Estuaries

Presentation OverviewPresentation Overview

Objectives

Data preparation

SQG calibration and development

Validation

Conclusions

Next stepsThresholds

Benthos

Page 67: Development of Chemistry Indicators Scientific Steering Committee Meeting July 26, 2005 Sediment Quality Objectives For California Enclosed Bays and Estuaries

Options for Threshold DevelopmentOptions for Threshold Development

Optimum statistical fit to effects in CA

– Toxicity only?

– Benthos only?

– Combination?

Based on accuracy or error rate

Consideration of national patterns

Page 68: Development of Chemistry Indicators Scientific Steering Committee Meeting July 26, 2005 Sediment Quality Objectives For California Enclosed Bays and Estuaries

National vs. CA dataNational vs. CA data

Narrower contamination range in CA

High range threshold (1.5) of limited utility

North South

Page 69: Development of Chemistry Indicators Scientific Steering Committee Meeting July 26, 2005 Sediment Quality Objectives For California Enclosed Bays and Estuaries

BenthosBenthos

How should benthic community response be incorporated into the chemical LOE

– In the SQG approach?

– In the thresholds?

Factors to consider:

– Strength of relationship between benthos and chemistry or toxicity

– Relative sensitivity of benthos and toxicity responses

– Nature of association with chemistry

– Are there different drivers?

Page 70: Development of Chemistry Indicators Scientific Steering Committee Meeting July 26, 2005 Sediment Quality Objectives For California Enclosed Bays and Estuaries

BenthosBenthos

Preliminary data analysis:

Used existing benthic response index (BRI) data for So. Calif. and San Francisco Bay

– South San Francisco Bay (North); n=83

– Southern California (South); n=203

Examined three aspects of relationship with chemistry

– Strength of relationship with SQGs and chemicals

– Relative sensitivity of response compared to toxicity

– Chemical drivers

Page 71: Development of Chemistry Indicators Scientific Steering Committee Meeting July 26, 2005 Sediment Quality Objectives For California Enclosed Bays and Estuaries

BenthosBenthos

BRI Score vs. Mean ERM Quotient for SouthernSan Francisco, California

m ERM q San Francisco South

0.01 0.10 1.00 10.00

BR

I Sco

re

0

20

40

60

80

100

Ref

Rl 1

RL 2

RL 3

BRI Index vs. Mean ERM Quotient for Southern California

m ERM q Southern CA

0.01 0.10 1.00 10.00B

RI S

core

0

20

40

60

80

100

Ref

RL 1

RL 2

RL 3

Page 72: Development of Chemistry Indicators Scientific Steering Committee Meeting July 26, 2005 Sediment Quality Objectives For California Enclosed Bays and Estuaries

BenthosBenthos

Significant correlations are present between BRI scores and SQGs or individual chemicals

BRI Score vs. Zinc in Southern San Francisco, California

Zn (mg/kg) San Francisco South

10 100 1000 10000

BR

I Sco

re

0

20

40

60

80

100

Ref

RL 1

RL 2

RL 3

BRI Score vs. Zinc in Southern California

Zn (mg/kg) Southern CA

10 100 1000 10000

BR

I Sco

re

0

20

40

60

80

100

Ref

RL 1

RL 2

RL 3

BRI Score vs. PAHs in Southern California

PAHs (ug/kg) Southern CA

10 100 1000 10000

BR

I Sco

re

0

20

40

60

80

100

Ref

RL 1

RL 2

RL 3

BRI Score vs. PAHs in Southern San Francisco, California

PAHs (mg/kg) San Francisco South

10 100 1000 10000

BR

I Sco

re

0

20

40

60

80

100

Ref

RL 1

RL 2

RL 3

Page 73: Development of Chemistry Indicators Scientific Steering Committee Meeting July 26, 2005 Sediment Quality Objectives For California Enclosed Bays and Estuaries

BenthosBenthos

Strong correlation between benthic response and amphipod mortality

Benthic response when no toxicity is evident

BRI versus percent mortality of Eohaustorius estuarius in sediment of Southern San Francisco

EE Mortality (%)

0 20 40 60 80 100

BR

I Sco

re S

an F

ranc

isco

Sou

th

0

20

40

60

80

100

Ref

RL 1

Rl 2

RL 3

BRI versus percent mortality of Eohaustorius estuarius in sediment of Southern CA

EE Mortality (%)

0 20 40 60 80 100

BR

I Sco

re S

outh

CA

0

20

40

60

80

100

Ref

RL 1

RL 2

RL 3

Page 74: Development of Chemistry Indicators Scientific Steering Committee Meeting July 26, 2005 Sediment Quality Objectives For California Enclosed Bays and Estuaries

Relative Sensitivity of Benthos ResponseRelative Sensitivity of Benthos Response

Use cumulative distribution function to indicate approximate thresholds for increased incidence of impacts (10th percentile) and likely impacts (50th percentile)

Compare results for toxicity and benthos (same dataset)

Pr o

por t

ion

of S

ampl

es

Pr o

por t

ion

of S

ampl

es

Page 75: Development of Chemistry Indicators Scientific Steering Committee Meeting July 26, 2005 Sediment Quality Objectives For California Enclosed Bays and Estuaries

Relative Sensitivity of Benthos ResponseRelative Sensitivity of Benthos Response

Toxicity and benthos responses occur over similar contamination ranges

North South SQG 10th

percentile 50th

percentile 10th

percentile 50th

percentile Toxicity Consensus 1.36 1.94 2.13 6.84 ERMq 0.13 0.17 0.13 0.29 SQGQ1 0.16 0.20 0.22 0.54 Benthos Consensus 1.19 1.94 1.83 8.83 ERMq 0.13 0.18 0.10 0.29 SQGQ1 0.18 0.21 0.22 0.62

Page 76: Development of Chemistry Indicators Scientific Steering Committee Meeting July 26, 2005 Sediment Quality Objectives For California Enclosed Bays and Estuaries

Chemical Correlations : NorthChemical Correlations : North

achi eved maxi um kappa wei ght ed scor el oc=Nor t h

per cmax SUM

0. 013

0. 025

0. 026

0. 025

0. 494

0. 025

0. 103

0. 090

0. 013

0. 089

0. 083

0. 025

0. 038

Chemi cal Name

Zi nc

TPAH_Z

Si l ver

PCB_Z

Ni ckel

Mer cur y

Lead

DDT_Z

Copper

Chr ysene

Chr omi um

Chl or dane_Z

Ar seni c

per cmax SUM

0. 00 0. 05 0. 10 0. 15 0. 20 0. 25 0. 30 0. 35 0. 40 0. 45 0. 50

Benthos

Toxicity

Chlordane, copper, and zinc show different relative influence on effects

achi eved maxi um kappa wei ght ed scor el oc=Nor t h

per cmax SUM

0. 329

0. 013

0. 052

0. 025

0. 077

0. 026

0. 063

0. 208

0. 025

0. 205

Chemi cal Name

Zi nc

TPAH_Z

Si l ver

PCB_Z

Ni ckel

Lead

DDT_Z

Copper

Chr omi um

Chl or dane_Z

per cmax SUM

0. 00 0. 05 0. 10 0. 15 0. 20 0. 25 0. 30 0. 35

S. Correlation

S. Correlation

Page 77: Development of Chemistry Indicators Scientific Steering Committee Meeting July 26, 2005 Sediment Quality Objectives For California Enclosed Bays and Estuaries

Chemical Correlations: SouthChemical Correlations: South

Benthos

Toxicity

Cadmium, DDTs, and zinc show different relative influence on effects

achi eved maxi um kappa wei ght ed scor el oc=Sout h

per cmax SUM

0. 006

0. 077

0. 174

0. 013

0. 090

0. 013

0. 032

0. 006

0. 052

0. 535

Chemi cal Name

Zi nc

TPAH_Z

Si l ver

PCB_Z

Mer cur y

Lead

DDT_Z

Copper

Chl or dane_Z

Cadmi um

per cmax SUM

0. 0 0. 1 0. 2 0. 3 0. 4 0. 5 0. 6

S. Correlation

achi eved maxi um kappa wei ght ed scor el oc=Sout h

per cmax SUM

0. 033

0. 399

0. 124

0. 007

0. 242

0. 131

0. 013

0. 007

0. 046

Chemi cal Name

Zi nc

TPAH_Z

PCB_Z

Ni ckel

DDT_Z

Chr omi um

Chl or dane_Z

Cadmi um

Ar seni c

per cmax SUM

0. 00 0. 05 0. 10 0. 15 0. 20 0. 25 0. 30 0. 35 0. 40

S. Correlation

Page 78: Development of Chemistry Indicators Scientific Steering Committee Meeting July 26, 2005 Sediment Quality Objectives For California Enclosed Bays and Estuaries

RecommendationsRecommendations

Develop thresholds of application specific to toxicity and benthos

– Need to incorporate both types of responses into assessment

Continue development of a SQG that is best predictor of benthic community impacts

– May respond to different chemical mixtures

– Need revised benthic index data to complete development and evaluation

– Determine whether toxicity and benthos SQGs are needed

– A method to combine the results will be needed to produce a single chemistry LOE score