expert system methodology for evaluating reductive dechlorination at tce sites

Expert System Methodology forEvaluating Reductive Dechlorinationat TCE SitesN E I L A . S T I B E R , *M A R I N A P A N T A Z I D O U , † A N DM I T C H E L L J . S M A L L ‡

Carnegie Mellon University, Pittsburgh, Pennsylvania 15213

An expert knowledge site-screening methodology hasbeen developed to evaluate naturally occurring reductivedechlorination as a remedial option for sites with TCE-contaminated groundwater. This methodology combines acausative model for the reductive dechlorination of TCEand expert knowledge within a Bayesian Belief Network.The knowledge base for this expert system was obtainedfrom 22 experts via an expert elicitation protocol. Theresulting expert system can be used to aid environmentaldecision making by evaluating the adequacy of reductivedechlorination at TCE-contaminated sites. Comparisonsbetween this expert system and a commonly used screeningtool show that this expert system produces predictivemodels that may better discriminate between locations thatwere sampled. The 22 elicitations revealed differentbeliefs and assumptions among experts about the biochemicalprocesses involved in reductive dechlorination. The decision-making value of some types of evidence is a matter ofdispute; however, findings about biodegradation daughter and/or end products have high decision-making value for allof the experts. The methodology demonstrated herein canprovide insights for other environmental decision-makingchallenges.

IntroductionGroundwater contaminated with trichloroethene (TCE) is acommon and persistent problem at thousands of sites in theU.S. (1). From 1925 to 1970, TCE, a versatile and effectivesolvent, was used without regulation (2), leaving a legacy ofTCE contamination at virtually every former industrial siteand many military installations in the U.S. Although itschronic toxicity is a matter of research and dispute, TCE’sacute toxicity is well documented (3). Trichloroethene andtwo of its degradation products, dichloroethene (DCE) andvinyl chloride, pose a risk to human and ecological healthand, consequently, are regulated by the U.S. EnvironmentalProtection Agency (EPA) (4, 5).

Trichloroethene is referred to as a recalcitrant compoundbecause of the technical problems associated with its cleanup.Attempts to remediate TCE-contaminated groundwater are

often complicated by heterogeneous subsurface environ-ments, contaminant sorption, and the persistent nature ofdense nonaqueous phase liquids (6). Groundwater reme-diation by pump and treat methods has proven to beexceedingly costly, time intensive, and minimally effective.Although it has been successful at containing contaminantplumes, pump and treat is rarely able to remediate aquifersto health-based standards (7). Furthermore, there has beenlittle progress in the development of innovative technologiesfor groundwater cleanup (6). Recently, environmental pro-fessionals have noted that naturally occurring processes oftenprovide an opportunity for achieving cleanups that areprotective of human and ecological health (8).

In 1999, the EPA finalized a directive that recognizesmonitored natural attenuation as “an appropriate remedia-tion option for contaminated soil and groundwater undercertain circumstances (9)”. According to this directive, naturalattenuation is a remediation approach that includes “a varietyof physical, chemical, or biological processes that, underfavorable conditions, act without human intervention toreduce the mass, toxicity, mobility, volume, or concentrationof contaminants in soil or groundwater. These in situprocesses include biodegradation; dispersion; dilution; sorp-tion; volatilization; radioactive decay; and chemical orbiological stabilization, transformation, or destruction ofcontaminants.”

Monitored natural attenuation is a promising remediationapproach because it may be less expensive than conventionalpump and treat systems and includes procedures fordemonstrating that cleanups are protective of human andecological health (10, 11).

Although natural attenuation includes many processes,its success often depends on the level of intrinsic bioreme-diation that occurs at the site. When evaluating sites fornatural attenuation, the EPA is most interested to determinethat the contaminant mass is being destroyed by biodeg-radation (12). Reductive dechlorination is an important, butby no means the only, process through which chlorinatedsolvents can be biodegraded at significant rates. In particular,enhancing aerobic processes has also proven successful forthe biodegradation of TCE (13, 14).

The expert system presented herein has been developedto evaluate the probability that reductive dechlorination isoccurring at TCE-contaminated sites. There are many othercomponents in the eventual selection of natural attenuationas a remedial option. The biodegradation potential of a siteshould be evaluated and a long-term monitoring programdesigned. There is also a need to investigate the fate of harmfulbyproducts (e.g., vinyl chloride) that are formed by reductivedechlorination. In addition, regulatory requirements, geo-graphic constraints, and social demands must be satisfied.

Recently, the need has been identified to develop rankingand/or scoring systems that evaluate TCE sites for reme-diation by natural attenuation. A number of protocols thatguide practitioners through the investigation, evaluation, anddemonstration of natural attenuation are being developedand used (15). The objective of this paper is to accompanythese protocols by (1) offering advice on what types ofevidence have the greatest decision-making impact, (2)providing a screening tool by which to structure this evidenceinto a persuasive argument for or against natural attenuation,and (3) identifying research needs that will engender betterdecision making in the future. Because there are so manyTCE-contaminated sites in the U.S. and owners and devel-opers are so interested in pursuing natural attenuationsolutions, it is important that site investigations measure the

* To whom correspondence should be addressed: McLaren/Hart,Inc., 5900 Landerbrook Drive, Suite 100, Cleveland, OH 44124.Phone: (440)684-8300;fax: (440)684-8320;e-mail: [email protected].

† Department of Civil and Environmental Engineering, CarnegieMellon University, Pittsburgh, PA 15213.

‡ Department of Civil and Environmental Engineering & Depart-ment of Engineering and Public Policy, Carnegie Mellon University,Pittsburgh, PA 15213.

Environ. Sci. Technol. 1999, 33, 3012-3020

3012 9 ENVIRONMENTAL SCIENCE & TECHNOLOGY / VOL. 33, NO. 17, 1999 10.1021/es981216s CCC: $18.00 1999 American Chemical SocietyPublished on Web 07/30/1999

proper quantities and extract the full value of informationfrom each piece of evidence.

Natural Attenuation and Decision Making. Relying onnatural attenuation can reduce cleanup costs if goodcandidate sites are correctly identified. However, if naturalattenuation is pursued inappropriately, additional site in-vestigation will eventually be required prior to selection andimplementation of other remedial approaches. In such cases,the initial efforts to demonstrate natural attenuation mayhave been a waste of time and money.

The expert system methodology presented herein hasproduced screening tools that are intended for use early inthe site investigation process, when few data are available.If the screening process indicates that naturally occurringreductive dechlorination is a viable option for the site, thenthe site investigation should proceed with a detailed sitecharacterization that will be used to further evaluate theacceptability of natural attenuation and to plan the moni-toring and compliance program. Before natural attenuationcan be accepted as a remedial strategy, it must be demon-strated that human health and ecological safety will beadequately protected. If natural attenuation does not appearpromising, the site investigation should proceed with theintention of selecting another remedial option.

Improving Site Screening for Natural Attenuation. TheU.S. Air Force and the EPA have developed a protocol (16,17) for evaluating natural attenuation at TCE sites. It uses ascoring system that awards points for 27 different measure-ments of groundwater quality parameters. Each measure-ment is compared with a cutoff value and, depending on itssignificance, results in 0, 1, 2, or 3 (and for a few cases, negativescores of -2 and -3) points being awarded to the site. Forexample, if [O2] is less than 0.5 mg/L, three points are awarded.Each site’s total score is interpreted based on a scale in whicha threshold of 20 points must be exceeded to indicate strongevidence that biodegradation of chlorinated organics isoccurring.

The Air Force/EPA protocol is attractive because it can beapplied immediately to many sites. It does an excellent jobof identifying critical groundwater quality parameters andsetting meaningful cutoff points for their measurement. Theexpert system presented herein uses this protocol as a startingpoint and builds on it in the following ways: (1) by capturingthe uncertain and probabilistic nature of data collection andevaluation, (2) by assigning weight to negative findings, and(3) by identifying which findings are most important todifferent experts.

One promising approach to improving site screening fornatural attenuation is to include new, more meaningfulmeasures, such as the chlorine isotope ratio (18). By contrast,the approach taken here is to use an expert system to betterdistinguish the value of geochemical parameters that arecommonly measured. In the future, a further improvementin evaluating natural attenuation might be to extend theexpert system methodology to incorporate the chlorineisotope ratio and/or other relevant measures.

Materials and MethodsThe methodology for this expert system is to combine acausative model for the reductive dechlorination of TCE andexpert knowledge within a Bayesian Belief Network (BBN)framework. Bayesian inferential methods have been used inother contexts to evaluate various types of environmentalinformation and improve decision making (19-21). The useof BBNs is particularly well suited for problems withconceptual uncertainty about the fundamentals of modelstructure.

The first step in the development of the BBN in this studywas to generalize the complex and incompletely understoodprocesses for the reductive dechlorination of TCE into a

conceptual causative model. Next, expert knowledge wasused to refine the model structure and to provide theprobabilistic relationships between variables. This approachwas selected because the state of experience with reductivedechlorination of TCE is evolving rapidly (17), and hence,subjective knowledge can be of great value. Examining theproblem through the knowledge of different experts permitsdifferent opinions and schools of thought to be identifiedand compared. When the state of science about the reductivedechlorination of TCE is more developed, a more quantitativeobjective approach might be better suited, e.g., addressingthe extent or rates of the kinetic processes.

While the BBN approach does help to identify differenttheories and alternative models, not all of the ideas could beincorporated into the final causative model. Using a commonstarting point makes the different expert opinions moredirectly comparable. However, some structural differenceswere permitted among the experts’ models, as discussed later.Also, since the complexity of the elicitation process for eachnode increases geometrically with additional inputs to thatnode, the elicitations were made more manageable bystructuring the nodes into convenient groupings. Thesegroups were designed to represent physical quantities.However, if the elicitation protocol had been developed bydifferent researchers with different perspectives, the modelmay have been structured differently. The expert system doesnot include sites with PCE contamination because doing sowould increase the causative model’s complexity by requiringTCE to be both a contaminant source and a daughter product.

Causative Model. The causative model in Figure 1 wasformulated on the basis of the available literature on thebiodegradation of chlorinated solvents (16, 22-26). Thismodel summarizes the essential components of the reductivedechlorination of TCE in groundwater and provides the basicframework for the expert elicitation. Each node in Figure 1represents a particular variable that plays a role in thereductive dechlorination process. Each arrow in Figure 1represents a causative link between nodes. Although thepresence of microorganisms is also critical for the successof reductive dechlorination, this component of the processhas been omitted to make the problem more manageable.For the purposes of a screening tool, this is a reasonableassumption; however, before reductive dechlorination isimplemented as a remedial strategy at a site, evidence ofmicrobial activity is necessary.

At the center of the diagram is the node of greatestinterest: anaerobic degradation by reductive dechlorination.Only reductive dechlorination is evaluated herein, becauseit is the most important process for the natural biodegradationof the more highly chlorinated solvents (17). There are tworeductive processes that contribute to the anaerobic bio-

FIGURE 1. Bayesian Belief Network for the reductive dechlorinationof TCE in groundwater.

VOL. 33, NO. 17, 1999 / ENVIRONMENTAL SCIENCE & TECHNOLOGY 9 3013

degradation of TCE (24). The first, co-metabolism, occurswhen the biodegradation of dissolved carbon producesenzymes that fortuitously degrade TCE. In this case, the activemicroorganisms derive no benefit from the degradation ofTCE. The second process, dehalorespiration, occurs whenmicroorganisms degrade TCE directly in an energy-conserv-ing process. Dehalorespiration has the potential to be a muchquicker and more complete process.

As structured in this model, three influences determineif anaerobic degradation by reductive dechlorination isoccurring: reducing conditions, electron donors, and en-vironmental conditions. The stronger the reducing condi-tions, the more electrons are potentially available forreductive dechlorination. In a less reducing environment,the TCE is at a competitive disadvantage in trying to acquireelectrons. The extent of reducing conditions can be assessedby the oxidation reduction potential (ORP, measured againstAg/AgCl) and the availability of molecular hydrogen, whichitself is a function of the terminal electron accepting process(TEAP).

The second influence on anaerobic degradation byreductive dechlorination is through electron donors. This ismeasured by the amount of available carbon in the ground-water from naturally occurring and anthropogenic sources,as determined by concentrations of dissolved organic carbon(DOC), total petroleum hydrocarbons (TPH), and benzene,toluene, ethylbenzene, and xylenes (BTEX). In co-metabolicand dehalorespirating systems, these carbon sources are anecessary food for the bacteria. The degradation of thesecarbons produces electrons that TCE can use for reductivedechlorination. Molecular hydrogen is often important asan intermediate electron donor; however, there is noagreement in the literature on the optimal range of hydrogenconcentration (27, 28). For this reason, hydrogen is used inthe model only as a measure of current reducing conditions.Such simplifications in model structure are customary in theBayesian Belief Network literature (29).

Environmental conditions is the third set of influenceson anaerobic degradation by reductive dechlorination. Thisincludes the temperature, pH, and dissolved oxygen contentof the groundwater. These inputs prescribe the range ofconditions under which the bacteria that biodegrade TCEare effective and conditions for which their activity is slowedor impossible.

Anaerobic degradation by reductive dechlorination hasa number of daughter and end products: dichloroethene(DCE), vinyl chloride, ethene and ethane, and chloride.Additionally, methane may accumulate in certain highlyreduced environments due to the transformation of carbondioxide and/or may be produced by the degradation of thecarbon sources. Methane is thereby included as a possibleindirect indicator of the reductive dechlorination mecha-nisms.

Note that the model, as formulated, assumes indepen-dence among the precursor nodes (e.g., the three indicatorsof electron donor presence), except as mediated by theirconditional dependence when a common descendent nodehas evidence. This assumption of independence ignoressecond-order interactions among the precursors, allows themodel to be computable, and helps to keep the overall modelmore tractable for experts by limiting the number ofconditional probabilities that must be elicited. Since mostof these precursor dependencies are likely to involve positivecorrelations (e.g., if BTEX > 0.1 mg/L, then DOC is morelikely to be greater than 20 mg/L), ignoring these depend-encies will tend to make the model more conservative inevaluating the impact of individual pieces of evidence. Forexample, a positive finding of BTEX > 0.1 mg/L will causethe posterior probability that anaerobic degradation byreductive dechlorination is occurring to increase by a lesser

amount than had the dependency between the BTEX andthe other precursor variables been considered. We anticipatethat these second-order effects are small compared to thedirect impacts of each precursor on the target prediction.This expectation is reinforced by the result, shown later, thatthe precursor nodes have a smaller impact on the predictionthan do the product nodes.

Bayesian Belief Networks. A BBN is a graphical proba-bilistic technique that is useful for modeling a causativenetwork. It is especially beneficial when individual nodes ofthe network will be updated with evidence (30, 31). BecauseBayes’ Rule is used to make inferences, the influence ofinformation flows in the causative direction (with the arrows)and in the diagnostic direction (against the arrows). Whenevidence is collected about any particular node, the prob-abilities of the other connected nodes adjust to the new stateof information. Netica, a user-friendly computer tool, wasused to develop, solve, and analyze the reductive dechlo-rination BBN (32). Figure 2 provides an example Neticaoutput. Note that the nodes for which evidence is availableare shaded and the states of these nodes are assumed to beknown with certainty. With each piece of evidence that iscollected, Netica computes the probabilities for all of theuncertain nodes.

Development of Expert Elicitation Protocol. The devel-opment of the expert elicitation protocol was an iterativeprocess. An elicitation is a structured interview used to acquireknowledge (often probabilistic) from expert subjects (33-35). The first versions of the causative model and theelicitation protocol were developed by the authors based ona review of the relevant literature. This version was testedwith the help of two researchers who are familiar withchemical processes in the aquatic environment. In addition,several social science researchers were consulted aboutmethodological issues concerning elicitation.

After making comprehensive revisions, a preliminaryround of elicitations was conducted with four experts (ahydrogeologist, a biochemist, a microbiologist, and anengineer). With the experience and knowledge gained fromthis preliminary round of elicitations, additional revisionsand improvements were made to the causative model andthe elicitation protocol. Although the feedback included somecontroversies, the objective was to develop a reasonableconsensus model that all could use. These changes includedrefining the model to better reflect the state of the science,rephrasing questions to be more precise, and better antici-pating experts’ problems during elicitation. The retailoredelicitation protocol, sent to the final group of experts, is foundin Appendix A of the Supporting Information.

The expert elicitees were selected on the basis of theirexpertise in environmental remediation in general and thebioremediation of TCE specifically. They were identifiedthrough their relevant research papers or by the recom-mendations of other experts. Although the elicitees are nota random group, they exhibit a range of educationalbackgrounds, varied professional experiences (academia,consulting, government, and industry), and regional diversity.The 22 experts who were elicited are listed, with theirapproval, in Appendix B of the Supporting Information. Theexperts were each assigned a letter by which they are referredto in this paper.

Conducting the Expert Elicitation. The elicitation pro-tocol includes six pages of preparatory materials that describethe research project and explain the elicitation process. Theelicitation asks for 94 probabilities and then goes through aseries of qualitative questions. Copies of the elicitationprotocol were mailed to 38 potential experts and followedup with phone calls. If the potential elicitee was willing toparticipate and suitable for this exercise, a time was scheduled

3014 9 ENVIRONMENTAL SCIENCE & TECHNOLOGY / VOL. 33, NO. 17, 1999

for conducting the elicitation via telephone. The averageelicitation time was about 1 h.

Each elicitation started with a discussion of the causativemodel. The experts were given the opportunity to makestructural changes in the model (Figure 1) to match theirconceptual understanding of reductive dechlorination. Forexample, four of the experts removed the hydrogen nodefrom their models. To completely specify their model, eachexpert had to provide both prior probabilities for the rootnodes (those nodes without any arrows going into them)and conditional probabilities for the causative relationshipsindicated by arrows. One of the experts (expert P) was unableto provide prior probabilities and hence there are 21individual expert models. However, the conditional prob-abilities of this 22nd expert were used in the developmentof an average model.

Each expert was asked to provide prior probabilities forthe model’s root nodes. For example, what is the priorprobability that [BTEX] > 0.1 mg/L?

A prior probability is the probability that you would expectfor an event without knowing anything about the specificsite. The experts were asked to estimate prior probabilitiesby thinking about the total population of all TCE-contam-inated sites in the U.S. In lieu of site-specific data for the rootnodes, these prior probabilities are used by the model.However, they may be changed to reflect knowledge aboutthe type of site or specific conditions at the site. The priorprobabilities elicited from expert L are illustrated in Table 1.The cutoff points for the measured parameters were adaptedfrom the Air Force/EPA protocol (16, 17).

The experts were asked to provide conditional prob-abilities for each of the model’s causative relationships. Many

FIGURE 2. Example Netica output. Expert L’s model with evidence: ORP < 50 mV, BTEX > 1 mg/L, oxygen > 1 mg/L, and methane >0.1 mg/L.

TABLE 1. Example of Prior Probabilities (Elicited from Expert L)

condition prior probability (%)

terminal electron accepting process is denitrification 5terminal electron accepting process is iron reduction 40terminal electron accepting process is sulfate reduction 15terminal electron accepting process is methanogenesis 40oxidation reduction potential < 50 mV 90[dissolved organic carbon] > 20 mg/L 70[benzene, toluene, ethylbenzene, & xylenes] > 0.1 mg/L 70[total petroleum hydrocarbons] > 1 mg/L 70[dissolved oxygen] < 0.5 mg/L 800.5 mg/L < [dissolved oxygen] < 1 mg/L 15[dissolved oxygen] > 1 mg/L 5temperature > 15 °C (59 °F) 705 < pH < 9 80


of the conditional probabilities were elicited as a functionof more than one parameter. For example, Table 2 showsexpert L’s completed 2×2×2 matrix for the adequacy ofelectron donors. As indicated, expert L assigns the highestprobability that the electron donors are adequate for reductivedechlorination (90%) to the case where DOC, TPH, and BTEXare all high and the lowest probability (30%) to the case wherethey are all low. Probabilities for the intermediate cases reflectexpert L’s beliefs concerning the importance of, and interac-tion among, these three sources of electrons. The conditionalprobabilities for products of reductive dechlorination arefunctions of a single parameter. Expert L’s responses to thispart of the elicitation are given in Table 3.

The elicitation process required the experts to providesubjective judgments about phenomena that they were oftenunaccustomed to thinking about in quantitative terms. Toreceive more complete feedback, the elicitation allowed theexperts to comment in support of their selected probabilitiesand concluded with open-ended questions through whichthe experts could express their uncertainties. During theelicitation process, each expert’s responses were checked toeliminate internal inconsistencies. For example, a positivefinding (e.g., detection of vinyl chloride) cannot diminishthe probability that reductive dechlorination is occurring.The expert may believe that the finding should not result inan appreciable increase in the probability of reductivedechlorination; however, a negative response would beunacceptable. When an inconsistency occurred, it waspointed out and the expert was asked to correct it or to providean alternative phenomenological explanation.

ResultsThese expert elicitations resulted in 21 individual screeningtools that can be used to evaluate prospective sites for naturalattenuation. In addition, an average model was developedby averaging the probabilities provided by each of the 22elicitees.

Comparison of Experts’ Models for a TCE Site. Thepredictive abilities of these 21 screening tools and the AirForce/EPA protocol’s scoring system are compared in Table4 for the “no evidence” case and with data from a site near

Niagara Falls, NY (36). An additional attribute of the expertsystem is that it calculates probabilities even without evidencebecause the experts have prior beliefs about the likelihoodof the precursor conditions (in this case, for a typical site inthe U.S.).

The limited data set presented in Table 4 is representativeof the amount of data that the expert system is designed touse. In addition to these limited data, this site had extensiveinvestigations, and hence, it is known that the first two wells(87-20 and 89-02) were within the plume and that reductivedechlorination was active. In contrast, well number 89-06was located in an area downgradient of the plume with littleevidence of active reductive dechlorination. However, whenusing the Air Force/EPA protocol all three of these wellsreceived a “limited” evidence score.

Most of the expert models do a better job of distinguishingbetween the wells within the plume and the one outside ofit. In addition, the expert models find the evidence at the twowells within the plume to be very compelling. At these twowells, the probability of reductive dechlorination ranges 0.38-1.00, with half of the probabilities being 0.99 or 1.00. For thedowngradient well, the expert models gave probabilitiesranging 0.00-0.71, with the majority of the probabilities being0.15 or lower. Two characteristics of the expert models mayaccount for this. First, these expert models make fuller useof negative evidence because the BBN can distinguishbetween it and lacking evidence. Second, these expert modelscan place great weight on individual pieces of evidence thatexperts find compelling. For example, the presence of vinylchloride was a compelling piece of evidence for nearly everyexpert. Consequently, its detection in wells 87-20 and 89-02 results in higher probabilities. It is possible for the expertmodels to conclude that reductive dechlorination is, or isnot, occurring with less evidence than is necessary for theAir Force/EPA protocol to do so.

An average model was constructed by averaging theprobabilities provided by each of the 22 experts. It is knownthat when experts use different paradigms, an average oftheir responses may not represent any valid paradigm forthe system (37). However, for this expert system, it is believedthat the paradigms of the experts are fundamentally similar.This belief is based on the fact that few structural changeswere made in the model and that there were no disagreementson the sign (positive or negative) of the influence of individualtypes of evidence. Although the group of experts is not acomprehensive set, they do represent a variety of scientificschools of thought and hence their average should be ofvalue (38). Furthermore, simple averaging methods foraggregating tend to be at least as good as more complexmethods for obtaining a consensus model (39). The resultsof this average model are also shown in Table 4.

Distribution of Expert Models’ Outcomes. Table 4 showsthat there is a large variance among the individual experts’models. This variance is most pronounced when there is noevidence. This means that before evidence is collected, the

TABLE 2. Example of a 2 × 2 × 2 Matrix for the Adequacy ofElectron Donors (Elicited from Expert L)a

[BTEX] >0.1 mg/L

[BTEX] <0.1 mg/L

[DOC] > 20 mg/L [TPH] > 1 mg/L 90% 80%[TPH] < 1 mg/L 80% 70%

[DOC] < 20 mg/L [TPH] > 1 mg/L 70% 70%[TPH] < 1 mg/L 40% 30%

a Given the following combinations for measured values of DOC,TPH, and BTEX, what is the Probability (adequate electron donors areavailable for reductive dechlorination)?

TABLE 3. Example of Conditional Probabilities for Products of Reductive Dechlorination (Elicited from Expert L)

given that reductive dechlorination is ... what is the probability that ...? probability (%)

occurring [cis-1,2-DCE] > 80% of [total 1,2-DCE] 90not occurring [cis-1,2-DCE] > 80% of [total 1,2-DCE] 10occurring vinyl chloride is detected 70not occurring vinyl chloride is detected 20occurring [ethene and ethane] > 0.01 mg/L 30not occurring [ethene and ethane] > 0.01 mg/L 10occurring [methane] > 0.1 mg/L 70not occurring [methane] > 0.1 mg/L 30occurring [chloride] > twice background 50not occurring [chloride] > twice background 10


experts have very different views about whether reductivedechlorination is occurring or not at a “typical” site.Furthermore, each expert has different beliefs and assump-tions about the chemical processes involved in reductivedechlorination. The distribution of expert models’ predictionsis examined for different cases of evidence in Figure 3.

The top line in the figure shows the range of predictedoutcomes, P d Probability (anaerobic degradation by reduc-tive dechlorination is occurring), in the absence of any site-specific data. The next three lines show that when increasingamounts of positive findings (e.g., [methane] > 0.1 mg/L)are collected, the expert models increase their confidence inthe occurrence of reductive dechlorination and move towardagreement (with increasing average P and eventually de-creasing standard deviation of P). Conversely, when increas-ing amounts of negative findings (e.g., [O2] > 1 mg/L) arecollected, the expert models move toward agreement thatreductive dechlorination is not taking place.

The last three rows of Figure 3 show what may occurunder certain situations of mixed evidence (i.e., positive andnegative findings). Because the experts disagree on theimportance of different types of evidence, the outcomes of

their models are in greater disagreement than prior to anyevidence (with little change in the average P; but, an increasein the standard deviation of P). The expert models have movedinto two “camps.” One group believes that the evidence ispersuasive for reductive dechlorination occurring; the othergroup believes the opposite.

Comparison of Experts’ Importance for Types of Evi-dence. Each of the 22 experts attributes different importanceto positive and negative findings for the 14 types of evidence.The importance of each positive or negative finding influencesthe predicted probability that reductive dechlorination isoccurring for a particular set of site conditions. This changein probability can be examined as the difference in thelogarithm of the odds ratios (LOR), that is

TABLE 4. Comparison of Expert Models’ Predictions for a TCE Site

Site Conditionstype of evidence no evidence well 87-20 in plume well 89-02 in plume well 89-06 downgradient

terminal electron accepting process (TEAP) - iron reduction iron reduction iron reductionhydrogen (H2) - <1 nM <1 nM <1 nMoxidation reduction potential (ORP) - - - -dissolved organic carbon (DOC) - <20 mg/L - <20 mg/Lbenzene, toluene, ethylbenzene, xylenes (BTEX) - - -total petroleum hydrocarbons (TPH) - - - -oxygen (O2) - <0.5 mg/L <0.5 mg/L <0.5 mg/Ltemperature - - - -pH - 5 < pH < 9 - 5 < pH < 9dichloroethene (DCE) - - - -vinyl chloride - detected detected not detectedethene & ethane - <0.01 mg/L - <0.01 mg/Lmethane - >0.1 mg/L - >0.1 mg/Lchloride - >2X background >2X background <2X backgroundAir Force/EPA protocol scorea 0 13 10 9

Predictions of Expert Modelsb

expert A 0.69 1.00 0.99 0.30expert B 0.39 0.92 0.95 0.09expert C 0.53 1.00 1.00 0.53expert D 0.63 1.00 1.00 0.71expert E 0.73 1.00 1.00 0.68expert F 0.72 0.99 0.98 0.04expert G 0.46 1.00 0.99 0.10expert H 0.18 0.73 0.78 0.06expert I 0.15 0.74 0.86 0.00expert J 0.82 0.96 0.98 0.13expert K 0.18 0.89 0.75 0.03expert L 0.77 0.98 0.97 0.40expert M 0.58 0.95 0.95 0.16expert N 0.60 0.96 0.95 0.40expert O 0.27 0.44 0.38 0.16expert Pc

expert Q 0.49 1.00 1.00 0.04expert R 0.25 0.99 0.97 0.02expert S 0.42 1.00 1.00 0.24expert T 0.45 1.00 1.00 0.02expert U 0.46 1.00 0.99 0.15expert V 0.56 1.00 0.98 0.39average model 0.47 0.96 0.95 0.13

Statistics for the Expert Models (A-V)average (P) 0.49 0.93 0.93 0.22standard deviation (P) 0.20 0.13 0.14 0.21

aWeight of evidence: 0-5 ) inadequate; 6-14 ) limited; 15-20 ) adequate; >20 ) strong. b P ) Probability (anaerobic degradation by reductivedechlorination is occurring). c Expert P’s model was incomplete because prior probabilities were not provided.

∆LOR ) log[posterior probability (true|evidence)

posterior probability (false|evidence)] -

log[prior probability (true|no evidence)

prior probability (false|no evidence)] (1)


When ∆LOR is positive, the evidence supports the event’soccurrence; negative evidence results in a negative ∆LOR.The greater the magnitude of ∆LOR, the greater the influenceand importance of the evidence. Figure 4 compares theimportance of different findings for three of the experts andthe average model. These experts were selected to illustratedifferences of opinion among the 22 experts. Some types ofevidence, like vinyl chloride, provide strong evidence to allof the experts. For other types of evidence, like oxidationreduction potential, the importance varies more markedlyamong the experts.

The importance of measuring the concentration ofmolecular hydrogen was a particularly divisive point amongthe elicitees. Because the equipment for performing this fieldtest has just recently become available at reasonable cost,many experts have little (or no) first-hand experience withit. However, even among those experts who were familiarwith it, its importance varied. Some experts believe that, ifperformed correctly, hydrogen measurements provide thebest means to assess the reducing conditions. Others feelthat there are too many variables in microbial communitiesfor hydrogen to be a reliable measurement. For example,

FIGURE 3. Distribution of expert models’ predictions for different cases of evidence. For each type of evidence, (+) is a positive findingand a (-) is a negative finding.

FIGURE 4. Importance of different findings for three expert models and the average model. Measured by ∆LOR(P) ) change in log (oddsratio) for probability (anaerobic degradation by reductive dechlorination is occurring).


according to expert B, measurements of molecular hydrogen(whether the findings are positive or negative) have noinfluence on the prediction of reductive dechlorination.

All of the experts agreed that groundwater temperaturehas little or no value as evidence. When reductive dechlo-rination was first studied, many thought that higher tem-peratures would lead to greater microbiological activity (16).However, it is believed by the vast majority of these expertsthat, regardless of ambient temperature, indigenous micro-organisms are nearly always capable of performing reductivedechlorination. In some cases, higher temperatures may leadto greater biodegradation rates; but, biological activity canbe adequate at any temperature.

The value of measuring pH varied among the experts. Forsome, pH is of little predictive value. These experts describedexperiences with low and high pH environments where themicrobial activity was adequate. However, other expertsbelieve that acidic or alkaline environments, especially inthe extremes, may inhibit reductive dechlorination. For theseexperts (e.g., expert B), a negative finding about pH (i.e., pH< 5 or pH > 9) can be somewhat influential; but, a positivefinding (i.e., 5 < pH < 9) is unlikely to contain valuableinformation.

Additionally, five of the experts believe that the micro-organisms for co-metabolic degradation and dehalorespi-ration are ubiquitous. Many of the other experts believe thatwhile nearly ubiquitous, there are some environments thatdo not support the necessary microorganisms.

In general, evidence that describes the conditions neces-sary for reductive dechlorination (TEAP, hydrogen, oxidationreduction potential, DOC, BTEX, TPH, oxygen, temperature,and pH) has less value for the experts than evidence aboutthe products of reductive dechlorination (DCE, vinyl chloride,ethene & ethane, and chloride) and related reactions(methane). Simply identifying that prerequisite conditionsare adequate is not sufficient; experts want to see productsto have confidence that reductive dechlorination is occurring.

DiscussionIn general the use of expert knowledge is desirable because(1) by acquiring expert knowledge, nonexperts can makebetter quality decisions, (2) by breaking up the decisionprocess into discrete components, experts can systematicallyspecify and integrate their knowledge, and (3) by combiningthe discrete elements of a decision and analyzing theoutcome, it is possible to identify which components aremost critical to the final evaluation, identify significantdifferences among experts, and determine the value ofadditional information (35).

All three of these benefits were demonstrated here. Bysharing their knowledge, the experts have contributed to adecision-making tool that can be used by less experiencedpractitioners. The experts themselves were challenged tothink in new (more explicitly probabilistic) terms than theywere accustomed. Their expert models can synthesizenumerous pieces of evidence differently, and possibly better,than the experts themselves. This process has providedconsensus information about which types of evidence aremost and least important. This can aid in the developmentof site investigation protocols and point to areas where furtherbiochemical research can be most fruitful.

In the future, the relative importance of the findings inthe average model (Figure 4) can be used to develop an easy-to-use scoring system similar to the Air Force/EPA protocol.In such a scoring system, each finding is awarded pointsbased on the magnitude of its column. For example, a positivefinding about DCE would be about 10 times as important asa positive finding about oxygen (40). Full development anddiscussion of such a system is planned for future publication.

AcknowledgmentsThe authors are grateful to the 22 experts whose results arepresented herein (see Supporting Information) and to thefour experts elicited in the preliminary round (James Gossett,Maureen Leahy, Jeff Schubert, and John Smith) for their time,knowledge, and willingness to participate. The comments ofthe three anonymous reviewers and the advice of MarekDruzdzel, David Dzombak, Paul Fischbeck, Baruch Fischhoff,Jay Kadane, Arifa Khandwalla, Richard Luthy, and GrangerMorgan contributed significantly to this project. This workwas supported in part by the NSF CAREER Grant CMS-9502546.

Supporting Information AvailableExpert elicitation protocol for an expert system to evaluatereductive dechlorination at sites with TCE-contaminatedground water and a list of expert elicitees. This informationis available free of charge via the Internet at http://pubs.acs.org.

Literature Cited(1) National Research Council. Alternatives for Groundwater Cleanup;

National Academy Press: Washington, DC, 1994.(2) Mertens, J. A. In Encyclopedia of Chemical Technology;

Kroschwitz, J. I., Ed.; Vol. 6, John Wiley and Sons: New York,1993.

(3) Steinberg, A. D.; DeSesso, J. M. Regul. Toxicol. Pharmacol. 1993,18, 137-153.

(4) U.S. EPA. Health Effects Assessment Summary Tables; U.S. EPAOffice of Solid Waste and Emergency Response: Washington,DC, 1994; EPA/540-R-94/020.

(5) U.S. EPA. Integrated Risk Information System; U.S. EPA Officeof Research and Development: Washington, DC, 1995.

(6) National Research Council. Innovations in Groundwater andSoil Cleanup; National Academy Press: Washington, DC, 1997.

(7) Doty, C. B.; Travis, C. C. The Effectiveness of GroundwaterPumping as a Restorative Technology; Oak Ridge NationalLaboratory: Oak Ridge, TN, 1991; ORNL/TM-11866.

(8) Renner, R. Environ. Sci. Technol. 1998, 32, 360A-363A.(9) U.S. EPA. Use of Monitored Natural Attenuation at Superfund,

RCRA Corrective Action, and Underground Storage Tank Sites;U.S. EPA Office of Solid Waste and Emergency Response:Washington, DC, 1999; Directive Number 9200.4-17P.

(10) Hart, S. Environ. Sci. Technol. 1996, 30, 398A-401A.(11) Powers, M. B.; Rubin, D. K. ENR. 1996, 237, 26-28.(12) U.S. EPA. Seminars: Monitored Natural Attenuation for Ground-

water; U.S. EPA Office of Research and Development: Wash-ington, DC, 1998; EPA/625/K-98/001.

(13) Munakata-Marr, J.; Matheson, V. G.; Forney, L. J.; Tiedje, J. M.;McCarty, P. L. Environ. Sci. Technol. 1997, 31, 786-791.

(14) Jenal-Wanner, U.; McCarty, P. L. Environ. Sci. Technol. 1997,31, 2915-2922.

(15) Renner, R. Environ. Sci. Technol. 1998, 32, 180A-182A.(16) Wiedemeier, T. H.; Swanson, M. A.; Moutoux, D. E.; Gordon, E.

K.; Wilson, J. T.; Wilson, B. H.; Kampbell, D. H.; Hansen, J. E.;Haas, P.; Chapelle, F. H. Technical Protocol for EvaluatingNatural Attenuation of Chlorinated Solvents in Groundwater;U.S. Air Force Center for Environmental Excellence: SanAntonio, TX, 1996.

(17) U.S. EPA. Technical Protocol for Evaluating Natural Attenuationof Chlorinated Solvents in Groundwater; U.S. EPA Office ofResearch and Development: Washington, DC, 1998; EPA/600/R-98/128.

(18) Sturchio, N. C.; Clausen, J. L.; Heraty, L. J.; Huang, L.; Holt, B.D.; Abrajano, Jr., T. A. Environ. Sci. Technol. 1998, 32, 3037-3042.

(19) Small, M. J. Water Resour. Res. 1997, 33, 957-969.(20) Freeze, R. A.; James, B.; Massman, J.; Sperling, T.; Smith, L.

Groundwater 1992, 30, 574-588.(21) Marin, C. M.; Medina, M. A.; Butcher, J. B. J. Contam. Hydrol.

1989, 5, 1-13.(22) Vogel, T. M. In Handbook of Bioremediation; Matthews, J. E.,

Ed.; CRC Press: Boca Raton, FL, 1994.(23) Chapelle, F. H.; McMahon, P. B.; Dubrovsky, N. M.; Fujii, R. F.;

Oaksford, E. T.; Vroblesky, D. A. Water Resour. Res. 1995, 31,359-371.

(24) Gossett, J. M.; Zinder, S. H. In Proceedings of the Symposium onNatural Attenuation of Chlorinated Organics in Groundwater;U.S. EPA: Washington, DC. 1997; EPA/540/R-97/504.


(25) McCarty, P. L. In Proceedings of the Symposium on NaturalAttenuation of Chlorinated Organics in Groundwater; U.S. EPA:Washington, DC, 1997; EPA/540/R-97/504.

(26) Remediation Technologies Development Forum. Natural At-tenuation of Chlorinated Solvents in Groundwater: Principlesand Practices; Version 3.0, 1997.

(27) Yang, Y.; McCarty, P. L. Environ. Sci. Technol. 1998, 32, 3591-3597.

(28) Carr, C. S.; Hughes, J. B. Environ. Sci. Technol. 1998, 32, 1817-1824.

(29) Chong, H. G.; Walley, W. J. Artif. Intell. Eng. 1996, 265-273.(30) Pearl, J. Probabilistic Reasoning in Intelligent Systems; Morgan

Kaufman: San Mateo, CA, 1988.(31) Charniak, E. AI Magazine. 1991, 12, 50-63.(32) Norsys. Netica Application for Belief Networks and Influence

Diagrams: User’s Guide; Versions 1.03 for Macintosh and 1.05for Windows; Norsys Software Corporation: Vancouver, Canada,1998.

(33) Morgan, G. M.; Henrion, M. Uncertainty; Cambridge UniversityPress: New York, 1990.

(34) Von Winterfeldt, D.; Edwards, W.; Decision Analysis andBehavioral Research; Cambridge University Press: New York,1986.

(35) Morgan, G. M.; Keith, D. W. Environ. Sci. Technol. 1995, 29,468A-476A.

(36) Yager, R. M.; Bilotta, S. E.; Mann, C. L.; Madsen, E. L. Environ.Sci. Technol. 1997, 31, 3138-3147.

(37) O’Leary, D. E. Decis. Sci. 1993, 24, 395-407.

(38) Dawes, R. M.; Faust, D.; Meehl, P. E. Science 1989, 243, 1668-1673.

(39) Clemen, R. T.; Winkler, R. L. Risk Anal., in press.

(40) Stiber, N. A. Ph.D. Dissertation, Carnegie Mellon University,1998.

Received for review November 23, 1998. Revised manuscriptreceived June 4, 1999. Accepted June 7, 1999.

ES981216S


expert system methodology for evaluating reductive dechlorination at tce sites

Documents