Office of the Science AdvisorRisk Assessment Forum

EPA/100/R-14/004 July 2014 www.epa.gov/raf

U.S. Environmental Protection Agency Office of Research and Development Washington, DC 20460

Official Business Penalty for Private Use $300

PRESORTED STANDARDPOSTAGE & FEES PAID

EPAPERMIT NO. G-35

EPA/100/R‐14/004

RiskAssessmentForumWhitePaper:

ProbabilisticRiskAssessmentMethodsandCaseStudies

July25,2014

U.S.EnvironmentalProtectionAgencyOfficeoftheScienceAdvisorRiskAssessmentForum

ProbabilisticRiskAnalysisTechnicalPanelWashington,D.C.20460

ii

Disclaimer ThisdocumenthasbeenreviewedinaccordancewithU.S.EnvironmentalProtectionAgency(EPA)policyandapprovedforpublication.Mentionoftradenamesorcommercialproductsdoesnotconstituteendorsementorrecommendationforuse.

ThisdocumentwasproducedbyaTechnicalPaneloftheEPARiskAssessmentForum(RAF).Theauthorsdrewontheirexperienceindoingprobabilisticassessmentsandinterpretingthemtoimproveriskmanagementofenvironmentalandhealthhazards.Interviews,presentationsanddialogueswithriskmanagersconductedbytheTechnicalPanelhavecontributedtotheinsightsandrecommendationsinthiswhitepaperandtheassociateddocumenttitledProbabilisticRiskAssessmenttoInformDecisionMaking:FrequentlyAskedQuestions.

U.S.EnvironmentalProtectionAgency(USEPA).2014.RiskAssessmentForumWhitePaper:ProbabilisticRiskAssessmentMethodsandCaseStudies.EPA/100/R‐09/001A.Washington,D.C.:RiskAssessmentForum,OfficeoftheScienceAdvisor,USEPA.

iii

Foreword ThroughoutmanyoftheU.S.EnvironmentalProtectionAgency’s(EPA)programofficesandregions,variousformsofprobabilisticmethodshavebeenusedtoanswerquestionsaboutexposureandrisktohumans,otherorganismsandtheenvironment.Riskassessors,riskmanagersandothers,particularlywithinthescientificandresearchdivisions,haverecognizedthatmoresophisticatedstatisticalandmathematicalapproachescouldbeutilizedtoenhancethequalityandaccuracyofAgencyriskassessments.Variousstakeholders,insideandoutsideoftheAgency,havecalledforamorecomprehensivecharacterizationofrisks,includinguncertainties,toimprovetheprotectionofsensitiveorvulnerablepopulationsandlifestages.

TheEPAidentifiedtheneedtoexaminetheuseofprobabilisticapproachesinAgencyriskassessmentsanddecisions.TheRAFdevelopedthispaperandthecompaniondocument,ProbabilisticRiskAssessmenttoInformDecisionMaking:FrequentlyAskedQuestions,toprovideageneraloverviewofthevalueofprobabilisticanalysesandsimilarorrelatedmethods,aswellasprovideexamplesofcurrentapplicationsacrosstheAgency.Draftsofbothdocumentswerereleased,withslightlydifferenttitles,forpubliccommentandexternalpeerreviewinAugust2009.AnexternalpeerreviewwasheldinArlington,VirginiainMay2010.

Thegoalofthesepublicationsisnotonlytodescribepotentialandactualusesofthesetools,butalsotoencouragetheirfurtherimplementationinhuman,ecologicalandenvironmentalriskanalysisandrelateddecisionmaking.Theenhanceduseofprobabilisticanalysestocharacterizeuncertaintyinassessmentswillnotonlyberesponsivetoexternalscientificadvice(e.g.,recommendationsfromtheNationalResearchCouncil)onhowtofurtheradvanceriskassessmentscience,butalsowillhelptoaddressspecificchallengesfacedbymanagersandincreasetheconfidenceintheunderlyinganalysisusedtosupportAgencydecisions.

____________________________________________RobertKavlockInterimScienceAdvisorU.S.EnvironmentalProtectionAgency

iv

ProbabilisticRiskAnalysisTechnicalPanel

HalûkÖzkaynak,EPAOfficeofResearchandDevelopment(Co‐Chair)RobertHetes,EPAOfficeofResearchandDevelopment(Co‐Chair)KathrynGallagher,EPAOfficeofWater(Co‐Chair)ChrisFrey,NorthCarolinaStateUniversity(whileontemporaryassignmentatEPA)(WhitePaperCo‐Lead)JohnPaul,EPAOfficeofResearchandDevelopment(WhitePaperCo‐Lead)PaskyPascual,EPAOfficeofResearchandDevelopment(WhitePaperCo‐Lead)MarianOlsen,EPARegion2(CaseStudyLead)MikeClipper,EPAOfficeofSolidWasteandEmergencyResponseMichaelMessner,EPAOfficeofWaterKeeveNachman,currentlyatTheJohnsHopkinsBloombergSchoolofPublicHealthZacharyPekar,EPAOfficeofAirandRadiationRitaSchoeny,EPAOfficeofResearchandDevelopmentCynthiaStahl,EPARegion3DavidHrdy,EPAOfficeofPesticideProgramsJohnLangstaff,EPAOfficeofAirandRadiationElizabethMargosches,EPAOfficeofChemicalSafetyandPollutionPrevention(retired)AudreyGalizia,EPAOfficeofResearchandDevelopmentNancyRios‐Jafolla,EPARegion3DonnaRandall,EPAOfficeofChemicalSafetyandPollutionPreventionKhoanDinh,EPAOfficeofChemicalSafetyandPollutionPrevention(retired)HarveyRichmond,EPAOfficeofAirandRadiation(retired)

OtherContributingAuthors

JonathanChen,EPAOfficeofChemicalSafetyandPollutionPreventionLisaConner,EPAOfficeofAirandRadiationJanetBurke,EPAOfficeofResearchandDevelopmentAllisonHess,EPARegion2KellySherman,EPAOfficeofChemicalSafetyandPollutionPreventionWoodrowSetzer,EPAOfficeofResearchandDevelopmentValerieZartarian,EPAOfficeofResearchandDevelopment

EPARiskAssessmentForumScienceCoordinators

JulieFitzpatrick,EPAOfficeoftheScienceAdvisorGaryBangs,EPAOfficeoftheScienceAdvisor(retired)

ExternalPeerReviewersScottFerson(Chair),AppliedBiomathematicsAnnetteGuiseppi‐Elie,DuPontEngineeringDaleHattis,ClarkUniversityIgorLinkov,U.S.ArmyEngineerResearchandDevelopmentCenterJohnToll,WindwardEnvironmentalLLC

v

Table of Contents Foreword.................................................................................................................................................................iii 

ListofFiguresandTables...............................................................................................................................viii 

ListofAcronymsandAbbreviations..............................................................................................................ix 

EXECUTIVESUMMARY.........................................................................................................................................1 

1.  INTRODUCTION:RELEVANCEOFUNCERTAINTYTODECISIONMAKING:HOWPROBABILISTICAPPROACHESCANHELP..............................................................................................4 

1.1.  EPADecisionMaking......................................................................................................................................4 1.2.  TheRoleofProbabilisticRiskAnalysisinCharacterizingUncertaintyand

Variability............................................................................................................................................................4 1.3.  GoalsandIntendedAudience......................................................................................................................5 1.4.  OverviewofThisDocument.........................................................................................................................5 1.5.  WhatAreCommonChallengesFacingEPARiskDecisionMakers?............................................5 1.6.  WhatAreKeyUncertaintyandVariabilityQuestionsOftenAskedbyDecision

Makers?.................................................................................................................................................................6 1.7.  WhyIstheImplementationofProbabilisticRiskAnalysisImportant?.....................................9 1.8.  HowDoesEPATypicallyAddressScientificUncertaintyandVariability?............................10 1.9.  WhatAretheLimitationsofRelyingonDefault‐BasedDeterministicApproaches?........11 1.10. WhatIsEPA’sExperiencewiththeUseofProbabilisticRiskAnalysis?.................................12 

2.  PROBABILISTICRISKANALYSIS..............................................................................................................15 

2.1.  WhatAreVariabilityandUncertainty,andHowAreTheyRelevanttoDecisionMaking?..............................................................................................................................................................15 

2.1.1.  Variability..................................................................................................................................15 2.1.2.  Uncertainty...............................................................................................................................15 

2.2.  WhenIsProbabilisticRiskAnalysisApplicableorUseful?..........................................................17 2.3.  HowCanProbabilisticRiskAnalysisBeIncorporatedIntoAssessments?...........................17 2.4.  WhatAretheScientificCommunity’sViewsonProbabilisticRiskAnalysis,and

WhatIstheInstitutionalSupportforItsUseinPerformingAssessments?..........................18 2.5.  AdditionalAdvantagesofUsingProbabilisticRiskAnalysisandHowItCan

ProvideMoreComprehensive,RigorousScientificInformationinSupportofRegulatoryDecisions...................................................................................................................................19 

2.6.  WhatAretheChallengestoImplementationofProbabilisticAnalyses?...............................19 2.7.  HowCanProbabilisticRiskAnalysisSupportSpecificRegulatoryDecision

Making?..............................................................................................................................................................20 2.8.  DoesProbabilisticRiskAnalysisRequireMoreResourcesThanDefault‐Based

DeterministicApproaches?.......................................................................................................................21 2.9.  DoesProbabilisticRiskAnalysisRequireMoreDataThanConventional

Approaches?....................................................................................................................................................21 2.10.  CanProbabilisticRiskAnalysisBeUsedtoScreenRisksorOnlyinComplexor

RefinedAssessments?..................................................................................................................................22 2.11. DoesProbabilisticRiskAnalysisPresentUniqueChallengestoModelEvaluation?.........23 2.12. HowDoYouCommunicatetheResultsofProbabilisticRiskAnalysis?.................................23 2.13.  AretheResultsofProbabilisticRiskAnalysisDifficulttoCommunicatetoDecision

MakersandStakeholders?.........................................................................................................................24 

vi

3.  ANOVERVIEWOFSOMEOFTHETECHNIQUESUSEDINPROBABILISTICRISKANALYSIS.........................................................................................................................................................26 

3.1.  WhatIstheGeneralConceptualApproachinProbabilisticRiskAnalysis?..........................26 3.2.  WhatLevelsandTypesofProbabilisticRiskAnalysesAreThereandHowAre

TheyUsed?.......................................................................................................................................................26 3.3.  WhatAreSomeSpecificAspectsofandIssuesRelatedtoMethodologyfor

ProbabilisticRiskAnalysis?......................................................................................................................29 3.3.1.  DevelopingaProbabilisticRiskAnalysisModel.......................................................29 3.3.2.  DealingWithDependenciesAmongProbabilisticInputs......................................29 3.3.3.  ConductingtheProbabilisticAnalysis...........................................................................30 

4.  SUMMARYANDRECOMMENDATIONS...................................................................................................33 

4.1.  ProbabilisticRiskAnalysisandRelatedAnalysesCanImprovetheDecision‐MakingProcessatEPA................................................................................................................................33 

4.2.  MajorChallengestoUsingProbabilisticRiskAnalysistoSupportDecisions......................34 4.3.  RecommendationsforEnhancedUtilizationofProbabilisticRiskAnalysisatEPA..........34 

GLOSSARY...............................................................................................................................................................37 

REFERENCES..........................................................................................................................................................42 

BIBLIOGRAPHY.....................................................................................................................................................48 

ProbabilisticRiskAnalysisMethodologyGeneral....................................................................................48 ProbabilisticRiskAnalysisandDecisionMaking.........................................................................................49 ProbabilisticRiskAnalysisMethodologySpecificAspects...................................................................49 SensitivityAnalysis...................................................................................................................................................50 CaseStudyExamplesofProbabilisticRiskAnalysisEPA(SeeAlsothePRACase

StudiesAppendix).........................................................................................................................................51 CaseStudyExamplesofProbabilisticRiskAnalysisOther..................................................................52 

APPENDIX:CASESTUDYEXAMPLESOFTHEAPPLICATIONOFPROBABILISTICRISKANALYSISINU.S.ENVIRONMENTALPROTECTIONAGENCYREGULATORYDECISIONMAKING............................................................................................................................................................54 

A.  OVERVIEW.......................................................................................................................................................55 

B.  INTRODUCTION.............................................................................................................................................55 

C.  OVERALLAPPROACHTOPROBABILISTICRISKANALYSISATTHEU.S.ENVIRONMENTALPROTECTIONAGENCY............................................................................................56 

C.1. U.S.EnvironmentalProtectionAgencyGuidanceandPoliciesonProbabilisticRiskAnalysis............................................................................................................................................................56 

C.2. CategorizingCaseStudies..........................................................................................................................56 

Group1CaseStudies..........................................................................................................................57 Group2CaseStudies..........................................................................................................................57 Group3CaseStudies..........................................................................................................................58 

vii

D.  CASESTUDYSUMMARIES..........................................................................................................................63 

D.1.  Group1CaseStudies....................................................................................................................63 

CaseStudy1:SensitivityAnalysisofKeyVariablesinProbabilisticAssessmentofChildren’sExposuretoArsenicinChromatedCopperArsenatePressure‐TreatedWood........................................................................................................................................63 CaseStudy2:AssessmentoftheRelativeContributionofAtmosphericDepositiontoWatershedContamination...................................................................................64 

D.2.  Group2CaseStudies....................................................................................................................65 

CaseStudy3:ProbabilisticAssessmentofAnglingDurationUsedintheAssessmentofExposuretoHudsonRiverSedimentsviaConsumptionofContaminatedFish...............................................................................................................................65 CaseStudy4:ProbabilisticAnalysisofDietaryExposuretoPesticidesforUseinSettingToleranceLevels..............................................................................................................65 CaseStudy5:One‐DimensionalProbabilisticRiskAnalysisofExposuretoPolychlorinatedBiphenylsviaConsumptionofFishFromaContaminatedSedimentSite.........................................................................................................................................66 CaseStudy6:ProbabilisticSensitivityAnalysisofExpertElicitationofConcentration‐ResponseRelationshipBetweenFineParticulateMatterExposureandMortality.....................................................................................................................69 CaseStudy7:EnvironmentalMonitoringandAssessmentProgram:UsingProbabilisticSamplingtoEvaluatetheConditionoftheNation’sAquaticResources................................................................................................................................................70 

D.3.  Group3CaseStudies....................................................................................................................71 

CaseStudy8:Two‐DimensionalProbabilisticRiskAnalysisofCryptosporidiuminPublicWaterSupplies,WithBayesianApproachestoUncertaintyAnalysis...........................................................................................................................71 CaseStudy9:Two‐DimensionalProbabilisticModelofChildren’sExposuretoArsenicinChromatedCopperArsenatePressure‐TreatedWood...................................72 CaseStudy10:Two‐DimensionalProbabilisticExposureAssessmentofOzone......74 CaseStudy11:AnalysisofMicroenvironmentalExposurestoFineParticulateMatterforaPopulationLivinginPhiladelphia,Pennsylvania..........................................76 CaseStudy12:ProbabilisticAnalysisinCumulativeRiskAssessmentofOrganophosphorusPesticides........................................................................................................77 CaseStudy13:ProbabilisticEcologicalEffectsRiskAssessmentModelsforEvaluatingPesticideUse...................................................................................................................78 CaseStudy14:ExpertElicitationofConcentration‐ResponseRelationshipBetweenFineParticulateMatterExposureandMortality.................................................80 CaseStudy15:ExpertElicitationofSea‐LevelRiseResultingFromGlobalClimateChange......................................................................................................................................81 CaseStudy16:KnowledgeElicitationforaBayesianBeliefNetworkModelofStreamEcology......................................................................................................................................82 

CASESTUDYREFERENCES................................................................................................................................85 

viii

List of Figures and Tables Figures

Figure1.GeneralPhasesoftheRiskAssessmentProcess.....................................................................................8Figure2.TieredApproachforRiskAssessment.......................................................................................................10Figure3.GraphicalDescriptionoftheLikelihood(Probability)ofRisk........................................................25Figure4.DiagrammaticComparisonofThreeAlternativeProbabilisticApproachesforthe

SameExposureAssessment...........................................................................................................................28 

AppendixFigures

FigureA‐1.MonteCarloCancerSummaryBasedonaOne‐DimensionalProbabilisticRiskAnalysisofExposuretoPolychlorinatedBiphenyls........................................................................72

FigureA‐2.MonteCarloNoncancerHazardIndexSummaryBasedonaOne‐DimensionalProbabilisticRiskAnalysisofExposuretoPolychlorinatedBiphenyls...................................73

FigureA‐3.UncertaintyDistributionModelResults...............................................................................................80

Tables

Table1.SelectedExamplesofEPAApplicationsofProbabilisticRiskAssessmentTechniques.........14

AppendixTable

TableA‐1.CaseStudyExamplesofEPAApplicationsofDeterministicandProbabilisticRiskAssessmentTechniques..................................................................................................................................59

ix

List of Acronyms and Abbreviations 1‐DMCA One‐DimensionalMonteCarloAnalysis2‐DMCA Two‐DimensionalMonteCarloAnalysisAPEX AirPollutantsExposureModelBBN BayesianBeliefNetworkCAA CleanAirActCASAC CleanAirScientificAdvisoryCommitteeCCA ChromatedCopperArsenateCPSC ConsumerProductSafetyCommissionCRA CumulativeRiskAssessmentCSFII ContinuingSurveyofFoodIntakebyIndividualsCWA CleanWaterActDEEM DietaryExposureEvaluationModelDRA DeterministicRiskAssessmentDRES DietaryRiskEvaluationSystemERA EcologicalRiskAssessmentEMAP EnvironmentalMonitoringandAssessmentProgramEPA U.S.EnvironmentalProtectionAgencyFACA FederalAdvisoryCommitteeActFAQ FrequentlyAskedQuestionsFDA U.S.FoodandDrugAdministrationFFDCA FederalFood,Drug,andCosmeticActFIFRA FederalInsecticide,Fungicide,andRodenticideActFQPA FoodQualityProtectionActGAO GovernmentAccountabilityOfficeHHRA HumanHealthRiskAssessmentHI HazardIndexIPCC IntergovernmentalPanelonClimateChangeIRIS IntegratedRiskInformationSystemLHS LatinHypercubeSamplingLOAEL Lowest‐Observed‐Adverse‐EffectLevelLT Long‐TermLT2 Long‐Term2EnhancedSurfaceWaterTreatmentRuleMCA MonteCarloAnalysisMCS MonteCarloSimulationMEE MicroexposureEventAnalysisMOEs MarginsofExposureNAAQS NationalAmbientAirQualityStandardsNAS NationalAcademyofSciencesNERL NationalExposureResearchLaboratoryNOAEL No‐Observed‐Adverse‐EffectLevelNRC NationalResearchCouncilOAQPS OfficeofAirQualityPlanningandStandardsOAR OfficeofAirandRadiationOCSPP OfficeofChemicalSafetyandPollutionPreventionOERR OfficeofEmergencyandRemedialResponseOGWDW OfficeofGroundwaterandDrinkingWaterOMB OfficeofManagementandBudgetOP OrganophosphorusPesticide

x

OPP OfficeofPesticideProgramsORD OfficeofResearchandDevelopmentOSA OfficeoftheScienceAdvisorOSTP OfficeofScienceandTechnologyPolicyOSWER OfficeofSolidWasteandEmergencyResponseOW OfficeofWaterPAH PolycyclicAromaticHydrocarbonPCB PolychlorinatedBiphenylPCCRARM Presidential/CongressionalCommissiononRiskAssessmentandRiskManagementPDP PesticideDataProgramPM ParticulateMatterPRA ProbabilisticRiskAssessmentRAF RiskAssessmentForumRfC ReferenceConcentration(Inhalation)RfD ReferenceDose(Oral)RIA RegulatoryImpactAnalysisRME ReasonableMaximumExposureSAB ScienceAdvisoryBoardSAP ScientificAdvisoryPanelSHEDS StochasticHumanExposureandDoseSimulationModelSHEDS‐PM StochasticHumanExposureandDoseSimulationModelforParticulateMatterSTPC ScienceandTechnologyPolicyCouncilTRIM.Expo TotalRiskIntegratedMethodology/ExposureModelUI UncertaintyIntervalUSDA U.S.DepartmentofAgricultureUSGS U.S.GeologicalSurveyWHO WorldHealthOrganization

1

EXECUTIVE SUMMARY Probabilisticriskassessment(PRA),initssimplestform,isagroupoftechniquesthatincorporateuncertaintyandvariabilityintoriskassessments.Variabilityreferstotheinherentnaturalvariation,diversityandheterogeneityacrosstime,spaceorindividualswithinapopulationorlifestage,whileuncertaintyreferstoimperfectknowledgeoralackofpreciseknowledgeofthephysicalworld,eitherforspecificvaluesofinterestorinthedescriptionofthesystem(USEPA2011c).Variabilityanduncertaintyhavethepotentialtoresultinoverestimatesorunderestimatesofthepredictedrisk.

PRAprovidesestimatesoftherangeandlikelihoodofahazard,exposureorrisk,ratherthanasinglepointestimate.StakeholdersinsideandoutsideoftheAgencyhaverecommendedamorecompletecharacterizationofrisks,includinguncertaintiesandvariability,inprotectingmoresensitiveorvulnerablepopulationsandlifestages.PRAcanbeusedtosupportriskmanagementbyassessmentofimpactsofuncertaintiesoneachofthepotentialdecisionalternatives.

Numerousadvisorybodies,suchastheScienceAdvisoryBoard(SAB)andtheNationalResearchCouncil(NRC)oftheNationalAcademyofSciences(NAS),haverecommendedthatEPAincorporateprobabilisticanalysesintotheAgency’sdecision‐makingprocess.EPA’sRiskAssessmentForum(RAF)formedaTechnicalPanel,consistingofrepresentativesfromtheAgency’sprogramandregionaloffices,todevelopthiswhitepaperanditscompaniondocument,titledProbabilisticRiskAssessmenttoInformDecisionMaking:FrequentlyAskedQuestions(FAQ).TheRAFisrecommendingthedevelopmentofAgencyresources,suchasaclearinghouseofPRAcasestudies,bestpractices,resourcesandseminars,toraisegeneralknowledgeabouthowtheseprobabilistictoolscanbeused.

TheintendedgoalofthiswhitepaperistoexplainhowEPAcanuseprobabilisticmethodstoaddressdata,modelandscenariouncertaintyandvariabilitybycapitalizingonthewidearrayoftoolsandmethodsthatcomprisePRA.ThiswhitepaperdescribeswherePRAcanfacilitatemoreinformedriskmanagementdecisionmakingthroughbetterunderstandingofuncertaintyandvariabilityrelatedtoAgencydecisions.Theinformationcontainedinthisdocumentisintendedforbothriskanalystsandmanagersfacedwithdeterminingwhenandhowtoapplythesetoolstoinformtheirdecisions.Thisdocumentdoesnotprescribeaspecificapproachbut,rather,describesthevariousstagesandaspectsofanassessmentordecisionprocessinwhichprobabilisticassessmenttoolsmayaddvalue.

ProbabilisticRiskAssessment

PRAisananalyticalmethodologyusedtoincorporateinformationregardinguncertaintyand/orvariabilityintoanalysestoprovideinsightregardingthedegreeofcertaintyofariskestimateandhowtheriskestimatevariesamongdifferentmembersofanexposedpopulation,includingsensitivepopulationsorlifestages.Traditionalapproaches,suchasdeterministicanalyses,oftenreportrisksas“centraltendency,”“highend”(e.g.,90thpercentileorabove)or“maximumanticipatedexposure,”butPRAcanbeusedtodescribemorecompletelytheuncertaintysurroundingsuchestimatesandidentifythekeycontributorstovariabilityoruncertaintyinpredictedexposuresorriskestimates.Thisinformationthencanbeusedbydecisionmakerstoachieveascience‐basedlevelofsafety,tocomparetherisksrelatedtodifferentmanagementoptions,ortoinvestinresearchwiththegreatestimpactonriskestimateuncertainty.

Tosupportregulatorydecisionmaking,PRAcanprovideinformationtodecisionmakersonspecificquestionsrelatedtouncertaintyandvariability.Forexample,inthecontextofadecisionanalysisthathasbeenconducted,PRAcan:identify“tippingpoints”wherethedecisionwouldbedifferentif

2

theriskestimatesweredifferent;estimatethedegreeofconfidenceinaparticulardecision;andhelptoestimatetrade‐offsrelatedtodifferentrisksormanagementoptions.PRAcanprovideuseful(evencritical)informationabouttheuncertaintiesandvariabilityinthedata,models,scenario,expertjudgmentsandvaluesincorporatedinriskassessmentstosupportdecisionmakingacrosstheAgency.

PRAisapplicabletobothhumanhealthriskassessment(HHRA)andecologicalriskassessment(ERA);however,therearedifferencesbetweenhowPRAisusedforthetwo.BothHHRAandERAhaveasimilarstructureandusethesameriskassessmentsteps,butHHRAfocusesonindividuals,asinglespecies,morbidityandmortality,butERAismoreconcernedwithmultiplepopulationsoforganisms(e.g.,individualspeciesoffishinariver)orecologicalintegrity(e.g.,willthetypesofspecieslivingintheriverchangeovertime).InERA,therealsoisarelianceonindicatorsofimpacts(e.g.,sentinelspeciesandothermetrics).

RiskAssessmentatEPA

PRAbeganplayinganincreasinglyimportantroleinAgencyriskassessmentsfollowingthe1997releaseofEPA’sPolicyforUseofProbabilisticAnalysisinRiskAssessmentattheU.S.EnvironmentalProtectionAgency(USEPA1997a)andpublicationoftheGuidingPrinciplesforMonte‐CarloAnalysis(USEPA1997b).PRAwasamajorfocusinanassociatedreviewofEPAriskassessmentpracticesbytheSAB(USEPA2007b).TheNRCrecommendedthatEPAadopta“tiered”approachforselectingthelevelofdetailusedinuncertaintyandvariabilityassessment(NRC2009).Furthermore,theNRCrecommendedthatadiscussionaboutthelevelofdetailusedforuncertaintyanalysisandvariabilityassessmentshouldbeanexplicitpartoftheplanning,scopingandproblemformulationstepintheriskassessmentprocess.BoththiswhitepaperandthecompanionFAQdocumenttakeintoaccountrecommendationsonriskassessmentprocessesdescribedintheNRC’sreportScienceandDecisions:AdvancingRiskAssessment(NRC2009)andEnvironmentalDecisionsintheFaceofUncertainty(IOM2013).

EPA’srecentriskassessmentpublications,includingthedocumenttitledFrameworkforHumanHealthRiskAssessmenttoInformDecisionMaking(UAEPA2014b)aswellasthiswhitepaper,emphasizetheimportanceofcommunicatingtheresultsofaPRAbecauseitprovidestherangeandlikelihoodestimatesforoneormoreaspectsofhazard,exposureorrisk,ratherthanasinglepointestimate.Riskassessorsareresponsibleforsharinginformationonprobabilisticresultssothatdecisionmakershaveaclearunderstandingofquantitativeassessmentsofuncertaintyandvariability,andhowthisinformationwillaffectthedecision.EffectivecommunicationbetweentheriskassessoranddecisionmakeriskeytopromoteunderstandinganduseoftheresultsfromthePRA.

PRAgenerallyrequiresmoreresourcesthanstandardAgencydefault‐baseddeterministicapproaches.Appropriatelytrainedstaffandtheavailabilityofadequatetools,methodsandguidanceareessentialfortheapplicationofPRA.Properapplicationofprobabilisticmethodsrequiresnotonlysoftwareanddata,butalsoguidanceandtrainingforanalystsusingthetools,andformanagersanddecisionmakerstaskedwithinterpretingandcommunicatingtheresults.Inmostcircumstances,probabilisticassessmentsmaytakemoretimeandefforttoconductthanconventionalapproaches,primarilybecauseofthecomprehensiveinclusionofavailableinformationonmodelinputs.Thepotentiallyhigherresourcecostsmaybeoffset,however,byamoreinformeddecisionthanwouldbeprovidedbyacomparabledeterministicanalysis.

ContentoftheWhitePaperandFrequentlyAskedQuestionsCompanionDocuments

ThesetwodocumentsdescribehowPRAcanbeappliedtoenhancethescientificfoundationofEPA’sdecisionmakingacrosstheAgency.ThiswhitepaperdescribesthechallengesfacedbyEPA

3

decisionmakers,definesandexplainsthebasicprinciplesofprobabilisticanalysis,brieflyhighlightsinstanceswherethesetechniqueshavebeenimplementedinEPAdecisions,anddescribescriteriathatmaybeusefulindeterminingwhetherandhowtheapplicationofprobabilisticmethodsmaybeusefuland/orapplicabletodecisionmaking.Thiswhitepaperalsodescribescommonlyemployedmethodstoaddressuncertaintyandvariability,includingthoseusedintheconsiderationofuncertaintyinscenariosanduncertaintyinmodels.Additionally,itaddressesuncertaintyandvariabilityintheinputsandoutputsofmodelsandtheimpactoftheseuncertaintiesoneachofthepotentialmanagementoptions.Ageneraldescriptionoftherangeofmethodsfromsimpletocomplex,rapidtomoretimeconsumingandleasttomostresourceintensiveisprovided,aswellasusesofthesemethods.

Bothdocumentsaddressissuessuchasuncertaintyandvariability,theirrelevancetodecisionmakingandthePRAgoaltoprovidequantitativecharacterizationoftheuncertaintyandvariabilityinestimatesofhazard,exposure,orrisk.ThedifferencebetweenthewhitepaperandtheFAQsdocumentisthelevelofdetailprovidedaboutPRAconceptsandpractices,andtheintendedaudience(e.g.,riskassessorsvs.decisionmakers).DetailedexamplesofapplicationsofthesemethodsareprovidedintheAppendixofthiswhitepaper,whichistitled“CaseStudyExamplesoftheApplicationofProbabilisticRiskAnalysisinU.S.EnvironmentalProtectionAgencyDecisionMaking.”ThewhitepaperAppendixincludes16casestudies—11HHRAand5ERAexamples—thatillustratehowEPA’sprogramandregionalofficeshaveusedprobabilistictechniquesinriskassessment.Toaidindescribinghowthesetoolswereapplied,the16casestudiesaresubdividedamong3categoriesforpurposesofthisdocument.Group1includes2casestudiesdemonstratingpointestimate,includingsensitivityanalysis;Group2iscomprisedof5casestudiesdemonstratingprobabilisticriskanalysis,includingone‐dimensionalMonteCarloanalysisandprobabilisticsensitivityanalysis;andGroup3includes9casestudiesdemonstratingadvancedprobabilisticriskanalysis,includingtwo‐dimensionalMonteCarloanalysiswithmicroexposure(microenvironments)modeling,Bayesianstatistics,geostatisticsandexpertelicitation.

TheFAQdocumentprovidesanswerstocommonquestionsregardingPRA,includingkeyconceptssuchasscientificandinstitutionalmotivationsfortheuseofPRA,andchallengesintheapplicationofprobabilistictechniques.TheprincipalreasonforincludingPRAasanoptionintheriskassessor’stoolboxisitsabilitytosupporttherefinementandimprovementoftheinformationleadingtodecisionmakingbyincorporatingknownuncertainties.

4

1. INTRODUCTION: RELEVANCE OF UNCERTAINTY TO DECISION MAKING: HOW PROBABILISTIC APPROACHES CAN HELP

1.1. EPA Decision Making TodiscusswhereprobabilisticapproachescanaidEPA’sdecisionmaking,itisimportanttogenerallydescribetheAgency’scurrentdecision‐makingprocessesandhowbetterunderstandingandimprovingelementswithintheseprocessescanclarifywhereprobabilisticapproachesmightprovidebenefits.TheenhanceduseofPRAandcharacterizationofuncertaintywouldallowEPAdecisionmakersopportunitiestouseamorerobustandtransparentprocess,whichmayallowgreaterresponsivenesstooutsidecommentsandrecommendations.SuchanapproachwouldsupporthigherqualityEPAassessmentsandimproveconfidenceinAgencydecisions.

Therearetwomajorareasinthedecision‐makingprocessthatmightbeimprovedwithPRA.Scientistscurrentlyaregenerallyfocusedonthefirstarea—theunderstandingofdata,modelandscenariouncertaintiesandvariability.Thesecondareaisonethathasnot,untilrecentlyandonlyinalimitedfashion,beenusedbyEPAdecisionmakers.Thisareaisformaldecisionanalysis.Withdecisionanalytictechniques,decisionmakerscanweightherelativeimportanceofriskinformationcomparedtootherinformationinmakingthedecision,understandhowuncertaintyaffectstherelativeattractivenessofpotentialdecisionalternatives,andassessoverallconfidenceinadecision.Inadditiontodata,modelandscenariouncertainty,thereisaseparatecategoryofuncertaintiesspecificallyassociatedwithhowthedecisioncriteriarelatetothedecisionalternatives.Althoughitisquiterelevanttoriskmanagementdecisions,thetopicanddecisionanalysisingeneralareoutsideofthescopeofthisreport.Thiswhitepaperfocusesontechnicalinformationthatwouldallowbetterunderstandingoftherelationshipsamongalternativedecisionsinassessingrisks.

1.2. The Role of Probabilistic Risk Analysis in Characterizing Uncertainty and Variability

Probabilisticanalysesincludetechniquesthatcanbeappliedformallytoaddressbothuncertaintyandvariability,typicallyarisingfromlimitationsofdata,modelsoradequatelyformulatingthescenariosusedinassessingrisks.Probabilityisusedinscience,business,economicsandotherfieldstoexamineexistingdataandestimatethechanceofanevent,fromhealtheffectstoraintomentalfatigue.Onecanuseprobability(chance)toquantifythefrequencyofoccurrenceorthedegreeofbeliefininformation.Forvariability,probabilitydistributionsareinterpretedasrepresentingtherelativefrequencyofagivenstateofthesystem(e.g.,thatthedataaredistributedinacertainway);foruncertainty,theyrepresentthedegreeofbelieforconfidencethatagivenstateofthesystemexists(e.g.,thatwehavetheappropriatedata;CullenandFrey1999).PRAoftenisdefinednarrowlytoindicateastatisticalorthoughtprocessusedtoanalyzeandevaluatethevariabilityofavailabledataortolookatuncertaintyacrossdatasets.

Forthepurposesofthisdocument,PRAisatermusedtodescribeaprocessthatemploysprobabilitytoincorporatevariabilityindatasetsand/ortheuncertaintyininformation(suchasdataormodels)intoanalysesthatsupportenvironmentalrisk‐baseddecisionmaking.PRAisusedherebroadlytoincludebothquantitativeandqualitativemethodsfordealingwithscenario,modelandinputuncertainty.Probabilistictechniquescanbeusedwithothertypesofanalysis,suchasbenefit‐costanalysis,regulatoryimpactanalysisandengineeringperformancestandards;thus,theycanbeusedforavarietyofapplicationsandbyexpertsinmanydisciplines.

5

1.3. Goals and Intended Audience TheprimarygoalsofthiswhitepaperaretointroducePRA,describehowitcanbeusedtobetterinformandimprovethedecision‐makingprocess,andprovidecasestudieswhereithasbeenusedinhumanhealthandecologicalanalysesatEPA(seetheAppendixforthecasestudies).AsecondarygoalofthispaperistobridgecommunicationgapsregardingPRAamonganalystsofvariousdisciplines,betweentheseanalystsandAgencydecisionmakers,andamongaffectedstakeholders.Thiswhitepaperalsoisintendedtoserveasacommunicationtooltointroducekeyconceptsandbackgroundinformationonapproachestoriskanalysisthatincorporateuncertaintyandprovideamorecomprehensivetreatmentofvariability.Riskanalysts,decisionmakersandaffectedstakeholderscanbenefitfromunderstandingthepotentialusesofPRA.PRAandrelatedapproachescanbeusedtoidentifyadditionalresearchthatmayreduceuncertaintyandmorethoroughlycharacterizevariabilityinariskassessment.ThiswhitepaperexplainshowPRAcanenhancethedecision‐makingprocessesfacedbymanagersatEPAbybettercharacterizingdata,model,scenarioanddecisionuncertainties.

1.4. Overview of This Document ThiswhitepaperprovidesanoverviewofEPA’sinterestandexperienceinaddressinguncertaintyandvariabilityusingprobabilisticmethodsinriskassessment;identifieskeyquestionsaskedorfacedbyAgencydecisionmakers;demonstrateshowconventionaldeterministicapproachestoriskanalysismaynotanswerthesequestionsfully;providesexamplesofapplications;andshowshowandwhy“probabilisticriskanalysis”(broadlydefined)couldprovideaddedvalue,comparedtotraditionalmethods,withregardtoregulatorydecisionmakingbymorefullycharacterizingriskestimatesandexploringdecisionuncertainties.Forthepurposesofthiswhitepaper,PRAandrelatedtoolsforbothhumanhealthandecologicalassessmentsincludearangeofapproaches,fromstatisticaltools,suchassensitivityanalysis,tomulti‐dimensionalMonteCarlomodels,geospatialapproachesandexpertelicitation.KeypointsaddressedbythisdocumentincludedefinitionsandkeyconceptspertainingtoPRA,benefitsandchallengesofPRA,ageneralconceptualframeworkforPRA,conclusionsregardingproductsandinsightsobtainedfromPRA,andexampleswhereEPAhasusedPRAinhumanhealthandecologicalanalyses.AGlossaryandaBibliographyalsoareprovided.

1.5. What Are Common Challenges Facing EPA Risk Decision Makers?

EPAoperatesunderstatutoryandregulatoryconstraintsthatoftenlimitthetypesofcriteriathatcanbeconsidered(includingwhethertheuseofPRAisappropriate)andimposestricttimeframesinwhichdecisionsmustbemade.Typically,thedecisionbeginswithunderstanding(1)whoorwhatwillbeprotected;(2)therelationshipbetweenthedataanddecisionalternatives;and(3)theimpactofdata,modelanddecisionuncertaintiesrelatedtoeachdecisionalternative.Theseareamongtheconsiderationsoftheplanningandscopingandproblemformulationphasesofriskassessment(USEPA2014).EPAdecisionmakersneedtoconsidermultipledecisioncriteria,whichareinformedbyvaryingdegreesofconfidenceintheunderlyinginformation.Decisionmakersneedtobalancetheregulatory/statutoryrequirementsandtimeframes,resources(i.e.,expertise,costsoftheanalysis,reviewtimes,etc.)toconducttheassessment,managementoptions,andstakeholderswhileatthesametimekeepingriskassessmentanddecisionmakingseparate.

Uncertaintycanbeintroducedintoanyassessmentatanystepintheprocess,evenwhenusinghighlyaccuratedatawiththemostsophisticatedmodels. Uncertaintycanbereducedorbettercharacterizedthroughknowledge.Variabilityornaturalheterogeneityisinherentinnaturalsystemsandthereforecannotbereduced,butcanbeexaminedanddescribed.Uncertaintyindecisionsisunavoidablebecausereal‐worldsituationscannotbeperfectlymeasured,modeledor

6

predicted.Asaresult,EPAdecisionmakersfacescientificallycomplexproblemsthatarecompoundedbyvaryinglevelsofuncertaintyandvariability.Ifuncertaintyandvariabilityhavenotbeenwellcharacterizedoracknowledged,potentialcomplicationsariseintheprocessofdecisionmaking.Increaseduncertaintycanmakeitmoredifficulttodetermine,withreasonableconfidence,thebalancepointbetweenthecostsofregulationandtheimplicationsforavoidingdamagesandproducingbenefits.Characterizationfacilitatedbyprobabilisticanalysescanprovideinsightintoweighingtherelativecostsandbenefitsofvaryinglevelsofregulationandalsocanassistinriskcommunicationactivities.

Decisionmakersoftenwanttoknowwhoisatriskandbyhowmuch,thetradeoffsbetweenalternativeactionsandthelikelyorpossibleconsequencesofdecisions.Tothisend,itisparticularlyusefulfordecisionmakerstounderstandthedistributionofriskacrosspotentiallyimpactedpopulationsandecologicalsystems.Itcanbeimportanttoknowthenumberofindividualsexperiencingdifferentmagnitudesofrisk,thedifferencesinriskmagnitudeexperiencedbyindividualsindifferentlifestagesorpopulationsortheprobabilityofaneventthatmayleadtounacceptablelevelsofrisk.Giventhelimitationsofdata,traditionalmethodsofriskanalysesarenotwellsuitedtoproducesuchestimates.Probabilisticanalyticalmethodsarecapableofaddressingtheseshortcomingsandcancontributetoamorethoroughrecognitionoftheimpactofdatagapsontheprojectedriskestimates.AlthoughPRAcanbeusedtocharacterizetheuncertaintyandvariabilityinsituationswithlimiteddata,currentlythereisnotextensiveexperienceusingPRAtocharacterizetherangeofeffectsordose‐responserelationshipsforpopulations,includingsensitivepopulationsandlifestages.

OtherchallengesfacingEPAdecisionmakersincludetheneedtoconsidermultipledecisioncriteria,whichareinformedbyvaryingdegreesofconfidenceintheunderlyinginformation,understandingtherelationshipbetweenandamongthosedecisioncriteria(includingmulti‐pollutantandmulti‐mediaeffects)andthedecisionalternatives,andthetimelinessofthedecisionmaking.Furthermore,evenwhenPRAisused,EPAdecisionmakersmustbemindfulofpotentialmisusesandobfuscationswhenconductingorpresentingPRAresults.DecisionmakersalsoneedtoconsidertheevolvingsciencebehindPRA.AstheuseofPRAincreasesdecisionmakerswillbecomemorefamiliarwiththetechniquesandtheirapplication.

Ariskassessmentprocessneedstoconsideruncertainties,variabilityandtherationaleorfactorsinfluencinghowtheymaybeaddressedbyadecisionmaker.Decisionmakersneedafoundationforestimatingthevalueofcollectingadditionalinformationtoallowforbetterinformeddecisions.Therearecostsassociatedwithignoringuncertainty(McConnell1997andToll1999),andafocusbydecisionmakersontheinformationprovidedbyuncertaintyanalysiscanstrengthentheirchoices.

1.6. What Are Key Uncertainty and Variability Questions Often Asked by Decision Makers?

Asdescribedabove,determiningthedecision‐makingcontextandspecificconcernsisacriticalfirststeptowarddevelopingausefulandresponsiveriskassessmentthatwillsupportthedecision.Forexample,theappropriatefocusandlevelofdetailoftheanalysisshouldbecommensuratewiththeneedsofthedecisionmakerandstakeholders,aswellastheappropriateuseofscience.Analysesoftenareconductedatalevelofdetaildictatedbytheissuebeingaddressed,thebreadthandqualityoftheavailableinformationuponwhichtobaseananalysis,andthesignificancesurroundingadecision.Theanalyticalprocesstendstobeiterative.Althoughaguidingsetofquestionsmayframetheinitialanalyses,additionalquestionscanarisethatfurtherdirectorevenreframetheanalyses.

7

BasedonaseriesofdiscussionswithAgencydecisionmakersandriskassessors,sometypicalquestionsaboutuncertaintyandvariabilityrelevanttoriskanalysesincluding:

Factorsinfluencingdecisionuncertainty:

Wouldmydecisionbedifferentifthedataweredifferent,improvedorexpanded?Wouldadditionaldatacollectionandresearchlikelyleadtoadifferentdecision?Howlongwillittaketocollecttheinformation,howmuchwoulditcost,andwouldtheresultingdecisionbesignificantlyaltered?

Whataretheliabilitiesandconsequencesofmakingadecisionunderthecurrentlevelofknowledgeanduncertainty?

Howdothealternativesandtheirassociateduncertaintyandvariabilityaffectthetargetpopulationorlifestage?

Considerationsforevaluatingdataormethoduncertainty:

Howrepresentativeorconservativeistheestimateduetodataormethoduncertainty(alsoincorporatingvariability)?

Whatarethemajorgapsinknowledge,andwhatarethemajorassumptionsusedintheassessment?Howreasonablearetheassumptions?

Issuesarisingwhenaddressingvariability:

Canaprobabilisticapproach(e.g.,tobettercharacterizeuncertaintiesandvariability)beaccomplishedinatimelymanner?

Whatisthedesiredpercentileofthepopulationtobeprotected?Bychoosingthispercentile,whomaynotbeprotected?

Thequestionsthatariseconcerninguncertaintyandvariabilitychangedependingonthestageandnatureofthedecision‐makingprocessandanalysis.GeneralphasesoftheriskassessmentprocessareillustratedinFigure1.Forfurtherinformationontheprocessofdecisionmaking,wesuggestreferringtothedescriptionprovidedbyEPARegion3ontheMulti‐CriteriaIntegratedResource

8

Figure 1. General Phases of the Risk Assessment Process. Risk assessment is an iterative process comprised of planning, scoping and problem formulation; analysis (e.g., hazard identification, dose‐response assessment and exposure assessment); interpretation and risk characterization; and risk communication. The highlighted boxes explain how PRA fits into the overall process. 

AssessmentInternetpageathttp://www.epa.gov/reg3esd1/data/mira.htm.TheutilityofvariouslevelsofanalysisandsophisticationinansweringthesequestionsisillustratedinthecasestudiesdescribedinSection1.10andpresentedintheAppendixofthiswhitepaper.ReferencestoexamplesbeyondtheseEPAcasestudiescanbefoundintheBibliography.Additionally,Lesteretal.(2007)identifiedmorethan20PRAapplicationcasestudies(includingEPAexamples)performedsince2000;thesecasestudyexamplesarecategorizedassite‐specificapplicationsandregionalriskassessments.

9

1.7. Why Is the Implementation of Probabilistic Risk Analysis Important?

TheprincipalreasonfortheinclusionofPRAasanoptionintheriskassessor’stoolboxisPRA’sabilitytosupportrefinementandimprovementoftheinformationleadingtodecisionmakingbyincorporatingknownuncertainties.Beginningasearlyasthe1980s,expertscientificadvisorygroups,suchastheNationalResearchCouncil(NRC),recommendedthatriskanalysesincludeacleardiscussionoftheuncertaintiesinriskestimation(NRC1983).TheNRCstatedtheneedtodescribeuncertaintyandtocapturevariabilityinriskestimates(NRC1994).ThePresidential/CongressionalCommissiononRiskAssessmentandRiskManagement(PCCRARM)recommendedagainstarequirementorneedfora“brightline”orsingle‐numberlevelofrisk(PCCRARM1997).SeeSection2.4formoreinformationregardingthescientificcommunity’sopinionontheuseofPRA.

Regulatoryscienceoftenrequiresselectionofalimitforacontaminant,yetthatlimitalwayscontainsuncertaintyastohowprotectiveitis.PRAandrelatedtoolsquantitativelydescribetheveryrealvariationsinnaturalsystemsandlivingorganisms,howtheyrespondtostressors,andtheuncertaintyinestimatingthoseresponses.

RiskcharacterizationbecameEPApolicyin1995(USEPA1995b),andtheprinciplesoftransparency,clarity,consistencyandreasonablenessareexplicatedinthe2000RiskCharacterizationHandbook(USEPA2000a).Transparency,clarity,consistencyandreasonablenesscriteriarequiredecisionmakerstodescribeandexplaintheuncertainties,variabilityandknowndatagapsintheriskanalysisandhowtheyaffecttheresultingdecision‐makingprocesses(USEPA1992,1995a,2000a).

TheuseofprobabilisticmethodsalsohasreceivedsupportfromsomedecisionmakerswithintheAgency,andthesemethodshavebeenincorporatedintoanumberofEPAdecisionstodate.Programoffices,suchastheOfficeofPesticidePrograms(OPP),OfficeofSolidWasteandEmergencyResponse(OSWER),OfficeofAirandRadiation(OAR),andOfficeofWater(OW),aswellastheOfficeofResearchandDevelopment(ORD),haveutilizedprobabilisticapproachesindifferentwaysandtovaryingextents,forbothhumanexposureandecologicalriskanalyses.Inaddition,OSWERhasprovidedexplicitguidanceontheuseofprobabilisticapproachesforexposureanalysis(USEPA2001).SomeprogramofficeshaveheldtrainingsessionsonMonteCarlosimulation(MCS)softwarethatisusedfrequentlyinprobabilisticanalyses.

TheNRCrecommendedthatEPAshouldadoptatieredapproachforselectingthelevelofdetailusedinuncertaintyandvariabilityassessment(NRC2009).Furthermore,NRCrecommendedthatadiscussionaboutthelevelofdetailusedforuncertaintyanalysisandvariabilityassessmentshouldbeanexplicitpartoftheplanning,scopingandproblemformulationstepintheriskassessmentprocess.ThewaythatPRAfitsintoagraduatedhierarchical(tiered)approachismorefullydescribedinSection2.10andillustratedinFigure2.

Whenitisbeneficialtorefineriskestimates,theuseofPRAcanhelpinthecharacterizationandcommunicationofuncertainty,variabilityandtheimpactofdatagapsinriskanalysesforassessors,decisionmakersandstakeholders(includingthetargetpopulationorlifestage).

10

Figure 2. Tiered Approach for Risk Assessment. The applicability of a probabilistic approach depends on the needs of decision makers and stakeholders. Assessments that are high in complexity and regulatory significance benefit from the application of probabilistic techniques.  Source: Adapted from USEPA 2004a and WHO 2008. 

1.8. How Does EPA Typically Address Scientific Uncertainty and Variability?

Environmentalassessmentscanbecomplex,suchascoveringexposuretomultiplechemicalsinmultiplemediaforawide‐rangingpopulation.TheAgencyhasdevelopedsimplifiedapproachestocharacterizerisksassociatedwithsuchcomplexassessmentsthroughtheuseofpointestimatesformodelvariablesorparameters.Suchanapproachtypicallyproducespointestimatesofrisks(e.g.,10‐5oralifetimeprobabilityofcancerriskofoneindividualin100,000).Theseoftenarecalled“deterministic”assessments.Asaresultoftheuseofpointestimatesforvariablesinmodelalgorithms,deterministicriskresultsusuallyarereportedaswhatareassumedtobeeitheraverageorworst‐caseestimates.Theydonotcontainanyquantitativeestimateoftheuncertaintyinthatestimate,norreportwhatpercentileoftheexposedpopulationtheestimateapplies.ThemethodstypicallyusedinEPAriskassessmentsrelyonacombinationofpointvalueswithpotentiallyvaryinglevelsofconservatismandcertainty,yieldingapointestimateofexposureatsomepointintherangeofpossiblerisks.

Becauseuncertaintyisinherentinallriskassessments,itisimportantthattheriskassessmentprocessenablehandlinguncertaintiesinalogicalwaythatistransparentandscientificallydefensible,consistentwiththeAgency’sstatutorymissionandresponsivetotheneedsofdecisionmakers(NRC1994).Uncertaintyisafactorinbothecologicalandhumanhealthriskassessments.

11

Forhumanhealthriskassessments,uncertaintiesariseforbothnoncancerandcancerendpoints.Thus,whendataaremissing,EPAoftenusesseveraloptionstoprovideboundariesonuncertaintyandvariabilityinanattempttoavoidriskunderestimation;attemptingtogiveasinglequantificationofhowmuchconfidencethereisintheriskestimatemaynotbeinformativeorfeasible.

Inexposureassessment,forexample,thepracticeatEPAistocollectnewdatawheretheyareneededandwheretimeandresourcesallow.Alternativeapproachestoaddressuncertaintyincludenarrowingthescopeoftheassessment;usingscreening‐leveldefaultassumptionsthatincludeupper‐endvaluesand/orcentraltendencyvaluesthataregenerallycombinedtogenerateriskestimatesthatfallwithinthehigherendofthepopulationriskrange(USEPA2004b);applyingmodelstoestimatemissingvalues;usingsurrogatedata(e.g.,dataonaparameterthatcomefromadifferentregionofthecountrythantheregionbeingassessed);orapplyingprofessionaljudgment.Theuseofindividualassumptionscanrangefromqualitative(e.g.,assumingthatoneissecuredtotheresidencelocationanddoesnotmovethroughtimeorspace)tomorequantitative(e.g.,usingthe95thpercentileofasampledistributionforaningestionrate).Thisapproachalsocanbeappliedtothepracticeofhazardidentificationanddose‐responseassessmentwhendataaremissing.Identifyingthesensitivityofexposureorriskestimatestokeyinputscanhelpfocuseffortstoreduceuncertaintybycollectingadditionaldata.

CurrentEPApracticestoaddressuncertaintyandvariabilityarefocusedontheevaluationofdata,model,andscenariouncertaintyandvariability.Inaddition,decisionmakersarefacedwithcombiningmanydifferentdecisioncriteriathatmaybeinformedbyscienceandPRAaswellasbyexpertjudgmentortheweightingofvaluestochooseadecisionalternative.Data,model,andscenariouncertaintiesandvariability(includingtheirprobabilitydistributions),aswellasexpertjudgment,canbeimportantconsiderationsintheselectionofonealternativeoveranother(Costanzaetal.1997;Morganetal.2009;StahlandCimorelli2005;Wrightetal.2002).

1.9. What Are the Limitations of Relying on Default-Based Deterministic Approaches?

Default‐baseddeterministicapproachesareappliedtodata,modelandscenariouncertainties.Deterministicriskassessment(DRA)oftenisconsideredatraditionalapproachtoriskanalysisbecauseoftheexistenceofestablishedguidanceandproceduresregardingitsuse,theeasewithwhichitcanbeperformed,anditslimiteddataandresourceneeds.TheuseofdefaultssupportingDRAprovidesaproceduralconsistencythatallowsforriskassessmentstobefeasibleandtractable.DecisionmakersandmembersofthepublictendtoberelativelyfamiliarwithDRA,andtheuseofsuchanapproachaddressesassessment‐relateduncertaintiesprimarilythroughtheincorporationofpredetermineddefaultvaluesandconservativeassumptions.Itaddressesvariabilitybycombininginputparametersintendedtoberepresentativeoftypicalorhigherendexposure(i.e.,consideredtobeconservativeassumptions).Theintentionoftenistoimplicitlyprovideamarginofsafety(i.e.,morelikelytooverestimateriskthanunderestimaterisk)orconstructascreening‐levelestimateofhigh‐endexposureandrisk(i.e.,anestimaterepresentativeofmorehighlyexposedandsusceptibleindividuals).

DRAprovidesanestimationoftheexposuresandresultingrisksthataddressesuncertaintiesandvariabilitiesinaqualitativemanner.ThemethodstypicallyusedinEPADRArelyonacombinationofpointvaluessomeconservativeandsometypicalyieldingapointestimateofexposurethatisatsomeunknownpointintherangeofpossiblerisks.AlthoughthisconservativebiasalignswiththepublichealthmissionofEPA(USEPA2004b),thedegreeofconservatismintheseriskestimates(andinanyconcomitantdecision)cannotbeestimatedwellorcommunicated(HattisandBurmaster1994).Typically,thisresultsinunquantifieduncertaintyinriskstatements.

12

Quantitativeinformationregardingtheprecisionorpotentialsystematicerrorandthedistributionofexposures,effectsandresultingrisksacrossdifferentmembersofanexposedpopulationareusuallynotprovidedwithestimatesgeneratedusingdefaultapproaches.AlthoughDRAmaypresentqualitativeinformationregardingtherobustnessoftheestimates,theimpactofdataandmodellimitationsonthequalityoftheresultscannotbequantified.Relianceondeterministicallyderivedestimationsofriskcanresultindecisionmakingbasedsolelyonpointestimateswithanunknowndegreeofconservatism,whichcancomplicatethecomparisonofrisksormanagementoptions.

Inriskassessmentsofnoncancerendpoints,metricssuchasanoralreferencedose(RfD)andaninhalationreferenceconcentration(RfC)aretypicallyused.Theuseofconservativedefaultslonghasbeenthetargetofcriticism(Finkel1989)andhasledtothepresumptionbycriticsthatEPAassessmentsareoverlyconservativeandunrealistic.TheuseofPRAwouldbeadvantageousineliminatingasinglevalueandmightbelesslikelytoimplyundueprecisionandlessentheneedforconservativeassumptions,therebyreducingbiasintheestimate.Intheprobabilisticframework,aprobabilitydistributionwouldbeusedtoexpressthebeliefthatanyparticularvaluerepresentsthedoseorexposureconcentrationthatwouldposenoappreciableriskofadverseeffects(NRC2009).EPAisinvestigatingtheuseofPRAtoderiveriskvaluesforRfDandRfCinEPA’sIntegratedRiskInformationSystem(IRIS)Database(www.epa.gov/IRIS/).

EPAcommissionedawhitepaper(HattisandLynch2010)presentedattheHazardousAirPollutantWorkshop,2009,illustratingtheimplementationofprobabilisticmethodsindefiningRfDsandassessingthebenefitsforreducingexposuretotoxicantsthatactinpartthroughtraditionalindividualthresholdprocesses.TheuseofPRA,amongotherthings,makesprovisionforinteractionswithbackgroundpathologicalprocesses,asrecommendedbytheNRC(2009),andshowshowthesystemcaninformassessmentsfor“data‐poor”toxicants.

PRAmaybemoresuitablethanDRAforcomplexassessments,includingthoseofaggregateandcumulativeexposuresandtime‐dependentindividualexposure,doseandeffectsanalyses.Identificationandprioritizationofcontributorysourcesofuncertaintycanbedifficultandtimeconsumingwhenusingdeterministicmethods,leadingtodifficultiesinmodelevaluationandthesubsequentappraisalofriskestimates(CullenandFrey1999).Quantitativeanalysesofmodelsensitivitiesareessentialfortheprioritizationofkeyuncertaintiesacriticalprocessinidentifyingstepsfordatacollectionorresearchtoimproveexposureorriskestimates.

1.10. What Is EPA’s Experience with the Use of Probabilistic Risk Analysis?

EPA’sexperiencewithPRAhas,todate,primarilybeenlimitedtotheevaluationofdata,modelandscenariouncertainties.Toassistwiththegrowingnumberofprobabilisticanalysesofexposuredataintheseuncertaintyareas,EPAissuedGuidingPrinciplesforMonteCarloAnalysis(USEPA1997b).Givenadequatesupportingdataandcredibleassumptions,probabilisticanalysistechniques,suchasMonteCarloanalysis,canbeviablestatisticaltoolsforanalyzinguncertaintyandvariabilityinriskassessments.EPA’spolicyfortheuseofprobabilisticanalysisinriskassessment,releasedin1997,isinclusiveofhumanexposureandecologicalriskassessmentsanddoesnotruleoutprobabilistichealtheffectsanalyses(USEPA1997a).Subsequently,EPA’sSABandScientificAdvisoryPanel(SAP)havereviewedPRAapproachestorisksusedbyEPAofficessuchasOAR,OPPandothers.SeveralprogramshavedevelopedspecificguidanceontheuseofPRA,includingOPPandOSWER(USEPA1998a,2001).

ToillustratethepracticalapplicationofPRAtoproblemsrelevanttotheAgency,severalexamplecasestudiesarebrieflydescribedhere.TheAppendixtitledCaseStudyExamplesofApplicationof

13

ProbabilisticRiskAnalysisinU.S.EnvironmentalProtectionAgencyRegulatoryDecisionMaking,discussestheseandothercasestudiesingreaterdetail,includingtheproceduresandoutcomes.TheAppendixincludes16casestudies—11HHRAand5ERAexamples—thatareintendedtoillustratehowsomeofEPA’sprogramsandofficescurrentlyutilizePRA.Toaidindescribinghowprobabilisticanalyseswereused,the16casestudiesaresubdividedamong3categoriesofPRAtools:Group1—pointestimate,includingsensitivityanalysis;Group2—probabilisticriskanalysis,includingone‐dimensionalMonteCarloanalysis(1‐DMCA)andprobabilisticsensitivityanalysis;andGroup3—advancedprobabilisticriskanalysis,includingtwo‐dimensionalMonteCarloanalysis(2‐DMCA)withmicroexposure(microenvironments)modeling,Bayesianstatistics,geostatisticsandexpertelicitation.

ItisusefultonotethattheNRC(2009)recommendedatieredapproachtoriskassessmentusingbothqualitativeandquantitative(deterministicandprobabilistic)tools,withthecomplexityoftheanalysisincreasingasprogressismadethroughthetiers.TheuseofPRAtoolstoaddressissuesofuncertaintyandvariabilityinatieredapproachisdescribedmorecompletelyinSection2.10andwasillustratedinFigure2.ThethreetiersillustratedinthatfigureapproximatelycorrespondtothethreegroupsofEPAcasestudiesdescribedintheAppendixthatprovideexamplesoftheuseofvariousPRAtools.

TableA‐1intheAppendixoffersasummaryofthe16casestudiesbasedonthetypeofriskassessment,thePRAtoolsusedintheassessment,andtheEPAprogramorregionalofficeresponsiblefortheassessment.Someoftheapproachesthatareprofiledinthesecasestudiescanbeusedintheplanningandscopingphasesofriskassessmentsandriskmanagement.Other,morecomplexPRAapproachesareusedtoanswermorespecificquestionsandprovidearicherdescriptionoftherisks.MoststudiesshowthatPRAcanimproveorexpandoninformationgeneratedbydeterministicmethods.Insomeofthecasestudies,theuseofmultiplePRAtoolsisillustrated.Forexample,CaseStudy1describestheuseofapointestimatesensitivityanalysistoidentifyexposurevariablescriticaltotheanalysissummarizedinCaseStudy9.Bothofthesecasestudiesfocusonchildren’sexposuretochromatedcopperarsenate(CCA)‐treatedwood.InCaseStudy9,anMCAwasusedasanexampleofatwo‐dimensional(i.e.,addressingbothvariabilityanduncertainty)probabilisticexposureassessment.

Overall,thecasestudiesillustratethattheAgencyalreadyhasappliedthescienceofPRAtoecologicalriskandhumanexposureestimationandhasbegunusingPRAtodescribehealtheffects.Someoftheapplicationshaveusedexisting“off‐the‐shelf”software,whereasothershaverequiredsignificanteffortandresources.Oncedeveloped,however,someofthemorecomplexmodelshavebeenusedmanytimesfordifferentassessments.Alloftheassessmentshavebeenvalidatedbyinternalandexternalpeerreview.Table1givessomehighlightsthecasestudiesfromdeterministictomorecomplexassessments,whicharedescribedinmoredetailintheAppendix.

14

Table 1. Selected Examples of EPA Applications of Probabilistic Risk Assessment Techniques 

Case Study No. Description Group Type of Risk

Assessment Office/Region

2

Atmospheric Deposition to Watershed Contamination: The Office of Research and Development (ORD) developed an analysis of nitrogen, mercury and polycyclic aromatic hydrocarbons (PAHs) depositions toward watershed contamination in the Casco Bay Estuary in southwestern Maine.

Group 1: Point Estimate

Ecological ORD

5

Hudson River Polychlorinated Biphenyl (PCB)-Contaminated Sediment Site: Region 2 evaluated the variability in risks to anglers who consume recreationally caught fish contaminated with PCBs from sediment contamination in the Hudson River.

Group 2: 1-D Monte Carlo Analysis

Human Health Superfund/ Region 2

(New York)

7

Environmental Monitoring and Assessment Program (EMAP): ORD developed and the Office of Water (OW) applied probabilistic sampling techniques to evaluate the Nation’s aquatic resources under the Clean Water Act (CWA) Section 305(b).

Group 2: Probabilistic Sensitivity Analysis

Ecological ORD/OW

9

Chromated Copper Arsenate (CCA) Risk Assessment: ORD and the Office of Pesticide Programs (OPP) conducted a probabilistic assessment of children’s exposure (addressing both variability and uncertainty) to arsenic and chromium from contact with CCA-treated wood play sets and decks.

Group 3: 2-D Monte Carlo Analysis

Human Health ORD/OPP

13

Evaluating Ecological Effects of Pesticide Uses: OPP developed a probabilistic model, which evaluates acute mortality levels in generic and specific ecological species for user-defined pesticide uses and exposures.

Group 3: Probabilistic Analysis

Ecological OPP

14

Fine Particulate Matter Health Impacts: ORD and the Office of Air and Radiation (OAR) used expert elicitation to more completely characterize, both qualitatively and quantitatively, the uncertainties associated with the relationship between reduction in fine particulate matter (PM2.5) and benefits of reduced PM2.5-related mortality.

Group 3: Expert Elicitation

Human Health ORD/OAR

15

2. PROBABILISTIC RISK ANALYSIS 2.1. What Are Uncertainty and Variability, and How Are They Relevant

to Decision Making? Theconceptsofuncertaintyandvariabilityareintroducedhere,andtherelevanceoftheseconceptstodecisionmakingisdiscussed.

2.1.1. Variability Variabilityreferstorealdifferencesovertime,spaceormembersofapopulationandisapropertyofthesystembeingstudied(e.g.,drinkingwaterconsumptionratesforeachofthemanyindividualadultresidentslivinginaspecificlocationordifferencesinbodylengthsorweightsforhumansorecologicalspecies)(CullenandFrey1999;USEPA2011c).Variabilitycanarisefrominherentlyrandomprocesses,suchasvariationsinwindspeedovertimeatagivenlocationorfromtruevariationacrossmembersofapopulationthat,inprinciple,couldbeexplained,butwhich,inpractice,maynotbeexplainableusingcurrentlyavailablemodelsordata(e.g.,therangeofleadlevelsinthebloodofchildren6yearsoldoryoungerfollowingaspecificdegreeofleadexposure).OfparticularinterestinbothHHRAandERAisinter‐individualvariability,whichtypicallyreferstodifferencesbetweenmembersofthesamepopulationineitherbehaviorrelatedtoexposure(e.g.,dietaryconsumptionratesforspecificfooditems),orbiokineticsrelatedtochemicaluptake(e.g.,gastrointestinaluptakeratesforleadfollowingintake)ortoxicresponse(e.g.,differencesamongindividualsorspeciesintheinternaldoseneededtoproduceaspecificamountofneurologicalimpairment).

Inter‐individualvariabilityisillustratedinCaseStudy5intheAppendix,whichassessesaPCB‐contaminatedsedimentsiteintheHudsonRiver.Inthiscasestudy,thequantificationofvariabilityisillustratedthroughtheuseofaPRAtool—1‐DMCA—todescribethevariabilityofexposureasafunctionofindividualexposurefactors(i.e.,youngchildren’sfishingestion).

2.1.2. Uncertainty Uncertaintyisthelackofknowledgeofthetruevalueofaquantityorrelationshipsamongquantities(USEPA2011c).Forexample,theremaybealackofinformationregardingthetruedistributionofvariabilitybetweenindividualsforconsumptionofcertainfooditems.Thereareanumberoftypesofuncertaintiesforbothriskanalysis.Thefollowingdescriptionsofthetypesofuncertainty(adaptedfromCullenandFrey1999)addressesuncertaintiesthatariseduringriskanalyses.Theseuncertaintiescanbeseparatedbroadlyintothreecategories:(1)scenariouncertainty;(2)modeluncertainty;and(3)inputorparameteruncertainty.Eachoftheseisexplainedintheparagraphsthatfollow.

Scenariouncertaintyreferstoerrors,typicallyofomission,resultingfromincorrectorincompletespecificationoftheriskscenariotobeevaluated.Theriskscenarioreferstoasetofassumptionsregardingthesituationtobeevaluated,suchas:(1)thespecificsourcesofchemicalemissionsorexposuretobeevaluated(e.g.,oneindustrialfacilityoraclusterofvariedfacilitiesimpactingthesamestudyarea);(2)thespecificreceptorpopulationsandassociatedexposurepathwaystobemodeled(e.g.,indoorinhalationexposure,track‐industorconsumptionofhome‐produceddietaryitems);and(3)activitiesbydifferentlifestagestobeconsidered(e.g.,exposureonlyathome,orconsiderationofworkplaceorcommutingexposure).Mis‐specificationoftheriskscenariocanresultinunderestimation,overestimationorothermischaracterizationofrisks.Underestimationmayoccurbecauseoftheexclusionofrelevantsituationsortheinclusionofirrelevantsituationswithrespecttoaparticularanalysis.Overestimationmayoccurbecauseoftheinclusionof

16

unrealisticorirrelevantsituations(e.g.,assumingcontinuousexposuretoanintermittentairbornecontaminantsourceratherthanaccountingformobilitythroughouttheday).

Modeluncertaintyreferstolimitationsinthemathematicalmodelsortechniquesthataredevelopedtorepresentthesystemofinterestandoftenstemsfrom:(1)simplifyingassumptions;(2)exclusionofrelevantprocesses;(3)mis‐specificationofmodelboundaryconditions(e.g.,therangeofinputparameters);or(4)misapplicationofamodeldevelopedforotherpurposes.Modeluncertaintytypicallyariseswhentheriskmodelreliesonmissingorimproperlyformulatedprocesses,structuresorequations.SourcesofmodeluncertaintyaredefinedintheGlossary.

Inputorparameteruncertaintytypicallyreferstoerrorsincharacterizingtheempiricalvaluesusedasinputstothemodel(e.g.,engineering,physical,chemical,biologicalorbehavioralvariables).Inputuncertaintycanoriginatefromrandomorsystematicerrorsinvolvedinmeasuringaspecificphenomenon(e.g.,biomarkermeasurements,suchastheconcentrationofmercuryinhumanhair);statisticalsamplingerrorsassociatedwithsmallsamplesizes(e.g.,ifthedataarebasedonsamplesselectedwitharandom,representativesamplingdesign);theuseofsurrogatedatainsteadofdirectlymeasureddata;theabsenceofanempiricalbasisforcharacterizinganinput(e.g.,theabsenceofmeasurementsforfugitiveemissionsfromanindustrialfacility);ortheuseofsummarymeasuresofcentraltendencyratherthanindividualobservations.Nonlinearrandomprocessescanexhibitabehaviorthat,forsmallchangesininputvalues,producesalargevariationinresults.

InputorparameteruncertaintyisillustratedinCaseStudy3intheAppendixtitled“ProbabilisticAssessmentofAnglingDurationUsedintheAssessmentofExposuretoHudsonRiverSedimentsviaConsumptionofContaminatedFish.”Inthiscasestudy,aprobabilisticanalysisofoneparameterinanexposureassessment—thetimeanindividualspendsfishinginalargeriversystem—wasassessedusingsensitivityanalysis.Thisanalysiswasconductedbecausetherewasuncertaintythattheindividualexposuredurationbasedonresidencedurationmayunderestimatethetimespentfishing(i.e.,anglingduration).Thefulldistributionofthecalculatedvalueswasusedinconductingthe1‐DMCAforthefishconsumptionpathway,whichispresentedinCaseStudy5.

Decisionuncertaintyreferstoadecisionanalysisthatwouldincludenotonlytheimpactofscenario,modelandinputuncertaintiesontherelativeattractivenessofpotentialdecisionalternatives,butalsowouldincludethedegreetowhichspecificchoices(suchasselectinginputdata,models,andscenarios,andevenhowtheproblemordecisionanalysisisframed)impacttherelativeattractivenessofpotentialdecisionalternatives.Indecisionmaking,analystsusedatatorepresentdecisioncriteriathatdecisionmakersandotherstakeholdersbelievewillhelpthemtoanswertheirdecisionquestion(s).ThesequestionsmightincludewhichpolicyalternativebestmeetsAgencygoals(thatmustbearticulated)orwhichriskassessmentscenariobestdescribestheobservedeffects.Data,modelandscenariouncertaintieswillinfluencetheriskassessmentresultsandthose,inturn,willinfluencetheriskmanagementoptions.Decisionmakerswhounderstandtheuncertaintyassociatedwiththeirspecificchoicescanbemoreconfidentthatthedecisionwillproducetheresultsthattheyseek.Inaddition,thesedecisionmakerswillbeabletodefendtheirdecisionsbetterandexplainhowthedecisionmeetsAgencyandstakeholdergoals.

Whilethisisbeyondthescopeofthisdocument,StahlandCimorelli(2005and2012)illustratehowuncertaintythroughoutthedecisionmakingprocesscanbeassessed.Thesecasestudiesexploredtheassessmentofozonemonitoringnetworksandairqualitymanagementpoliciesthatseektominimizetheadverseimpactsfromozone,fineparticulatematterandairtoxicssimultaneously.Thesecasestudiesdemonstratetheimportanceandfeasibilityofbetterunderstandingtheuncertaintyintroducedbyspecificchoices(e.g.,selectinginputdata,models,andscenarios)whenmakingpublicpolicydecisions.

17

2.2. When Is Probabilistic Risk Analysis Applicable or Useful? PRAmaybeparticularlyuseful,forexample,inthefollowing(Cooke1991;CullenandFrey1999;NRC2009;USEPA2001):

Whenascreening‐levelDRAindicatesthatrisksarepossiblyhigherthanalevelofconcernandamorerefinedassessmentisneeded.

Whentheconsequencesofusingpointestimatesofriskareunacceptablyhigh.

Whensignificantequityorenvironmentaljusticeissuesareraisedbyinter‐individualvariability.

Toestimatethevalueofcollectingadditionalinformationtoreduceuncertainty.

Toidentifypromisingcriticalcontrolpointsandlevelswhenevaluatingmanagementoptions.

Torankexposurepathways,sites,contaminantsandsoonforthepurposesofprioritizingmodeldevelopmentorfurtherresearch.

Whencombiningexpertjudgmentsonthesignificanceofthedata.

Whenexploringtheimpactoftheprobabilitydistributionsofstakeholderanddecision‐makervaluesontheattractivenessofpotentialdecisionalternatives(Fischhoff1995;Illing1999;KunreutherandSlovic1996;USEPA2000b).

Whenexploringtheimpactoftheprobabilitydistributionsofthedata,modelandscenariouncertainties,andvariabilitytogethertocomparepotentialdecisionalternatives.

PRAmayaddminimalvaluetotheassessmentinthefollowingtypesofsituations(CullenandFrey1999;USEPA1997a):

Whenascreening‐leveldeterministicriskassessmentindicatesthatrisksarenegligible,presumingthattheassessmentisknowntobeconservativeenoughtoproduceoverestimatesofrisk.

Whenthecostofavertingtheexposureandriskissmallerthanthecostofaprobabilisticanalysis.

Whenthereislittleuncertaintyorvariabilityintheanalysis(thisisararesituation).

2.3. How Can Probabilistic Risk Analysis Be Incorporated Into Assessments?

AsillustratedintheaccompanyingcasestudiesintheAppendix,probabilisticapproachescanbeincorporatedintoanystageofariskassessment,fromproblemformulationorplanningandscopingtotheanalysisofalternativedecisions.Insomesituations,PRAcanbeusedselectivelyforcertaincomponentsofanassessment.Itiscommoninassessmentsthatsomemodelinputsareknownwithhighconfidence(i.e.,basedonsite‐specificmeasurements),whereasvaluesforotherinputsarelesscertain(i.e.,basedonsurrogatedatacollectedforadifferentpurpose).Forexample,anexposuremodelermaydeterminethatrelevantairqualitymonitoringdataexists,butthereisalackofdetailedinformationonhumanactivitypatternsindifferentmicroenvironments.Thus,anassessmentofthevariabilityinexposuretoairbornepollutantsmightbebasedondirectuseofthemonitoringdata,whereasassessmentofuncertaintyandvariabilityintheinhalationexposurecomponentmightbebasedonstatisticalanalysisofsurrogatedataoruseofexpertjudgment.Theuncertaintiesarelikelytobelargerforthelatterthantheformercomponentoftheassessment;

18

effortstocharacterizeuncertaintiesassociatedwithpollutantexposureswouldfocusonthelatter.PRAalsodealswithdependencyissues;adescriptionoftheseissuesisavailableinSection3.3.2.

2.4. What Are the Scientific Community’s Views on Probabilistic Risk Analysis, and What Is the Institutional Support for Its Use in Performing Assessments?

TheNRCandIOMrecentlyemphasizedtheirlong‐standingadvocacyforPRA(NRC2007aandb;IOM2013).Datingfromits1983RiskAssessmentintheFederalGovernment:ManagingtheProcess(NRC1983)—whichfirstformalizedtheriskassessmentparadigm—throughreportsreleasedfromthelate1980sthroughtheearly2000s,variousNRCpanelshavemaintainedconsistentlythatbecauseriskanalysisinvolvessubstantialuncertainties,theseuncertaintiesshouldbeevaluatedwithinariskassessment.Thesepanelsnotedthat:

1. Whenevaluatingthetotalpopulationrisk,EPAshouldconsiderthedistributionofexposureandsensitivityofresponseinthepopulation(NRC1989).

2. Whenassessinghumanexposuretoairpollutants,EPAshouldpresentmodelresultsalongwithestimateduncertainties(NRC1991).

3. WhenconductingERA,EPAshoulddiscussthoroughlyuncertaintyandvariabilitywithintheassessment(NRC1993).

4. “Uncertaintyanalysisistheonlywaytocombatthe‘falsesenseofcertainty,’whichiscausedbyarefusaltoacknowledgeand[attemptto]quantifytheuncertaintyinriskpredictions,”asstatedintheNRCreport,ScienceandJudgmentinRiskAssessment(NRC1994).

5. EPA’sestimationofhealthbenefitswasnotwhollycrediblebecauseEPAfailedtodealformallywithuncertaintiesinitsanalyses(NRC2002).

6. EPAshouldadopta“tiered”approachforselectingthelevelofdetailusedinuncertaintyandvariabilityassessment.Furthermore,theNRCrecommendedthatadiscussionofthelevelofdetailusedforuncertaintyanalysisandvariabilityassessmentshouldbeanexplicitpartoftheplanning,scopingandproblemformulationphaseoftheriskassessmentprocess(NRC2009).

7. EPAshoulddevelopmethodstosystematicallydescribeandaccountforuncertaintiesindecision‐relevantfactorsinadditiontoestimatesofhealthriskinitsdecision‐makingprocess(IOM2013).

AskedtorecommendimprovementstotheAgency’sHHRApractices,EPA’sSABechoedtheNRC’ssentimentsandurgedtheAgencytocharacterizeuncertaintyandvariabilitymorefullyandsystematicallyandtoreplacesingle‐pointuncertaintyfactorswithasetofdistributionsusingprobabilisticmethods(ParkinandMorgan2007).ThekeyprinciplesofriskassessmentcitedbytheOfficeofScienceandTechnologyPolicy(OSTP)andtheOfficeofManagementandBudget(OMB)include“explicit”characterizationoftheuncertaintiesinriskjudgments;theyproceedtocitetheNationalAcademyofScience’s(NAS)2007recommendationtoaddressthe“variabilityofeffectsacrosspotentiallyaffectedpopulations”(OSTP/OMB2007).

19

2.5. Additional Advantages of Using Probabilistic Risk Analysis and How It Can Provide More Comprehensive, Rigorous Scientific Information in Support of Regulatory Decisions.

ExternalstakeholderspreviouslyhaveusedtheAdministrativeProcedureActandtheDataQualityActtochallengetheAgencyforalackoftransparencyandconsistencyorfornotfullyanalyzingandcharacterizingtheuncertaintiesinriskassessmentsordecisions(Fisheretal.2006).ThemorecompleteimplementationofPRAandrelatedapproachestodealwithuncertaintiesindecisionmakingwouldaddressstakeholderconcernsinregardtocharacterizinguncertainties.

Theresultsofanyassessment,includingPRA,aredependentontheunderlyingmethodsandassumptions.Accompaniedbytheappropriatedocumentation,PRAmaycommunicateamorerobustrepresentationofrisksandcorrespondinguncertainties.Thischaracterizationmaybeintheformofarangeofpossibleestimatesasopposedtothemoretraditionallypresentedsingle‐pointvalues.Dependingontheuseoftheassessment,rangescanbederivedforvariabilityanduncertainty(oracombinationofthetwo)inbothmodelinputsandresultingestimationsofrisk.

PRAquantifieshowexposures,effectsandrisksdifferamonghumanpopulationsorlifestagesortargetecologicalorganisms.PRAalsoprovidesanestimationofthedegreeofconfidencewithwhichtheseestimatesmaybemade,giventhecurrentuncertaintyinscientificknowledgeandavailabledata.A2007NRCpanelstatedthattheobjectiveofPRAsisnottodecide“howmuchevidenceissufficient”toadoptanalternativebut,rather,todescribethescientificbasesofproposedalternativessothatscientificandpolicyconsiderationsmaybemorefullyevaluated(NRC2007a).EPA’sSABsimilarlynotedthatPRAsprovidemore“valueofinformation”throughaquantitativeassessmentofuncertaintyandclarifythescienceunderlyingAgencydecisions(USEPA2007b).

TheSABarticulatedanumberofadvantagesforEPAdecisionmakersfromtheutilizationofprobabilisticmethods(ParkinandMorgan2007):

AprobabilisticreferencedosecouldhelpreducethepotentiallyinaccurateimplicationofzeroriskbelowtheRfD.

Byunderstandingandexplicitlyaccountingforuncertaintiesunderlyingadecision,EPAcanestimateformallythevalueofgatheringmoreinformation.Bydoingso,theAgencycanbetterprioritizeitsinformationneedsbyinvestinginareasthatyieldthegreatestinformationvalue.

StrategicuseofPRAwouldallowEPAtosendtheappropriatesignaltotheintellectualmarketplace,therebyencouraginganalyststogatherdataanddevelopmethodologiesnecessaryforassessinguncertainties.

2.6. What Are the Challenges to Implementation of Probabilistic Analyses?

Currently,EPAisusingPRAinavarietyofprogramstosupportdecisions,butchallengesremainregardingtheexpandeduseofthesetoolswithintheAgency.Thechallengesinclude:

AlackofunderstandingofthevalueofPRAfordecisionmaking.PRAhelpstoimprovetherigorofthedecision‐makingprocessbyallowingdecisionmakerstoexploretheimpactsofuncertaintyandvariabilityonthedecisionchoices.

Aclearinstitutionalunderstandingofhowtoincorporatetheresultsofprobabilisticanalysesintodecisionmakingislacking.

20

PRAtypicallyrequiresadifferentskillsetthanusedincurrentevaluations,andlimitedresources(staff,time,trainingormethods)toconductPRAareavailable.

Communicatingprobabilisticanalysisresultsandtheimpactofthoseresultsonthedecision/policyoptionscanbecomplex.

Communicationwithstakeholdersisoftendifficultandresultsintheappearanceofregulatorydelaysduethenecessityofanalyzingnumerousscenariosusingvariousmodels.

PRAcomplicatesdecisionmakingandriskcommunicationininstanceswhereamorecomprehensivecharacterizationoftheuncertaintiesleadstoadecreaseinclarityregardinghowtoestimateriskforthescenariounderconsideration.ThesechallengesarediscussedinmoredetailinSections2.7through2.13.

2.7. How Can Probabilistic Risk Analysis Support Specific Regulatory Decision Making?

Decisionmakerssometimesperceivethatthebinarynatureofregulatorydecisions(e.g.,Doesanexposureexceedareferencedoseornot?DoemissionscomplywithAgencystandardsornot?)precludestheuseofariskrangedevelopedthroughPRA.Generally,itisnecessarytoexplaintherationaleunderlyingaparticulardecision.PRA’sprimarypurposeistoprovideinformationtoenhancetheabilitytomaketransparentdecisionsbasedonthebestavailablescience.Byconductingasensitivityanalysisoftheinfluenceoftheuncertaintyonthedecision‐makingprocess,itcanbedeterminedhoworifPRAcanhelptoimprovetheprocess.

PRAcanprovideinformationtodecisionmakersonspecificquestionsrelatedtouncertaintyandvariability.Forquestionsofuncertaintyandtominimizethelikelihoodofunintendedconsequences,PRAcanhelptoprovidethefollowingtypesofinformation:

Characterizationoftheuncertaintyinestimates(i.e.,Whatisthedegreeofconfidenceintheestimate?).Couldthepredictionbeoffbyafactorof2,afactorof10orafactorof1,000?

Criticalparametersandassumptionsthatmostaffectorinfluenceadecisionandtheriskassessment.

“Tippingpoints”wherethedecisionwouldbealterediftheriskestimatesweredifferent,orifadifferentassumptionwasvalid.

Estimatethelikelihoodthatvaluesforcriticalparameterswilloccurortestthevalidityofassumptions.

Estimatethedegreeofconfidenceinaparticulardecisionand/orthelikelihoodofspecificdecisionerrors.

Thepossibilityofalternativeoutcomeswithadditionalinformation,orestimatetradeoffsrelatedtodifferentrisksorrisk‐managementdecisions.

Theimpactofadditionalinformationondecisionmaking,consideringthecostandtimetoobtaintheinformationandtheresultingchangeindecision(i.e.,thevalueoftheinformation).

Fortheconsiderationofvariability,PRAcanhelptoprovidethefollowingtypesofinformationforexposures:

Explicitlydefinedexposuresforvariouspopulationsorlifestages(i.e.,Whoarewetryingtoprotect?).Thatis,willtheregulatoryactionkeep50percent,90percent,99.9percentorsomeotherfractionofthepopulationbelowaspecifiedexposure,doseorrisktarget?

21

Variabilityintheexposures,amongvariouspopulationsorlifestages,andinformationonthepercentileofthepopulationthatisbeingevaluatedintheriskassessment(e.g.,variationsinthenumberoflitersofwaterperkilogram[kg]bodyweightperdayconsumedbythepopulation).Thisinformationishelpfulinaddressingcomments:

OntheconservatismofEPA’sriskassessments;

Concernsaboutwhethertheirparticularexposureswereevaluatedintheriskassessment;

Whomorwhatisbeingprotectedbyimplementingadecision;and

Whetherandwhatadditionalresearchmaybeneededtoreduceuncertainty.

PRAhelpstoinformdecisionsbycharacterizingthealternativesavailabletothedecisionmakerandtheuncertaintyheorshefaces,andbyprovidingevaluationmeasuresofoutcomes.Uncertaintiesoftenarerepresentedasprobabilitiesorprobabilitydistributionsnumericallyoringraphs.Aspartofadecisionanalysis,stakeholderscanmorefullyexaminehowuncertaintiesinfluencethepreferenceamongalternatives.

2.8. Does Probabilistic Risk Analysis Require More Resources Than Default-Based Deterministic Approaches?

PRAgenerallycanbeexpectedtorequiremoreresourcesthanstandardAgencydefault‐baseddeterministicapproaches.ThereisextensiveexperiencewithinEPAinconductingandreviewingDRA.Theseassessmentstendtofollowstandardizedmethodsthatminimizetheeffortrequiredtoconductthemandtocommunicatetheresults.Probabilisticassessmentsoftenentailamoredetailedanalysis,andasaresult,theseassessmentsrequiresubstantiallymoreresources,includingtimeandeffort,thandodeterministicapproaches.

Appropriatelytrainedstaffandtheavailabilityofadequatetools,methodsandguidanceareessentialfortheapplicationofPRA.Properapplicationofprobabilisticmethodsrequiresnotonlysoftwareanddata,butalsoguidanceandtrainingforanalystsusingthetoolsandformanagersanddecisionmakerstaskedwithinterpretingandcommunicatingtheresults.

Anupfrontincreaseinresourcesneededtoconductaprobabilisticassessmentcanbeexpected,butdevelopmentofstandardizedapproachesand/ormethodscanleadtotheroutineincorporationofPRAinAgencyapproaches(e.g.,OPP’suseoftheDietaryExposureEvaluationModel[DEEM;http://www.epa.gov/pesticides/science/deem/],aprobabilisticdietaryexposuremodel).Theinitialand,insomecases,ongoingresourcecost(e.g.,fordevelopmentofsite‐specificmodelsforsiteassessments)maybeoffsetbyamoreinformeddecisionthanacomparabledeterministicanalysis.Probabilisticmethodsareusefulforidentifyingeffectivemanagementoptionsandprioritizingadditionaldatacollectionorresearchaimedatimprovingriskestimation,ultimatelyresultingindecisionsthatenableimprovedenvironmentalprotectionwhilesimultaneouslyconservingmoreresources.

2.9. Does Probabilistic Risk Analysis Require More Data Than Conventional Approaches?

TherearedifferencesofopinionwithinthetechnicalcommunityastowhetherPRArequiresmoredatathanothertypesofanalyses.AlthoughsomeemphaticallybelievethatPRArequiresmoredata,othersarguethatprobabilisticassessmentsmakebetteruseofalloftheavailabledataandinformation.StahlandCimorelli(2005)discusswhenandhowmuchdataarenecessaryforadecision.PRAcanbenefitfrommoredatathanmightbeusedinaDRA.Forexample,whereDRA

22

mightemployselectedpointestimates(e.g.,themeanor95thpercentilevalues)fromavailabledatasetsforuseinmodelinputs,PRAfacilitatestheuseoffrequency‐weighteddatadistributions,allowingforamorecomprehensiveconsiderationoftheavailabledata.Inmanycases,thedatathatwereusedtodevelopthepresumptive95thpercentilecanbeemployedinthedevelopmentofprobabilisticdistributions.

RestrictionofPRAtoprincipallydata‐richsituationsmaypreventitsbroaderapplicationwhereitismostuseful.BecausePRAincorporatesinformationondataquality,variabilityanduncertaintyintoriskmodels,theinfluenceofthesefactorsonthecharacterizationofriskcanbecomeagreaterfocusofdiscussionanddebate.

AkeybenefitofusingPRAisitsabilitytorevealthelimitationsaswellasthestrengthsofdatathatoftenaremaskedbyadeterministicapproach.Indoingso,PRAcanhelptoinformresearchagendas,aswellassupportregulatorydecisionmaking,basedonthestateofthebestavailablescience.Insummary,PRAtypicallyrequiresmoretimefordevelopinginputassumptionsthanaDRA,butwhenincorporatedintotherelevantstepsoftheriskassessmentprocess,PRAcandemonstrateaddedbenefits.Insomecases,PRAcanprovideadditionalinterpretationsthatcompensatefortheextraeffortrequiredtoconductaPRA.

2.10. Can Probabilistic Risk Analysis Be Used to Screen Risks or Only in Complex or Refined Assessments?

Probabilisticmethodstypicallyarenotnecessarywheretraditionaldefault‐baseddeterministicmethodsareadequateforscreeningrisks.Suchmethodsarerelativelylowcost,intendedtoproduceconservativelybiasedestimates,andusefulforidentifyingsituationsinwhichrisksaresolowthatnofurtheractionisneeded.Theapplicationofprobabilisticmethodscanbetargetedtosituationsinwhichascreeningapproachindicatesthatariskmaybeofconcernorwhenthecostofmanagingtheriskishigh,creatinganeedforinformationtohelpinformdecisionmaking.PRAfitsdirectlyintoagraduatedhierarchicalapproachtoriskanalysis.Thistieredapproach,depictedinFigure2,isaprocessforasystematicinformedprogressiontoincreasinglymorecomplexriskassessmentmethods,dependingonthedecision‐makingcontextandneed.Highertiersreflectincreasingcomplexityandoftenwillrequiremoretimeandresources.Ananalysismighttypicallystartatalowertierandonlyprogresstoahighertierifthereisaneedforamoresophisticatedassessmentcommensuratewiththeimportanceoftheproblem.Highertiersalsoreflectincreasingcharacterizationofvariabilityand/oruncertaintyintheriskestimate,whichmaybeimportantforrisk‐managementdecisions.ThecasestudiesdescribedintheAppendixarepresentedinthreegroupsthatgenerallycorrespondtothetiersidentifiedinFigure2.Group1casestudiesarepointestimate(sensitivityanalysis)examples(Tier1);Group2casestudiesincludemostmoderate‐complexityPRAexamples(Tier2);andGroup3casestudiesareadvanced(highcomplexity)PRAexamples(Tier3).

ThetieredapproachinFigure2depictsacontinuumfromscreeninglevelpointestimatethatisdonewithlittledataandconservativeassumptionstoPRAthatrequiresanextensivedatasetandmorerealistic(lessconservative)assumptions.Inbetween,therecanbeawidevarietyoftiersofincreasingcomplexity,ortheremaybeonlyafewreasonablechoicesbetweenscreeningmethodsandhighlyrefinedanalyses(USEPA2004a).Asimilarfour‐tieredapproachforcharacterizingthevariabilityand/oruncertaintyintheestimatedexposureorriskanalysis(WHO2008)hasbeenadaptedbyEPAintheriskandexposureassessmentsconductedfortheNationalAmbientAirQualityStandards(NAAQS).

PRAalsocouldbeusedtoexaminemorefullytheexistingdefault‐basedmethodsbasedonthecurrentstateofinformationandknowledgetodetermineifsuchmethodsaretrulyconservative

23

andadequateforscreening(e.g.,indose‐responseanalysesdealingwithhazardcharacterization)(Swartoutetal.1998;Hattisetal.2002).

Theuseofaspectrumofdatashouldbeemployedbothindeterminingscreeningrisksandinmorecomplexassessments.ForHHRA,datafromhuman,animal,mechanisticandotherstudiesshouldbeusedtodevelopaprobabilisticcharacterizationofcancerandnoncancerrisksandtoidentifyuncertainties.TheNRCrecommendedthatEPAfacilitatethisapproachbyredefiningRfDandRfCwithintheprobabilisticframeworktotakeintoaccounttheprobabilityofharm(NRC2009).ItislikelythatbothDRAandPRAwillbepartofthisframework.

2.11. Does Probabilistic Risk Analysis Present Unique Challenges to Model Evaluation?

Theconceptof“validation”ofmodelsusedforregulatorydecisionmakinghasbeenatopicofintensediscussion.Inarecentreportontheuseofmodelsinenvironmentalregulatorydecisionmaking,theNRCrecommendedusingthenotionofmodel“evaluation”ratherthan“validation,”suggestingthatuseofaprocessthatencompassestheentirelifecycleofthemodelandincorporatesthespectrumofinterestedpartiesintheapplicationofthemodeloftenextendsbeyondthemodelbuilderanddecisionmaker.Suchaprocesscanbedesignedtoensurethatjudgmentofthemodelapplicationisbasednotonlyonitspredictivevaluedeterminedfromcomparisonwithhistoricaldata,butalsoonitscomprehensiveness,rigorindevelopment,transparencyandinterpretability(NRC2007b).

Modelevaluationisimportantinallriskassessments.InthecaseofPRA,thereisanadditionalquestionastothevalidityoftheassumptionsregardingprobabilityandfrequencydistributionsformodelinputsandtheirdependencies.Probabilisticinformationcanbeaccountedforduringevaluationanalysesbyconsideringtherangeofuncertaintyinthemodelpredictionandwhethersucharangeoverlapswiththe“true”valuebasedonindependentdata.Thus,probabilisticinformationcanaidincharacterizingtheprecisionofthemodelpredictionsandwhetherapredictionissignificantlydifferentfromabenchmarkofinterest.Forexample,comparisonsofprobabilisticmodelresultsandmonitoringdatawereperformedformultiplemodelsindevelopingthecumulativepesticideexposuremodel.ConcurrentPRAmodelevaluationsusingaBayesiananalysisalsohavebeenpublished(Clyde2000).

Whenriskassessorsdevelopmodelsofrisk,theyrelyontwopredominantstatisticalmethods.Bothmethodsarisefromaxiomsofprobability,buteachappliestheseaxiomsdifferently.Underthefrequentistapproach,onedevelopsandevaluatesamodelbytestingwhetherthemodel—asappliedtotheobservations—conformstoidealizeddistributions.UndertheBayesianapproach,onedevelopsandevaluatesamodelbytestingwhich—amongalternativemodels—bestyieldstheunderlyingdistributiondescribingthedata.Thepracticaldifferencesbetweenthesetwoapproachescanperhapsbestbeappreciatedwhenconsideringthestructuraluncertaintyinmodels(Section3.3.3).BecauseBayesiansestimatemodelparameterswiththeexpectationthattheseparameters—orevenmodelstructures—willbeupdatedasnewdatabecomeavailable,theyhavedevelopedformaltechniquestoprovideuncertaintyboundsaroundtheseparameterestimates,selectmodelsthatbestexplainthegivendata,orcombinetheresultsofalternativemodels.

2.12. How Do You Communicate the Results of Probabilistic Risk Analysis?

Effectivecommunicationmakesiteasierforregulatorsandstakeholderstounderstandthedecisioncriteriadrivingthedecision‐makingprocess.Inotherwords,communicationofPRAresultswithinthedecision‐makingcontextfacilitatesunderstanding.Thespecificapproachesforreportingresults

24

fromPRAvarydependingontheassessmentobjectiveandtheintendedaudience.Beyondthebasic1997principlesandthepolicyfromthesameyear(USEPA1997aandb),theRiskAssessmentGuidanceforSuperfund:VolumeIII—PartA,ProcessforConductingProbabilisticRiskAssessmentalsoprovidessomeguidanceonthequalityandcriteriaforacceptanceaswellascommunicationbasics(USEPA2001).TherehavebeenlimitedstudiesofhowinformationfromPRAregardinguncertaintyandvariabilitycanorshouldbecommunicatedtokeyaudiences,suchasdecisionmakersandstakeholders(e.g.,MorganandHenrion1990;Bloometal.1993;Krupnicketal.2006).Amongtheanalystcommunity,thereoftenisaninterestinvisualizationofthestructureofascenarioandmodelusinginfluencediagramsanddepictionoftheuncertaintyandvariabilityinmodelinputsandoutputsusingprobabilitydistributionsintheformofcumulativedensityfunctionsorprobabilitydistributionfunctions(Figure3).Sensitivityofthemodeloutputtouncertaintyandvariabilityinmodelinputscanbedepictedusinggraphicaltools.

Insomecases,thesegraphicalmethodscanbeusefulforthoselessfamiliarwithPRA,butinmanycasesthereisaneedtotranslatethequantitativeresultsintoamessagethatextractsthekeyinsightswithoutburdeningthedecisionmakerwithobscuretechnicaldetails.Inthisregard,theuseofrangesofvaluesforaparticularmetricofdecision‐makingrelevance(e.g.,therangeofuncertaintyassociatedwithaparticularestimateofrisk)maybeadequate.ThepresentationofPRAresultstoadecisionmakermaybeconductedbestasaninteractivediscussion,inwhichaprincipalmessageisconveyed,followedbyexplorationofissuessuchasthesource,qualityanddegreeofconfidenceassociatedwiththeinformation.ThereisaneedforthedevelopmentofrecommendationsandacommunicationplanregardinghowtocommunicatetheresultsofPRAtodecisionmakersandstakeholders,buildingontheexperienceofvariousprogramsandregions.

2.13. Are the Results of Probabilistic Risk Analysis Difficult to Communicate to Decision Makers and Stakeholders?

Researchhasshownthattheabilityofdecisionmakerstodealwithconceptsofprobabilityanduncertaintyvaries.Bloometal.(1993)surveyedagroupofseniormanagersatEPAandfoundthatmanycouldinterpretinformationaboutuncertaintyifitwascommunicatedinamannerresponsivetodecision‐makerinterests,capabilitiesandneeds.Inamorerecentsurveyofex‐EPAofficials,Krupnicketal.(2006)concludedthatmosthaddifficultyunderstandinginformationonuncertaintywithconventionalscientificpresentationapproaches.ThefindingsofthesestudieshighlighttheneedforpracticalstrategiesforthecommunicationofresultsofPRAanduncertaintyinformationbetweenriskanalystsanddecisionmakers,aswellasbetweendecisionmakersandotherstakeholders.TheOfficeofEmergencyandRemedialResponse(OERR)hascompiledguidancetoassistanalystsandmanagersinunderstandingandcommunicatingtheresultsofPRA(USEPA2001).

Riskanalystsneedtofocusonhowtouseuncertaintyanalysistocharacterizehowconfidentdecisionmakersshouldbeintheirchoices.AsWilson(2000)explained,“…uncertaintyisthebaneofanydecisionmaker’sexistence.Thus,anyonewhowantstoinformdecisionsusingscientificinformationneedstoassurethattheiranalysestransformuncertaintyintoconfidenceinconclusions.”Hence,althoughenvironmentalriskassessmentsarecomplicatedanditiseasytogetlostinthedetails,presentinganddiscussingtheseresultswithinthecontextofthedecisionfacilitatesunderstanding.Thetranslationofuncertaintyintoconfidencestatementsforcesa“top‐down”perspectivethatpromotesaccountingforwhetherandhowuncertaintiesaffectchoices(Tolletal.1997).

25

 Figure 3. Graphical Description of the Likelihood (Probability) of Risk. Hypothetical fitted data distribution with upper and lower confidence intervals are depicted for the output of a 2‐D MCA model. 

Example of 2-D MCA Output Graphical

Example 2-D MCA Output

0.0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1.0

1.0E-06 1.0E-05 1.0E-04 1.0E-03

Risk

Cum

ulat

ive P

roba

bility

Lower confidence bound

Best Estimate

Upper confidence bound

26

3. AN OVERVIEW OF SOME OF THE TECHNIQUES USED IN PROBABILISTIC RISK ANALYSIS

3.1. What Is the General Conceptual Approach in Probabilistic Risk Analysis?

PRAincludesseveralmajorsteps,whichparalleltheacceptedenvironmentalhealthriskassessmentprocess.Theseinclude:(1)problemand/ordecisioncriteriaidentification;(2)gatheringinformation;(3)interpretingtheinformation;(4)selectingandapplyingmodelsandmethodsforquantifyingvariabilityand/oruncertainty;(5)quantifyinginter‐individualorpopulationuncertaintyandvariabilityinmetricsrelevanttodecisionmaking;(6)sensitivityanalysistoidentifykeysourcesofvariabilityanduncertainty;and(7)interpretingandreportingresults.

Problemformulationentailsidentifyingtheassessmentendpointsorissuesthatarerelevanttothedecision‐makingprocessandstakeholders,andthatcanbeaddressedinascientificassessmentprocess.Followingproblemformulation,informationisneededfromstakeholdersandexpertsregardingthescenariostoevaluate.Basedonthescenariosandassessmentendpoints,theanalystsselectordevelopmodels,whichinturnleadstoidentificationofmodelinputdatarequirementsandacquisitionofdataorotherinformation(e.g.,expertjudgmentencodedastheresultofaformalelicitationprocess)thatcanbeusedtoquantifyinputstothemodels.Thedataorotherinformationformodelinputsisinterpretedintheprocessofdevelopingprobabilitydistributionstorepresentvariability,uncertaintyorbothforaparticularinput.Thus,steps(1)through(4)listedabovearehighlyinteractiveanditerativeinthatthedatainputrequirementsandhowinformationistobeinterpreteddependonthemodelformulation,whichdependsonthescenarioandthatinturndependsontheassessmentobjective.Theassessmentobjectivemayhavetoberefineddependingontheavailabilityofinformation.

Onceascenario,modelandinputsarespecified,themodeloutputisestimated.AcommonapproachistouseMonteCarloAnalysis(MCA)orotherprobabilisticmethodstogeneratesamplesfromtheprobabilitydistributionsofeachmodelinput,runthemodelbasedononerandomvaluefromeachprobabilisticinput,andproduceonecorrespondingestimateofthemodeloutputs.Thisprocessisrepeatedtypicallyhundredsorthousandsoftimestocreateasyntheticstatisticalsampleofmodeloutputs.Theseoutputdataareinterpretedasaprobabilitydistributionoftheoutputofinterest.Sensitivityanalysiscanbeperformedtodeterminewhichmodelinputdistributionsaremosthighlyassociatedwiththerangeofvariationinthemodeloutputs.Theresultsmaybereportedinawidevarietyofformsdependingontheintendedaudience,rangingfromqualitativesummariestotables,graphsanddiagrams.

DetailedintroductionstoPRAmethodologyareavailableelsewhere,suchasAngandTang(1984),CullenandFrey(1999),EPA(2001),andMorganandHenrion(1990).AfewkeyaspectsofPRAmethodologyarebrieflymentionedhere.ReaderswhoseekmoredetailshouldconsultthesereferencesandseetheBibliographyforadditionalreferences.

3.2. What Levels and Types of Probabilistic Risk Analyses Are There and How Are They Used?

Therearemultiplelevelsandtypesofanalysisusedtoconductriskassessments(illustratedinFigure2andTable1,respectively).Graduatedapproachestoanalysisarewidelyrecognized(e.g.,USEPA1997a,2001;WHO2008).Theideaofagraduatedapproachistochoosealevelofdetailandrefinementforananalysisthatisappropriatetotheassessmentobjective,dataquality,informationavailableandimportanceofthedecision(e.g.,resourceimplications).

27

Asdiscussedinsection1.8,thereisavarietyofapproachestoriskassessmentthatdifferintheircomplexityandthemannerinwhichtheyaddressuncertaintyandvariability.InDRAonedoesnotformallycharacterizeuncertaintyorvariabilitybutrathertypicallyreliesonusingdefault‐basedassumptionsandfactorstogenerateasingleestimateofrisk.InPRAthereisavarietyofapproachestoexplicitlyaddressorcharacterizeuncertaintyorvariabilityinriskestimatesandthesedifferintermsofhowtheyaccomplishthis,thedataused,andtheoverallcomplexity.Someexamplesare:

Sensitivityanalysis

MonteCarloanalysisofvariabilityinexposuredata

Humanhealthorecologicaleffectsdata

MonteCarloanalysisofuncertainty

“Cumulative”PRAmulti‐pathwayormulti‐chemical

Two‐dimensionalPRAofuncertaintyandvariability

Decisionuncertaintyanalysis

Geospatialanalysis

Expertelicitation

TheDRAapproachesdescribedinSection1.8areexamplesoflowerlevelsinagraduatedapproachtoanalysis.Riskatthelowerlevelsofanalysisisassessedbyconservative,boundingassumptions.Iftheriskestimateisfoundtobeverylowdespitetheuseofconservativeassumptions,thenthereexistsagreatdealofcertaintythattheactualriskstothepopulationofinterestforthegivenscenarioarebelowthelevelofconcernandnofurtherinterventionisrequired,assumingthatthescenarioandmodelspecificationsarecorrect.WhenaconservativeDRAindicatesthatariskmaybehigh,itispossiblethattheriskestimateisbiasedandtheactualriskmaybelower.Insuchasituation,dependingontheresourceimplicationsofthedecision,itmaybeappropriatetoproceedwithamorerefinedorhigherlevelofanalysis.Therelativecostsofinterventionversusfurtheranalysisshouldbeconsideredwhendecidingwhethertoproceedwithadecisionbasedonalowerlevelanalysisortoescalatetoahigherlevelofanalysis.Insomedeterministicassessments(e.g.,ecologicalrisks),theassumptionsarenotwellassuredofconservatism,andtheestimatedrisksmightbebiasedtoappearlowerthantheunseenactualrisk.

AmorerefinedanalysiscouldinvolvetheapplicationofDRAmethods,butwithalternativesetsofassumptionsintendedtocharacterizecentraltendencyandreasonableupperboundsofexposure,effectsandriskestimates,suchthattheestimatescouldbeforanactualindividualinthepopulationofinterestratherthanahypotheticalmaximallyexposedindividual.Suchanalysesarenotlikelytoprovidequantificationregardingtheproportionofthepopulationatorbelowaparticularexposureorrisklevelofconcern,uncertaintiesforanygivenpercentileoftheexposedpopulation,orprioritiesamonginputassumptionswithrespecttotheircontributionstouncertaintyandvariabilityintheestimates.

Tomorefullyanswerthequestionsoftenaskedbydecisionmakers,theanalysiscanbefurtherrefinedbyincorporatingquantitativecomparisonsofalternativemodelingstrategies(torepresentstructuraluncertaintiesassociatedwithscenariosormodels),quantifyingrangesofuncertaintyandvariabilityinmodeloutputs,andprovidingthecorrespondingrangesformodeloutputsofinterest.Whenperformingprobabilisticanalyses,choicesaremaderegardingwhethertofocusonthequantificationofvariabilityonly,uncertaintyonly,bothvariabilityanduncertaintytogether(representingarandomlyselectedindividual),orvariabilityanduncertaintyindependently(e.g.,in

28

atwo‐dimensionaldepictionofprobabilitybandsforestimatesofinter‐individualvariability;seeFigure4).Thesimultaneousbutdistinctpropagationofuncertaintyandvariabilityinatwo‐dimensionalframeworkenablesquantificationofuncertaintyintheriskforanypercentileofthepopulation.Forexample,onecouldestimatetherangeofuncertaintyintheriskfacedbythemedianmemberofthepopulationorthe95thpercentilememberofthepopulation.Suchinformationcanbeusedbyadecisionmakertogaugetheconfidencethatshouldbeplacedinanyparticularestimateofrisk,aswellastodeterminewhetheradditionaldatacollectionorinformationmightbeusefultoreducetheuncertaintyintheestimates.TheOPPassessmentofChromatedCopperArsenate‐treatedwoodusedsuchanapproach.(SeeCaseStudy9intheAppendix.)

Figure 4. Diagrammatic Comparison of Three Alternative Probabilistic Approaches for the Same Exposure Assessment. In Option 1 (one dimensional Monte Carlo analysis), only variability is quantified. In Option 2 (one dimensional Monte Carlo analysis), both uncertainty and variability are combined. In Option 3 (two dimensional Monte Carlo analysis), variability and uncertainty are analyzed separately. Source: WHO 2008. 

Whenconductingananalysisforthefirsttime,itmaynotbeknownorclear,priortoanalysis,whichcomponentsofthemodelorwhichmodelinputscontributethemosttotheestimatedriskoritsuncertaintyandvariability.Asaresultofcompletingananalysis,however,theanalystoftengainsinsightintothestrengthsandweaknessesofthemodelsandinputinformation.Probabilisticanalysisandsensitivityanalysiscanbeusedtogethertoidentifythekeysourcesofquantifieduncertaintyinthemodeloutputstoinformdecisionsregardingprioritiesforadditionaldatacollection.Ideally,timeshouldbeallowedforcollectingsuchinformationandrefiningtheanalysistoarriveatamorerepresentativeandrobustestimateofuncertaintyandvariabilityinrisk.Thus,thenotionofiterationindevelopingandimprovingananalysisiswidelyrecommended.

0%

20%

40%

60%

80%

100%

% o

f spe

cies

100 1000 104 105

95% Conf. IntervalMedianEC50

2D

MLE

1DPerc

entil

e co

nsum

er

Exposure (g/day)

1 10 100 1000

Option 1. Only variability

Option 3. Separating variability & uncertainty (2D MC)

Option 2. Combining variability & uncertainty (1D MC)

0%

20%

40%

60%

80%

100%

% o

f spe

cies

100 1000 104 105

95% Conf. IntervalMedianEC50

2D

MLE

1DPerc

entil

e co

nsum

er

Exposure (g/day)

1 10 100 1000

Option 1. Only variability

Option 3. Separating variability & uncertainty (2D MC)

Option 2. Combining variability & uncertainty (1D MC)

29

Thenotionofiterationcanbeappliedbroadlytotheriskassessmentframework.Forexample,afirstefforttoperformananalysismayleadtoinsightthattheassessmentquestionsmightbeimpossibletoaddress,orthatthereareadditionalassessmentquestionsthatmaybeequallyormoreimportant.Thus,iterationcanincludereconsiderationoftheinitialassessmentquestionsandthecorrespondingimplicationsfordefinitionofscenarios,selectionofmodelsandprioritiesforobtainingdataformodelinputs.Alternatively,inatime‐limiteddecisionenvironment,probabilisticandsensitivityanalysesmayofferinsightintotheeffectofmanagementoptionsonriskestimates.

3.3. What Are Some Specific Aspects of and Issues Related to Methodology for Probabilistic Risk Analysis?

ThissectionbrieflydescribesafewkeyaspectsofPRA,modeldevelopmentandassociateduncertainties.DetailedintroductionstoPRAmethodologyareavailableelsewhere,suchasAngandTang(1984),MorganandHenrion(1990),CullenandFrey(1999)andEPA(2001).Formoredetailedinformation,consultthesereferencesandseetheBibliographyforadditionalsources.

3.3.1. Developing a Probabilistic Risk Analysis Model TherearetwokeyissuesthatshouldbeconsideredindevelopingaPRAmodel;asdiscussedbelow.

StructuralUncertaintyinScenarios

Apotentiallykeysourceofuncertaintyinananalysisisthescenario,whichincludesspecificationofpollutantsources,transportpathways,exposureroutes,timingandlocations,geographicextentandrelatedissues.Thereisnoformalizedmethodologyfordealingquantitativelywithuncertaintyandvariabilityinscenarios.Decisionsregardingwhattoincludeorexcludefromascenariocouldberecastashypothesesregardingwhichagents,pathways,microenvironments,etc.,contributesignificantlytotheoverallexposureandriskofinterest.Inpractice,however,theuseofqualitativemethodstoframeanassessmenttendstobemorecommon,giventheabsenceofaformalquantitativemethodology.

CoupledModels

Forsource‐to‐outcomeriskassessments,itoftenisnecessarytoworkwithmultiplemodels,eachofwhichrepresentsadifferentcomponentofascenario.Forexample,theremaybeseparatemodelsforemissions,airquality,exposure,doseandeffects.Suchmodelsmayhavedifferentspatialandtemporalscales.Whenconductinganintegratedassessment,theremaybesignificantchallengesandbarrierstocouplingsuchmodelsintoonecoherentframework.Sometimes,thecouplingisdonedynamicallyinasoftwareenvironment.Inothercases,theoutputofonemodelmightbeprocessedmanuallytopreparetheinformationforinputtothenextmodel.Furthermore,theremaybefeedbackbetweencomponentsofthescenario(e.g.,poorairqualitymightaffecthumanactivity,which,inturn,couldaffectbothemissionsandexposures)thatareincompletelycapturedornotincluded.Thus,thecouplingofmultiplemodelscanbeapotentiallysignificantsourceofstructuraluncertainty(Özkaynak2009).

3.3.2. Dealing With Dependencies Among Probabilistic Inputs Whenrepresentingtwoormoreinputstoamodelasprobabilitydistributions,thequestionarisesastowhetheritisreasonabletoassumethatthedistributionsarestatisticallyindependent.Ifthereisadependence,itcouldbeassimpleasalinearcorrelationbetweentwoinputs,oritcouldbemorecomplicated,suchasnonlinearornonmonotonicrelationships.Dependenciestypicallyarenotimportantiftheriskestimateorothermodeloutputissensitivetooneornoneoftheprobabilisticinputsthatmighthaveinterdependence.Furthermore,dependenciestypicallyarenotofpracticalimportanceiftheyareweak.Whendependenciesexistandmightsignificantlyinfluence

30

theriskestimate,theycanbetakenintoaccountusingavarietyofstatisticalsimulationmethodsor,perhapsmoreappropriately,bymodelingthedependenceanalyticallywherepossible.DetailsonmethodsforassessingtheimportanceofpossibledependenciesandofquantifyingthemwhenneededaredescribedinFersonetal.(2004and2009).

Forsometypesofmodels,suchasairqualitymodels,itisnotpossibletointroduceaprobabilitydistributiontooneinput(e.g.,ambienttemperatureataparticularlocation)withoutaffectingvariablesatotherlocationsortimes(e.g.,temperaturesinotherlocationsatthesametimesortemporaltrendsintemperature).Insuchcases,itisbettertoproducean“ensemble”ofalternativetemperaturefields,eachofwhichisinternallyconsistent.Individualmembersofanensembleusuallyarenotinterpretedasrepresentingaprobabilitysample;however,comparisonofmultipleensemblesofmeteorologicalconditions,forexample,canprovideinsightintonaturalsourcesofvariabilityinambientconcentrations.

3.3.3. Conducting the Probabilistic Analysis

QuantifyingUncertaintyandVariabilityinModelInputsandParameters

Afterthemodelsareselectedordevelopedtosimulateascenarioofinterest,attentiontypicallyturnstothedevelopmentofinputdataforthemodel.Thereisasubstantialamountofliteratureregardingtheapplicationofstatisticalmethodsforquantifyinguncertaintyandvariabilityinmodelinputsandparametersbasedonempiricaldata(e.g.,AngandTang1984;CullenandFrey1999;MorganandHenrion1990;USEPA2001).Forexample,acommonlyusedmethodforquantifyingvariabilityinamodelinputistoobtainasampleofdata,selectatypeofparametricprobabilitydistributionmodeltofittothedata(e.g.,normal,lognormalorotherform),estimatetheparametersofthedistributionbasedonthedata,critiquethegoodness‐of‐fitusinggraphical(e.g.,probabilityplot)andstatistical(e.g.,Anderson‐Darling,Chi‐SquareorKolmogorov‐Smirnovtests)methodsandchooseapreferredfitteddistribution.Thismethodologycanbeadjustedtoaccommodatevarioustypesofdata,suchasdatathataresamplesfrommixturesofdistributionsorthatcontainnon‐detected(censored)values.Uncertaintiescanbeestimatedbasedonconfidenceintervalsforstatisticsofinterest,suchasmeanvalues,ortheparametersoffrequencydistributionsforvariability.Varioustextsandguidancedocuments,bothAgencyandprogrammatic,describetheseapproaches,includingtheGuidingPrinciplesforMonteCarloAnalysis(USEPA1997b).

Themostcommonmethodforestimatingaprobabilitydistributionintheoutputofamodel,basedonprobabilitydistributionsspecifiedformodelinputs,isMCS(CullenandFrey1999;MorganandHenrion1990).MCSispopularbecauseitisveryflexible.MCScanbeusedwithawidevarietyofprobabilitydistributionsaswellasdifferenttypesofmodels.ThemainchallengeforMCSisthatitrequiresrepetitivemodelcalculationstoconstructasetofpseudo‐randomnumbersformodelinputsandthecorrespondingestimatesformodeloutputsofinterest.TherearealternativestoMCSthataresimilarbutmorecomputationallyefficient,suchasLatinHypercubeSampling(LHS).TechniquesareavailableforsimulatingcorrelationsbetweeninputsinbothMCSandLHS.Formodelswithverysimplefunctionalforms,itmaybepossibletouseexactorapproximateanalyticalcalculations,butsuchsituationsareencounteredinfrequentlyinpractice.Theremaybesituationsinwhichthedatadonotconformtoawell‐definedprobabilitydistribution.Insuchcases,algorithms(suchasMarkovChainMonteCarlo)canestimateaprobabilitydistributionbycalculatingamathematicalformdescribingthepatternofobserveddata.Thisform,calledthelikelihoodfunction,isakeycomponentofBayesianinferenceand,therefore,servesasthebasisforsomeoftheanalyticalapproachestouncertaintyandvariabilitydescribedbelow.

Theuseofempiricaldatapresumesthatthedatacomprisearepresentative,randomsample.Ifknownbiasesorotherdataqualityproblemsexist,orifthereisascarcityorabsenceofrelevantdata,thennaïverelianceonavailableempiricaldataislikelytoresultinmisleadinginferencesin

31

theanalysis.Alternatively,estimatesofuncertaintyandvariabilitycanbeencoded,usingformalprotocols,basedonelicitationofexpertjudgment(e.g.,MorganandHenrion1990,USEPA2011a).Elicitationofexpertjudgmentforsubjectiveprobabilitydistributionsisusedinsituationswherethereareinsufficientdatatosupportastatisticalanalysisofuncertainty,butinwhichthereissufficientknowledgeonthepartofexpertstomakeaninferenceregardinguncertainty.Forexample,EPAconductedanexpertelicitationstudyontheconcentration‐responserelationshipbetweentheannualaverageambientlessthan2.5micrometer(µm)diameterparticulatematter(PM2.5)exposureandannualmortality(IEC2006;seealsoCaseStudies6and14intheAppendix).Subjectiveprobabilitydistributionsthatarebasedonexpertjudgmentcanbe“updated”withnewdataastheybecomeavailableusingBayesianstatisticalmethods.

StructuralUncertaintyinModels

Theremaybesituationsinwhichitprovesusefultoevaluatenotjusttheuncertaintiesininputsandparametervalues,butalsouncertaintiesregardingwhetheramodeladequatelycaptures—inahypothesized,mathematical,structuredform—therelationshipunderinvestigation.Aqualitativeapproachtoevaluatingthestructuraluncertaintyinamodelincludesdescribingthecriticalassumptionswithinamodel,thedocumentationofamodelorthemodelquality,andhowthemodelfitsthepurposeoftheassessment.Quantitativeapproachestoevaluatingstructuraluncertaintyinmodelsaremanifold.Theseincludeparameterizationofageneralmodelthatcanbereducedtoalternativefunctionalforms(e.g.,MorganandHenrion1990),enumerationofalternativemodelsinaprobabilitytree(e.g.,Evansetal.1994),comparingalternativemodelsbyevaluatinglikelihoodfunctions(e.g.,Royall1997;BurnhamandAnderson2002),poolingresultsofmodelalternativesusingBayesianmodelaveraging(e.g.,Hoetingetal.1999)ortestingthecausalrelationshipswithinalternativemodelsusingBayesianNetworks(Pearl2009).

SensitivityAnalysis:IdentifyingtheMostImportantModelInputs

Probabilisticmethodstypicallyfocusonhowuncertaintyorvariabilityinamodelinputaffect[orresultin]withrespecttouncertaintyorvariabilityinamodeloutput.Afteraprobabilisticanalysisiscompleted,sensitivityanalysistypicallytakestheperspectiveoflookingbacktoevaluatehowmuchofthevariationinthemodeloutputisattributabletoindividualmodelinputs(e.g.,FreyandPatil2002;Mokhtarietal.2006;Saltellietal.2004).

Therearemanytypesofsensitivityanalysismethods,includingsimpletechniquesthatinvolvechangingthevalueofoneinputatatimeandassessingtheeffectonanoutput,andstatisticalmethodsthatevaluatewhichofmanysimultaneouslyvaryinginputscontributethemosttothevarianceofthemodeloutput.Sensitivityanalysiscananswerthefollowingkeyquestions:

Whatistheimpactofchangesininputvaluesonmodeloutput?

Howcanvariationinoutputvaluesbeapportionedamongmodelinputs?

Whataretherangesofinputsassociatedwithbestorworstoutcomes?

Whatarethekeycontrollablesourcesofvariability?

Whatarethecriticallimits(e.g.,theemissionreductiontarget)?

Whatarethekeycontributorstotheoutputuncertainty?

Thus,sensitivityanalysiscanbeusedtoinformdecisionmaking.

Iteration

Therearetwomajortypesofiterationinriskassessmentmodeling.Oneisiterativerefinementofthetypeofanalysis,perhapsstartingwitharelativelysimpleDRAasascreeningstepinaninitial

32

levelofanalysisandproceedingtomorerefinedtypesofassessmentsasneededinsubsequentlevelsofanalysis.ExamplesofmorerefinedlevelsofassessmentincludeapplicationofsensitivityanalysistoDRA;theuseofprobabilisticmethodstoquantifyvariabilityonly,uncertaintyonly,orcombinedvariabilityanduncertainty(torepresentarandomlyselectedindividual);ortheuseoftwo‐dimensionalprobabilisticmethodsfordistinguishingandsimultaneouslycharacterizingbothuncertaintyandvariability.

Theothertypeofiterationoccurswithinaparticularlevelandincludesiterativeeffortstoformulateamodel,obtaindataandevaluatethemodeltoprioritizedataneeds.Forexample,amodelmayrequirealargenumberofinputassumptions.Toprioritizeeffortsofspecifyingdistributionsforuncertaintyandvariabilityformodelinputs,itisusefultodeterminewhichmodelinputsarethemostinfluentialwithrespecttotheassessmentendpoint.Therefore,sensitivitycanbeusedbasedonpreliminaryassessmentsofrangesordistributionsforeachmodelinputtodeterminewhichinputsarethemostimportanttotheassessment.Refinedeffortstocharacterizedistributionsthencanbeprioritizedtothemostimportantinputs.

33

4. SUMMARY AND RECOMMENDATIONS 4.1. Probabilistic Risk Analysis and Related Analyses Can Improve

the Decision-Making Process at EPA PRAcanprovideuseful(evencritical)informationabouttheuncertaintiesandvariabilityinthedata,models,scenario,expertjudgmentsandvaluesincorporatedinriskassessmentstosupportdecisionmakingacrosstheAgency.AsdiscussedinthispaperPRAisananalyticalmethodologycapableofincorporatinginformationregardinguncertaintyand/orvariabilityinriskanalysestoprovideinsightonthedegreeofcertaintyofariskestimateandhowtheriskestimatevarieswithintheexposedpopulation.TraditionalapproachessuchasDRA,oftenreportrisksusingdescriptorssuchas“centraltendency,”“highend”(e.g.,90thpercentileorabove)or“maximumanticipatedexposure”.BycontrastPRAcanbeusedtodescribemorecompletelytheuncertaintysurroundingsuchestimates,aswellastoidentifythekeycontributorstouncertaintyandvariabilityinpredictedexposuresorriskestimates.Thisinformationthencanbeusedbydecisionmakerstoweighalternatives,ortomakedecisionsonwhethertocollectadditionaldata,ortoconductadditionalresearchinordertoreducetheuncertaintyandfurthercharacterizevariabilitywithintheexposedpopulation.Informationonuncertaintiesandvariabilityinexposureandresponsecanultimatelyimprovetheriskestimates.

PRAcanbeusedtoobtaininsightonwhetheronemanagementalternativeismorelikelytoreduceriskscomparedtoanother.Inaddition,PRAcanfacilitatethedevelopmentofmodelingscenariosandthesimultaneousconsiderationofmultiplemodelalternatives.Probabilisticmethodsofferanumberoftoolsdesignedtoincreaseconfidenceindecisionmakingthroughtheincorporationofinputuncertaintyandvariabilitycharacterizationandprioritizationinriskanalyses.Forexample,onePRAtool,sensitivityanalysescanbeusedtoidentifyinfluentialknowledgegapsintheestimationofrisk;thisimprovestransparencyinthepresentationoftheseuncertaintiesandimprovestheabilitytocommunicatethemostrelevantinformationmoreclearlytodecisionmakersandstakeholders.PRAallowsonetoinvestigatepotentialchangesindecisionsthatcouldresultfromthecollectionofadditionalinformation.However,theadditionalresources(e.g.,time,costs,orexpertise)toundertakeneedtobeweighedagainstthepotentialimprovementsinthedecisionmakingprocess.Ultimately,PRAmayenhancethescientificfoundationoftheEPA’sapproachtodecisionmaking.

Thevarioustoolsandmethodsdiscussedinthiswhitepapercanbeutilizedatallstagesofriskanalysisandalsocanaidthedecision‐makingprocessby,forexample,characterizinginter‐individualvariabilityanduncertainties.

PRAandrelatedmethodsareemployedinvaryingdegreesacrosstheAgency.BasicguidanceexistsatEPAontheuseandacceptabilityofPRAforriskestimation,butimplementationvariesgreatlywithinprograms,officesandregions.TheuseofMonteCarloorotherprobability‐basedtechniquestoderivearangeofpossibleoutputsfromuncertaininputsisafairlywell‐developedapproachwithinEPA.Althoughhighlysophisticatedhumanexposureassessmentandecologicalriskapplicationshavebeendeveloped,theuseofPRAmodelstoassesshumanhealtheffectsanddose‐responserelationshipshasbeenmorelimitedattheAgency.

TheevaluationoftheapplicationofPRAtechniquesunderspecificlawsandregulationsvariesbyprogram,officeandregion.Movingforward,itisimportanttobroadendiscussionsbetweenriskassessorsandriskmanagersregardinghowPRAtoolscanbeusedtosupportspecificdecisionsandhowtheycanbeusedwithintheregulatoryframeworkusedbyprograms,offices,andregionstomakedecisions.Thiscanbeaccomplishedbyexpandingthedialoguebetweenassessorsand

34

managesatalllevelsregardinghowthePRAtoolshavebeenusedandhowtheyhaveenhanceddecisionmaking.

IncreaseduseofPRAandconsistentapplicationofPRAtoolsinsupportofEPAdecisionmakingrequiresenhancedinternalcapacityforconductingtheseassessments,aswellasimprovedinterpretationandcommunicationofsuchinformationinthecontextofdecisions.ImprovementsofAgencycapacitycouldbeaccomplishedthroughsharingofexperiences,knowledgeandtrainingandincreasedavailabilityoftoolsandmethods.

4.2. Major Challenges to Using Probabilistic Risk Analysis to Support Decisions

ThechallengesforEPAaretwo‐fold.AsanAgencyresponsibleforprotectinghumanhealthandtheenvironment,EPAmakesregulatoryandpolicydecisions,eveninthepresenceofconflictingstakeholderpositionsandtheinevitableuncertaintiesinthescience.ThefirstchallengeforEPAistodeterminehowtoconductitsdecision‐makingresponsibilities,weighingdeterminationsofwhatconstitutestoomuchuncertaintytomakeadecision,againstpotentialadverseconsequencesofpostponingdecisions.

Thesecondchallenge,isthatalthoughcurrentPRAtechniquesareavailablethatwouldhelptoinformEPAdecision‐makingprocesses,researchandguidanceareneededtoimprovethesemethodsforamorecompleteimplementationofPRAinHHRAandERA.Inparticular,additionalguidanceisneededtohelpanalystsanddecisionmakersbetterunderstandhowtoincorporatePRAapproachesintothedecision‐makingprocess.Thisincludes,guidanceonwhichstatisticaltoolstouseandwhentousethem,andhowprobabilisticinformationcanhelptoinformthescientificbasisofdecisions.BothDRAandPRAaswellasappropriatestatisticalmethodsmaybeusefulatanystageoftheriskanalysisanddecision‐makingprocess,fromplanningandscopingtocharacterizingandcommunicatinguncertainty.

AsnotedinSection3.3,therearesignificantchallengesinproperlyaccountingforuncertaintyandvariabilitywhenmultiplemodelsarecoupledtogethertorepresentthesource‐to‐outcomecontinuum.Moreover,thecouplingofmultiplemodelsmightneedtoinvolveinputsandcorrespondinguncertaintiesthatareincorporatedintomorethanonemodel,potentiallyresultingincomplexdependencies.Integrativeresearchoncoupledmodeluncertaintieswillbequitevaluable.

Theremaybemismatchesinthetemporalandspatialresolutionofeachmodelthatconfoundtheabilitytopropagateuncertaintyandvariabilityfromonemodeltoanother.Forsomemodels,thekeyuncertaintiesmaybeassociatedwithinputs,whereasforothermodels,thekeyuncertaintiesmaybeassociatedwithstructureorparameterizationalternatives.Modelintegrationandharmonizationactivitieswillbeimportanttoaddressingthesetechnicalissues.

4.3. Recommendations for Enhanced Utilization of Probabilistic Risk Analysis at EPA

Someexamplesofareaswhereneworupdatedguidancewouldbehelpfularethese:

IdentificationofdifferenttypesofinformationrequiredforthevariousAgencydecision‐makingprocesses,suchasdataanalysis,tools,models,anduseofexperts.

Useofprobabilisticapproachestoevaluatehealtheffectsdata.

UseofprobabilisticapproachesforERA.

35

Integratingprobabilisticexposureandriskestimatesandcommunicatinguncertaintyandvariability.

Inordertosupportthedevelopmentofguidanceontheseorrelatedtopics,followingstudiesorresearcharerecommended:

TheuseofPRAmodelstoevaluatetoxicitydatahasbeenverylimited.Scientific,technicalandpolicy‐baseddiscussionsareneededinthisarea.

Additionalresearchonformalmethodsfortreatingmodeluncertaintieswillbevaluable.

Somestepstoimproveimplementationincludethese:

InformingdecisionmakersabouttheadvantagesanddisadvantagesofusingPRAtechniquesintheirdecision‐makingprocessesthroughlectures,webinarsandcommunicationsregardingthetechniquesandtheiruseinEPA.

IncorporatingadiscussionofPRAtoolsduringPlanningandScopingforHHRAsandERAs.

ContinuingthedialoguebetweenassessorsandmanagersonhowtousePRAwithintheregulatorydecisionmakingprocess.

ConductingmeetingsanddiscussionsofPRAtechniquesandtheirapplicationwithbothmanagersandassessorstoaidinprovidinggreaterconsistencyandtransparencyinEPA’sriskassessmentandriskmanagementprocessandindevelopingEPA’sinternalcapacity.

Developinga“CommunityofPractice”forfurtherdiscussionregardingtheapplicationofPRAtechniquesandtheuseofthesetoolsindecisionmaking.

Riskassessorsandriskmanagersneedinformationandtrainingsothattheycanbetterutilizethesetools.Educationandexperiencewillgeneratefamiliaritywiththesetools,whichwillhelpanalystsanddecisionmakersbetterunderstandandconsidermorefullyutilizingthesetechniqueswithintheirregulatoryprograms.Increasedtrainingisneededtofacilitateunderstandingonalllevelsandmayincludethefollowing:

ProvidingintroductoryaswellasadvancedtrainingtoallEPAoffices.

TrainingriskassessorsandriskmanagersinthePRAtechniquessothattheycanlearnaboutthevarioustoolsavailable,theirapplications,softwareandreviewconsiderations,andresourcesforadditionalinformation(e.g.,expertsandsupportserviceswithintheAgency).

Providingeasilyavailable,flexible,modulartrainingforalllevelsofexperiencetofamiliarizeEPAemployeeswiththemenuoftoolsandtheircapacities.

Providingliveandrecordedseminarsandwebinarsforintroductoryandsupplementaleducation,aswellasperiodic,centralizedhands‐ontrainingsessionsdemonstratinghowtoutilizesoftwareprograms.

TrainingiscriticalbothforanimprovedunderstandingbutalsotobuildincreasedcapacityintheAgencyandexplicitstepscouldincludethese:

Demonstrating,throughinformationalopportunitiesandresourcelibraries,thevarioustoolsandmethodsthatcanbeusedatallstagesofriskanalysistoaidthedecision‐makingprocessbycharacterizinginter‐individualvariabilityanduncertainties.

Promotingthesharingofexperience,knowledge,modelsandbestpracticesviameetingsofriskassessorsandmanagers;electronicexchanges,suchastheEPAPortalEnvironmental

36

ScienceConnector(https://ssoprod.epa.gov/sso/jsp/obloginESCNew.jsp);andmoredetaileddiscussionsofthecasestudies.

AsEPAworkstowardthemoreintegratedevaluationofenvironmentalproblems,thiswillincludenotjusttheimprovedunderstandingofsinglepollutants/singlemedia,butmulti‐pollutant,multi‐mediaandmulti‐receptoranalysiswithinadecisionanalyticframework.EPAisbeginningtobuildsuchintegratedcapabilityintoanalyticaltoolslikePRA(BabendreierandCastleton2005;Stahletal.2011).

TheRAFwillbetakingaleadershiprolethroughtheUncertaintyandVariabilityWorkgrouptomorefullyevaluatetheapplicationanduseofPRAtoolsandbroadeningthedialoguebetweenassessorsandmanagers.UpdatesontheprogressofthisTechnicalPanelwillbeprovidedontheRAFwebpageat:www.epa.gov/raf.

37

GLOSSARY Analysis.Examinationofanythingcomplextounderstanditsnatureortodetermineitsessentialfeatures(WHO2004).

Assessment.Adeterminationorappraisalofpossibleconsequencesresultingfromananalysisofdata(2011b).

Assessmentendpoint.Anexplicitexpressionoftheenvironmentalvaluethatistobeprotected,operationallydefinedbyanecologicalentityanditsattributes.Forexample,salmonarevaluedecologicalentities;reproductionandageclassstructurearesomeoftheirimportantattributes.Together,salmon“reproductionandageclassstructure”formanassessmentendpoint(USEPA1998b).

Bayesianprobability.Anapproachtoprobability,representingapersonaldegreeofbeliefthatavalueofrandomvariablewillbeobserved.Alternatively,theuseofprobabilitymeasurestocharacterizethedegreeofuncertainty(Gelmanetal.2004).

BayesianAnalysis.Bayesiananalysisisamethodofstatisticalinferenceinwhichtheknowledgeofprioreventsisusedtopredictfutureevents(USEPA2011b).

Correlation.Anestimateofthedegreetowhichtwosetsofvariablesvarytogether,withnodistinctionbetweendependentandindependentvariables.Correlationreferstoabroadclassofstatisticalrelationshipsinvolvingdependence(USEPA2012).

Criticalcontrolpoint.Acontrollablevariablethatcanbeadjustedtoreduceexposureandrisk.Forexample,acriticalcontrolpointmightbetheemissionratefromaparticularemissionsource.Theconceptofcriticalcontrolpointisfromthehazardassessmentandcriticalcontrolpointconceptforriskmanagementthatisusedinspaceandfoodsafetyapplications,amongothers(USEPA2006c).

Criticallimit.Anumericalvalueofacriticalcontrolpointatorbelowwhichriskisconsideredtobeacceptable.Acriterionthatseparatesacceptabilityfromunacceptability(USEPA2006c).

Deterministic.Amethodologyrelyingonpoint(i.e.,exact)valuesasinputstoestimaterisk;thisobviatesquantitativeestimatesofuncertaintyandvariability.Resultsalsoarepresentedaspointvalues.Uncertaintyandvariabilitymaybediscussedqualitativelyorsemi‐quantitativelybymultipledeterministicriskestimates(USEPA2006b).

Deterministicriskassessment(DRA).Riskevaluationinvolvingthecalculationandexpressionofriskasasinglenumericalvalueor“singlepoint”estimateofrisk,withuncertaintyandvariabilitydiscussedqualitatively(USEPA2012).

Ecologicalriskassessment.Theprocessthatevaluatesthelikelihoodthatadverseecologicaleffectsmayoccurorareoccurringasaresultofexposuretooneormorestressors(USEPA1998b).

Ecosystem.Thebioticcommunityandabioticenvironmentwithinaspecifiedlocationinspaceandtime(USEPA1998b).

Ensemble.Amethodforpredictivemodelingbasedonmultiplemeasuresofthesameeventovertime(e.g.,theamountofcarbondioxidepresentintheatmosphereatselectedtimepoints).Thecollectionofdatainputisknownasanensembleandcanbeusedtodevelopaquantificationofpredictionvariabilitywithinthemodel.Ensemblemodelingisusedmostcommonlyinatmosphericpredictioninforecasting,althoughensemblemodelinghasbeenappliedtobiologicalsystemstobetterquantifyrisksofeventsorperturbationswithinbiologicalsystems(FuentesandFoley2012).

Environment.Thesumofallexternalconditionsaffectingthelife,developmentandsurvivalofanorganism(USEPA2010a).

38

Expertelicitation.Asystematicprocessofformalizingandquantifying,typicallyinprobabilisticterms,expertjudgmentsaboutuncertainquantities(USEPA2011a).

Frequentist(orfrequency)probability.Aviewofprobabilitythatconcernsitselfwiththefrequencywith which an event occurs givenalongsequenceofidenticalandindependenttrials(USEPA1997b).

Hazardidentification.Theriskassessmentprocessofdeterminingwhetherexposuretoastressorcancauseanincreaseintheincidenceorseverityofaparticularadverseeffect,andwhetheranadverseeffectislikelytooccur(USEPA2012).

Humanhealthriskassessment.1.Theprocesstoestimatethenatureandprobabilityofadversehealtheffectsinhumanswhomaybeexposedtochemicalsincontaminatedenvironmentalmedia,noworinthefuture(USEPA2010b).2.Theevaluationofscientificinformationonthehazardouspropertiesofenvironmentalagents(hazardcharacterization),thedose‐responserelationship(dose‐responseassessment),andtheextentofhumanexposuretothoseagents(exposureassessment).Theproductoftheriskassessmentisastatementregardingtheprobabilitythatpopulationsorindividualssoexposedwillbeharmedandtowhatdegree(riskcharacterization)(USEPA2006a).

Inputs.Quantitiesthatareappliedtoamodel(WHO2008).

LikelihoodFunction.Anapproachtomodelingexposureinwhichlong‐termexposureofanindividualissimulatedasthesumofseparateshort‐termexposureevents(USEPA2001).

Microenvironment.Well‐definedsurroundingssuchasthehome,office,automobile,kitchen,store,etc.,thatcanbetreatedashomogenous(orwellcharacterized)intheconcentrationsofachemicalorotheragent(USEPA1992).

Microexposureevent(MEE)analysis.Anapproachtomodelingexposureinwhichlong‐termexposureofanindividualissimulatedasthesumofseparateshort‐termexposureevents(USEPA2001).

Model.Amathematicalrepresentationofanaturalsystemintendedtomimicthebehavioroftherealsystem,allowingdescriptionofempiricaldata,andpredictionsaboutuntestedstatesofthesystem(USEPA2006b).

Modelboundaries.1.Decisionsregardingthetime,space,numberofchemicals,etc.,usedinguidingmodelingofthesystem.Riskscanbeunderstatedoroverstatedifthemodelboundaryismis‐specified.Forexample,ifastudyareaisdefinedtobetoolargeandincludesasignificantnumberoflow‐exposureareas,thenapopulation‐levelriskdistributioncanbedilutedbyincludinglessexposedindividuals,whichcan,inturn,resultinarisk‐baseddecisionthatdoesnotprotectsufficientlythemostexposedindividualsinthestudyarea.2.Designatedareasofcompetenceofthemodel,includingtime,space,pathogens,pathways,exposedpopulations,andacceptablerangesofvaluesforeachinputandjointlyamongallinputsforwhichthemodelmeetsdataqualityobjectives(WHO2008).

Modeling.Developmentofamathematicalorphysicalrepresentationofasystemortheorythataccountsforallorsomeofitsknownproperties.Modelsoftenareusedtotesttheeffectofchangesofcomponentsontheoverallperformanceofthesystem(USEPA2010a).

Modeluncertainty(sourcesof):

Modelstructure.Asetofassumptionsandinferenceoptionsuponwhichamodelisbased,includingunderlyingtheoryaswellasspecificfunctionalrelationships(WHO2008).

39

Modeldetail.Levelofsimplicityordetailassociatedwiththefunctionalrelationshipsassumedinthemodelcomparedtotheactualbutunknownrelationshipsinthesystembeingmodeled(WHO2008).

Extrapolation.Useofmodelsoutsideoftheparameterspaceusedintheirderivationmayresultinerroneouspredictions.Forexample,athresholdforhealtheffectsmayexistatexposurelevelsbelowthosecoveredbyaparticularepidemiologicalstudy.Ifthatstudyisusedinmodelinghealtheffectsatthoselowerlevels(anditisassumedthatthelevelofresponseseeninthestudyholdsforlowerlevelsofexposure),thendiseaseincidencemaybeoverestimated(USEPA2007a).

MonteCarloanalysis(MCA)orsimulation(MCS).Arepeatedrandomsamplingfromthedistributionofvaluesforeachoftheparametersinagenericexposureorriskequationtoderiveanestimateofthedistributionofexposuresorrisksinthepopulation(USEPA2006b).

One‐dimensionalMonteCarloanalysis(1‐DMCA).Anumericalmethodofsimulatingadistributionforanendpointofconcernasafunctionofprobabilitydistributionsthatcharacterizevariabilityoruncertainty.Distributionsusedtocharacterizevariabilityaredistinguishedfromdistributionsusedtocharacterizeuncertainty(WHO2008).

Parameter.Aquantityusedtocalibrateorspecifyamodel,suchas‘parameters’ofaprobabilitymodel(e.g.,meanandstandarddeviationforanormaldistribution).Parametervaluesoftenareselectedbyfittingamodeltoacalibrationdataset(WHO2008).

Probability.Afrequentistapproachconsidersthefrequencywithwhichsamplesareobtainedwithinaspecifiedrangeorforaspecifiedcategory(e.g.,theprobabilitythatanaverageindividualwithaparticularmeandosewilldevelopanillness)(WHO2008).

Probabilisticriskanalysis(PRA).Calculationandexpressionofhealthrisksusingmultipleriskdescriptorstoprovidethelikelihoodofvariousrisklevels.Probabilisticriskresultsapproximateafullrangeofpossibleoutcomesandthelikelihoodofeach,whichoftenispresentedasafrequencydistributiongraph,thusallowinguncertaintyorvariabilitytobeexpressedquantitatively(USEPA2012).

Problemformulation.Theinitialstageofariskassessmentwherethepurposeoftheassessmentisarticulated,exposureandriskscenariosareconsidered,aconceptualmodelisdeveloped,andaplanforanalyzingandcharacterizingriskisdetermined(USEPA2004a).

Referenceconcentration(RfC).Anestimate(withuncertaintyspanningapproximatelyanorderofmagnitude)ofacontinuousinhalationexposuretothehumanpopulation(includingsensitivesubgroups)thatislikelytobewithoutanappreciableriskofdeleteriouseffectsduringalifetime.ItcanbederivedfromaNo‐Observed‐Adverse‐EffectLevel(NOAEL),Lowest‐Observed‐Adverse‐EffectLevel(LOAEL),orbenchmarkconcentration,withuncertaintyfactorsgenerallyappliedtoreflectlimitationsofthedataused.ItisgenerallyusedinEPA’snoncancerhealthassessments(USEPA2007a).

Referencedose(RfD).Anestimate(withuncertaintyspanningapproximatelyanorderofmagnitude)ofadailyoralexposuretothehumanpopulation(includingsensitivesubgroups)thatislikelytobewithoutanappreciableriskofdeleteriouseffectsduringalifetime.ItcanbederivedfromaNOAEL,LOAELorbenchmarkdose,withuncertaintyfactorsgenerallyappliedtoreflectlimitationsofthedataused.ItistypicallyusedinEPA’snoncancerhealthassessments(USEPA2011c).

Risk.1.Riskincludesconsiderationofexposuretothepossibilityofanadverseoutcome,thefrequencywithwhichoneormoretypesofadverseoutcomesmayoccur,andtheseverityor

40

consequencesoftheadverseoutcomesifsuchoccur.2.Thepotentialforrealizationofunwanted,adverseconsequencestohumanlife,health,propertyortheenvironment.3.Theprobabilityofadverseeffectsresultingfromexposuretoanenvironmentalagentormixtureofagents.4.Thecombinedanswersto:Whatcangowrong?Howlikelyisit?Whataretheconsequences?(USEPA2011c).

Riskanalysis.Aprocessforidentifying,characterizing,controllingandcommunicatingrisksinsituationswhereanorganism,system,subpopulationorpopulationcouldbeexposedtoahazard.Riskanalysisisaprocessthatincludesriskassessment,riskmanagementandriskcommunication(WHO2008).

Riskassessment.1.Aprocessintendedtocalculateorestimatetherisktoagiventargetorganism,system,subpopulationorpopulation,includingtheidentificationofattendantuncertaintiesfollowingexposuretoaparticularagent,takingintoaccounttheinherentcharacteristicsoftheagentofconcern,aswellasthecharacteristicsofthespecifictargetsystem(WHO2008).2.Theevaluationofscientificinformationonthehazardouspropertiesofenvironmentalagents(hazardcharacterization),thedose‐responserelationship(dose‐responseassessment),andtheextentofhumanexposuretothoseagents(exposureassessment)(NRC1983).Theproductoftheriskassessmentisastatementregardingtheprobabilitythatpopulationsorindividualssoexposedwillbeharmedandtowhatdegree(riskcharacterization;USEPA2000a).3.Qualitativeandquantitativeevaluationoftheriskposedtohumanhealthortheenvironmentbytheactualorpotentialpresenceoruseofspecificpollutants(USEPA2012).

Risk‐baseddecisionmaking.Aprocessthroughwhichdecisionsaremadeaccordingtotheriskeachposedtohumanhealthandtheenvironment(USEPA2012).

Riskmanagement.Adecision‐makingprocessthattakesintoaccountenvironmentallaws;regulations;andpolitical,social,economic,engineeringandscientificinformation,includingariskassessment,toweighpolicyalternativesassociatedwithahazard(USEPA2011c).

Scenario.Asetoffacts,assumptionsandinferencesabouthowexposuretakesplacethataidstheexposureassessorinevaluating,estimatingorquantifyingexposures(USEPA1992).Scenariosmightincludeidentificationofpollutants,pathways,exposureroutesandmodesofaction,amongothers.

Sensitivityanalysis.Theprocessofchangingonevariablewhileleavingtheothersconstanttodetermineitseffectontheoutput.Thisprocedurefixeseachuncertainquantityatitscrediblelowerandupperbounds(holdingallothersattheirnominalvalues,suchasmedians)andcomputestheresultsofeachcombinationofvalues.Theresultshelptoidentifythevariablesthathavethegreatesteffectonexposureestimatesandhelpfocusfurtherinformation‐gatheringefforts(USEPA2011b).

Tieredapproach.Referstovarioushierarchicaltiers(levels)ofcomplexityandrefinementfordifferenttypesofmodelingapproachesthatcanbeusedinriskassessment.Adeterministicriskassessmentwithconservativeassumptionsisanexampleofalowerleveltypeofanalysis(Tier0)thatcanbeusedtodeterminewhetherexposuresandrisksarebelowlevelsofconcern.Examplesofprogressivelyhigherlevelsincludetheuseofdeterministicriskassessmentcoupledwithsensitivityanalysis(Tier1),theuseofprobabilistictechniquestocharacterizeeithervariabilityoruncertaintyonly(Tier2),andtheuseoftwo‐dimensionalprobabilistictechniquestodistinguishbetweenbutsimultaneouslycharacterizebothvariabilityanduncertainty(Tier3)(USEPA2004aandWHO2008).

Two‐dimensionalMonteCarloanalysis(2‐DMCA).Anadvancednumericalmodelingtechniquethatusestwostagesofrandomsampling,alsocallednestedloops,todistinguishbetween

41

variabilityanduncertaintyinexposureandtoxicityvariables.Thefirststage,oftencalledtheinnerloop,involvesacomplete1‐DMCAsimulationofvariabilityinrisk.Inthesecondstage,oftencalledtheouterloop,parametersoftheprobabilitydistributionsareredefinedtoreflectuncertainty.Theseloopsarerepeatedmanytimesresultinginmultipleriskdistributions,fromwhichconfidenceintervalsarecalculatedtorepresentuncertaintyinthepopulationdistributionofrisk(WHO2008).

Uncertainty.Uncertaintyoccursbecauseofalackofknowledge.Itisnotthesameasvariability.Forexample,ariskassessormaybeverycertainthatdifferentpeopledrinkdifferentamountsofwaterbutmaybeuncertainabouthowmuchvariabilitythereisinwaterintakeswithinthepopulation.Uncertaintyoftencanbereducedbycollectingmoreandbetterdata,whereasvariabilityisaninherentpropertyofthepopulationbeingevaluated.Variabilitycanbebettercharacterizedwithmoredatabutitcannotbereducedoreliminated.Effortstoclearlydistinguishbetweenvariabilityanduncertaintyareimportantforbothriskassessmentandriskcharacterization,althoughtheybothmaybeincorporatedintoanassessment(USEPA2011c).

Uncertaintyanalysis.Adetailedexaminationofthesystematicandrandomerrorsofameasurementorestimate;ananalyticalprocesstoprovideinformationregardinguncertainty(USEPA2006b).

Valueofinformation.Ananalysisthatinvolvesestimatingthevaluethatnewinformationcanhavetoariskmanagerbeforetheinformationisactuallyobtained.Itisameasureoftheimportanceofuncertaintyintermsoftheexpectedimprovementinariskmanagementdecisionthatmightcomefrombetterinformation(USEPA2001).

Variability.Referstotrueheterogeneityordiversity,asexemplifiedinnaturalvariation.Forexample,amongapopulationthatdrinkswaterfromthesamesourceandwiththesamecontaminantconcentration,therisksfromconsumingthewatermayvary.Thismayresultfromdifferencesinexposure(e.g.,differentpeopledrinkingdifferentamountsofwaterandhavingdifferentbodyweights,exposurefrequenciesandexposuredurations),aswellasdifferencesinresponse(e.g.,geneticdifferencesinresistancetoachemicaldose).Thoseinherentdifferencesarereferredtoasvariability.Differencesamongindividualsinapopulationarereferredtoasinter‐individualvariability,anddifferencesforoneindividualovertimearereferredtoasintra‐individualvariability(USEPA2011c).

42

REFERENCES Ang,A.H‐S.,andW.H.Tang.1984.ProbabilityConceptsinEngineeringPlanningandDesign,Volume2:Decision,Risk,andReliability.NewYork:JohnWiley&Sons.

Babendreier,J.E.,andK.J.Castleton.2005.“InvestigatingUncertaintyandSensitivityinIntegrated,MultimediaEnvironmentalModels:ToolsforFRAMES‐3MRA.”EnvironmentalModeling&Software20:1043–1055.

Bloom,D.L.,D.M.Byrne,andJ.M.Andreson.1993.CommunicatingRisktoSeniorEPAPolicy‐Makers:AFocusGroupStudy.PreparedbyBloomResearchandtheOfficeofAirQualityPlanningandStandards.ResearchTrianglePark,NC:USEPA.

Burnham,K.P.,andD.R.Anderson.2002.ModelSelectionandInference:APracticalInformation‐TheoreticApproach.2nded.NewYork:Springer‐Verlag.

Clyde,M.,P.Guttorp,andE.Sullivan.2000.EffectsofAmbientFineandCoarseParticlesonMortalityinPhoenix,Arizona.NationalResearchCenterforStatisticsandtheEnvironmentTechnicalReportSeries,No.40.

Cooke,R.M.1991.ExpertsinUncertainty:OpinionandSubjectiveProbabilityinScience.NewYork:OxfordUniversityPress.

Costanza,R.,S.O.Funtowicz,andJ.R.Ravetz.1997.“AssessingandCommunicatingDataQualityinPolicy‐RelevantResearch.”InFrontiersinEcologicalEconomics:TransdisciplinaryEssays,byRobertCostanza,283–308.Cheltenham,UK:E.Elgar.

Cullen,A.C.,andH.C.Frey.1999.ProbabilisticExposureAssessment:AHandbookforDealingwithVariabilityandUncertaintyinModelsandInputs.NewYork:PlenumPress.

Evans,J.S.,J.D.Graham,G.M.Gray,andR.L.Sielken.1994.“ADistributionalApproachtoCharacterizingLow‐DoseCancerRisk.”RiskAnalysis14(1):25–34.

Ferson,S.,R.Nelsen,J.Hajagos,D.Berleant,J.Zhang,W.T.Tucker,L.Ginzburg,andW.L.Oberkampf.2004.DependenceinProbabilisticModeling,Dempster‐ShaferTheory,andProbabilityBoundsAnalysis.SAND20043072.Albuquerque,NM:SandiaNationalLaboratories.www.ramas.com/depend.pdf.

Ferson,S.,W.L.Oberkampt,andL.Ginzburg.2009.“ValidationofImpreciseProbabilityModels.”InternationalJournalofReliabilityandSafety3(1–3):3–22.

Finkel,A.M.1989.“IsRiskAssessmentReallyTooConservative?:RevisingtheRevisionists.”JournalofEnvironmentalLaw14:427–67.

Fisher,L.,P.Pascual,andW.Wagner.2006.MappingtheRoleofModelsinU.S.andE.U.RiskRegulation:ALegalBackgroundPaper.Washington,D.C.:TheAssociationofUnionConstructorsWorkshop.

Fischhoff,B.1995.“RankingRisks.”Risk:Health,Safety&Environment6:191–202.

Frey,H.C.,andS.R.Patil.2002.“IdentificationandReviewofSensitivityAnalysisMethods.”RiskAnalysis22(3):553–78.

Fuentes,M.andK.Foley.2012.“EnsembleModels.”InAbdel,H.andWalterW.El‐Shaarawai(eds.)EncyclopediaofEnvironmetrics.Indianapolis:JohnWileyandSons.

Gelman,A.,J.B.Carlin,H.S.Stern,andD.B.Rubin.2004.BayesianDataAnalysis.2nded.BocaRaton,FL:ChapmanandHall/CRC.

43

Hattis,D.,S.Baird,andR.Goble.2002.“AStrawManProposalforaQuantitativeDefinitionoftheRfD.”DrugandChemicalToxicology25(4):403–36.

Hattis,D.B.,andD.E.Burmaster.1994.“AssessmentofVariabilityandUncertaintyDistributionsforPracticalRiskAnalyses.”RiskAnalysis14(5):713–30.

Hattis,D.,andM.K.Lynch.2010.AlternativestoPollutant‐by‐PollutantDose‐ResponseEstimationforAirToxics(FinalReport).EPA‐W‐05‐022,WA3‐80.Washington,D.C.:USEPA.

Hoeting,J.A.,D.Madigan,A.E.Raftery,andC.T.Volinsky.1999.“BayesianModelAveraging:ATutorial.”StatisticalScience14(4):382–417.

IEC(IndustrialEconomics,Inc.).2006.ExpandedExpertJudgmentAssessmentoftheConcentration‐ResponseRelationshipBetweenPM2.5ExposureandMortality(FinalReport).ResearchTrianglePark,NC:USEPA.http://www.epa.gov/ttn/ecas/regdata/Uncertainty/pm_ee_report.pdf.

InstituteofMedicine.2013.EnvironmentalDecisionsintheFaceofUncertainty.Washington,D.C.:TheNationalAcademiesPress.

Illing,H.P.A.1999.“AreSocietalJudgmentsBeingIncorporatedIntotheUncertaintyFactorsUsedinToxicologicalRiskAssessment?”RegulatoryToxicologyandPharmacology29(3):300–308.

Krupnick,A.,R.Morgenstern,M.Batz,P.Nelson,D.Burtraw,J.Shih,andM.McWilliams.2006.NotaSureThing:MakingRegulatoryChoicesUnderUncertainty.Washington,D.C.:ResourcesfortheFuture.http://www.rff.org/rff/Documents/RFF‐Rpt‐RegulatoryChoices.pdf.

Kunreuther,H.C.,andP.Slovic.1996.“Science,ValuesandRisk.”AnnalsoftheAmericanAcademyofPoliticalandSocialScience545:116–25.

Lester,R.R.,L.C.Green,andI.Linkov.2007.“Site‐SpecificApplicationsofProbabilisticHealthRiskAssessment:ReviewoftheLiteratureSince2000.”RiskAnalysis27:635–58.

McConnell,K.E.1997.“UsingCost‐BenefitAnalysisintheManagementofContaminatedSediments(AppendixE).”InContaminatedSedimentsinPortsandWaterways:CleanupStrategiesandTechnologies,bytheCommissiononEngineeringandTechnicalSystemsoftheNationalResearchCouncil,239–56.Washington,D.C.:TheNationalAcademiesPress.

Mokhtari,A.,H.C.Frey,andJ.Zheng.2006.“EvaluationandRecommendationofSensitivityAnalysisMethodsforApplicationtoStochasticHumanExposureandDoseSimulation(SHEDS)Models.”JournalofExposureScienceandEnvironmentalEpidemiology16(6):491–506.

Morgan,G.,H.Dowlatabadi,M.Henrion,D.Keith,R.Lempert,S.McBrid,M.Small,andT.Wilbanks,eds.2009.BestPracticeApproachesforCharacterizing,Communicating,andIncorporatingScientificUncertaintyinDecisionMaking.FinalReport.Washington,D.C.:U.S.ClimateChangeScienceProgram,NationalOceanicandAtmosphericAdministration.http://library.globalchange.gov/sap‐5‐2‐best‐practice‐approaches‐for‐characterizing‐communicating‐and‐incorporating‐scientific‐uncertainty‐in‐decisionmaking.

Morgan,M.G.,andM.Henrion.1990.Uncertainty:AGuidetoDealingwithUncertaintyinQuantitativeRiskandPolicyAnalysis.NewYork:CambridgeUniversityPress.

NRC(NationalResearchCouncil).1983.RiskAssessmentintheFederalGovernment:ManagingtheProcess.CommitteeontheInstitutionalMeansforAssessmentofRiskstoPublicHealth.Washington,D.C.:TheNationalAcademiesPress.

NRC.1989.RiskAssessmentintheFederalGovernment:ManagingtheProcess.Washington,D.C.:TheNationalAcademiesPress.

44

NRC.1991.RethinkingtheOzoneProbleminUrbanandRegionalAirPollution.Washington,D.C.:TheNationalAcademiesPress.

NRC.1993.IssuesinRiskAssessment.Washington,D.C.:TheNationalAcademiesPress.

NRC.1994.ScienceandJudgmentinRiskAssessment.Washington,D.C.:TheNationalAcademiesPress.

NRC.2002.EstimatingthePublicHealthBenefitsofProposedAirPollutionRegulations.Washington,D.C.:TheNationalAcademiesPress.

NRC.2007a.ScientificReviewoftheProposedRiskAssessmentBulletinFromtheOfficeofManagementandBudget.Washington,D.C.:TheNationalAcademiesPress.

NRC.2007b.ModelsinEnvironmentalRegulatoryDecisionMaking.Washington,D.C.:TheNationalAcademiesPress.

NRC.2009.ScienceandDecisions:AdvancingRiskAssessment.Washington,D.C.:TheNationalAcademiesPress.

OSTP(OfficeofScienceandTechnologyPolicy)/OMB(OfficeofManagementandBudget).2007.Memorandum(M‐07‐24)fortheHeadsofExecutiveDepartmentsandAgencies;fromSusanE.Dudley,Administrator,OfficeofInformationandRegulatoryAffairs,OfficeofManagementandBudget,toSharonL.Hays,AssociateDirectorandDeputyDirectorforScience,OfficeofScienceandTechnologyPolicy:UpdatedPrinciplesforRiskAnalysis.September19,2007.

ÖzkaynakH.,H.C.Frey,J.Burke,andR.W.Pinder.2009.“AnalysisofCoupledModelUncertaintiesinSource‐to‐DoseModelingofHumanExposurestoAmbientAirPollution:APM2.5Case‐Study.”AtmosphericEnvironment43:1641–49.

Parkin,R.T.,andM.G.Morgan.2007.ConsultationonEnhancingRiskAssessmentPracticesandUpdatingEPA’sExposureGuidelines.EPA/SAB‐07/003.Washington,D.C.:USEPA.USEPA.2007.LetterFromM.G.MorganandR.T.Parkin,ScienceAdvisoryBoard,toS.Johnson,U.S.EnvironmentalProtectionAgency.February28,2007.EPA/SAB‐07/003.http://yosemite.epa.gov/sab/sabproduct.nsf/55E1B2C78C6085EB8525729C00573A3E/$File/sab‐07‐003.pdf.

PCCRARM(Presidential/CongressionalCommissiononRiskAssessmentandRiskManagement).1997.FrameworkforEnvironmentalHealthRiskManagementFinalReport,Volume1.Washington,D.C.:PCCRARM.http://www.riskworld.com/nreports/1997/risk‐rpt/pdf/EPAJAN.PDF.

Pearl,J.2009.Causality:Models,Reasoning,andInference.2nded.NewYork:CambridgeUniversityPress.

Royall,R.M.1997.StatisticalEvidence:ALikelihoodParadigm.London:Chapman&Hall.

Saltelli,A.,S.Tarantola,F.Campolongo,andM.Rattoet.2004.SensitivityAnalysisinPractice.NewYork:JohnWiley&Sons.

Stahl,C.H.,andA.J.Cimorelli.2005.“HowMuchUncertaintyIsTooMuchandHowDoWeKnow?ACaseExampleoftheAssessmentofOzoneMonitorNetworkOptions.”RiskAnalysis25(5):1109–20.

Stahl,C.H.,andA.J.Cimorelli.2012.“TheDemonstrationoftheNecessityandFeasibilityofUsingaClumsyDecisionAnalyticApproachinWickedEnvironmentalProblems.”IntegratedEnvironmentalAssessmentandManagement9(1):17.

45

Stahl,C.H.,A.J.Cimorelli,C.Mazzarella,andB.Jenkins.2011.“TowardSustainability:ACaseStudyDemonstratingTransdisciplinaryLearning.”IntegratedEnvironmentalAssessmentandManagement7(3):483–98.

Swartout,J.C.,P.S.Price,M.L.Dourson,H.L.Carlson‐Lynch,andR.E.Keenan.1998.“ProbabilisticFrameworkfortheReferenceDose(ProbabilisticRfD).”RiskAnalysis18(3):271–82.

Toll,J.,S.Pavlou,D.Lee,L.Zaragosa,andP.Shelley.1997.“UsingDecisionAnalysisintheManagementofContaminatedSediments(AppendixE).”InContaminatedSedimentsinPortsandWaterways:CleanupStrategiesandTechnologies,bytheCommissiononEngineeringandTechnicalSystemsoftheNationalResearchCouncil,257–84.Washington,D.C.:TheNationalAcademiesPress.

Toll,J.E.1999.“ElementsofEnvironmentalProblem‐Solving.”HumanandEcologicalRiskAssessment5(2):275–80.

USEPA(U.S.EnvironmentalProtectionAgency).1992.GuidelinesforExposureAssessment.EPA/600/Z‐92/001.Washington,D.C.:RiskAssessmentForum,USEPA.http://cfpub.epa.gov/ncea/cfm/recordisplay.cfm?deid=15263.

USEPA.1995a.GuidanceforRiskCharacterization.Washington,D.C.:SciencePolicyCouncil,USEPA.http://www.epa.gov/spc/pdfs/rcguide.pdf.

USEPA.1995b.PolicyforRiskCharacterization.Washington,D.C.:USEPA.http://www.epa.gov/oswer/riskassessment/pdf/1995_0521_risk_characterization_program.pdf.

USEPA.1997a.PolicyforUseofProbabilisticAnalysisinRiskAssessmentattheU.S.EnvironmentalProtectionAgency.Washington,D.C.:USEPA.http://www.epa.gov/spc/pdfs/probpol.pdf.

USEPA.1997b.GuidingPrinciplesforMonteCarloAnalysis.EPA/630/R‐97/001.Washington,D.C.:USEPA.http://www.epa.gov/raf/publications/pdfs/montecar.pdf.

USEPA.1998a.GuidanceforSubmissionofProbabilisticHumanHealthExposureAssessmentstotheOfficeofPesticidePrograms(Draft).Washington,D.C.:USEPA.http://www.epa.gov/fedrgstr/EPA‐PEST/1998/November/Day‐05/6021.pdf.

USEPA.1998b.GuidelinesforEcologicalRiskAssessment.EPA/630/R‐95/002F.Washington,D.C.:RiskAssessmentForum,USEPA.http://www.epa.gov/raf/publications/pdfs/ECOTXTBX.PDF.

USEPA.2000a.EPASciencePolicyCouncilHandbook:RiskCharacterization.EPA/100/B‐00/002.Washington,D.C.:USEPA.http://www.epa.gov/spc/pdfs/rchandbk.pdf.

USEPA.2000b.TowardIntegratingEnvironmentalDecision‐Making.EPA/SAB/EC‐00/011.Washington,D.C.:ScienceAdvisoryBoard,USEPA.

USEPA.2001.RiskAssessmentGuidanceforSuperfund:VolumeIII—PartA,ProcessforConductingProbabilisticRiskAssessment.EPA/540/R‐02/002.Washington,D.C.:USEPA.http://www.epa.gov/oswer/riskassessment/rags3adt/pdf/rags3adt_complete.pdf.

USEPA.2004a.AirToxicsRiskAssessmentReferenceLibrary.VolumeI:TechnicalResourceManual.EPA/453/K‐04/001A.ResearchTrianglePark,NC:USEPA.http://www.epa.gov/ttn/fera/data/risk/vol_1/title_page_volume_1.pdf.

USEPA.2004b.AnExaminationofEPARiskAssessmentPrinciplesandPractices.EPA/100/B‐04/001.Washington,D.C.:USEPA.http://www.epa.gov/osa/pdfs/ratf‐final.pdf.

USEPA.2006a.AFrameworkforAssessingHealthRisksofEnvironmentalExposurestoChildren.EPA/600/R‐05/093F.Washington,D.C.:USEPA.http://ofmpub.epa.gov/eims/eimscomm.getfile?p_download_id=459047.

46

USEPA.2006b.AirToxicsRiskAssessmentReferenceLibrary,Volume3:Community‐ScaleAssessment.EPA/452/K‐06/001C.Washington,D.C.:USEPA.http://www.epa.gov/ttn/fera/risk_atra_vol3.html.

USEPA.2006c.HazardAnalysisCriticalControlPointStrategiesforDistributionSystemMonitoring,HazardAssessmentandControl.Washington,D.C.:USEPA.http://www.epa.gov/ogwdw/disinfection/tcr/pdfs/issuepaper_tcr_haccp‐strategies.pdf.

USEPA.2007a.Concepts,MethodsandDataSourcesforCumulativeHealthRiskAssessmentofMultipleChemicals,ExposuresandEffects:AResourceDocument.EPA/600/R‐06/013F.Washington,D.C.:USEPA.http://cfpub.epa.gov/ncea/risk/recordisplay.cfm?deid=190187.

USEPA.2007b.LetterFromM.G.MorganandR.T.Parkin,ScienceAdvisoryBoard,toS.Johnson,U.S.EnvironmentalProtectionAgency.February28,2007.EPA/SAB‐07/003.http://yosemite.epa.gov/sab/sabproduct.nsf/55E1B2C78C6085EB8525729C00573A3E/$File/sab‐07‐003.pdf.

USEPA.2010a.EnvironmentalMonitoringandAssessmentProgramMasterGlossary.LastmodifiedNovember8.http://www.epa.gov/emap

2/html/pubs/docs/resdocs/mglossary.html#ee.

USEPA.2010b.HumanHealthRiskAssessmentOnondagaLake,LakeBottomSubsite:SedimentConsolidationArea,Camillus,NY.Washington,D.C.:USEPA.http://www.epa.gov/region2/superfund/npl/onondagalake/healthriskassess.pdf.

USEPA.2011a.ExpertElicitationTaskForceWhitePaper.Washington,D.C.:USEPA.http://www.epa.gov/stpc/pdfs/ee‐white‐paper‐final.pdf.

USEPA.2011b.ExposureFactorsHandbook:2011Edition.EPA/600/R‐09/052F.Washington,D.C.:NationalCenterforEnvironmentalAssessment,OfficeofResearchandDevelopment,USEPA.http://cfpub.epa.gov/ncea/risk/recordisplay.cfm?deid=236252.

USEPA.2011c.IntegratedRiskInformationSystemGlossary.LastmodifiedAugust21.http://ofmpub.epa.gov/sor_internet/registry/termreg/searchandretrieve/glossariesandkeywordlists/search.do?details=&glossaryName=IRIS%20Glossary.

USEPA.2012.WasteandCleanupRiskAssessmentGlossary.OfficeofSolidWasteandEmergencyResponse,USEPA.LastmodifiedDecember24.http://www.epa.gov/oswer/riskassessment/glossary.htm.

USEPA.2014a.ProbabilisticRiskAssessmenttoInformDecisionMaking:FrequentlyAskedQuestions.EPA/100/R‐09/001B.Washington,D.C.:RiskAssessmentForum,OfficeoftheScienceAdvisor,USEPA.

USEPA.2014b.FrameworkforHumanHealthRiskAssessmenttoInformDecisionMaking.EPA/100/R‐14/001.Washington,D.C.:RiskAssessmentForum,OfficeoftheScienceAdvisor,USEPA.

Wilson,J.D.2000.“RiskManagementCannotAbideUncertainty.”InJ.H.ExnerandM.L.Trehy(eds.)SymposiapaperspresentedbeforetheDivisionofEnvironmentalChemistry,AmericanChemicalSociety,August20–24,2000.PreprintsofExtendedAbstracts40(2):268–69.Washington,D.C.:AmericanChemicalSociety.

WHO(WorldHealthOrganization).2004.InternationalProgrammeonChemicalSafetyRiskAssessmentTerminology.Geneva:WHO.http://www.who.int/ipcs/methods/harmonization/areas/ipcsterminologyparts1and2.pdf.

47

WHO.2008.UncertaintyandDataQualityinExposureAssessment.Geneva:WHO.http://www.who.int/ipcs/publications/methods/harmonization/exposure_assessment.pdf.

Wright,G.,F.Bolger,andG.Rowe.2002.“AnEmpiricalTestoftheRelativeValidityofExpertandLayJudgmentsofRisk.”RiskAnalysis22(6):1107–22.

48

BIBLIOGRAPHY

Probabilistic Risk Analysis MethodologyGeneral Cullen,A.C.,andH.C.Frey.1999.TheUseofProbabilisticTechniquesinExposureAssessment:AHandbookforDealingwithVariabilityandUncertaintyinModelsandInputs.NewYork:Plenum.

Dakins,M.E.,J.E.Toll,M.J.Small,andK.P.Brand.1996.“Risk‐BasedEnvironmentalRemediation:BayesianMonteCarloAnalysisandtheExpectedValueofSampleInformation.”RiskAnalysis16(1):67–79.

Ferson,S.,andV.Kreinovich.2006.ModelingCorrelationandDependenceAmongIntervals.UniversityofTexasatElPasoDepartmentalTechnicalReports(CS),Paper131.http://digitalcommons.utep.edu/cs_techrep/131.

Finkel,A.M.1990.ConfrontingUncertaintyinRiskManagement:AGuideforDecision‐Makers.Washington,D.C.:ResourcesfortheFuture.

Frey,H.C.,D.Crawford‐Brown,J.Zheng,andD.Loughlin.2003.HierarchyofMethodstoCharacterizeUncertainty:StateofScienceofMethodsforDescribingandQuantifyingUncertainty(Draft).ResearchTrianglePark,NC:USEPA.

Frey,C.,J.Penman,L.Hanle,S.Monni,andS.Ogle.2006.Chapter3,“Uncertainties.”InS.Eggleston,L.Buendia,K.Miwa,T.Ngara,andK.Tanabe(eds.)2006IPCCGuidelinesforNationalGreenhouseGasInventories,VolumeI:GeneralGuidanceandReporting.Kanagawa,Japan:NationalGreenhouseGasInventoriesProgramme,Inter‐GovernmentalPanelonClimateChange.

Hattis,D.1996.“HumanInterindividualVariabilityinSusceptibilitytoToxicEffects—FromAnnoyingDetailtoaCentralDeterminantofRisk.”Toxicology111:5–14.

Hattis,D.,andE.Anderson.1999.“WhatShouldBetheImplicationsofUncertainty,Variability,andInherent‘Biases’/‘Conservatism’forRiskManagementDecisionMaking?”RiskAnalysis19:95–107.

Hattis,D.,andM.K.Lynch.2007.“EmpiricallyObservedDistributionsofPharmacokineticandPharmacodynamicVariabilityinHumans—ImplicationsfortheDerivationofSinglePointComponentUncertaintyFactorsProvidingEquivalentProtectionasExistingRfDs.”InJ.C.LipscombandE.V.Ohanian(eds.)ToxicokineticsinRiskAssessment,69–93.NewYork:InformaHealthcareUSA,Inc.

Hattis,D.2009.“FailuretoCommunicate—Comment:Regulator/RiskPerspectiveonBurzalaandMazzuchi’sPaper.”InR.M.Cooke(ed.).UncertaintyModelinginDoseResponse:BenchTestingEnvironmentalToxicity,153–59.NewYork:JohnWiley&Sons,Inc.

Loughlin,D.,H.C.Frey,K.Hanisak,andA.Eyth.2003.ImplementationRequirementsfortheDevelopmentofaSensitivity/UncertaintyAnalysisToolforMIMS(Draft).ResearchTrianglePark,NC:USEPA.

Morgan,M.G.,andM.Henrion.1990.Uncertainty:AGuidetoDealingwithUncertaintyinQuantitativeRiskandPolicyAnalysis.NewYork:CambridgeUniversityPress.

NARSTO(NorthAmericanResearchStrategyforTroposphericOzone).2005.ImprovingEmissionInventoriesforEffectiveAirQualityManagementAcrossNorthAmerica,ANARSTOAssessment.NARSTO‐05‐001.Pasco,WA:NARSTO.

NRC(NationalResearchCouncil).1994.ScienceandJudgmentinRiskAssessment.Washington,D.C.:TheNationalAcademiesPress.

49

USEPA(U.S.EnvironmentalProtectionAgency).1997.GuidingPrinciplesforMonteCarloAnalysis.EPA/630/R‐97/001.Washington,D.C.:USEPA.http://www.epa.gov/raf/publications/pdfs/montecar.pdf.

Probabilistic Risk Analysis and Decision Making Bernardini,A.,andF.Tonon.2010.BoundingUncertaintyinCivilEngineering:TheoreticalBackground.NewYork:Springer.

Bloom,D.L.,D.M.Byrne,andJ.M.Andreson.1993.CommunicatingRisktoSeniorEPAPolicy‐Makers:AFocusGroupStudy.ResearchTrianglePark,NC:USEPA.

Krupnick,A.,R.Morgenstern,M.Batz,P.Nelson,D.Burtraw,J.‐S.Shih,andM.McWilliams.2006.NotaSureThing:MakingRegulatoryChoicesUnderUncertainty.Washington,D.C.:ResourcesfortheFuture.

McGarity,T.O.,andW.E.Wagner.2003.“LegalAspectsoftheRegulatoryUseofEnvironmentalModeling.”EnvironmentalLawReporter33:10751–74.

NRC.1996.UnderstandingRisk:InformingDecisionsinaDemocraticSociety.Washington,D.C.:TheNationalAcademiesPress.

NRC.2009.ScienceandDecisions:AdvancingRiskAssessment.Washington,D.C.:TheNationalAcademiesPress.

Pascual,P.,N.Steiber,andE.Sunderland.2003.DraftGuidanceontheDevelopment,Evaluation,andApplicationofRegulatoryEnvironmentalModels.Washington,D.C.:USEPA.

Thompson,K.M.,andJ.D.Graham.1996.“GoingBeyondtheSingleNumber:UsingProbabilisticRiskAssessmenttoImproveRiskManagement.”HumanandEcologicalRiskAssessment2(4):1008–34.

USEPA.1997.MemofromFredHansen.UseofProbabilisticTechniques(includingMonteCarloAnalysis)inRiskAssessment,datedMay15,1997.Washington,D.C.:USEPA.

Vick,S.G.2002.DegreesofBelief:SubjectiveProbabilityandEngineeringJudgment.Reston,VA:ASCEPress.

Probabilistic Risk Analysis MethodologySpecific Aspects Abdel‐Aziz,A.,andH.C.Frey.2003.“DevelopmentofHourlyProbabilisticUtilityNOxEmissionInventoriesUsingTimeSeriesTechniques:PartI—UnivariateApproach.”AtmosphericEnvironment37:5379–89.

Abdel‐Aziz,A.,andH.C.Frey.2003.“DevelopmentofHourlyProbabilisticUtilityNOxEmissionInventoriesUsingTimeSeriesTechniques:PartII—MultivariateApproach.”AtmosphericEnvironment37:5391–401.

Abdel‐Aziz,A.,andH.C.Frey.2004.“PropagationofUncertaintyinHourlyUtilityNOxEmissionsThroughaPhotochemicalGridAirQualityModel:ACaseStudyfortheCharlotte,NCModelingDomain.”EnvironmentalScience&Technology38(7):2153–60.

Casman,E.A.,M.G.Morgan,andH.Dowlatabadi.1999.“MixedLevelsofUncertaintyinComplexPolicyModels.”RiskAnalysis19(1):33–42.

50

Crawford‐Brown,D.J.,andK.G.Brown.1987.“AFrameworkforAssessingtheRationalityofJudgmentsinCarcinogenicityHazardIdentification.”Risk:Health,SafetyandEnvironment8:307–37.

Draper,D.1995.“AssessmentandPropagationofModelUncertainty.”JournaloftheRoyalStatisticalSociety57:45–97.

Evans,J.S.,J.D.Graham,G.M.Gray,andR.L.Sielken.1994.“ADistributionalApproachtoCharacterizingLow‐DoseCancerRisk.”RiskAnalysis14(1):25–34.

Frey,H.C.,andD.E.Burmaster.1999.“MethodsforCharacterizingVariabilityandUncertainty:ComparisonofBootstrapSimulationandLikelihood‐BasedApproaches.”RiskAnalysis19(1):109–30.

Frey,H.C.,andD.S.Rhodes.1996.“Characterizing,Simulating,andAnalyzingVariabilityandUncertainty:AnIllustrationofMethodsUsinganAirToxicsEmissionsExample.”HumanandEcologicalRiskAssessment:AnInternationalJournal2(4):762–97.

Frey,H.C.,andY.Zhao.2004.“QuantificationofVariabilityandUncertaintyforAirToxicEmissionInventoriesWithCensoredEmissionFactorData.”EnvironmentalScience&Technology38(22):6094–100.

Landis,W.G.2005.RegionalScaleEcologicalRiskAssessmentUsingtheRelativeRiskModel.BocaRaton,FL:CRCPress.

Maginnis,C.M.2006.AScreeningLevelIntegratedEcologicalandHumanHealthRiskAssessmentforLakeWhatcom,WhatcomCounty,Washington.Master’sThesis,WesternWashingtonUniversity.

Shlyakhter,A.I.1994.“AnImprovedFrameworkforUncertaintyAnalysis:AccountingforUnsuspectedErrors.”RiskAnalysis14:441–47.

Zheng,J.,andH.C.Frey.2004.“QuantificationofVariabilityandUncertaintyUsingMixtureDistributions:EvaluationofSampleSize,MixingWeightsandSeparationbetweenComponents.”RiskAnalysis24(3):553–71.

Zheng,J.,andH.C.Frey.2005.“QuantitativeAnalysisofVariabilityandUncertaintywithKnownMeasurementError:MethodologyandCaseStudy.”RiskAnalysis25(3):663–76.

Sensitivity Analysis Agro,K.E.,C.A.Bradley,andN.Mittmann.1997.“SensitivityAnalysisinHealthEconomicandPharmacoeconomicStudies—anAppraisaloftheLiterature.”Pharmacoeconomics11(1):75–88.

Cukier,R.I.,H.B.Levine,andK.E.Shuler.1978.“NonlinearSensitivityAnalysisofMulti‐parameterModelSystems.”JournalofChemicalPhysics26(1):1–42.

Frey,H.C.,andR.Patil.2002.“IdentificationandReviewofSensitivityAnalysisMethods.”RiskAnalysis22(3):553–77.

Frey,H.C.,A.Mokhtari,andJ.Zheng.2004.RecommendedPracticeRegardingSelection,Application,andInterpretationofSensitivityAnalysisMethodsAppliedtoFoodSafetyProcessRiskModels.Washington,D.C.:U.S.DepartmentofAgriculture.http://www.ce.ncsu.edu/risk/Phase3Final.pdf

Helton,J.C.,andF.J.Davis.2002.“IllustrationofSampling‐BasedMethodsforSensitivityAnalysis.”ReliabilityEngineering&SystemSafety81(1):23–69.

51

Mokhtari,A.,andH.C.Frey.2005.“RecommendedPracticeRegardingSelectionofSensitivityAnalysisMethodsAppliedtoMicrobialFoodSafetyProcessRiskModels.”HumanandEcologicalRiskAssessment11(3):591–605.

Mokhtari,A.,H.C.Frey,andJ.Zheng.2006.“EvaluationandRecommendationofSensitivityAnalysisMethodsforApplicationtoEPA‐StochasticHumanExposureandDoseSimulation(SHEDS)Models.”JournalofExposureScienceandEnvironmentalEpidemiology16(6):491–506.

Saltelli,A.,K.Chan,andM.Scott.2000.SensitivityAnalysis,ProbabilityandStatisticsSeries.NewYork:JohnWiley&Sons.

Sobol,I.M.1993.“SensitivityEstimatesforNonlinearMathematicalModels.”MathematicalModelingandComputation1(4):407–14.

Xue,J.,V.G.Zartarian,H.Özkaynak,W.Dang,G.Glen,L.Smith,andC.Stallings.2006.“AProbabilisticArsenicExposureAssessmentforChildrenWhoContactChromatedCopperArsenate(CCA)‐TreatedPlaysetsandDecks,Part2:SensitivityandUncertaintyAnalyses.”RiskAnalysis26(2):533–41.

Case Study Examples of Probabilistic Risk AnalysisEPA (See Also the PRA Case Studies Appendix) Babendreier,J.E.,andK.L.Castleton.2005.“InvestigatingUncertaintyandSensitivityinIntegratedMultimediaEnvironmentalModels:ToolsforFRAMES‐3MRA.”EnvironmentalModeling&Software20:1043–55.

Blancato,J.N.,F.W.Power,R.N.Brown,andC.C.Dary.2006.ExposureRelatedDoseEstimatingModel(ERDEM):APhysiologically‐BasedPharmacokineticandPharmacodynamic(PBPK/PD)ModelforAssessingHumanExposureandRisk.EPA/600/R‐06/061.ResearchTrianglePark,NC:USEPA.http://www.epa.gov/heasd/products/erdem/237edrb05‐Report.pdf

Burke,J.2005.EPAStochasticHumanExposureandDoseSimulationforParticulateMatter(SHEDS‐PM)User’sGuide.ResearchTrianglePark,NC:USEPA.

Frey,H.C.2003.EvaluationofanApproximateAnalyticalProcedureforCalculatingUncertaintyintheGreenhouseGasVersionoftheMulti‐ScaleMotorVehicleandEquipmentEmissionsSystem.AnnArbor,MI:USEPA.

Frey,H.C.,andY.Zhao.2003.DevelopmentofProbabilisticEmissionInventoriesofBenzene,FormaldehydeandChromiumfortheHoustonDomain.ResearchTrianglePark,NC:USEPA.

Frey,H.C.,R.Bharvirkar,andJ.Zheng.1999.QuantitativeAnalysisofVariabilityandUncertaintyinEmissionsEstimation.ResearchTrianglePark,NC:USEPA.

Frey,H.C.,J.Zheng,Y.Zhao,S.Li,andY.Zhu.2002.TechnicalDocumentationoftheAuvToolSoftwareforAnalysisofVariabilityandUncertainty.ResearchTrianglePark,NC:USEPA.

ÖzkaynakH.,H.C.Frey,J.Burke,andR.W.Pinder.2009.“AnalysisofCoupledModelUncertaintiesinSource‐to‐DoseModelingofHumanExposurestoAmbientAirPollution:APM2.5Case‐Study.”AtmosphericEnvironment43:1641–49.

Zartarian,V.G.,J.Xue,H.A.Özkaynak,W.Dang,G.Glen,L.Smith,andC.Stallings.2005.AProbabilisticExposureAssessmentforChildrenWhoContactCCA‐TreatedPlaysetsandDecksUsingtheStochasticHumanExposureandDoseSimulationModelfortheWoodPreservativeScenario(SHEDS‐WOOD),FinalReport.EPA/600/X‐05/009.Washington,D.C.:USEPA.http://www.epa.gov/heasd/sheds/CCA_all.pdf

52

Zartarian,V.G.,J.Xue,H.Özkaynak,W.Dang,G.Glen,L.Smith,andC.Stallings.2006.“AProbabilisticArsenicExposureAssessmentforChildrenWhoContactCCA‐TreatedPlaysetsandDecks,Part1:ModelMethodology,VariabilityResults,andModelEvaluation.”RiskAnalysis26(2):515–32.

Zheng,J.,andH.C.Frey.2002.AuvToolUser’sGuide.ResearchTrianglePark,NC:USEPA.

Case Study Examples of Probabilistic Risk AnalysisOther

Bloyd,C.,J.Camp,G.Conzelmann,J.Formento,J.Molburg,J.Shannon,M.Henrion,etal.1996.TrackingandAnalysisFramework(TAF)ModelDocumentationandUser’sGuide.ANL/DIS/TM‐36.Washington,D.C.:ArgonneNationalLaboratory.

Frey,H.C.,andS.Li.2003.“QuantificationofVariabilityandUncertaintyinAP‐42EmissionFactors:CaseStudiesforNaturalGas‐FueledEngines.”JournaloftheAir&WasteManagementAssociation53(12):1436–47.

Hanna,S.R.,Z.Lu,H.C.Frey,N.Wheeler,J.Vukovich,S.Arunachalam,M.Fernau,andD.A.Hansen.2001.“UncertaintiesinPredictedOzoneConcentrationsDuetoInputUncertaintiesfortheUAM‐VPhotochemicalGridModelAppliedtotheJuly1995OTAGDomain.”AtmosphericEnvironment35(5):891–903.

Munns,W.R.,Jr.,H.A.Walker,andJ.F.Paul.1989.“AnEcologicalRiskAssessmentFrameworkforExaminingtheImpactsofOceanicDisposalStatusandTrendsProgram.”InOceans‘89,Volume2:OceanPollution,bytheMarineTechnologySocietyandtheOceanicEngineeringSociety(U.S.),664–69.IEEEPublicationNo.89,CH2780‐5.NewYork:IEEEPress.

Munns,W.R.,Jr.,H.A.Walker,J.F.Paul,andJ.H.Gentile.1996.“AProspectiveAssessmentofEcologicalRiskstoUpperWaterColumnPopulationsfromOceanDisposalatthe106‐MileDumpsite.”JournalofMarineEnvironmentalEngineering3:279–97.

Nocito,J.A.,H.A.Walker,J.F.Paul,andC.Menzie.1988.“ApplicationofaRiskAssessmentforMarineDisposalofSewageSludgeatMidshelfandOffshelfSites.”InProceedingsofthe7thInternationalOceanDisposalSymposium,644–63.Ottawa:EnvironmentCanada.

Nocito,J.A.,H.A.Walker,J.F.Paul,andC.A.Menzie.1989.“ApplicationofaRiskAssessmentFrameworkforMarineDisposalofSewageSludgeatMidshelfandOffshelfSites.”InG.W.SuterIIandM.A.Lewis(eds.).AquaticToxicologyandEnvironmentalFate:EleventhVolume,101–20.ASTMSTP1007.Philadelphia,PA:AmericanSocietyforTestingandMaterials.

Paul,J.F.,H.A.Walker,andV.J.Bierman,Jr.1983.“ProbabilisticApproachfortheDeterminationofthePotentialAreaofInfluenceforWasteDisposedatthe106‐MileOceanDisposalSite.”InJ.B.Pearce,D.C.Miller,andC.Berman(eds.).106‐MileWasteDisposalSiteCharacterizationUpdateReport,A–1toA–8.NOAATechnicalMemorandumNMFS‐F/NEC‐26.WoodsHole,MA:NationalMarineFisheriesService.

Paul,J.F.,V.J.Bierman,Jr.,W.R.Davis,G.L.Hoffman,W.R.Munns,C.E.Pesch,P.F.Rogerson,andS.C.Schimmel.1988.“TheApplicationofaHazardAssessmentResearchStrategytotheOceanDisposalofaDredgedMaterial:ExposureAssessmentComponent.”InD.A.WolfeandT.P.O’Connor(eds.).OceanicProcessesinMarinePollution,Volume5,123–35.Melbourne,FL:KreigerPublishingCo.

Prager,J.C.,V.J.Bierman,Jr.,J.F.Paul,andJ.S.Bonner.1986.“SamplingtheOceansforPollution:ARiskAssessmentApproachtoEvaluatingLow‐LevelRadioactiveWasteDisposalatSea.”DangerousPropertiesofIndustrialMaterialsReport6(3):2–26.

53

Walker,H.A.,J.F.Paul,J.A.Nocito,andJ.H.Gentile.1988.“EcologicalandHumanHealthRisksforSewageSludgeDisposalatthe106‐MileSite.”InProceedingsofthe7thInternationalOceanDisposalSymposium,625–43.Ottawa:EnvironmentCanada.

54

APPENDIX: CASE STUDY EXAMPLES OF THE APPLICATION OF PROBABILISTIC RISK ANALYSIS IN U.S.

ENVIRONMENTAL PROTECTION AGENCY REGULATORY DECISION MAKING

55

A. OVERVIEW ThisAppendixfocusesonexamplesofhowprobabilisticriskanalysis(PRA)approacheshavebeenusedatEPAtoinformregulatorydecisions.TheAppendixwaspreparedbyrepresentativesfromvariousEPAprogramofficesandregionscurrentlyinvolvedinthedevelopmentandapplicationofPRAtechniques.TheTechnicalPanelselectedthecasestudyexamplesbasedonthemembers’knowledgeofthespecificPRAprocedures,thetypesoftechniquesdemonstrated,theavailabilitytothereaderthroughtheInternetandtheconditionofhavingbeenpeerreviewed;theyalsowereselectedtobeillustrativeofaspectrumofPRAusedatEPA.ThecasestudiesarenotdesignedtoprovideanexhaustivediscussionofthewidevarietyofapplicationsofPRAusedwithintheAgency,buttohighlightspecificexamplesreflectingtherangeofapproachescurrentlyappliedwithinEPA.

ThisAppendixisintendedtoserveasaresourceformanagersfacedwithdecisionsregardingwhentoapplyPRAtechniquestoinformenvironmentaldecisions,andforexposureandriskassessorswhomaynotbefamiliarwiththewidevarietyofavailablePRAapproaches.ThedocumentoutlinescategoriesofPRAsclassifiedbythecomplexityofanalysistoaidthedecision‐makingprocess.ThisapproachidentifiesvariousPRAtools,whichincludetechniquesrangingfromasimplesensitivityanalysis(e.g.,identificationofkeyexposureparametersordatavisualization)requiringlimitedtime,resourcesandexpertisetodevelop(Group1);toprobabilisticapproaches,includingMonteCarloanalysis,thatprovidetoolsforevaluatingvariabilityanduncertaintyseparatelyandthatrequiremoreresourcesandspecializedexpertise(Group2);tosophisticatedtechniquesofexpertelicitationthatgenerallyrequiresignificantinvestmentofemployeetime,additionalexpertiseandexternalpeerreview(Group3).

ThecasestudiesinthisAppendixusedPRAtechniqueswithinthisrankedframeworktoprovideadditionalinformationformanagers.ThecasestudysummariesareprovidedinaformatdesignedtohighlighthowtheresultsofthePRAswereconsideredindecisionmaking.Thesesummariesincludespecificinformationontheconductoftheanalysesasanaidindeterminingwhattoolsmightbeappropriatetodevelopspecificexposureorriskassessmentsforothersites.

Thecasestudiesrangefromexamplesoflessresource‐intensiveanalyses,whichmightassistinidentifyingkeyexposureparametersortheneedformoredata,tomoredetailedandresource‐intensiveapproaches.Examplesofapplicationsinhumanhealthandecologicalriskassessmentincludetheexposureofchildrentochromatedcopperarsenate(CCA)‐treatedwood,therelationbetweenparticulatesinairandhealth,dietaryexposurestopesticides,modelingsealevelchange,samplingwatersheds,andmodelingbirdandanimalexposures.

B. INTRODUCTION Historically,EPAhasuseddeterministicriskassessments,orpointestimatesofrisk,toevaluatecancerrisksandnoncancerhealthhazardstohigh‐endexposedindividuals(90thpercentileorhigher)andtheaverageexposedindividual(50thpercentile)and,whereappropriate,risksandhazardstopopulations,asrequiredbyspecificenvironmentallaws(USEPA1992a).Theuseofdefaultvaluesforexposureparametersinriskassessmentsprovidesaproceduralconsistencythatallowsriskassessmentstobefeasibleandtractable(USEPA2004).ThemethodstypicallyusedinEPAdeterministicriskassessments(DRA)relyonacombinationofpointvalues―someconservativeandsometypical―yieldingapointestimateofexposurethatisatsomeunknownpointintherangeofpossiblerisks(USEPA2004).

ThisAppendixpresentscasestudiesofPRAconductedbyEPAoverthepast10to15years.TableA‐1summarizesthecasestudiesbytitle,techniquedemonstrated,classificationasahumanhealthriskassessment(HHRA)orecologicalriskassessment(ERA),andtheprogramorregional

56

officeresponsiblefordevelopingthecasestudies.ThisAppendix,providesa“snapshot”oftheutilizationofPRAacrossvariousprogramsinEPA.

C. OVERALL APPROACH TO PROBABILISTIC RISK ANALYSIS AT THE U.S. ENVIRONMENTAL PROTECTION AGENCY

C.1. U.S. Environmental Protection Agency Guidance and Policies on Probabilistic Risk Analysis

ThecasestudiespresentedherebuildontheprinciplesofPRAoutlinedinEPA’s1997PolicyforUseofProbabilisticAnalysisinRiskAssessmentattheU.S.EnvironmentalProtectionAgency(USEPA1997a)andGuidingPrinciplesforMonteCarloAnalysis(USEPA1997b),aswellassubsequentguidancedocumentsondevelopingandusingPRA.GuidancehasbeendevelopedfortheAgencyandindividualprograms.SpecificdocumentsthatrefertotheuseofPRAincludetheRiskAssessmentGuidanceforSuperfund:VolumeIII(USEPA2001);RiskAssessmentForum(RAF)FrameworkforEcologicalRiskAssessment(USEPA1992b);GuidelinesforEcologicalRiskAssessment(USEPA1998);GuidanceforRiskCharacterization(USEPA1995a);PolicyonEvaluatingHealthRiskstoChildren(USEPA1995b);PolicyforUseofProbabilisticAnalysisinRiskAssessment(USEPA1997a);GuidanceonCumulativeRiskAssessment,Part1:PlanningandScoping(USEPA1997c);andRiskCharacterizationHandbook(USEPA2000a);andFrameworkforHumanHealthRiskAssessmenttoInformDecisionMaking(USEPA2014).

Asshownintheindividualcasestudies,therangeandscopeofthePRAwilldependontheoverallobjectivesofthedecisionthattheanalysiswillinform.TheGuidingPrinciplesforMonteCarloAnalysis(USEPA1997b)layoutthegeneralapproachthatshouldbetakeninallcases,beginningwithdefiningtheproblemandscopeoftheassessmenttoselectingthebesttoolsandapproach.TheGuidingPrinciplesalsodescribetheprocessofestimatingandcharacterizingvariabilityanduncertaintyaroundriskestimates.StahlandCimorelli(2005)andtheRiskAssessmentGuidanceforSuperfund:VolumeIII(USEPA2001)highlighttheimportanceofcommunicationbetweentheriskassessorandmanager.StahlandCimorelli(2005)andJamieson(1996)indicatethatitisimportanttodeterminewhetheraparticularlevelofuncertaintyisacceptableornot.Theauthorsalsosuggestthatthisdecisiondependsoncontext,valuesandregulatorypolicy.TheRiskAssessmentGuidanceforSuperfund:VolumeIII(Chapter2andAppendixFinUSEPA2001)describesaprocessfordeterminingtheappropriatelevelofPRAusingarankedapproachfromthelessresource‐andtime‐intensiveapproachestomoresophisticatedanalyses.Furthermore,theRiskAssessmentGuidanceforSuperfund:VolumeIIIoutlinesaprocessfordevelopingaPRAworkplanandachecklistforPRAreviewers(Chapter2andAppendixFinUSEPA2001).ThisguidancealsoprovidesinformationregardinghowtocommunicatePRAresultstodecisionmakersandstakeholders(Chapter6inUSEPA2001).

C.2. Categorizing Case Studies

Therankedapproachusedforcategorizationisaprocessforasystematic,informedprogressiontoincreasinglycomplexriskassessmentmethodsofPRA,whichisoutlinedintheRiskAssessmentGuidanceforSuperfund(USEPA2001).TheuseofcategoriesprovidesaframeworkforevaluatingthevarioustechniquesofPRA.Highercategoriesreflectincreasingcomplexityandoftenwillrequiremoretimeandresources.Highercategoriesalsoreflectincreasingcharacterizationofvariabilityanduncertaintyintheriskestimate,whichmaybeimportantformakingspecificriskmanagementdecisions.Centraltotheapproachisasystematic,informedprogressionusingan

57

iterativeprocessofevaluation,deliberation,datacollection,planningandscoping,development,andupdatestotheworkplanandcommunication.Allofthesestepsfocusondeciding:

1. Whetherornottheriskassessment,initscurrentstate(e.g.,DRA)issufficienttosupportdecisions(i.e.,aclearpathtoexitingtheprocessisavailableateachstep).

2. Iftheassessmentisdeterminedtobeinsufficient,whetherornotprogressiontoahigherlevelofcomplexity(orrefinementofthecurrentanalyses)wouldprovideasufficientbenefittowarranttheadditionaleffortofperformingaPRA.

ThisAppendixgroupscasestudiesaccordingtolevelofeffortandcomplexityoftheanalysisandtheincreasingsophisticationofthemethodsused(TableA‐1).Althougheachgroupgenerallyrepresentsincreasingeffortandcost,thismaynotalwaysbetrue.Thegroupsalsoareintendedtoreflecttheprogressionfromsimpletocomplexanalysisthatisdeterminedbytheinteractiveplanningandscopingeffortsoftheriskassessorsandmanagers.Theuseofparticulartermstodescribethegroups,including“tiers,”wasavoidedduetospecificprogrammaticandregulatoryconnotations.

Group 1 Case Studies Assessmentswithinthisgrouptypicallyinvolveasensitivityanalysisandserveasaninitialscreeningstepintheriskassessment.Sensitivityanalysesidentifyimportantparametersintheassessmentwhereadditionalinvestigationmaybehelpful(KurowickaandCooke2006).Sensitivityanalysiscanbesimpleorinvolvemorecomplexmathematicalandstatisticaltechniques,suchascorrelationandregressionanalysis,todeterminewhichfactorsinariskmodelcontributemosttothevarianceintheriskestimate.

Withinthesensitivityanalyses,arangeoftechniquesisavailable:simple,“back‐of‐the‐envelope”calculations,wheretheriskparametersareevaluatedusingarangeofexposureparameterstodeterminetheparameterthatcontributesmostsignificantlytotherisk(CaseStudy1);analysestoranktherelativecontributionsofvariablestotheoverallrisk(CaseStudy2);anddatavisualizationusinggraphicaltechniquestoarraythedataorMonteCarlosimulations(e.g.,scatterplots).

Moresophisticatedanalysesmayincludesensitivityratios(e.g.,elasticity);sensitivityscores(e.g.,weightedsensitivityratios);correlationcoefficientorcoefficientofdetermination;r2(e.g.,Pearsonproductmoment,Spearmanrank);normalizedmultipleregressioncoefficients;andgoodness‐of‐fittestsforsubsetsoftheriskdistribution(USEPA2001).

Thesensitivityanalysestypicallyrequireminimalresourcesandtime.ResultsofthesensitivityanalysesareusefulinidentifyingkeyparameterswhereadditionalGroup2orGroup3analysesmaybeappropriate.Sensitivityanalysesalsoarehelpfulinidentifyingkeyparameterswhereadditionalresearchwillhavethehighestimpactontheriskassessment.

Group 2 Case Studies Casestudieswithinthisgroupincludeamoresophisticatedapplicationofprobabilistictools,includingPRAofspecificexposureparameters(CaseStudies3and4),one‐dimensionalanalyses(CaseStudy5)andprobabilisticsensitivityanalysis(CaseStudies6and7).

TheGroup2casestudiesrequirelargertimecommitmentsfordevelopment,specializedexpertiseandadditionalanalysisofexposureparameterdatasources.Dependingonthenatureoftheanalysis,peerinvolvementorpeerreviewmaybeappropriatetoevaluatetheproductsoftheanalysis.

58

Group 3 Case Studies Assessmentswithinthisgrouparethemostresource‐andtime‐intensiveanalysesofthethreecategories.Riskanalysesincludetwo‐dimensionalMonteCarloanalysis(2‐DMCA)thatevaluatesmodelvariabilityanduncertainty(CaseStudies8,9and10);microexposureeventanalysis(MEE),inwhichlong‐termexposureofanindividualissimulatedasthesumofseparateshort‐termexposureevents(CaseStudy11);andprobabilisticanalysis(CaseStudies12and13).

Othertypesofanalyseswithinthisgroupincludetheexpertelicitationmethodthatisasystematicprocessofformalizingandquantifying,intermsofprobabilities,experts’judgmentsaboutuncertainquantities(CaseStudies14and15);Bayesianstatistics,whichisaspecializedbranchofstatisticsthatviewstheprobabilityofaneventoccurringasthedegreeofbelieforconfidenceinthatoccurrence(CaseStudy16);andgeostatisticalanalysis,whichisanotherspecializedbranchofstatisticsthatexplicitlytakesintoaccountthegeo‐referencedcontextofthedataandtheinformation(e.g.,attributes)attachedtothedata.

TheGroup3analysesrequireadditionaltimeandexpertiseintheplanningandanalysisoftheassessment.Withinthisgroup,thelevelofexpertiseandresourcecommitmentsmayvarywiththetechniques.Expertelicitation,forexample,requiressignificantlymoretimeforplanning,identificationofexpertsandmeetings,whencomparedwiththeothertechniques.

 

59

Table A‐1. Case Study Examples of EPA Applications of Deterministic and Probabilistic Risk Assessment Techniques  

Case Study Number

Title and Case Study Description Type of Risk Assessment

Office/Region

Group 1: Point Estimate—Sensitivity Analysis

1

Sensitivity Analysis of Key Variables in Probabilistic Assessment of Children’s Exposure to Arsenic in Chromated Copper Arsenate (CCA) Pressure-Treated Wood. This case study demonstrates use of a point estimate sensitivity analysis to identify exposure variables critical to the analysis summarized in Case Study 9. The sensitivity analysis identified critical areas for future research and data collection and better characterized the amount of dislodgeable residue that exists on the wood surface.

Human Health

Office of Research and Development (ORD) and Office of Pesticide Programs (OPP)

2

Assessment of the Relative Contribution of Atmospheric Deposition to Watershed Contamination. An example of a workbook that demonstrates how “back-of-the-envelope” analysis of potential exposure rates can be used to target resources to identify other inputs before further analysis of air inputs in watershed contamination. Identification of key variables aided in identifying uncertainties and data gaps to target resource expenditures for further analysis. A case study example of the application of this technique also is identified.

Ecological ORD

Group 2: Probabilistic Risk Analysis, One-Dimensional Monte Carlo Analysis (1-D MCA) and Probabilistic Sensitivity Analysis

Group 2: Probabilistic Risk Analysis

3

Probabilistic Assessment of Angling Duration Used in the Assessment of Exposure to Hudson River Sediments via Consumption of Contaminated Fish. A probabilistic analysis of one parameter in an exposure assessment―the time an individual fishes in a large river system. Development of site-specific information regarding exposure, with an existing data set for this geographic area, was needed to represent this exposed population. This information was used in the one-dimensional PRA described in Case Study 5.

Human Health

Superfund/ Region 2 (New York)

4

Probabilistic Analysis of Dietary Exposure to Pesticides for Use in Setting Tolerance Levels. The probabilistic Dietary Exposure Evaluation Model (DEEM) provides more accurate information on the range and probability of possible exposures.

Human Health OPP

60

Table A‐1. Case Study Examples of EPA Applications of Deterministic and Probabilistic Risk Assessment Techniques  

Case Study Number

Title and Case Study Description Type of Risk Assessment

Office/Region

Group 2: One-Dimensional Monte Carlo Analysis (1-D MCA)

5

One-Dimensional Probabilistic Risk Analysis of Exposures to Polychlorinated Biphenyls (PCBs) via Consumption of Fish From a Contaminated Sediment Site. An example of a one-dimensional PRA (1-D MCA) of the variability of exposure as a function of the variability of individual exposure factors to evaluate the risks to anglers who consume recreationally caught fish from a PCB-contaminated river.

Human Health

Superfund/ Region 2

(New York)

Group 2: Probabilistic Sensitivity Analysis

6

Probabilistic Sensitivity Analysis of Knowledge Elicitation of the Concentration-Response Relationship Between PM2.5 Exposure and Mortality. An example of how the probabilistic analysis tools can be used to conduct a probabilistic sensitivity analysis following an expert elicitation (Group 3) presented in Case Study 14.

Human Health Office of Air

and Radiation (OAR)

7

Environmental Monitoring and Assessment Program (EMAP): Using Probabilistic Sampling Techniques to Evaluate the Nation’s Ecological Resources. A probability-based sampling program designed to provide unbiased estimates of the condition of an aquatic resource over a large geographic area based on a small number of samples.

Ecological ORD

Group 3: Advanced Probabilistic Risk Analysis―Two-Dimensional Monte Carlo Analysis (2-D MCA) Including Microexposure Modeling, Bayesian Statistics, Geostatistics and Expert Elicitation

Group 3: Two-Dimensional Probabilistic Risk Analysis

8

Two-Dimensional Probabilistic Risk Analysis of Cryptosporidium in Public Water Supplies, With Bayesian Approaches to Uncertainty Analysis. An analysis of the variability in the occurrence of Cryptosporidium in raw water supplies and in the treatment efficiency, as well as the uncertainty in these inputs. This case study includes an analysis of the dose-response relationship for Cryptosporidium infection.

Human Health Office of

Water (OW)

9

Two-Dimensional Probabilistic Model of Children’s Exposure to Arsenic in Chromated Copper Arsenate (CCA) Pressure-Treated Wood. A two-dimensional model that addresses both variability and uncertainty in the exposures of children to CCA pressure-treated wood. The analysis was built on the sensitivity analysis described in Case Study 2.

Human Health OPP/ORD

61

Table A‐1. Case Study Examples of EPA Applications of Deterministic and Probabilistic Risk Assessment Techniques  

Case Study Number

Title and Case Study Description Type of Risk Assessment

Office/Region

10

Two-Dimensional Probabilistic Exposure Assessment of Ozone. A probabilistic exposure assessment that addresses short-term exposures to ozone. Population exposure to ambient ozone levels was evaluated using EPA’s Air Pollutants Exposure (APEX) model, also referred to as the Total Risk Integrated Methodology/Exposure (TRIM.Expo) model.

Human Health OAR

Group 3: Microexposure Event Analysis

11

Analysis of Microenvironmental Exposures to Fine Particulate Matter (PM2.5) for a Population Living in Philadelphia, Pennsylvania. A microexposure event analysis to simulate individual exposures to PM2.5 in specific microenvironments, including the outdoors, indoor residences, offices, schools, stores and a vehicle.

Human Health Region 3

(Philadelphia) and ORD

Group 3: Probabilistic Analysis

12

Probabilistic Analysis in Cumulative Risk Assessment of Organophosphorus Pesticides. A probabilistic computer software program used to integrate various pathways, while simultaneously incorporating the time dimensions of the input data to calculate margins of exposure.

Human Health OPP

13

Probabilistic Ecological Effects Risk Assessment Models for Evaluating Pesticide Uses. A multimedia exposure/ effects model that evaluates acute mortality levels in generic or specific avian species over a user-defined exposure window.

Ecological OPP

Group 3: Expert Elicitation and Bayesian Belief Network

14

Expert Elicitation of Concentration-Response Relationship Between Fine Particulate Matter (PM2.5) Exposure and Mortality. A knowledge elicitation used to derive probabilistic estimates of the uncertainty in one element of a cost-benefit analysis used to support the PM2.5 regulations.

Human Health ORD/ OAR

15

Expert Elicitation of Sea-Level Change Resulting From Global Climate Change. An example of a PRA that describes the probability of sea level rise and parameters that predict sea level change.

Ecological

Office of Policy,

Planning, and Evaluation

62

Table A‐1. Case Study Examples of EPA Applications of Deterministic and Probabilistic Risk Assessment Techniques  

Case Study Number

Title and Case Study Description Type of Risk Assessment

Office/Region

16

Knowledge Elicitation for Bayesian Belief Network Model of Stream Ecology. An example of a Bayesian belief network model of the effect of increased fine-sediment load in a stream on macroinvertebrate populations.

Ecological ORD

63

D. CASE STUDY SUMMARIES D.1. Group 1 Case Studies

Case Study 1: Sensitivity Analysis of Key Variables in Probabilistic Assessment of Children’s Exposure to Arsenic in Chromated Copper Arsenate Pressure-Treated Wood ThiscasestudyprovidesanexampleoftheapplicationofsensitivityanalysistoidentifyimportantvariablesforpopulationexposurevariabilityforaGroup2assessment(CaseStudy9)andtoindicateareasforfurtherresearch.Specifically,EPA’sOfficeofResearchandDevelopment(ORD),incollaborationwiththeOfficeofPesticidePrograms(OPP),usedsensitivityanalysestoidentifythekeyvariablesinchildren’sexposuretoCCA‐treatedwood.

Approach.Thesensitivityanalysesusedtwoapproaches.Thefirstapproachestimatedbaselineexposurebyrunningtheexposuremodelwitheachinputvariablesettoitsmedian(50thpercentile)value.Next,alternativeexposureestimatesweremadebysettingeachinputtoits25thor75thpercentilevaluewhileholdingallotherinputsattheirmedianvalues.Theratiooftheexposureestimatecalculatedwhenaninputwasestimatedatits25thor75thpercentiletotheexposureestimatecalculatedwhentheinputwasatitsmedianvalueprovidedameasureofthatinput’simportancetotheoverallexposureassessment.Thesecondapproachappliedamultiplestepwiseregressionanalysistothedatapointsgeneratedfromthefirstapproach.Thecorrelationbetweentheinputvariablesandtheexposureestimatesprovidedanalternativemeasureoftheinputvariable’srelativeimportanceintheexposureassessment.Thesetwoapproacheswereusedintandemtoidentifythecriticalinputstotheexposureassessmentmodel.

ResultsofAnalysis.Thetwosensitivityanalysestogetheridentifiedsixcriticalinputvariablesthatmostinfluencedtheexposureassessment.Thecriticalinputvariableswere:woodsurfaceresidue‐to‐skintransferefficiency,woodsurfaceresiduelevels,fractionofhandsurfaceareamouthedpermouthingevent,averagefractionofnonresidentialoutdoortimespentplayingonaCCA‐treatedplayset,frequencyofhandwashingandfrequencyofhand‐to‐mouthactivity.

ManagementConsiderations.Theresultsofthesensitivityanalyseswereusedtoidentifythemostimportantinputparametersinthetreatedwoodriskassessments.Theprocessalsoidentifiedcriticalareasforfutureresearch.Inparticular,theassessmentpointedtoaneedtocollectdataontheamountofdislodgeableresiduethatistransferredfromthewoodsurfacetoachild’shanduponcontact,andtobettercharacterizetheamountofdislodgeableresiduethatexistsonthewoodsurface.

SelectedReferences.ThefinalreportontheprobabilisticexposureassessmentofCCA‐treatedwood:

Zartarian,V.G.,J.Xue,H.A.Özkaynak,W.Dang,G.Glen,L.Smith,andC.Stallings.AProbabilisticExposureAssessmentforChildrenWhoContactCCA‐TreatedPlaysetsandDecksUsingtheStochasticHumanExposureandDoseSimulationModelfortheWoodPreservativeScenario(SHEDS‐WOOD),FinalReport.EPA/600/X‐05/009.Washington,D.C.:USEPA.

Seealso:Xue,J.,V.G.Zartarian,H.Özkaynak,W.Dang,G.Glen,L.Smith,andC.Stallings.2006.“AProbabilisticExposureAssessmentforChildrenWhoContactChromatedCopperArsenate(CCA)‐TreatedPlaysetsandDecks,Part2:SensitivityandUncertaintyAnalyses.”RiskAnalysis26:533–41.

64

Case Study 2: Assessment of the Relative Contribution of Atmospheric Deposition to Watershed Contamination Watershedcontaminationcanresultfromseveraldifferentsources,includingthedirectreleaseofpollutionintoawaterbody,inputfromupstreamwaterbodiesanddepositionfromairbornesources.Effortstocontrolwaterbodycontaminationbeginwithananalysisoftheenvironmentalsourcestoidentifytheparametersthatprovidethegreatestcontributionanddeterminewheremitigationand/oranalysisresourcesshouldbedirected.

Approach.Thiscasestudyprovidesanexampleofa“back‐of‐the‐envelope”deterministicanalysisofthecontributionofairdepositiontooverallwatershednitrogen?Nutrient?contaminationtoidentifyuncertaintiesand/ordatagaps,aswellastotargetresourceexpenditures.NitrogeninputshavebeenstudiedinseveraleastandGulfCoastestuariesduetoconcernsabouteutrophication.Nitrogenfromatmosphericdepositionisestimatedtobeashighas10to40percentofthetotalinputofnitrogentomanyoftheseestuaries,andperhapshigherinafewcases.Forawatershedthathasnotbeenstudiedyet,aback‐of‐the‐envelopecalculationcouldbepreparedbasedoninformationaboutthenitrogendepositionratesmeasuredinasimilararea.Toestimatethedepositionloaddirectlytothewaterbody,onewouldmultiplythenitrogendepositionratebytheareaofthewaterbody.Theanalystthencouldestimatethenitrogenloadfromothersources(e.g.,pointsourcedischargesandrunoff)toestimateatotalnitrogenloadforthewaterbody.Theestimateofloadingduetoatmosphericdepositionthencouldbedividedbythetotalnitrogenloadforthewaterbodytoestimatethepercentofcontributiondirectlytothewaterbodyfromatmosphericdeposition.

TheMay2003reportbytheCascoBayAirDepositionStudyTeamtitledEstimatingPollutantLoadingFromAtmosphericDepositionUsingCascoBay,MaineasaCaseStudyisananalysisusingthemethodologydescribedabove.TheCascoBayEstuary,locatedinsouthwesternMaine,isusedasacasestudy.ThepaperalsoincludestheresultsofafieldairdepositionmonitoringprogramconductedinCascoBayfrom1998to2000andfavorablycomparestheestimatesdevelopedfortherateofdepositionofnitrogen,mercuryandpolycyclicaromatichydrocarbons(PAHs)tothefieldmonitoringresults.Theestimationapproachisausefulstartingpointforunderstandingthesourcesofpollutantsenteringwaterbodiesthatcannotbeaccountedforthroughrunofforpointsourcedischarges.

ResultsofAnalysis.TheapproachoutlinedabovewasappliedtotheCascoBayEstuaryinMaine.Resources,toolsandstrategiesforpollutionabatementcanbeeffectivelytargetedatprioritysourcesifestuariesaretobeprotected.Understandingthesourcesandannualloadingofcontaminantstoanestuaryfacilitatesgoodwaterqualitymanagementbydefiningtherangeofcontrolsofbothairandwaterpollutionneededtoachieveadesiredresult.Thecostofconductingmonitoringtodetermineatmosphericloadingtoawaterbodycanbeprohibitivelyhigh.Also,collectionofmonitoringdataisalong‐termundertakingbecauseaminimumof3yearsofdataisadvisableto“smoothout”inter‐annualvariability.Theestimationtechniquesdescribedinthispapercanserveasausefulandinexpensive“first‐cut”atunderstandingtheimportanceoftheatmosphereasapollutionsourceandcanhelptoidentifyareaswherefieldmeasurementsareneededtoguidefuturemanagementdecisions.

ManagementConsiderations.Ifareviewofinformationonairdepositionavailablefortheanalysisindicatesawiderangeofpotentialdepositionrates,furtherstudyofthisinputwouldleadtobettercharacterizationoftheaircontributiontooverallcontamination.Iftheback‐of‐theenvelopeanalysissuggeststhatairdepositionisverysmallrelativetootherinputs,thenresourcesshouldbetargetedatstudyingorreducingotherinputsbeforeproceedingwithfurtheranalysisoftheairinputs.

65

SelectedReferences.Theback‐of‐the‐envelopecalculationisoutlinedinFrequentlyAskedQuestionsaboutAtmosphericDeposition:AHandbookforWatershedManagers.http://www.epa.gov/air/oaqps/gr8water/handbook/airdep_sept.pdf.

FurtheranalysisisavailableinDepositionofAirPollutantstotheGreatWaters—ThirdReporttoCongress.http://www.epa.gov/air/oaqps/gr8water/3rdrpt/index.html.

TheCascoBayEstuaryexampleisavailableathttp://epa.gov/owow/airdeposition/cascobay.pdf.

D.2. Group 2 Case Studies

Case Study 3: Probabilistic Assessment of Angling Duration Used in the Assessment of Exposure to Hudson River Sediments via Consumption of Contaminated Fish InassessingthehealthimpactofcontaminatedSuperfundsites,exposuredurationtypicallyisassumedtobethesameasthelengthoftimethatanindividuallivesinaspecificarea(i.e.,residenceduration).InconductingtheHHRAfortheHudsonRiverPolychlorinatedBiphenyl(PCB)SuperfundSite,however,therewasconcernthatexposuredurationbasedonresidencedurationmayunderestimatethetimespentfishing(i.e.,anglingduration).

RiskAnalysis.Anindividualmaymovefromoneresidencetoanotherandcontinuetofishinthesamelocation,oranindividualmaychoosetostopfishingirrespectiveofthelocationofhisorherhome.EPARegion2developedasite‐specificdistributionofanglingdurationusingthefishingpatternsreportedinaNewYorkState‐wideanglingsurvey(Connellyetal.1990)andmigrationdataforthefivecountiessurroundingmorethan40milesoftheUpperHudsonRivercollectedaspartoftheU.S.Census.

ResultsofAnalysis.The50thand95thpercentilevaluesfromthedistributionofanglingdurationswerehigherthanthedefaultvaluesbasedonresidencedurationusingstandarddefaultexposureassumptionsforresidentialscenarios.Thesevalueswereusedasabaseforthecentraltendencyandreasonablemaximumexposurepointestimates,respectively,inthedeterministicassessment.

ManagementConsiderations.Theinformationprovidedinthisanalysiswasusedinthepointestimateanalysis.ThefulldistributionwasusedinconductingaGroup2PRAforthefishconsumptionpathway,whichispresentedasCaseStudy5.

SelectedReferences.ThefinalriskassessmentwasreleasedinNovember2000andisavailableathttp://www.epa.gov/hudson/reports.htm.

Furtherinformation,includingEPA’sJanuary2002responsetocommentsontheriskassessment,isavailableathttp://www.epa.gov/hudson/ResponsivenessSummary.pdf.

Case Study 4: Probabilistic Analysis of Dietary Exposure to Pesticides for Use in Setting Tolerance Levels UndertheFederalFood,Drug,andCosmeticAct(FFDCA),EPAmayauthorizeatoleranceorexemptionfromtherequirementofatolerancetoallowapesticideresidueinfood,onlyiftheAgencydeterminesthatsuchresidueswouldbe“safe.”Thisdeterminationismadebyestimatingexposuretothepesticideandcomparingtheestimatedexposuretoatoxicologicalbenchmarkdose.Until1998,theOPPusedasoftwareprogramcalledtheDietaryRiskEvaluationSystem(DRES)toconductitsacutedietaryriskassessmentsforpesticideresiduesinfoods.AcuteassessmentsconductedwithDRESassumedthat100percentofagivencropwithregistereduseofapesticide

66

wastreatedwiththatpesticideandallsuchtreatedcropitemscontainedpesticideresiduesatthemaximumlegal(tolerance)level,matchingthistoareasonablyhighconsumptionvalue(aroundthe95thpercentile).TheresultingDRESacuteriskestimateswereconsidered“high‐end”or“bounding”estimates.Itwasnotpossible,however,toknowwherethepesticideexposureestimatesfromtheDRESsoftwarefitintheoveralldistributionofexposuresduetothelimitsofthetoolsbeingused.

Toaddressthesedeficiencies,OPPdevelopedanacuteprobabilisticdietaryexposureguidancetouseamodelthatwouldestimatetheexposuretopesticidesinthefoodsupply.Ratherthanthecrude“high‐end,”single‐pointestimatesprovidedbydeterministicassessments,theprobabilisticDietaryExposureEvaluationModel(DEEM)providesspecificinformationabouttherangeandprobabilityofpossibleexposures.Dependingonthecharacterizationoftheinput,thiscouldincludethe95thpercentileregulation—generallyforlowertiersthatdonotincludethepercentofcroptreated—tothe99.9thpercentileforthemorerefinedassessments,whichwouldincludethepercentofcroptreatedinformation.

ProbabilisticAnalysis.Thiscasestudyprovidesanexampleofaone‐dimensionalPRAofdietaryexposuretopesticides(Group2).TheDEEMgeneratesacute,probabilisticdietaryexposureassessmentsusingdataon:(1)thedistributionofdailyconsumptionofspecificcommodities(e.g.,wheat,cornandapples)byspecificindividuals;and(2)thedistributionofconcentrationsofaspecificpesticideinthosefoodcommodities.DataoncommodityconsumptionarecollectedbytheU.S.DepartmentofAgriculture(USDA)initsContinuingSurveyofFoodIntakebyIndividuals(CSFII).Pesticideresidueconcentrationsonfoodcommoditiesaregenerallyobtainedfromcropfieldtrials,USDA’sPesticideDataProgram(PDP),U.S.FoodandDrugAdministration(FDA)monitoringdata,ormarketbasketsurveys.Usingthesedata,DEEMisabletocalculateanestimateoftherisktothegeneralU.S.population,inadditionto26populationsubgroups,including5subgroupsforinfantsandchildren(infantslessthan1,children1to2,children3to5,youths6to12andteens13to19yearsofage).

ResultsofAnalysis.DEEMhasbeenusedinriskassessmentstosupporttolerancelevelsforseveralpesticides(e.g.,phosalone)andaspartofcumulativeriskassessmentsfororganophosphoruscompounds(seeCaseStudy12)andotherpesticides.

ManagementConsiderations.UsingtheDRES,decisionswerebeingmadewithoutacompleterepresentationofthedistributionofriskamongthepopulationandwithoutfullknowledgeofwhereinthedistributionofrisktheDRESriskestimatelay.Thiswasofconcernnotonlyforregulatorsinterestedinpublichealthprotection,butalsoforthepesticideregistrantswhocouldarguethattheAgencywasarbitrarilyselectingthelevelatwhichtoregulate.FormostcasesreviewedbyOPPtodate,estimatedexposureatthe99.9thpercentilecalculatedbyDEEMprobabilistictechniquesissignificantlylowerthanexposurecalculatedusingDRES‐typedeterministicassumptionsattheunknownpercentile.

SelectedReferences.AlinktotheDEEMmodelisavailableathttp://www.epa.gov/pesticides/science/deem/index.html.

Case Study 5: One-Dimensional Probabilistic Risk Analysis of Exposure to Polychlorinated Biphenyls via Consumption of Fish From a Contaminated Sediment Site EPARegion2conductedapreliminarydeterministicHHRAattheHudsonRiverPCBsSuperfundsite.TheDRAdemonstratedthatconsumptionofrecreationallycaughtfishprovidedthehighest

67

exposureamongrelevantexposurepathways,whichresultedincancerrisksandnoncancerhealthhazardsthatexceededregulatorybenchmarks.

ProbabilisticAnalysis.Becauseofthesize,complexityandhighlevelofpublicinterestinthissite,EPARegion2implementedaGroup2probabilisticassessmenttocharacterizethevariabilityinrisksassociatedwiththefishconsumptionexposurepathway.Theanalysiswasaone‐dimensionalMonteCarloanalysis(1‐DMCA)ofthevariabilityofexposureasafunctionofthevariabilityofindividualexposurefactors.Uncertaintywasassessedusingsensitivityanalysisoftheinputvariables.Datatocharacterizethedistributionsofexposureparametersweredrawnfromthepublishedliterature(e.g.,fishconsumptionrate)orfromexistingdatabases,suchastheU.S.Censusdata(e.g.,anglingduration,seeCaseStudy3).Mathematicalmodelsoftheenvironmentalfate,transportandbioaccumulationofPCBsintheHudsonRiverpreviouslydevelopedwereusedtoforecastchangesinPCBconcentrationovertime.

ResultsofAnalysis.TheresultsofthePRAwereconsistentwiththedeterministicresults.Forthecentraltendencyindividual,pointestimateswerenearthemedian(50thpercentile).Forthereasonablemaximumexposure(RME)individual,pointestimatevalueswereatorabovethe95thpercentileoftheprobabilisticanalysis.TheDRAandPRAwerethesubjectofaformalpeerreviewbyapanelofindependentexperts.

TheMonteCarlo‐basedcasescenarioistheonefromwhichpointestimateexposurefactorsforfishingestionweredrawn;thus,thepointestimateRMEsandtheMonteCarlo‐basedcaseestimatescanbecompared.Similarly,thepointestimatecentraltendency(average)andtheMonteCarlo‐basedcasemidpoint(50thpercentile)arecomparable.Forcancerrisk,thepointestimateRMEforfishingestion(1x10‐3)fallsapproximatelyatthe95thpercentilefromtheMonteCarlo‐basedcaseanalysis.Thepointestimatecentraltendencyvalue(3x10‐5)andtheMonteCarlo‐basedcase50thpercentilevalue(6x10‐5)aresimilar.Fornoncancerhealthhazards,thepointestimateRMEforfishingestion(104forayoungchild1to6yearsofage)fallsbetweenthe95thand99thpercentilesoftheMonteCarlo‐basedcase.Thepointestimatecentraltendencyhazardindex(HI;12forayoungchild)isapproximatelyequaltothe50thpercentileoftheMonteCarlo‐basedcaseHIof11.

FiguresA‐1andA‐2provideacomparisonofresultsfromtheprobabilisticanalysiswiththatoftheDRAforcancerrisksandnoncancerhealthhazards.FiguresA‐1andA‐2plotpercentilesfor72combinationsofexposurevariables(e.g.,distributionsfromcreelanglersurveys’residenceduration,fishinglocationsandcookinglosses)ofthenoncancerHIvaluesandthecancerrisks,respectively.Ineachofthesefigures,thevariabilityofcancerriskornoncancerHIsforanglerswithintheexposedpopulationisplottedonthey‐axisforparticularpercentileswithinthepopulation.Thisvariabilityisafunctionofthevariationsinfishconsumptionrates,fishingduration,differencesinfishspeciesingestedandsoforth.TheuncertaintyintheestimatesisindicatedbytherangeofeithercancerriskornoncancerHIvaluesplottedonthex‐axis.Thisuncertaintyisafunctionofthe72combinationsoftheexposurefactorinputsexaminedinthesensitivityanalysis.Thisanalysisprovidesasemi‐quantitativeconfidenceintervalforthecancerrisksandHIvaluesatanyparticulatepercentile.Asthesefiguresshow,theintervalsspansomewhatlessthantwoordersofmagnitude(e.g.,<100‐fold).Theverticallinesindicatethedeterministicendpoints.

ManagementConsiderations.Earlyandcontinuedinvolvementofthecommunityimprovedpublicacceptanceoftheresults.Inaddition,carefulconsiderationofthemethodsusedtopresenttheprobabilisticresultstothepublicleadtogreaterunderstandingofthefindings.

SelectedReferences.ThefinalriskassessmentwasreleasedinNovember2000andisavailableathttp://www.epa.gov/hudson/reports.htm.

68

Furtherinformation,includingEPA’sJanuary2002responsetocommentsontheriskassessment,isavailableathttp://www.epa.gov/hudson/ResponsivenessSummary.pdf.

 Figure A‐1. Monte Carlo Cancer Summary Based on a One‐Dimensional Probabilistic Risk Analysis of Exposure to Polychlorinated Biphenyls. The estimated cancer rate was calculated based on the consumption of fish from a contaminated sediment site. Source: USEPA 2000b. 

  Figure A‐2. Monte Carlo Noncancer Hazard Index Summary Based on a One‐Dimensional Probabilistic Risk Analysis of Exposure to Polychlorinated Biphenyls. The incremental individual hazard index (HI) was calculated based on the consumption of fish from a contaminated sediment site. Source: USEPA 2000b. 

Fraction of Anglers

With HI ≤

Indicated Value

Fraction of Anglers

With Risk ≤

Indicated Value

69

Case Study 6: Probabilistic Sensitivity Analysis of Expert Elicitation of Concentration-Response Relationship Between Fine Particulate Matter Exposure and Mortality In2002,theNationalResearchCouncil(NRC)recommendedthatEPAimproveitscharacterizationofuncertaintyinthebenefitsassessmentforproposedregulationsofairpollutants.NRCrecommendedthatprobabilitydistributionsforkeysourcesofuncertaintybedevelopedusingavailableempiricaldataorthroughformalelicitationofexpertjudgments.Inresponsetothisrecommendation,EPAconductedanexpertelicitationevaluationoftheconcentration‐responserelationshipbetweenfineparticulatematter(PM2.5)exposureandmortality,akeycomponentofthebenefitsassessmentofthePM2.5regulation.FurtherinformationontheexpertelicitationprocedureandresultsisprovidedinCaseStudy14.Toevaluatethedegreetowhichtheresultsoftheassessmentdependedonthejudgmentsofindividualexpertsoronthemethodsofexpertelicitation,aprobabilisticsensitivityanalysiswasperformedontheresults.

ProbabilisticAnalysis.Theexpertelicitationprocedureusedcarefullyconstructedinterviewstoelicitfromeachof12expertsanestimateoftheprobabilisticdistributionfortheaverageexpecteddecreaseinU.S.annual,adult,all‐causemortalityassociatedwitha1microgrampercubicmeter(μg/m3)decreaseinannualaveragePM2.5levels.Thiscasestudyprovidesanexampleoftheuseofprobabilisticsensitivityanalysis(Group2)asoneelementoftheoverallassessment.Forthesensitivityanalysis,asimplifiedhealthbenefitsanalysiswasconductedtoassessthesensitivityoftheresultstotheresponsesofindividualexpertsandtothreefactorsinthestudydesign:(1)theuseofparametricornonparametricapproachesbyexpertstocharacterizetheiruncertaintyinthePM2.5mortalitycoefficient;(2)participationinthePre‐ElicitationWorkshop;and(3)allowingexpertstochangetheirjudgmentsafterthePost‐ElicitationWorkshop.Theindividualquantitativeexpertjudgmentswereusedtoestimateadistributionofbenefits,intheformofthenumberofdeathsavoided,associatedwithareductioninambient,annualaveragePM2.5concentrationsfrom12to11μg/m3.The12individualdistributionsofestimatedavoideddeathswerepooledusingequalweightstocreateasingleoveralldistributionreflectinginputfromeachexpert.Thisdistributionservedasthebaselineforthesensitivityanalysis,whichcomparedthemeansandstandarddeviationsofthebaselinedistributionwithseveralvariants.

ResultsofAnalysis.Thefirstanalysisexaminedthesensitivityofthemeanandstandarddeviationoftheoverallmortalitydistributiontotheremovalofindividualexperts’distributions.Ingeneral,theresultssuggestedafairlyequaldivisionbetweenthoseexpertswhoseremovalshiftedthedistributionmeanupandthosewhoshifteditdown.Therewererelativelymodestimpactsofindividualexperts.Thestandarddeviationofthecombineddistributionalsowasnotaffectedstronglybytheremovalofindividualexperts.Thesecondanalysisevaluatedwhethertheuseofparametricornonparametricapproachesaffectedtheoverallresults.Theresultssuggestedthattheuseofparametricdistributionsledtodistributionswithsimilarorslightlyincreaseduncertaintycomparedwithdistributionsprovidedbyexpertswhoofferedpercentilesofanonparametricdistribution.ThelastanalysisevaluatedwhetherparticipationinthePre‐orPost‐ElicitationWorkshopsaffectedtheresults.Participationineitherworkshopdidnotappeartohaveasignificanteffectonexperts’judgmentsbasedonmeasuresofchangeinthebaselinedistribution.Overall,thesensitivityanalysesdemonstratedthattheassessmentwasrobust,withlittledependenceonindividualexperts’judgmentsoronthespecificelicitationmethodsevaluated.

ManagementConsiderations.ThesensitivityanalysisdemonstratedtherobustnessofthePM2.5expertelicitation‐basedassessmentbyshowingthatthepanelofexpertswasgenerallywellbalancedandthatalternativeelicitationmethodswouldnothavemarkedlyalteredtheoverallresults.

70

SelectedReferences.ThedetailsofthisanalysisareprovidedintheIndustrialEconomics,Inc.,documenttitled:ExpandedExpertJudgmentAssessmentoftheConcentration‐ResponseRelationshipBetweenPM2.5ExposureandMortality,FinalReport,September21,2006.Thisdocumentisavailableathttp://www.epa.gov/ttn/ecas/regdata/Uncertainty/pm_ee_report.pdf.

Theexpertelicitationassessment,alongwiththeRegulatoryImpactAnalysis(RIA)ofthePM2.5standard,isavailableathttp://www.epa.gov/ttn/ecas/ria.html.

Case Study 7: Environmental Monitoring and Assessment Program: Using Probabilistic Sampling to Evaluate the Condition of the Nation’s Aquatic Resources Monitoringisakeytoolusedtoidentifythelocationswheretheenvironmentisinahealthybiologicalconditionandrequiresprotection,andwhereenvironmentalproblemsareoccurringandneedremediation.Mostmonitoring,however,isnotperformedinawaythatallowsforstatisticallyvalidassessmentsofwaterqualityconditionsinunmonitoredwaters(USGAO2000).StatesthuscannotadequatelymeasurethestatusandtrendsinwaterqualityasrequiredbytheCleanWaterAct(CWA)Section305(b).

TheEnvironmentalMonitoringandAssessmentProgram’s(EMAP)focushasbeentodevelopunbiasedstatisticalsurveydesignframeworksandsensitiveindicatorsthatallowtheconditionofaquaticecosystemstobeassessedatstate,regionalandnationalscales.AcornerstoneofEMAPhasbeentheuseofprobabilisticsamplingtoallowrepresentative,unbiased,cost‐effectiveconditionassessmentsforaquaticresourcescoveringlargeareas.EMAP’sstatisticalsurveymethodsareveryefficient,requiringrelativelyfewsamplelocationstomakevalidscientificstatementsabouttheconditionofaquaticresourcesoverlargeareas(e.g.,theconditionofallofthewadeablestreamsinthewesternUnitedStates).

ProbabilisticAnalysis.Thisresearchprogramprovidesmultiplecasestudiesusingprobabilisticsamplingdesignsfordifferentaquaticresources(estuaries,streamsandrivers).AnEMAPprobability‐basedsamplingprogramdeliversanunbiasedestimateoftheconditionofanaquaticresourceoveralargegeographicareafromasmallnumberofsamples.Theprincipalcharacteristicsofaprobabilisticsamplingdesignare:thepopulationbeingsampledisunambiguouslydescribed;everyelementinthepopulationhastheopportunitytobesampledwithaknownprobability;andsampleselectionisconductedbyarandomprocess.Thisapproachallowsstatisticalconfidencelevelstobeplacedontheestimatesandprovidesthepotentialtodetectstatisticallysignificantchangesandtrendsinconditionwithrepeatedsampling.Inaddition,thisapproachpermitstheaggregationofdatacollectedfromsmallerareastopredicttheconditionofalargegeographicarea.

TheEMAPdesignframeworkallowstheselectionofunbiased,representativesamplingsitesandspecifiestheinformationtobecollectedatthesesites.Thevalidityoftheoverallinferencerestsonthedesignandsubsequentanalysistoproduceregionallyrepresentativeinformation.TheEMAPusestheapproachoutlinedintheEPA’sGeneralizedRandomTessellationStratifiedSpatially‐BalancedSurveyDesignsforAquaticResources(Olsen2012).Thespatiallybalancedaspectspreadsoutthesamplinglocationsgeographically,butstillensuresthateachelementhasanequalchanceofbeingselected.

ResultsofAnalysis.DatacollectedusingtheEMAPapproachhasallowedtheAgencytomakescientificallydefensibleassessmentsoftheecologicalconditionoflargegeographicareasforreportingtoCongressunderCWA305(b).TheEMAPapproachhasbeenusedtoprovidethefirstreportsontheconditionofthenation’sestuaries,streams,riversandlakes,anditisscheduledtobe

71

usedforwetlands.EMAPfindingshavebeenincludedinEPA’sReportontheEnvironmentandtheHeinzCenter’sTheStateoftheNation’sEcosystems.DatacollectedthroughanEMAPapproachimprovetheabilitytoassessecologicalprogressinenvironmentalprotectionandrestoration,andprovidevaluableinformationfordecisionmakersandthepublic.TheuseofprobabilisticanalysismethodsallowsmeaningfulassessmentandregionalcomparisonsofaquaticecosystemconditionsacrosstheUnitedStates.Finally,theprobabilisticapproachprovidesscientificcredibilityforthemonitoringnetworkandaidsinidentifyingdatagaps.

ManagementConsiderations.UseofanEMAPapproachaddressescriticismsfromtheGovernmentAccountabilityOffice(GAO),theNationalAcademyofSciences(NAS),theHeinzCenter(anonprofitenvironmentalpolicyinstitution),andothersthatnotedthenationlackedthedatatomakescientificallyvalidcharacterizationsofwaterqualityregionallyandacrosstheUnitedStates.Theprogramprovidescost‐effective,scientificallydefensibleandrepresentativedata,topermittheevaluationofquantifiabletrendsinecosystemcondition,tosupportperformance‐basedmanagementandfacilitatebetterpublicdecisionsregardingecosystemmanagement.EMAP’sapproachnowhasbeenadoptedbyEPA’sOfficeofWater(OW)tocollectdataontheconditionofallthenation’saquaticresources.OW,OfficeofAirandRadiation(OAR)andOfficeofChemicalSafetyandPollutionPrevention(OCSPP;formerlytheOfficeofPrevention,Pesticides,andToxicSubstances)nowhaveenvironmentalaccountabilityendpointsusingEMAPapproachesintheirAgencyperformancegoals.

SelectedReferences.GeneralinformationconcerningEMAPisavailableathttp://www.epa.gov/emap/index.html.

InformationonEMAPmonitoringdesignsisavailableathttp://www.epa.gov/nheerl/arm/designpages/monitdesign/monitoring_design_info.htm.

EPA’sGeneralizedRandomTessellationStratifiedSpatially‐BalancedSurveyDesignsforAquaticResourcesdocumentisavailableathttp://www.epa.gov/nheerl/arm/documents/presents/grts_ss.pdf.

USGAO(U.S.GovernmentAccountabilityOffice).2000.WaterQuality:KeyEPAandStateDecisionsLimitedbyInconsistentandIncompleteData.GAO/RCED‐00‐54.Washington,D.C.:USGAO.http://www.environmental‐auditing.org/Portals/0/AuditFiles/useng00ar_ft_key_epa.pdf.

USEPA(U.S.EnvironmentalProtectionAgency).2002.EMAPResearchStrategy.ResearchTrianglePark,NC:EnvironmentalMonitoringandAssessmentProgram,NationalHealthandEnvironmentalEffectsResearchLaboratory(NHEERL),USEPA.http://www.epa.gov/nheerl/emap/files/emap_research_strategy.pdf.

D.3. Group 3 Case Studies

Case Study 8: Two-Dimensional Probabilistic Risk Analysis of Cryptosporidium in Public Water Supplies, With Bayesian Approaches to Uncertainty Analysis ProbabilisticassessmentoftheoccurrenceandhealtheffectsassociatedwithCryptosporidiumbacteriainpublicdrinkingwatersupplieswasusedtosupporttheeconomicanalysisofthefinalLong‐Term2EnhancedSurfaceWaterTreatmentRule(LT2).EPA’sOfficeofGroundWaterandDrinkingWater(OGWDW)conductedthisanalysisandestablishedabaselinediseaseburdenattributabletoCryptosporidiuminpublicwatersuppliesthatusesurfacewatersources.Next,itmodeledsourcewatermonitoringandfinishedwaterimprovementsthatwillberealizedasaresult

72

oftheLT2.Post‐RuleriskisestimatedandtheLT2’shealthbenefitistheresultofsubtractingthisfromthebaselinediseaseburden.

ProbabilisticAnalysis.ProbabilisticassessmentwasusedforthisanalysisasameansofaddressingthevariabilityintheoccurrenceofCryptosporidiuminrawwatersupplies,thevariabilityinthetreatmentefficiency,andtheuncertaintyintheseinputsandinthedose‐responserelationshipforCryptosporidiuminfection.ThiscasestudyprovidesanexampleofaPRAthatevaluatesbothvariabilityanduncertaintyatthesametimeandisreferredtoasatwo‐dimensionalPRA.Theanalysisalsoincludedprobabilistictreatmentsofuncertaindose‐responseandoccurrenceparameters.MarkovChainMonteCarlosamplesofparametersetsfilledthisfunction.ThisBayesianapproach(treatingtheunknownparametersasrandomvariables)differsfromclassicaltreatments,whichwouldregardtheparametersasunknown,butfixed(Group3:AdvancedPRA).Theriskanalysisusedexistingdatasets(e.g.,theoccurrenceofCryptosporidiumandtreatmentefficacy)toinformthevariabilityoftheseinputs.UncertaintydistributionswerecharacterizedbasedonprofessionaljudgmentorbyanalyzingdatausingBayesianstatisticaltechniques.

ResultsofAnalysis.TheriskanalysisidentifiedtheCryptosporidiumdose‐responserelationshipasthemostcriticalmodelparametersintheassessment,followedbytheoccurrenceofthepathogenandtreatmentefficiency.BysimulatingimplementationoftheRuleusingimprecise,biasedmeasurementmethods,theassessmentprovidedestimatesofthenumberofpublicwatersupplysystemsthatwouldrequirecorrectiveactionandthenatureoftheactionslikelytobeimplemented.Thisinformationaffordedarealisticmeasureofthebenefits(inreduceddiseaseburden)expectedwiththeLT2.InresponsetoScienceAdvisoryBoard(SAB)comments,additionalCryptosporidiumdose‐responsemodelswereaddedtomorefullyreflectuncertaintyinthiselementoftheassessment.

ManagementConsiderations.TheLT2underwentexternalpeerreview,reviewbyEPA’sSABandintensereviewbytheOfficeofManagementandBudget(OMB).Occurrenceanddose‐responsecomponentsoftheriskanalysismodelwerecommunicatedtostakeholdersduringtheRule’sFederalAdvisoryCommitteeAct(FACA)process.DuetotherigoroftheanalysisandthesignedFACA“AgreementinPrinciple,”theOMBreviewwasstraightforward.

SelectedReferences.ThefinalassessmentofoccurrenceandexposuretoCryptosporidiumwasreleasedinDecember2005andisavailableathttp://www.epa.gov/safewater/disinfection/lt2/regulations.html.

Case Study 9: Two-Dimensional Probabilistic Model of Children’s Exposure to Arsenic in Chromated Copper Arsenate Pressure-Treated Wood ProbabilisticmodelsweredevelopedinresponsetoEPA’sOctober2001FederalInsecticide,Fungicide,andRodenticideAct(FIFRA)ScientificAdvisoryPanel(SAP)recommendationstouseprobabilisticmodelingtoestimatechildren’sexposurestoarsenicinCCA‐treatedplaysetsandhomedecks.

ProbabilisticAnalysis.EPA’sORD,incollaborationwiththeOfficeofPesticidePrograms(OPP),developedandappliedaprobabilisticexposureassessmentofchildren’sexposuretoarsenicandchromiumfromcontactwithCCA‐treatedwoodplaysetsanddecks.Thiscasestudyprovidesanexampleoftheuseoftwo‐dimensional(i.e.,addressingbothvariabilityanduncertainty)probabilisticexposureassessment(Group3:AdvancedPRA).Thetwo‐dimensionalassessmentemployedamodificationofORD’sStochasticHumanExposureandDoseSimulation(SHEDS)model

73

tosimulatechildren’sexposuretoarsenicandchromiumfromCCA‐treatedwoodplaysetsanddecks,aswellasthesurroundingsoil.StafffrombothORDandOPPcollaboratedinthedevelopmentoftheSHEDS‐Woodmodel.

ResultsofAnalysis.AdraftoftheprobabilisticexposureassessmentreceivedSAPreviewinDecember2003;thefinalreportwasreleasedin2005.Theresultsoftheprobabilisticexposureassessmentwereconsistentwithorintherangeoftheresultsofdeterministicexposureassessmentsconductedbyseveralotherorganizations.Themodelresultswereusedtocompareexposuresunderavarietyofscenarios,includingcoldversuswarmweatheractivitypatterns,useofwoodsealantstoreducetheavailabilityofcontaminantsonthesurfaceofthewood,anddifferenthand‐washingfrequencies.Themodelingofalternativemitigationscenariosindicatedthattheuseofsealantscouldresultinthegreatestexposurereduction,whileincreasedfrequencyofhandwashingalsocouldreduceexposure.

OPPusedtheSHEDS‐Woodmodelexposureresultsinitsprobabilisticchildren’sriskassessmentforCCA(USEPA2008).Thisincludedrecommendationsforriskreduction(useofsealantsandcarefulattentiontochildren’shandwashing)tohomeownerswithexistingCCAwoodstructures.Inaddition,theexposureassessmentwasusedtoidentifyareasforfurtherresearch,including:theefficacyofwoodsealantsinreducingdislodgeablecontaminantresidues,thefrequencywithwhichchildrenplayonoraroundCCAwood,andtransferefficiencyandresidueconcentrations.Tobettercharacterizetheefficacyofsealantsinreducingexposure,EPAandtheConsumerProductSafetyCommission(CPSC)conducteda2‐yearstudyofhowdislodgeablecontaminantresiduelevelschangedwiththeuseofvarioustypesofcommerciallyavailablewoodsealants.

ManagementConsiderations.TheOPPusedSHEDSresultsdirectlyinitsfinalriskassessmentforchildrenplayingonCCA‐treatedplaygroundequipmentanddecks.Themodelenhancedriskassessmentandmanagementdecisionsandprioritizeddataneedsrelatedtowoodpreservatives.Themodelingproductwaspivotalintheriskmanagementandre‐registrationeligibilitydecisionsforCCA,andinadvisingthepublichowtominimizehealthrisksfromexistingtreatedwoodstructures.IndustryalsoisusingSHEDStoestimateexposurestoCCAandotherwoodpreservatives.SomestatesareusingtheriskassessmentasguidanceinsettingtheirregulationsforCCA‐relatedplaygroundequipment.

SelectedReferences.ThefinalprobabilisticriskassessmentbasedontheSHEDS‐Woodexposureassessmentisavailableathttp://www.epa.gov/oppad001/reregistration/cca/final_cca_factsheet.htm.

ThemodelresultswereincludedinthefinalreportontheprobabilisticexposureassessmentofCCA‐treatedwoodsurfaces:Zartarian,V.G.,J.Xue,H.A.Özkaynak,W.Dang,G.Glen,L.Smith,andC.Stallings.2006.AProbabilisticExposureAssessmentforChildrenWhoContactCCA‐TreatedPlaysetsandDecksUsingtheStochasticHumanExposureandDoseSimulationModelfortheWoodPreservativeScenario(SHEDS‐Wood),FinalReport.EPA/600/X‐05/009.Washington,D.C.:USEPA.

ResultsofthesealantstudieswerereleasedinJanuary2007andareavailableathttp://www.epa.gov/oppad001/reregistration/cca/index.htm#reviews.

Theresultsoftheanalysiswerepublishedas:Zartarian,V.G.,J.Xue,H.Özkaynak,W.Dang,G.Glen,L.Smith,andC.Stallings.2006.“AProbabilisticArsenicExposureAssessmentforChildrenwhoContactCAA‐TreatedPlaysetsandDecks,Part1:ModelMethodology,VariabilityResults,andModelEvaluation.”RiskAnalysis26:515–31.

Moreinformationontheanalysiscanbefoundbyconsultingthefollowingresource:

74

USEPA(U.S.EnvironmentalProtectionAgency).2008.CaseStudyExamplesoftheApplicationofProbabilisticRiskAnalysisinU.S.EnvironmentalProtectionAgencyRegulatoryDecision‐Making(InReview).Washington,D.C.:RiskAssessmentForum,USEPA

Case Study 10: Two-Dimensional Probabilistic Exposure Assessment of Ozone AspartofEPA’srecentreviewoftheozoneNationalAmbientAirQualityStandards(NAAQS),theOfficeofAirQualityPlanningandStandards(OAQPS)conducteddetailedprobabilisticexposureandriskassessmentstoevaluatepotentialalternativestandardsforozone.Atdifferentstagesofthisreview,theCleanAirScientificAdvisoryCommittee(CASAC)OzonePanel(anindependentscientificreviewcommitteeofEPA’sSAB)andthepublicreviewedandprovidedcommentsonanalysesanddocumentspreparedbyEPA.Ascopeandmethodsplanfortheexposureandriskassessmentswasdevelopedin2005(USEPA2005).ThisplanwasintendedtofacilitateconsultationwiththeCASAC,aswellaspublicreview,andtoobtainadviceontheoverallscope,approachesandkeyissuesinadvanceofthecompletionoftheanalyses.Thiscasestudydescribestheprobabilisticexposureassessment,whichaddressesshort‐termexposurestoozone.TheexposureestimateswereusedasaninputtotheHHRAforlungfunctiondecrementsinallchildrenandasthmaticschool‐agedchildrenbasedonexposure‐responserelationshipsderivedfromcontrolledhumanexposurestudies.

ProbabilisticAnalysis.PopulationexposuretoambientozonelevelswasevaluatedusingEPA’sAirPollutantsExposure(APEX)model,alsoreferredtoastheTotalRiskIntegratedMethodology/Exposure(TRIM.Expo)model.Exposureestimatesweredevelopedforrecentozonelevels,basedon2002to2004airqualitydata,andforozonelevelssimulatedtojustmeettheexisting0.08ppm,8‐hourozoneNAAQSandseveralalternativeozonestandards,basedonadjustingthe2002to2004airqualitydata.Exposureestimatesweremodeledfor12urbanareaslocatedthroughouttheUnitedStatesforthegeneralpopulation,allschool‐agechildrenandasthmaticschool‐agechildren.Thisexposureassessmentisdescribedinatechnicalreport(USEPA2007b).TheexposuremodelAPEXisdocumentedinauser’sguideandtechnicaldocument(USEPA2006).AMonteCarloapproachwasusedtoproducequantitativeestimatesoftheuncertaintyintheAPEXmodelresults,basedonestimatesoftheuncertaintiesforthemostimportantmodelinputs.Thequantificationofmodelinputuncertainties,adiscussionofmodelstructureuncertainties,andtheresultsoftheuncertaintyanalysisaredocumentedinLangstaff(2007).

ResultsofAnalysis.UncertaintyintheAPEXmodelpredictionsresultsfromuncertaintiesinthespatialinterpolationofmeasuredconcentrations,themicroenvironmentmodelsandparameters,people’sactivitypatterns,andtoalesserextent,modelstructure.Thepredominantsourcesofuncertaintyappeartobethehumanactivitypatterninformationandthespatialinterpolationofambientconcentrationsfrommonitoringsitestootherlocations.Theprimarypolicy‐relevantfindingwasthattheuncertaintyintheexposureassessmentissmallenoughtolendconfidencetotheuseofthemodelresultsforthepurposeofinformingtheAdministrator’sdecisionontheozonestandard.

FigureA‐3illustratestheuncertaintydistributionforonemodelresult,thepercentofchildrenwithexposuresabove0.08ppm,8‐hourwhileatmoderateexertion.Thepointestimateof20percentistheresultfromtheAPEXsimulationusingthebestestimatesofthemodelinputs.ThecorrespondingresultfromtheMonteCarlosimulationsrangesfrom17to26percent,witha95percentuncertaintyinterval(UI)of19to24percent.NotethattheUIsarenotsymmetricbecausethedistributionsareskewed.

75

ManagementConsiderations.Theexposureanalysisalsoprovidedinformationonthefrequencywithwhichpopulationexposuresexceededseveralpotentialhealtheffectbenchmarklevelsthatwereidentifiedbasedontheevaluationofhealtheffectsinclinicalstudies.

TheexposureandriskassessmentsaresummarizedinChapters4and5,respectively,oftheOzoneStaffPaper(USEPA2007a).TheexposureestimatesoverthesepotentialhealtheffectbenchmarkswerepartofthebasisfortheAdministrator’sMarch27,2008,decisiontorevisetheozoneNAAQSfrom0.08to0.075ppm,8‐houraverage(seethefinalrulefortheozoneNAAQS1).

Figure A‐3. Uncertainty Distribution Model Results. The estimated percentage of children with 8‐hour exposures above 0.08 ppm at moderate exertion (the point estimate is 20%). 

SelectedReferences.Moreinformationontheanalysiscanbefoundbyconsultingthefollowingresources:

Langstaff,J.E.2007.AnalysisofUncertaintyinOzonePopulationExposureModeling.OfficeofAirQualityPlanningandStandardsStaffMemorandumtoOzoneNAAQSReviewDocket.OAR‐2005‐0172.http://www.epa.gov/ttn/naaqs/standards/ozone/s_ozone_cr_td.html

USEPA(U.S.EnvironmentalProtectionAgency).2005.OzoneHealthAssessmentPlan:ScopeandMethodsforExposureAnalysisandRiskAssessment.ResearchTrianglePark,NC:OfficeofAirQualityPlanningandStandards,USEPA.http://www.epa.gov/ttn/naaqs/standards/ozone/s_o3_cr_pd.html

USEPA.2006.TotalRiskIntegratedMethodology(TRIM)—AirPollutantsExposureModelDocumentation(TRIM.Expo/APEX,Version4)VolumeI:User’sGuide;VolumeII:TechnicalSupportDocument.ResearchTrianglePark,NC:OfficeofAirQualityPlanningandStandards,USEPA.June2006.http://www.epa.gov/ttn/fera/human_apex.html

USEPA.2007a.ReviewofNationalAmbientAirQualityStandardsforOzone:PolicyAssessmentofScientificandTechnicalInformation—OAQPSStaffPaper.ResearchTrianglePark,NC:OfficeofAirQualityPlanningandStandards,USEPA.http://www.epa.gov/ttn/naaqs/standards/ozone/s_ozone_cr_sp.html

1NationalAmbientAirQualityStandardsforOzone,FinalRule.73Fed.Reg.16436(Mar.27,2008).

0

50

100

150

200

250

300

350

400

450

17% 18% 19% 20% 21% 22% 23% 24% 25% 26%

Percent of children

Num

ber o

f Mon

te C

arlo

sim

ulat

ions

76

USEPA.2007b.OzonePopulationExposureAnalysisforSelectedUrbanAreas.ResearchTrianglePark,NC:OfficeofAirQualityPlanningandStandards,USEPA.http://www.epa.gov/ttn/naaqs/standards/ozone/s_ozone_cr_td.html

Case Study 11: Analysis of Microenvironmental Exposures to Fine Particulate Matter for a Population Living in Philadelphia, Pennsylvania ThiscasestudyusedtheStochasticHumanExposureandDoseSimulationmodelforParticulateMatter(SHEDS‐PM)developedbyEPA’sNationalExposureResearchLaboratory(NERL)toprepareaprobabilisticassessmentofpopulationexposuretoPM2.5inPhiladelphia,Pennsylvania.Thiscasestudysimulationwaspreparedtoaccomplishthreegoals:(1)estimatethecontributionofPM2.5ofambient(outdoor)origintototalPM2.5exposure;(2)determinethemajorfactorsthatinfluencepersonalexposuretoPM2.5;and(3)identifyfactorsthatcontributethegreatestuncertaintytomodelpredictions.

ProbabilisticAnalysis.Thetwo‐dimensionalprobabilisticassessmentusedamicroexposureeventtechniquetosimulateindividualexposurestoPM2.5inspecificmicroenvironments(outdoors,indoorresidence,office,school,store,restaurantorbar,andinavehicle).Thepopulationforthesimulationwasgeneratedusingdemographicdataatthecensus‐tractlevelfromtheU.S.Census.Characteristicsofthesimulatedindividualswereselectedrandomlytomatchthedemographicproportionswithinthecensustractforgender,age,employmentstatusandhousingtype.TheassessmentusedspatiallyandtemporallyinterpolatedambientPM2.5measurementsfrom1992to1993and1990U.S.CensusdataforeachcensustractinPhiladelphia.Thiscasestudyprovidesanexampleofbothtwo‐dimensional(variabilityanduncertainty)probabilisticassessmentandmicroexposureeventassessment(Group3:AdvancedPRA).

ResultsofAnalysis.ResultsoftheanalysisshowedthathumanactivitypatternsdidnothaveasstronganinfluenceonambientPM2.5exposuresaswasobservedforexposuretoindoorPM2.5

sources.ExposuretoPM2.5ofambientorigincontributedasignificantpercentofthedailytotalPM2.5

exposures,especiallyforthesegmentofthepopulationwithoutexposuretoenvironmentaltobaccosmokeintheresidence.DevelopmentoftheSHEDS‐PMmodelusingthePhiladelphiaPM2.5casestudyalsoprovidedusefulinsightsintodataneedsforimprovinginputsintotheSHEDS‐PMmodel,reducinguncertaintyandfurtherrefinementofthemodelstructure.Someoftheimportantdataneedsidentifiedfromtheapplicationofthemodelinclude:dailyPM2.5measurementsovermultipleseasonsandacrossmultiplesiteswithinanurbanarea,improvedcapabilityofdispersionmodelstopredictambientPM2.5concentrationsatgreaterspatialresolutionandovera1‐yeartimeperiod,measurementstudiestobettercharacterizethephysicalfactorsgoverninginfiltrationofambientPM2.5intoresidentialmicroenvironments,furtherinformationonparticle‐generatingsourceswithintheresidence,anddatafortheotherindoormicroenvironmentsnotspecifiedinthemodel.

ManagementConsiderations.TheapplicationoftheSHEDS‐PMmodeltothePhiladelphiapopulationgaveinsightsintodataneedsandareasformodelrefinement.ThecontinueddevelopmentandevaluationoftheSHEDS‐PMpopulationexposuremodelarebeingconductedaspartofORD’sefforttodevelopasource‐to‐dosemodelingsystemforPMandairtoxics.Thistypeofexposureanddosemodelingsystemisconsideredtobeimportantforthescientificandpolicyevaluationofthecriticalpathways,aswellastheexposurefactorsandsourcetypesinfluencinghumanexposurestoavarietyofenvironmentalpollutants,includingPM.

77

SelectedReferences.Theresultsoftheanalysiswerepublishedin:

Burke,J.,M.Zufall,andH.Özkaynak.2001.“APopulationExposureModelforParticulateMatter:CaseStudyResultsforPM2.5inPhiladelphia,PA.”JournalofExposureAnalysisandEnvironmentalEpidemiology11(6):470–89.

Georgepoulos,P.G.,S.W.Wang,V.M.Vyas,Q.Sun,J.Burke,R.Vedantham,T.McCurdy,andH.Özkaynak.2005.“ASource‐to‐DoseAssessmentofPopulationExposuretoFinePMandOzoneinPhiladelphia,PA,DuringaSummer1999Episode.”JournalofExposureAnalysisandEnvironmentalEpidemiology15(5):439–57.

Case Study 12: Probabilistic Analysis in Cumulative Risk Assessment of Organophosphorus Pesticides In1996,CongressenactedtheFoodQualityProtectionAct(FQPA),whichrequiresEPAtoconsider“availableevidenceconcerningthecumulativeeffectsoninfantsandchildrenofsuchresiduesandothersubstancesthathaveacommonmechanismoftoxicity”whensettingpesticidetolerances(i.e.,themaximumamountofpesticideresiduelegallyallowedtoremainonfoodproducts).FQPAalsomandatedthatEPAcompletelyreassessthesafetyofallexistingpesticidetolerances(thoseineffectsinceAugust1996)toensurethattheyaresupportedbycurrentscientificdataandmeetcurrentsafetystandards.Becauseorganophosphoruspesticides(OPs)wereassignedpriorityfortolerancereassessment,thesepesticideswerethefirst“commonmechanism”groupidentifiedbyEPA’sOPP.Theultimategoalassociatedwiththiscumulativeriskassessment(CRA)wastoprovideabasisforthedecisionmakertoestablishsafetolerancelevelsforthisgroupofpesticides,whilemeetingtheFQPAstandardforprotectinginfantsandchildren.

ProbabilisticAnalysis.Thiscasestudyprovidesanexampleofanadvancedprobabilisticassessment(Group3).In2006,EPAanalyzedexposuresto30OPsthroughfoodconsumption,drinkingwaterintake,andexposureviapesticideapplication.Distributionsofhumanexposurefactors,suchasbreathingrates,bodyweightandsurfaceareasusedintheassessment,camefromtheAgency’sExposureFactorsHandbook(USEPA1997d).EPAusedCalendex,aprobabilisticcomputersoftwareprogram(availableathttp://www.epa.gov/pesticides/science/deem/)tointegratevariouspathways,whilesimultaneouslyincorporatingthetimedimensionsoftheinputdata.Basedontheresultsoftheexposureassessment,EPAcalculatedmarginsofexposure(MOEs)forthetotalcumulativeriskfromallpathwaysforeachagegroup(infantlessthan1;children1–2,3–5,6–12;youth13–19;andadults20–49and50+yearsofage).

ThefoodcomponentoftheOPsCRAwashighlyrefined,asitwasbasedonresiduemonitoringdatafromtheUSDA’sPDPandsupplementedwithinformationfromtheFDA’sSurveillanceMonitoringProgramsandTotalDietStudy.TheresiduedatawerecombinedwithactualconsumptiondatafromUSDA’sContinuingSurveyofFoodIntakesbyIndividuals(CSFII)usingprobabilistictechniques.TheCRAevaluateddrinkingwaterexposuresonaregionalbasis.TheassessmentfocusedonareaswherecombinedOPexposureislikelytobehighestwithineachregion.Primarily,theanalysisusedprobabilisticmodelingtoestimatetheco‐occurrenceofOPresiduesinwater.Monitoringdatawerenotavailablewithenoughconsistencytobethesolebasisfortheassessment;however,theywereusedtocorroboratethemodelingresults.DatasourcesforthewatercomponentoftheassessmentincludedUSDAAgriculturalUsageReportsforFieldCrops,FruitsandVegetables;USDATypicalPlantingandHarvestingDatesforFieldCropsandFreshMarketandProcessingVegetables;localsourcesforrefinements;andmonitoringstudiesfromtheU.S.GeologicalSurvey(USGS)andothersources.Finally,exposureviatheoral,dermalandinhalationroutesresultingfromapplicationsofOPsinandaroundhomes,schools,officesandotherpublicareaswereassessedprobabilisticallyforeachofthesevenregions.Thedatasourcesforthispartof

78

theassessmentincludedinformationfromsurveysandtaskforces,specialstudiesandreportsfrompublishedscientificliterature,EPA’sExposureFactorsHandbook(USEPA1997d),andothersources.

ResultsofAnalysis.TheOPsCRApresentedpotentialriskfromsingle‐day(acute)exposuresacross1yearandfromaseriesof21‐dayrollingaveragesacrosstheyear.MOEsatthe99.9thpercentilewereapproximately100orgreaterforallpopulationsforthe21‐dayaverageresults.Theonlyexceptionisabriefperiod(roughly2weeks)inwhichdrinkingwaterexposures(identifiedfromtheExposureFactorsHandbook,USEPA1997d)attributedtophorateuseonsugarcaneresultedinMOEsnear80forchildrenages1to2years.Generally,exposuresthroughthefoodpathwaydominatedtotalMOEs,andexposuresthroughdrinkingwaterweresubstantiallylowerthroughoutmostoftheyear.Residentialexposuresweresubstantiallysmallerthanexposuresthroughbothfoodanddrinkingwater.

TheOPsCRAwasveryresourceintensive.Workbeganin1997withthepreparationofguidancedocumentsandthedevelopmentofaCRAmethodology.Over2to3years,morethan25peoplespent50to100percentoftheirtimeworkingontheassessment,withupto50peopleworkingontheCRAatcriticalperiods.EPAhasspentapproximately$1milliononthisassessment(e.g.,forcomputers,modelsandcontractorsupport).

ManagementConsiderations.TheOPsCRAwasalandmarkdemonstrationofthefeasibilityofaregulatory‐levelassessmentoftheriskfrommultiplechemicals.Uponcompletion,EPApresentedtheCRAatnumerouspublictechnicalbriefingsandFIFRASAPmeetings,andmadeallofthedatainputsavailabletothepublic.TheOPP’ssubstantialefforttocommunicatemethodologies,approachesandresultstothestakeholdersaidedinthesuccessoftheOPsCRA.ThestakeholdersexpressedappreciationforthetransparentnatureoftheOPsCRAandrecognizedtheinnovationandhardworkthatwentintodevelopingtheassessments.

SelectedReferences.The2006assessmentandrelateddocumentsareavailableathttp://www.epa.gov/pesticides/cumulative/common_mech_groups.htm#op.

USEPA(U.S.EnvironmentalProtectionAgency).1997d.ExposureFactorsHandbook.Washington,D.C.:NationalCenterforEnvironmentalAssessment,USEPA.http://cfpub.epa.gov/ncea/cfm/recordisplay.cfm?deid=12464.

Case Study 13: Probabilistic Ecological Effects Risk Assessment Models for Evaluating Pesticide Use AspartoftheprocessofdevelopingandimplementingaprobabilisticapproachforERA,anillustrativecasewascompletedin1996.ThiscaseinvolvedbothDRAandPRAfortheeffectsofahypotheticalchemicalX(ChemX)onbirdsandaquaticspecies.FollowingtherecommendationsoftheSAPandinresponsetoissuesraisedbyOPPriskmanagers,theAgencybegananinitiativetorefinetheERAprocessforevaluatingtheeffectsofpesticidestoterrestrialandaquaticspecieswithinthecontextofFIFRA,themainstatutoryauthorityforregulatingpesticidesatthefederallevel.ThekeygoalsandobjectivesofEPA’sinitiativewereto:

Incorporateprobabilistictoolsandmethodstoprovideanestimateonthemagnitudeandprobabilityofeffects.

Buildonexistingdatarequirementsforregistration.

Utilize,whereverpossible,existingdatabasesandcreatenewonesfromexistingdatasourcestominimizetheneedtogenerateadditionaldata.

Focusadditionaldatarequirementsonreducinguncertaintyinkeyareas.

79

Afterproposingafour‐levelriskassessmentscheme,withhigherlevelsreflectingmorerefinedriskassessmenttechniques,theAgencydevelopedpilotmodelsforbothterrestrialandaquaticspecies.Refinedriskassessmentmodels(LevelII)thenweredevelopedandusedinagenericchemicalcasestudythatwaspresentedtotheSAPin2001.

ProbabilisticAnalysis.Thiscasestudydescribesanadvancedprobabilisticmodelfortheecologicaleffectsofpesticides(Group3).TheterrestrialLevelIImodel(version2.0)isamultimediaexposure/effectsmodelthatevaluatesacutemortalitylevelsingenericorspecificavianspeciesoverauser‐definedexposurewindow.Thespatialscaleisatthefieldlevel,whichincludesthefieldandsurroundingarea.Bothareasareassumedtomeetthehabitatrequirementsforeachspecies,andcontaminationofedgeoradjacenthabitatfromdriftisassumedtobezero.ForeachindividualbirdconsideredinarunoftheLevelIImodel,arandomselectionofvaluesismadeforthemajorexposureinputparameterstoestimateanexternaloraldoseequivalentforthatindividual.Theestimateddoseequivalentiscomparedtoarandomlyassignedtolerancefortheindividualpreselectedfromthedose‐responsedistribution.Ifthedoseisgreaterthanthetolerance,theindividualisscored“dead,”ifnot,theindividualisscored“notdead.”Aftermultipleiterationsofindividuals,aprobabilitydensityfunctionofpercentmortalityisgenerated.

FromMay29to31,1996,theAgencypresentedtwoERAcasestudiestotheSAPforreviewandcomment.AlthoughrecognizingandgenerallyreaffirmingtheutilityofEPA’scurrentdeterministicassessmentprocess,theSAPofferedanumberofsuggestionsforimprovement.Foremostamongtheirsuggestionswasarecommendationtomovebeyondtheexistingdeterministicassessmentapproachbydevelopingthetoolsandmethodologiesnecessarytoconductaprobabilisticassessmentofeffects.Suchanassessmentwouldestimatethemagnitudeandprobabilityoftheexpectedimpactanddefinethelevelofcertaintyandvariationinvolvedintheestimate;riskmanagerswithinEPA’sOPPalsohadrequestedthisinformationinthepast.

TheaquaticLevelIImodelisatwo‐dimensionalMonteCarloriskmodelconsistingofthreemaincomponents:(1)exposure,(2)effectsand(3)risk.TheexposurescenariosusedatLevelIIareintendedtoprovideestimatesforvulnerableaquatichabitatsacrossawiderangeofgeographicalconditionsunderwhichapesticideproductisused.TheLevelIIriskevaluationprocessyieldsestimatesoflikelihoodandmagnitudeofeffectsforacuteexposures.Fortheestimateofacuterisks,adistributionofestimatedexposureandadistributionoflethaleffectsarecombinedthrougha2‐DMCAtoobtainadistributionofjointprobabilityfunctions.Fortheestimateofchronicrisks,adistributionofexposureconcentrationsiscomparedtoachronicmeasurementendpoint.Theriskanalysisforchroniceffectsprovidesinformationonlyontheprobabilitythatthechronicendpointassessedwillbeexceeded,notonthemagnitudeofthechroniceffectexpected.

ResultsofAnalysis.AspartoftheprocessofdevelopingandimplementingaprobabilisticapproachforERA,acasestudywascompleted.ThecasestudyinvolvedbothDRAsandPRAsforeffectsofChemXonbirdsandaquaticspecies.ThedeterministicscreenconductedonChemXconcludedqualitativelythatitcouldposeahighrisktobothfreshwaterfishandinvertebratesandshowedthatPRAwaswarranted.Basedontheprobabilisticanalysis,itwasconcludedthattheuseofChemXwasexpectedtoinfrequentlyresultinsignificantfreshwaterfishmortalitiesbutroutinelyresultinreducedgrowthandotherchroniceffectsinexposedfish.Substantialmortalitiesandchroniceffectstosensitiveaquaticinvertebrateswerepredictedtooccurroutinelyafterpeakexposures.

ManagementConsiderations.Initsreviewofthecasestudy,theFIFRASAPcongratulatedtheAgencyontheeffortmadetoconductPRAofpesticideeffectsinecosystems.Thepanelcommentedthattheapproachhadprogressedgreatlyfromearlierefforts,andthattheintricacyofthemodelswassurprisinglygood,giventhetimeintervalinwhichtheAgencyhadtocompletethetask.

80

Followingthecasestudy,EPArefinedthemodelsbasedontheSAPcomments.Inaddition,theterrestrialLevelIImodelwasrefinedtoincludedermalandinhalationexposure.

SelectedReferences.Anoverviewofthemodelsisavailableathttp://www.epa.gov/oppefed1/ecorisk/fifrasap/rra_exec_sum.htm#Terrestrial.

Case Study 14: Expert Elicitation of Concentration-Response Relationship Between Fine Particulate Matter Exposure and Mortality In2002,theNRCrecommendedthatEPAimproveitscharacterizationofuncertaintyinthebenefitsassessmentforproposedregulationsofairpollutants.NRCrecommendedthatprobabilitydistributionsforkeysourcesofuncertaintybedevelopedusingavailableempiricaldataorthroughformalelicitationofexpertjudgments.AkeycomponentofEPA’sapproachforassessingpotentialhealthbenefitsassociatedwithairqualityregulationstargetingemissionsofPM2.5anditsprecursorsistheeffectofchangesinambientPM2.5levelsonmortality.Avoidedprematuredeathsconstitute,onamonetarybasis,between85and95percentofthemonetizedbenefitsreportedinEPA’sretrospectiveandprospectiveSection812Abenefit‐costanalysesoftheCleanAirAct(CAA;USEPA1997eand1999)andinRegulatoryImpactAnalysis(RIA)forrulessuchastheHeavyDutyDieselEngine/FuelRule(USEPA2000c)andtheNon‐RoadDieselEngineRule(USEPA2004).InresponsetotheNRCrecommendation,EPAconductedanexpertelicitationevaluationoftheconcentration‐responserelationshipbetweenPM2.5exposureandmortality.

ProbabilisticAnalysis.Thiscasestudyprovidesanexampleoftheuseofexpertelicitation(Group3)toderiveprobabilisticestimatesoftheuncertaintyinoneelementofacost‐benefitanalysis.Expertelicitationusescarefullystructuredinterviewstoelicitfromeachexpertabestestimateofthetruevalueforanoutcomeorvariableofinterest,aswellastheiruncertaintyaboutthetruevalue.Thisuncertaintyisexpressedasasubjectiveprobabilisticdistributionofvaluesandreflectseachexpert’sinterpretationoftheoryandempiricalevidencefromrelevantdisciplines,aswellastheirbeliefsaboutwhatisknownandnotknownaboutthesubjectofthestudy.ForthePM2.5expertelicitation,theprocessconsistedofdevelopmentofanelicitationprotocol,selectionofexperts,developmentofabriefingbook,conductofelicitationinterviews,theuseofexpertinputpriortoandfollowingindividualelicitationofjudgmentsandtheexpertjudgmentsthemselves.Theelicitationinvolvedpersonalinterviewswith12healthexpertswhohadconductedresearchontherelationshipbetweenPM2.5exposuresandmortality.

ThemainquantitativequestionaskedeachexperttoprovideaprobabilisticdistributionfortheaverageexpecteddecreaseinU.S.annual,adultandall‐causemortalityassociatedwitha1μg/m3decreaseinannualaveragePM2.5levels.Whenansweringthemainquantitativequestion,eachexpertwasinstructedtoconsiderthatthetotalmortalityeffectofa1μg/m3decreaseinambientannualaveragePM2.5mayreflectreductionsinbothshort‐termpeakandlong‐termaverageexposurestoPM2.5.Eachexpertwasaskedtoaggregatetheeffectsofbothtypesofchangesintheiranswers.Theexpertsweregiventheoptiontointegratetheirjudgmentsaboutthelikelihoodofacausalrelationshiporthresholdintheconcentration‐responsefunctionintotheirowndistributionsortoprovideadistribution“conditionalon”oneorbothofthesefactors.

ResultsofAnalysis.TheprojectteamdevelopedtheinterviewprotocolbetweenOctober2004andJanuary2006.DevelopmentoftheprotocolwasinformedbyanApril2005symposiumheldbytheprojectteam,wherenumeroushealthscientistsandanalystsprovidedfeedback;detailedpretestingwithindependentEPAscientistsinNovember2005;anddiscussionwiththeparticipatingexpertsatapre‐elicitationworkshopinJanuary2006.TheelicitationinterviewswereconductedbetweenJanuaryandApril2006.Followingtheinterviews,theexpertsreconvenedforapost‐elicitation

81

workshopinJune2006,inwhichtheprojectteamanonymouslysharedtheresultsofallexpertswiththegroup.

ThemedianestimatesforthePM2.5mortalityrelationshipweregenerallysimilartoestimatesderivedfromthetwoepidemiologicalstudiesmostoftenusedinbenefitsassessment.However,inalmostallcases,thespreadoftheuncertaintydistributionselicitedfromtheexpertsexceededthestatisticaluncertaintyboundsreportedbythemostinfluentialepidemiologicstudies,suggestingthattheexpertelicitationprocesswassuccessfulindevelopingmorecomprehensiveestimatesofuncertaintyforthePM2.5mortalityrelationship.TheuncertaintydistributionsforPM2.5

concentration‐responseresultingfromtheexpertelicitationprocesswereusedintheRIAfortherevisedNAAQSstandardforPM2.5(promulgatedonSeptember21,2006).BecausetheNAAQSareexclusivelyhealth‐basedstandards,thisRIAplayednopartinEPA’sdeterminationtorevisethePM2.5NAAQS.BenefitsestimatesintheRIAwerepresentedasrangesandincludedadditionalinformationonthequantifieduncertaintydistributionssurroundingthepointsontheranges,derivedfrombothepidemiologicalstudiesandtheexpertelicitationresults.OMB’sreviewoftheRIAwascompletedinMarch2007.

ManagementConsiderations.TheNAAQSareexclusivelyhealth‐basedstandards,sotheseanalyseswerenotusedinanymannerbyEPAindeterminingwhethertorevisetheNAAQSforPM2.5.Moreover,therequesttoengageintheexpertelicitationdidnotcomefromtheCASAC,theofficialpeerreviewbodyfortheNAAQS;adecisiontoconducttheanalysesdoesnotreflectCASACadvicethatsuchanalysesinformNAAQSdeterminations.TheanalysesaddressedcommentsfromtheNRCthatrecommendedthatprobabilitydistributionsforkeysourcesofuncertaintybeaddressed.TheanalyseswereusedinEPA’sretrospectiveandprospectiveSection812Abenefit‐costanalysesoftheCAA(USEPA1997eand1999)andinRIAsforrulessuchastheHeavyDutyDieselEngine/FuelRule(USEPA2000c)andtheNon‐RoadDieselEngineRule(USEPA2004).InresponsetotheNRCrecommendation,EPAconductedanexpertelicitationevaluationoftheconcentration‐responserelationshipbetweenPM2.5exposureandmortality.

SelectedReference.Theassessmentisavailableathttp://www.epa.gov/ttn/ecas/ria.html.

Case Study 15: Expert Elicitation of Sea-Level Rise Resulting From Global Climate Change TheUnitedNationsFrameworkConventiononClimateChangerequiresnationstoimplementmeasuresforadaptingtorisingsealevelandothereffectsofchangingclimate.Todecideonanappropriateresponse,coastalplannersandengineersweighthecostofthesemeasuresagainstthelikelycostoffailingtoprepare,whichdependsontheprobabilityofthesearisingaparticularamount.TheU.S.NationalAcademyofEngineeringrecommendedthatassessmentsofsealevelriseshouldprovideprobabilityestimates.Coastalengineersregularlyemployprobabilityinformationwhendesigningstructuresforfloods,andcourtsuseprobabilitiestodeterminethevalueoflandexpropriatedbyregulations.This1995casestudydescribesthedevelopmentofaprobabilitydistributionforsealevelrise,usingmodelsemployedbypreviousassessments,aswellastheexpertopinionsof20climateandglaciologyreviewersabouttheprobabilitydistributionsforparticularmodelcoefficients.

ProbabilisticAnalysis.Thiscasestudyprovidesanexamplebothofananalysisdescribingtheprobabilityofsealevelrise,aswellasanexpertelicitationofthelikelihoodofparticularmodelsandprobabilitydistributionsofthecoefficientsusedbythosemodelstopredictfuturesealevelrise(Group3).Theassessmentoftheprobabilityofsealevelriseusedexistingmodelsdescribingthechangeinfivecomponentsofsealevelriseassociatedwithgreenhousegas‐relatedclimatechange(thermalexpansion,smallglaciers,polarprecipitation,meltingandicedischargefromGreenland

82

andicedischargefromAntarctica).Toprovideastartingpointfortheexpertelicitation,initialprobabilitydistributionswereassignedtoeachmodelcoefficientbasedonthepublishedliterature.

Aftertheinitialprobabilisticassessmentwascompleted,thedraftreportwascirculatedtoexpertreviewersconsideredmostqualifiedtorenderjudgmentsonparticularprocessesforrevisedestimatesofthelikelihoodofparticularmodelsandthemodelcoefficients’probabilitydistributions.Expertsrepresentingbothextremesofclimatechangescience(thosewhopredictedtrivialconsequencesandthosewhopredictedcatastrophiceffects;individualswhosethoughtshadbeenexcludedfrompreviousassessments)wereincluded.Theexpertswereaskedtoprovidesubjectiveassessmentsoftheprobabilitiesofvariousmodelsandmodelcoefficients.Thesesubjectiveprobabilityestimateswereusedinplaceoftheinitialprobabilitiesinthefinalmodelsimulations.Differentrevieweropinionswerenotcombinedtoproduceasingleprobabilitydistributionforeachparameter;instead,eachreviewer’sopinionswereusedinindependentiterationsofthesimulation.Thegroupofsimulationsresultedintheprobabilitydistributionofsealevelrise.

ResultsofAnalysis.Theanalysis,completedwithabudgetof$100,000,providedaprobabilisticestimateofsealevelriseforusebycoastalengineersandregulators.Theresultssuggestedthatthereisa65percentchancethatthesealevelwillrise1millimeter(mm)peryearmorerapidlyinthenext30yearsthanithasbeenrisinginthelastcentury.Undertheassumptionthatnonclimaticfactorsdonotchange,theprojectionssuggestedthatthereisa50percentchancethattheglobalsealevelwillrise45centimeters(cm),anda1percentchanceofa112cmrisebytheyear2100.Themedianpredictionofsealevelrisewassimilartothemidpointestimateof48cmpublishedbytheIntergovernmentalPanelonClimateChange(IPCC)shortlythereafter(IPCC1996).Thereportalsofounda1percentchanceofa4to5meterriseoverthenext2centuries.

ManagementConsiderations.Therearetworeports(USEPA1995c;TitusandNarayanan1996)thatdiscussseveralusesoftheresultsofthisstudy.Byprovidingaprobabilisticrepresentationofsealevelrise,theassessmentallowscoastalresidentstomakedecisionswithrecognitionoftheuncertaintyassociatedwithpredictedchange.Rollingeasementsthatvestwhenthesearisestoaparticularlevelcanbeproperlyvaluedinboth“arms‐length”transactionsalesorwhencalculatingtheallowabledeductionforacharitablecontributionoftheeasementtoaconservancy.Cost‐benefitassessmentsofalternativeinfrastructuredesigns—whichalreadyconsiderfloodprobabilities—alsocanconsidersealevelriseuncertainty.Assessmentsofthebenefitsofpreventinganaccelerationofsealevelrisecanincludemorereadilylow‐probabilityoutcomes,whichcanprovideabetterassessmentofthetrueriskwhenthedamagefunctionisnonlinear,whichoftenisthecase.

SelectedReferences.

USEPA(U.S.EnvironmentalProtectionAgency).1995c.TheProbabilityofSeaLevelRise.EPA/230/R‐95/008.Washington,D.C.:ClimateChangeDivision,USEPA.http://nepis.epa.gov/Exe/ZyPURL.cgi?Dockey=20011G1O.txt

IPCC(IntergovernmentalPanelonClimateChange).1996.ClimateChange1995:TheScienceofClimateChange.ContributionofWorkingGroupItotheSecondAssessmentoftheIntergovernmentalPanelonClimateChange.Cambridge:CambridgeUniversityPress.

Titus,J.G.,andV.Narayanan.1996.“TheRiskofSeaLevelRise.”ClimaticChange33(2):151–212.

Case Study 16: Knowledge Elicitation for a Bayesian Belief Network Model of Stream Ecology Theidentificationofthecausalpathwaysleadingtostreamimpairmentisacentralchallengetounderstandingecologicalrelationships.Bayesianbeliefnetworks(BBNs)areapromisingtoolfor

83

modelingpresumedcausalrelationships,providingamodelingstructurewithinwhichdifferentfactorsdescribingtheecosystemcanbecausallylinkedandcalculatinguncertaintiesexpressedforeachlinkage.

BBNscanbeusedtomodelcomplexsystemsthatinvolveseveralinterdependentorinterrelatedvariables.Ingeneral,aBBNisamodelthatevaluatessituationswheresomeinformationalreadyisknown,andincomingdataareuncertainorpartiallyunavailable.Theinformationisdepictedwithinfluencediagramsthatpresentasimplevisualrepresentationofadecisionproblem,forwhichquantitativeestimatesofeffectprobabilitiesareassigned.Assuch,BBNshavethepotentialforrepresentingecologicalknowledgeanduncertaintyinaformatthatisusefulforpredictingoutcomesfrommanagementactionsorfordiagnosingthecausesofobservedconditions.Generally,specificationofaBBNcanbeperformedusingavailableexperimentaldata,literaturereviewinformation(secondarydata)andexpertelicitation.AttemptstospecifyaBBNforthelinkagebetweenfinesedimentloadandmacroinvertebratepopulationsusingdatafromliteraturereviewshavefailedbecauseoftheabsenceofconsistentconceptualmodelsandthelackofquantitativedataorsummarystatisticsneededforthemodel.Inlightofthesedeficiencies,aneffortwasmadetouseexpertelicitationtospecifyaBBNfortherelationshipbetweenfinesedimentloadresultingfromhumanactivityandpopulationsofmacroinvertebrates.ThegoalsofthiseffortweretoexaminewhetherBBNsmightbeusefulformodelingstreamimpairmentandtoassesswhetherexpertopinioncouldbeelicitedtomaketheBBNapproachusefulasamanagementtool.

ProbabilisticAnalysis.ThiscasestudyprovidesanexampleofexpertelicitationinthedevelopmentofaBBNmodeloftheeffectofincreasedfinesedimentloadinastreamonmacroinvertebratepopulations(Group3).Forthepurposeofthisstudy,astreamsetting(aMidwestern,low‐gradientstream)andascenarioofimpairment(introductionofexcessfinesediment)wereused.Fivestreamecologistswithexperienceinthespecifiedgeographicsettingwereinvitedtoparticipateinanelicitationworkshop.Aninitialmodelwasdepictedusinginfluencediagrams,withthegoalsofstructuringandspecifyingthemodelusingexpertelicitation.Theecologistswereguidedthroughaknowledgeelicitationsessioninwhichtheydefinedfactorsthatdescribedrelevantchemical,physicalandbiologicalaspectsoftheecosystem.Theecologiststhendescribedhowthesefactorswereconnectedandwereaskedtoprovidesubjective,quantitativeestimatesofhowdifferentattributesofthemacroinvertebrateassemblagewouldchangeinresponsetoincreasedlevelsoffinesediment.Elicitedinputwasusedtorestructurethemodeldiagramandtodevelopprobabilisticestimatesoftherelationshipsamongthevariables.

ResultsofAnalysis.Theelicitedinputwascompiledandpresentedintermsofthemodelasstructuredbythestreamecologistsandtheirmodelspecifications.TheresultswerepresentedbothasrevisedinfluencediagramsandwithBayesianprobabilistictermsrepresentingtheelicitedinput.Thestudyyieldedseveralimportantlessons.Amongthesewerethattheelicitationprocesstakestime(includinganinitialsessionbyteleconferenceaswellasa3‐dayworkshop),definingascenariowithanappropriatedegreeofdetailiscriticalandelicitationofcomplexecologicalrelationshipsisfeasible.

ManagementConsiderations.Thestudywasconsideredsuccessfulforseveralreasons.First,thefeasibilityoftheelicitationapproachtobuildingstreamecosystemmodelswasdemonstrated.Thestudyalsoresultedinthedevelopmentofnewgraphicaltechniquestoperformtheelicitation.TheelicitedinputwasinterpretedintermsofpredictivedistributionstosupportfittingacompleteBayesianmodel.Furthermore,thestudywassuccessfulinbringingtogetheragroupofexpertsinaparticularsubjectareaforthepurposeofsharinginformationandlearningaboutexpertelicitationinsupportofmodelbuilding.Theexerciseprovidedinsightsintohowbesttoadaptknowledgeelicitationmethodstoecologicalquestionsandinformedtheassembledstreamecologistsontheelicitationprocessandonthepotentialbenefitsofthismodelingapproach.Theexplicit

84

quantificationofuncertaintyinthemodelnotonlyenhancestheutilityofthemodelpredictions,butalsocanhelpguidefutureresearch.

SelectedReferences.

Black,P.,T.Stockton,L.Yuan,D.Allan,W.Dodds,L.Johnson,M.Palmer,B.Wallace,andA.Stewart.2005.“UsingKnowledgeElicitationtoInformaBayesianBeliefNetworkModelofaStreamEcosystem.”Eos,Transactions,AmericanGeophysicalUnion86(18),JointAssemblySupplement,Abstract#NB41E‐05.

Yuan,L.2005.“TI:ABayesianApproachforCombiningDataSetstoImproveEstimatesofTaxonOptima.”Eos,Transactions,AmericanGeophysicalUnion86(18),JointAssemblySupplement,Abstract#NB41E‐04.

85

CASE STUDY REFERENCES Black,P.,T.Stockton,L.Yuan,D.Allan,W.Dodds,L.Johnson,M.Palmer,B.Wallace,andA.Stewart.2005.“UsingKnowledgeElicitationtoInformaBayesianBeliefNetworkModelofaStreamEcosystem.”Eos,Transactions,AmericanGeophysicalUnion86(18),JointAssemblySupplement,Abstract#NB41E‐05.

Burke,J.,M.Zufall,andH.Özkaynak.2001.“APopulationExposureModelforParticulateMatter:CaseStudyResultsforPM2.5inPhiladelphia,PA.”JournalofExposureAnalysisandEnvironmentalEpidemiology11(6):470–89.

Connelly,N.A.,T.L.Brown,andB.A.Knuth.1990.NewYorkStatewideAnglerSurvey1988.Albany,NY:BureauofFisheries,NewYorkDepartmentofEnvironmentalConservation.

Georgepoulos,P.G.,S.W.Wang,V.M.Vyas,Q.Sun,J.Burke,R.Vedantham,T.McCurdy,andH.Özkaynak.2005.“ASource‐to‐DoseAssessmentofPopulationExposuretoFinePMandOzoneinPhiladelphia,PA,DuringaSummer1999Episode.”JournalofExposureAnalysisandEnvironmentalEpidemiology15(5):439–57.

IPCC(IntergovernmentalPanelonClimateChange).1996.ClimateChange1995:TheScienceofClimateChange.ContributionofWorkingGroupItotheSecondAssessmentoftheIPPC.Cambridge:CambridgeUniversityPress.

Jamieson,D.1996.“ScientificUncertaintyandthePoliticalProcess.”AnnalsoftheAmericanAcademyofPoliticalandSocialScience545:35–43.

Kurowicka,D.,andR.Cooke.2006.UncertaintyAnalysisWithHighDimensionalDependentModeling.WileySeriesinProbabilityandStatistics.NewYork:JohnWiley&Sons.

Langstaff,J.E.2007.AnalysisofUncertaintyinOzonePopulationExposureModeling.OfficeofAirQualityPlanningandStandardsStaffMemorandumtoOzoneNAAQSReviewDocket.OAR‐2005‐0172.Washington,D.C.:USEPA.http://www.epa.gov/ttn/naaqs/standards/ozone/s_ozone_cr_td.html.

OlsenAR,KincaidTM,PaytonQ.2012.Spatiallybalancedsurveydesignsfornaturalresources.In:GitzenRA,MillspaughJJ,CooperAB,LichtDS(eds)DesignandAnalysisofLong‐TermEcologicalMonitoringStudies.CambridgeUniversityPress,Cambridge,UK,pp126‐150

Stahl,C.H.,andA.J.Cimorelli.2005.“HowMuchUncertaintyIsTooMuchandHowDoWeKnow?ACaseExampleoftheAssessmentofOzoneMonitorNetworkOptions.”RiskAnalysis25(5):1109–20.

Titus,J.G.andV.Narayanan.1996.“TheRiskofSeaLevelRise:ADelphicMonteCarloAnalysisinWhichTwentyResearchersSpecifySubjectiveProbabilityDistributionsforModelCoefficientsWithinTheirRespectiveAreasofExpertise.”ClimaticChange33(2):151–212.

USEPA(U.S.EnvironmentalProtectionAgency).1992a.GuidelinesforExposureAssessment.EPA/600/Z‐92/001.Washington,D.C.:RiskAssessmentForum,USEPA.http://cfpub.epa.gov/ncea/cfm/recordisplay.cfm?deid=15263.

USEPA.1992b.FrameworkforEcologicalRiskAssessment.EPA/630/R‐92/001.Washington,D.C.:RiskAssessmentForum,USEPA.http://www.epa.gov/raf/publications/framework‐eco‐risk‐assessment.htm.

USEPA.1995a.GuidanceforRiskCharacterization.Washington,D.C.:SciencePolicyCouncil,USEPA.http://www.epa.gov/spc/pdfs/rcguide.pdf.

86

USEPA.1995b.PolicyonEvaluatingHealthRiskstoChildren.Washington,D.C.:SciencePolicyCouncil,USEPA.http://www.epa.gov/spc/pdfs/memohlth.pdf.

USEPA.1995c.TheProbabilityofSeaLevelRise.EPA/230/R‐95/008.Washington,D.C.:OfficeofPolicy,Planning,andEvaluation,USEPA.http://nepis.epa.gov/Exe/ZyPURL.cgi?Dockey=20011G1O.txt.

USEPA.1997a.PolicyforUseofProbabilisticAnalysisinRiskAssessmentattheU.S.EnvironmentalProtectionAgency.Washington,D.C.:SciencePolicyCouncil,USEPA.http://www.epa.gov/spc/pdfs/probpol.pdf.

USEPA.1997b.GuidingPrinciplesforMonteCarloAnalysis.EPA/630/R‐97/001.Washington,D.C.:RiskAssessmentForum,USEPA.http://www.epa.gov/raf/publications/pdfs/montecar.pdf.

USEPA.1997c.GuidanceonCumulativeRiskAssessment.Part1:PlanningandScoping.Washington,D.C.:SciencePolicyCouncil,USEPA.http://www.epa.gov/spc/pdfs/cumrisk2.pdf.

USEPA.1997d.ExposureFactorsHandbook.Washington,D.C.:NationalCenterforEnvironmentalAssessment,USEPA.http://cfpub.epa.gov/ncea/cfm/recordisplay.cfm?deid=12464.

USEPA.1997e.FinalReporttoCongressonBenefitsandCostsoftheCleanAirAct,1970to1990.EPA/410/R‐97/002.Cincinnati,OH:OfficeofAirandRadiation,USEPA.

USEPA.1998.GuidelinesforEcologicalRiskAssessment.EPA/630/R‐95/002F.Washington,D.C.:RiskAssessmentForum,USEPA.http://www.epa.gov/raf/publications/pdfs/ECOTXTBX.PDF.

USEPA.1999.TheBenefitsandCostsoftheCleanAirAct1990to2010:EPAReporttoCongress.EPA/410/R‐99/001.Washington,D.C.:OfficeofAirandRadiation,USEPA.http://www.epa.gov/oar/sect812/1990‐2010/chap1130.pdf.

USEPA.2000a.EPASciencePolicyCouncilHandbook:RiskCharacterization.EPA/100/B‐00/002.Washington,D.C.:SciencePolicyCouncil,USEPA.http://www.epa.gov/spc/pdfs/rchandbk.pdf.

USEPA.2000b.Phase2Report:FurtherSiteCharacterizationandAnalysis.Volume2F—RevisedHumanHealthRiskAssessment,HudsonRiverPCBsReassessmentRI/FS.NewYork:Region2,USEPA.http://www.epa.gov/hudson/revisedhhra‐text.pdf.

USEPA.2000c.FinalHeavyDutyEngine/DieselFuelRule:AirQualityEstimation,SelectedHealthandWelfareBenefitsMethods,andBenefitAnalysisResults.ResearchTrianglePark,NC:OfficeofAirQualityPlanningandStandards,USEPA.http://www.epa.gov/ttnecas1/regdata/Benefits/tsdhddv8.pdf.

USEPA.2001.RiskAssessmentGuidanceforSuperfund:VolumeIII—PartA,ProcessforConductingProbabilisticRiskAssessment.EPA/540/R‐02/002.Washington,D.C.:OfficeofEmergencyandRemedialResponse,USEPA.http://www.epa.gov/oswer/riskassessment/rags3adt/pdf/rags3adt_complete.pdf.

USEPA.2002.EMAPResearchStrategy.EPA620/R‐02/002.ResearchTrianglePark,NC:EnvironmentalMonitoringandAssessmentProgram,NHEERL,USEPA.http://www.epa.gov/nheerl/emap/files/emap_research_strategy.pdf.

USEPA.2004.AnExaminationofEPARiskAssessmentPrinciplesandPractices.EPA/100/B‐04/001.Washington,D.C.:OfficeoftheScienceAdvisor,USEPA.http://www.epa.gov/osa/pdfs/ratf‐final.pdf.

USEPA.2005.OzoneHealthAssessmentPlan:ScopeandMethodsforExposureAnalysisandRiskAssessment.ResearchTrianglePark,NC:OfficeofAirQualityPlanningandStandards,USEPA.http://www.epa.gov/ttn/naaqs/standards/ozone/data/o3_health_assessment_plan_april05.pdf.

87

USEPA.2006.TotalRiskIntegratedMethodology(TRIM)—AirPollutantsExposureModelDocumentation(TRIM.Expo/APEX,Version4)VolumeI:User’sGuide;VolumeII:TechnicalSupportDocument.ResearchTrianglePark,NC:OfficeofAirQualityPlanningandStandards,USEPA.http://www.epa.gov/ttn/fera/human_apex.html.

USEPA.2007a.ReviewofNationalAmbientAirQualityStandardsforOzone:PolicyAssessmentofScientificandTechnicalInformation—OAQPSStaffPaper.ResearchTrianglePark,NC:OfficeofAirQualityPlanningandStandards,USEPA.http://www.epa.gov/ttn/naaqs/standards/ozone/data/2007_07_ozone_staff_paper.pdf.

USEPA.2007b.OzonePopulationExposureAnalysisforSelectedUrbanAreas.EPA/452/R‐07‐010.ResearchTrianglePark,NC:OfficeofAirQualityPlanningandStandards,USEPA.http://www.epa.gov/ttn/naaqs/standards/ozone/data/2007_07_o3_exposure_tsd.pdf.

USEPA.2014.FrameworkforHumanHealthRiskAssessmenttoInformDecisionMaking.EPA/100/R‐14/001.Washington,D.C.:RiskAssessmentForum,OfficeoftheScienceAdvisor,USEPA.http://www.epa.gov/raf/frameworkhhra.htm

USGAO(U.S.GovernmentAccountabilityOffice).2000.WaterQuality:KeyEPAandStateDecisionsLimitedbyInconsistentandIncompleteData.GAO/RCED‐00‐54.Washington,D.C.:USGAO.http://www.environmental‐auditing.org/Portals/0/AuditFiles/useng00ar_ft_key_epa.pdf.

Xue,J.,V.G.Zartarian,H.Özkaynak,W.Dang,G.Glen,L.Smith,andC.Stallings.2006.“AProbabilisticArsenicExposureAssessmentforChildrenWhoContactChromatedCopperArsenate(CCA)‐TreatedPlaysetsandDecks,Part2:SensitivityandUncertaintyAnalyses.”RiskAnalysis26(2):533–41.

Yuan,L.2005.“TI:ABayesianApproachforCombiningDataSetstoImproveEstimatesofTaxonOptima.”Eos,Transactions,AmericanGeophysicalUnion86(18),JointAssemblySupplement,Abstract#NB41E‐04.

Zartarian,V.G.,J.Xue,H.A.Özkaynak,W.Dang,G.Glen,L.Smith,andC.Stallings.2006.AProbabilisticExposureAssessmentforChildrenWhoContactCCA‐TreatedPlaysetsandDecksUsingtheStochasticHumanExposureandDoseSimulationModelfortheWoodPreservativeScenario(SHEDS‐Wood),FinalReport.EPA/600/X‐05/009.Washington,D.C.:USEPA.

Zartarian,V.G.,J.Xue,H.A.Özkaynak,W.Dang,G.Glen,L.Smith,andC.Stallings.2006.“AProbabilisticArsenicExposureAssessmentforChildrenWhoContactChromatedCopperArsenate(CAA)‐TreatedPlaysetsandDecks,Part1:ModelMethodology,VariabilityResults,andModelEvaluation.”RiskAnalysis26(2):515–31.