reasoning with bayesian networks. overview bayesian belief networks (bbns) can reason with networks...
TRANSCRIPT
![Page 1: Reasoning with Bayesian Networks. Overview Bayesian Belief Networks (BBNs) can reason with networks of propositions and associated probabilities Useful](https://reader036.vdocuments.net/reader036/viewer/2022081519/56649e155503460f94afe9bd/html5/thumbnails/1.jpg)
Reasoning with Bayesian Networks
![Page 2: Reasoning with Bayesian Networks. Overview Bayesian Belief Networks (BBNs) can reason with networks of propositions and associated probabilities Useful](https://reader036.vdocuments.net/reader036/viewer/2022081519/56649e155503460f94afe9bd/html5/thumbnails/2.jpg)
Overview
• Bayesian Belief Networks (BBNs) can reason with networks of propositions and associated probabilities
• Useful for many AI problems– Diagnosis– Expert systems– Planning– Learning
![Page 3: Reasoning with Bayesian Networks. Overview Bayesian Belief Networks (BBNs) can reason with networks of propositions and associated probabilities Useful](https://reader036.vdocuments.net/reader036/viewer/2022081519/56649e155503460f94afe9bd/html5/thumbnails/3.jpg)
Recall Bayes Rule
)()|()()|(),( HPHEPEPEHPEHP
)(
)()|()|(
EP
HPHEPEHP
Note the symmetry: we can compute the probability of a hypothesis given its evidence and vice versa.
![Page 4: Reasoning with Bayesian Networks. Overview Bayesian Belief Networks (BBNs) can reason with networks of propositions and associated probabilities Useful](https://reader036.vdocuments.net/reader036/viewer/2022081519/56649e155503460f94afe9bd/html5/thumbnails/4.jpg)
Simple Bayesian Network
CancerSmoking heavylightnoS ,,
malignantbenignnoneC ,,P(S=no) 0.80P(S=light) 0.15P(S=heavy) 0.05
Smoking= no light heavyP(C=none) 0.96 0.88 0.60P(C=benign) 0.03 0.08 0.25P(C=malig) 0.01 0.04 0.15
![Page 5: Reasoning with Bayesian Networks. Overview Bayesian Belief Networks (BBNs) can reason with networks of propositions and associated probabilities Useful](https://reader036.vdocuments.net/reader036/viewer/2022081519/56649e155503460f94afe9bd/html5/thumbnails/5.jpg)
More Complex Bayesian Network
Smoking
GenderAge
Cancer
LungTumor
SerumCalcium
Exposureto Toxics
![Page 6: Reasoning with Bayesian Networks. Overview Bayesian Belief Networks (BBNs) can reason with networks of propositions and associated probabilities Useful](https://reader036.vdocuments.net/reader036/viewer/2022081519/56649e155503460f94afe9bd/html5/thumbnails/6.jpg)
More Complex Bayesian Network
Smoking
GenderAge
Cancer
LungTumor
SerumCalcium
Exposureto Toxics
Links represent“causal” relations
Nodesrepresentvariables
![Page 7: Reasoning with Bayesian Networks. Overview Bayesian Belief Networks (BBNs) can reason with networks of propositions and associated probabilities Useful](https://reader036.vdocuments.net/reader036/viewer/2022081519/56649e155503460f94afe9bd/html5/thumbnails/7.jpg)
More Complex Bayesian Network
Smoking
GenderAge
Cancer
LungTumor
SerumCalcium
Exposureto Toxics
predispositions
![Page 8: Reasoning with Bayesian Networks. Overview Bayesian Belief Networks (BBNs) can reason with networks of propositions and associated probabilities Useful](https://reader036.vdocuments.net/reader036/viewer/2022081519/56649e155503460f94afe9bd/html5/thumbnails/8.jpg)
More Complex Bayesian Network
Smoking
GenderAge
Cancer
LungTumor
SerumCalcium
Exposureto Toxics
condition
![Page 9: Reasoning with Bayesian Networks. Overview Bayesian Belief Networks (BBNs) can reason with networks of propositions and associated probabilities Useful](https://reader036.vdocuments.net/reader036/viewer/2022081519/56649e155503460f94afe9bd/html5/thumbnails/9.jpg)
More Complex Bayesian Network
Smoking
GenderAge
Cancer
LungTumor
SerumCalcium
Exposureto Toxics
observable symptoms
![Page 10: Reasoning with Bayesian Networks. Overview Bayesian Belief Networks (BBNs) can reason with networks of propositions and associated probabilities Useful](https://reader036.vdocuments.net/reader036/viewer/2022081519/56649e155503460f94afe9bd/html5/thumbnails/10.jpg)
Independence
Age and Gender are independent.
P(A|G) = P(A) A G P(G|A) = P(G) G A
GenderAge
P(A,G) = P(G|A) P(A) = P(G)P(A)P(A,G) = P(A|G) P(G) = P(A)P(G)
P(A,G) = P(G)P(A)
![Page 11: Reasoning with Bayesian Networks. Overview Bayesian Belief Networks (BBNs) can reason with networks of propositions and associated probabilities Useful](https://reader036.vdocuments.net/reader036/viewer/2022081519/56649e155503460f94afe9bd/html5/thumbnails/11.jpg)
Conditional Independence
Smoking
GenderAge
Cancer
Cancer is independent of Age and Gender given Smoking.
P(C|A,G,S) = P(C|S) C A,G | S
![Page 12: Reasoning with Bayesian Networks. Overview Bayesian Belief Networks (BBNs) can reason with networks of propositions and associated probabilities Useful](https://reader036.vdocuments.net/reader036/viewer/2022081519/56649e155503460f94afe9bd/html5/thumbnails/12.jpg)
Conditional Independence: Naïve Bayes
Cancer
LungTumor
SerumCalcium
Serum Calcium is independent of Lung Tumor, given Cancer
P(L|SC,C) = P(L|C)
Serum Calcium and Lung Tumor are dependent
Naïve Bayes assumption: evidence (e.g., symptoms) is indepen-dent given the disease. This make it easy to combine evidence
![Page 13: Reasoning with Bayesian Networks. Overview Bayesian Belief Networks (BBNs) can reason with networks of propositions and associated probabilities Useful](https://reader036.vdocuments.net/reader036/viewer/2022081519/56649e155503460f94afe9bd/html5/thumbnails/13.jpg)
Explaining Away
Exposure to Toxics is dependent on Smoking, given Cancer
Exposure to Toxics and Smoking are independentSmoking
Cancer
Exposureto Toxics
P(E = heavy | C = malignant) >
P(E = heavy | C = malignant, S=heavy)
“Explaining away” is like abductive inference in that it moves from observation to possible causes or explanations.
![Page 14: Reasoning with Bayesian Networks. Overview Bayesian Belief Networks (BBNs) can reason with networks of propositions and associated probabilities Useful](https://reader036.vdocuments.net/reader036/viewer/2022081519/56649e155503460f94afe9bd/html5/thumbnails/14.jpg)
Conditional Independence
Smoking
GenderAge
Cancer
LungTumor
SerumCalcium
Exposureto Toxics Cancer is independent
of Age and Gender given Exposure to Toxics and Smoking.
Descendants
Parents
Non-Descendants
A variable (node) is conditionally independent of its non-descendants given its parents
![Page 15: Reasoning with Bayesian Networks. Overview Bayesian Belief Networks (BBNs) can reason with networks of propositions and associated probabilities Useful](https://reader036.vdocuments.net/reader036/viewer/2022081519/56649e155503460f94afe9bd/html5/thumbnails/15.jpg)
Another non-descendant
Diet Cancer is independent of Diet given Exposure to Toxics and Smoking
Smoking
GenderAge
Cancer
LungTumor
SerumCalcium
Exposureto Toxics
A variable is conditionally independent of its non-descendants given its parents
![Page 16: Reasoning with Bayesian Networks. Overview Bayesian Belief Networks (BBNs) can reason with networks of propositions and associated probabilities Useful](https://reader036.vdocuments.net/reader036/viewer/2022081519/56649e155503460f94afe9bd/html5/thumbnails/16.jpg)
BBN Construction
The knowledge acquisition process for a BBN involves three steps
– Choosing appropriate variables
– Deciding on the network structure
– Obtaining data for the conditional probability tables
![Page 17: Reasoning with Bayesian Networks. Overview Bayesian Belief Networks (BBNs) can reason with networks of propositions and associated probabilities Useful](https://reader036.vdocuments.net/reader036/viewer/2022081519/56649e155503460f94afe9bd/html5/thumbnails/17.jpg)
Risk of Smoking Smoking
They should be values, not probabilities
(1) Choosing variables
Variables should be collectively exhaustive, mutually exclusive values
4321 xxxx
jixx ji )(
Error Occurred
No Error
![Page 18: Reasoning with Bayesian Networks. Overview Bayesian Belief Networks (BBNs) can reason with networks of propositions and associated probabilities Useful](https://reader036.vdocuments.net/reader036/viewer/2022081519/56649e155503460f94afe9bd/html5/thumbnails/18.jpg)
Heuristic: Knowable in Principle
Example of good variables
– Weather {Sunny, Cloudy, Rain, Snow}
– Gasoline: Cents per gallon
– Temperature { 100F , < 100F}
– User needs help on Excel Charting {Yes, No}
– User’s personality {dominant, submissive}
![Page 19: Reasoning with Bayesian Networks. Overview Bayesian Belief Networks (BBNs) can reason with networks of propositions and associated probabilities Useful](https://reader036.vdocuments.net/reader036/viewer/2022081519/56649e155503460f94afe9bd/html5/thumbnails/19.jpg)
(2) Structuring
LungTumor
SmokingExposureto Toxic
GenderAgeNetwork structure correspondingto “causality” is usually good.
CancerGeneticDamage
Initially this uses the designer’sknowledge but can be checked with data
![Page 20: Reasoning with Bayesian Networks. Overview Bayesian Belief Networks (BBNs) can reason with networks of propositions and associated probabilities Useful](https://reader036.vdocuments.net/reader036/viewer/2022081519/56649e155503460f94afe9bd/html5/thumbnails/20.jpg)
(3) The numbers
• Zeros and ones are often enough
• Order of magnitude is typical: 10-9 vs 10-6
• Sensitivity analysis can be used to decide accuracy needed
• Second decimal usually doesn’t matter
• Relative probabilities are important
![Page 21: Reasoning with Bayesian Networks. Overview Bayesian Belief Networks (BBNs) can reason with networks of propositions and associated probabilities Useful](https://reader036.vdocuments.net/reader036/viewer/2022081519/56649e155503460f94afe9bd/html5/thumbnails/21.jpg)
Predictive Inference
How likely are elderly malesto get malignant cancer?
P(C=malignant | Age>60, Gender=male)
Smoking
GenderAge
Cancer
LungTumor
SerumCalcium
Exposureto Toxics
![Page 22: Reasoning with Bayesian Networks. Overview Bayesian Belief Networks (BBNs) can reason with networks of propositions and associated probabilities Useful](https://reader036.vdocuments.net/reader036/viewer/2022081519/56649e155503460f94afe9bd/html5/thumbnails/22.jpg)
Predictive and diagnostic combined
How likely is an elderly male patient with high Serum Calcium to have malignant cancer?
P(C=malignant | Age>60, Gender= male, Serum Calcium = high)
Smoking
GenderAge
Cancer
LungTumor
SerumCalcium
Exposureto Toxics
![Page 23: Reasoning with Bayesian Networks. Overview Bayesian Belief Networks (BBNs) can reason with networks of propositions and associated probabilities Useful](https://reader036.vdocuments.net/reader036/viewer/2022081519/56649e155503460f94afe9bd/html5/thumbnails/23.jpg)
Explaining away
Smoking
GenderAge
Cancer
LungTumor
SerumCalcium
Exposureto Toxics
• If we see a lung tumor, the probability of heavy smoking and of exposure to toxics both go up.
• If we then observe heavy smoking, the probability of exposure to toxics goes back down.
Smoking
![Page 24: Reasoning with Bayesian Networks. Overview Bayesian Belief Networks (BBNs) can reason with networks of propositions and associated probabilities Useful](https://reader036.vdocuments.net/reader036/viewer/2022081519/56649e155503460f94afe9bd/html5/thumbnails/24.jpg)
Decision making
• Decision - an irrevocable allocation of domain resources
• Decision should be made so as to maximize expected utility.
• View decision making in terms of– Beliefs/Uncertainties– Alternatives/Decisions– Objectives/Utilities
![Page 25: Reasoning with Bayesian Networks. Overview Bayesian Belief Networks (BBNs) can reason with networks of propositions and associated probabilities Useful](https://reader036.vdocuments.net/reader036/viewer/2022081519/56649e155503460f94afe9bd/html5/thumbnails/25.jpg)
A Decision Problem
Should I have my party inside or outside?
in
out
Regret
Relieved
Perfect!
Disaster
dry
wet
dry
wet
![Page 26: Reasoning with Bayesian Networks. Overview Bayesian Belief Networks (BBNs) can reason with networks of propositions and associated probabilities Useful](https://reader036.vdocuments.net/reader036/viewer/2022081519/56649e155503460f94afe9bd/html5/thumbnails/26.jpg)
Value Function
A numerical score over all possible states of the world allows BBN to be used to make decisions
Location? Weather? Valuein dry $50in wet $60out dry $100out wet $0
![Page 27: Reasoning with Bayesian Networks. Overview Bayesian Belief Networks (BBNs) can reason with networks of propositions and associated probabilities Useful](https://reader036.vdocuments.net/reader036/viewer/2022081519/56649e155503460f94afe9bd/html5/thumbnails/27.jpg)
Netica
• Software for working with Bayesian belief networks and influence diagrams
• A commercial product but free for small networks
• Includes a graphical editor, compiler, inference engine, etc.
• http://www.norsys.com/
![Page 28: Reasoning with Bayesian Networks. Overview Bayesian Belief Networks (BBNs) can reason with networks of propositions and associated probabilities Useful](https://reader036.vdocuments.net/reader036/viewer/2022081519/56649e155503460f94afe9bd/html5/thumbnails/28.jpg)
![Page 29: Reasoning with Bayesian Networks. Overview Bayesian Belief Networks (BBNs) can reason with networks of propositions and associated probabilities Useful](https://reader036.vdocuments.net/reader036/viewer/2022081519/56649e155503460f94afe9bd/html5/thumbnails/29.jpg)
Predispositions or causes
![Page 30: Reasoning with Bayesian Networks. Overview Bayesian Belief Networks (BBNs) can reason with networks of propositions and associated probabilities Useful](https://reader036.vdocuments.net/reader036/viewer/2022081519/56649e155503460f94afe9bd/html5/thumbnails/30.jpg)
Conditions or diseases
![Page 31: Reasoning with Bayesian Networks. Overview Bayesian Belief Networks (BBNs) can reason with networks of propositions and associated probabilities Useful](https://reader036.vdocuments.net/reader036/viewer/2022081519/56649e155503460f94afe9bd/html5/thumbnails/31.jpg)
Functional Node
![Page 32: Reasoning with Bayesian Networks. Overview Bayesian Belief Networks (BBNs) can reason with networks of propositions and associated probabilities Useful](https://reader036.vdocuments.net/reader036/viewer/2022081519/56649e155503460f94afe9bd/html5/thumbnails/32.jpg)
Symptoms or effects
Dyspnea is shortness of breath
![Page 33: Reasoning with Bayesian Networks. Overview Bayesian Belief Networks (BBNs) can reason with networks of propositions and associated probabilities Useful](https://reader036.vdocuments.net/reader036/viewer/2022081519/56649e155503460f94afe9bd/html5/thumbnails/33.jpg)
![Page 34: Reasoning with Bayesian Networks. Overview Bayesian Belief Networks (BBNs) can reason with networks of propositions and associated probabilities Useful](https://reader036.vdocuments.net/reader036/viewer/2022081519/56649e155503460f94afe9bd/html5/thumbnails/34.jpg)
![Page 35: Reasoning with Bayesian Networks. Overview Bayesian Belief Networks (BBNs) can reason with networks of propositions and associated probabilities Useful](https://reader036.vdocuments.net/reader036/viewer/2022081519/56649e155503460f94afe9bd/html5/thumbnails/35.jpg)
![Page 36: Reasoning with Bayesian Networks. Overview Bayesian Belief Networks (BBNs) can reason with networks of propositions and associated probabilities Useful](https://reader036.vdocuments.net/reader036/viewer/2022081519/56649e155503460f94afe9bd/html5/thumbnails/36.jpg)
![Page 37: Reasoning with Bayesian Networks. Overview Bayesian Belief Networks (BBNs) can reason with networks of propositions and associated probabilities Useful](https://reader036.vdocuments.net/reader036/viewer/2022081519/56649e155503460f94afe9bd/html5/thumbnails/37.jpg)