![Page 1: Evaluating a Bayesian Network using - abnms.org...ABNMS 2012 4th Annual Conference of the Australasian Bayesian Network Modelling Society 26th th– 30 November 2012 7 Sensitivity](https://reader033.vdocuments.net/reader033/viewer/2022051901/5ff0b9c0ebb2c77b790effa8/html5/thumbnails/1.jpg)
ABNMS 2012 4th Annual Conference of the Australasian Bayesian Network Modelling Society
26th – 30th November 2012 1
Ann E. Nicholson, David Albrecht, Lucas Azzola, Michael Gill and Stuart Lloyd
Monash University
Evaluating a Bayesian Network using “Sensitivity to Findings”:
How useful is it?
![Page 2: Evaluating a Bayesian Network using - abnms.org...ABNMS 2012 4th Annual Conference of the Australasian Bayesian Network Modelling Society 26th th– 30 November 2012 7 Sensitivity](https://reader033.vdocuments.net/reader033/viewer/2022051901/5ff0b9c0ebb2c77b790effa8/html5/thumbnails/2.jpg)
ABNMS 2012 4th Annual Conference of the Australasian Bayesian Network Modelling Society
26th – 30th November 2012 2
Overview
• Sensitivity Analysis
• Mutual Information
• A Simple Metric
• Examples
![Page 3: Evaluating a Bayesian Network using - abnms.org...ABNMS 2012 4th Annual Conference of the Australasian Bayesian Network Modelling Society 26th th– 30 November 2012 7 Sensitivity](https://reader033.vdocuments.net/reader033/viewer/2022051901/5ff0b9c0ebb2c77b790effa8/html5/thumbnails/3.jpg)
ABNMS 2012 4th Annual Conference of the Australasian Bayesian Network Modelling Society
26th – 30th November 2012 3
Sensitivity Analysis
• Evidence
• Parameters
• Structure
• Decisions
![Page 4: Evaluating a Bayesian Network using - abnms.org...ABNMS 2012 4th Annual Conference of the Australasian Bayesian Network Modelling Society 26th th– 30 November 2012 7 Sensitivity](https://reader033.vdocuments.net/reader033/viewer/2022051901/5ff0b9c0ebb2c77b790effa8/html5/thumbnails/4.jpg)
ABNMS 2012 4th Annual Conference of the Australasian Bayesian Network Modelling Society
26th – 30th November 2012 4
Chest Clinic Network
Tuberculosis
presentabsent
1.0499.0
XRay Result
abnormalnormal
11.089.0
Tuberculosis or Cancer
truefalse
6.4893.5
Lung Cancer
presentabsent
5.5094.5
Dyspnea
presentabsent
43.656.4
Bronchitis
presentabsent
45.055.0
Visit To Asia
visitno visit
1.0099.0
Smoking
smokernon smoker
50.050.0
Contributing Factors
Diseases
Symptoms
Chest Clinic
![Page 5: Evaluating a Bayesian Network using - abnms.org...ABNMS 2012 4th Annual Conference of the Australasian Bayesian Network Modelling Society 26th th– 30 November 2012 7 Sensitivity](https://reader033.vdocuments.net/reader033/viewer/2022051901/5ff0b9c0ebb2c77b790effa8/html5/thumbnails/5.jpg)
ABNMS 2012 4th Annual Conference of the Australasian Bayesian Network Modelling Society
26th – 30th November 2012 5
Sensitivity of “Lung Cancer”
Node Mutual Information Percentage
Lung Cancer 0.30727 100
Tuberculosis or Cancer 0.26747 87
Xray Result 0.18481 60.1
Smoking 0.03237 10.5
Dyspnea 0.02538 8.26
Bronchitis 0.00254 0.827
Visit to Asia 0 0
Tuberculosis 0 0
![Page 6: Evaluating a Bayesian Network using - abnms.org...ABNMS 2012 4th Annual Conference of the Australasian Bayesian Network Modelling Society 26th th– 30 November 2012 7 Sensitivity](https://reader033.vdocuments.net/reader033/viewer/2022051901/5ff0b9c0ebb2c77b790effa8/html5/thumbnails/6.jpg)
ABNMS 2012 4th Annual Conference of the Australasian Bayesian Network Modelling Society
26th – 30th November 2012 6
Sensitivity of “Lung Cancer”
Node Mutual Information Percentage
Lung Cancer 0.30727 100
Tuberculosis or Cancer 0.26747 87
Xray Result 0.18481 60.1
Smoking 0.03237 10.5
Dyspnea 0.02538 8.26
Bronchitis 0.00254 0.827
Visit to Asia 0 0
Tuberculosis 0 0
0.03237/0.30727 * 100% ≈ 10.5%
![Page 7: Evaluating a Bayesian Network using - abnms.org...ABNMS 2012 4th Annual Conference of the Australasian Bayesian Network Modelling Society 26th th– 30 November 2012 7 Sensitivity](https://reader033.vdocuments.net/reader033/viewer/2022051901/5ff0b9c0ebb2c77b790effa8/html5/thumbnails/7.jpg)
ABNMS 2012 4th Annual Conference of the Australasian Bayesian Network Modelling Society
26th – 30th November 2012 7
Sensitivity of “Lung Cancer”
Tuberculosis
presentabsent
1.0499.0
XRay Result
abnormalnormal
11.089.0
Tuberculosis or Cancer
truefalse
6.4893.5
Lung Cancer
presentabsent
5.5094.5
Dyspnea
presentabsent
43.656.4
Bronchitis
presentabsent
45.055.0
Visit To Asia
visitno visit
1.0099.0
Smoking
smokernon smoker
50.050.0
Contributing Factors
Diseases
Symptoms
Chest Clinic
Node MI
Lung Cancer 0.30727
Tuberculosis or Cancer
0.26747
Xray Result 0.18481
Smoking 0.03237
Dyspnea 0.02538
Bronchitis 0.00254
Visit to Asia 0
Tuberculosis 0 Independent of Lung Cancer
Entropy
![Page 8: Evaluating a Bayesian Network using - abnms.org...ABNMS 2012 4th Annual Conference of the Australasian Bayesian Network Modelling Society 26th th– 30 November 2012 7 Sensitivity](https://reader033.vdocuments.net/reader033/viewer/2022051901/5ff0b9c0ebb2c77b790effa8/html5/thumbnails/8.jpg)
ABNMS 2012 4th Annual Conference of the Australasian Bayesian Network Modelling Society
26th – 30th November 2012 8
Mutual Information
x1 … xn
y1 P(x1,y1) … P(xn,y1)
: : :
ym P(x1,ym)
… P(xn,ym)
M (X,Y ) = P(xij=1
m
åi=1
n
å , y j )log(P(xi )P(y j )
P(xi, y j ))
Y
X
![Page 9: Evaluating a Bayesian Network using - abnms.org...ABNMS 2012 4th Annual Conference of the Australasian Bayesian Network Modelling Society 26th th– 30 November 2012 7 Sensitivity](https://reader033.vdocuments.net/reader033/viewer/2022051901/5ff0b9c0ebb2c77b790effa8/html5/thumbnails/9.jpg)
ABNMS 2012 4th Annual Conference of the Australasian Bayesian Network Modelling Society
26th – 30th November 2012 9
Some Properties
• M(X, Y) = M(Y, X)
• 0 ≤ M(X, Y) ≤ M(X, X)
• If X Y Z then M(X, Y) ≥ M(X, Z)
• If X Y Z then M(X, Y) ≥ M(X, Z)
• If X Y Z then M(X, Y) ≥ M(X, Z)
![Page 10: Evaluating a Bayesian Network using - abnms.org...ABNMS 2012 4th Annual Conference of the Australasian Bayesian Network Modelling Society 26th th– 30 November 2012 7 Sensitivity](https://reader033.vdocuments.net/reader033/viewer/2022051901/5ff0b9c0ebb2c77b790effa8/html5/thumbnails/10.jpg)
ABNMS 2012 4th Annual Conference of the Australasian Bayesian Network Modelling Society
26th – 30th November 2012 10
0.00 0.05 0.10 0.15
0.0
0.1
0.2
0.3
0.4
M(Q, T)
M(Q
, A
)
T
A B
Q
M(Q,A) ≤ M(Q,T)
![Page 11: Evaluating a Bayesian Network using - abnms.org...ABNMS 2012 4th Annual Conference of the Australasian Bayesian Network Modelling Society 26th th– 30 November 2012 7 Sensitivity](https://reader033.vdocuments.net/reader033/viewer/2022051901/5ff0b9c0ebb2c77b790effa8/html5/thumbnails/11.jpg)
ABNMS 2012 4th Annual Conference of the Australasian Bayesian Network Modelling Society
26th – 30th November 2012 11
Structure Metrics
• Measure the component of sensitivity of evidence that depends on structure.
• Depends on the number of:
– Nodes in a path
– Paths between nodes
• Investigated a simple metric:
– Distance Weighted Influence (DWI)
![Page 12: Evaluating a Bayesian Network using - abnms.org...ABNMS 2012 4th Annual Conference of the Australasian Bayesian Network Modelling Society 26th th– 30 November 2012 7 Sensitivity](https://reader033.vdocuments.net/reader033/viewer/2022051901/5ff0b9c0ebb2c77b790effa8/html5/thumbnails/12.jpg)
ABNMS 2012 4th Annual Conference of the Australasian Bayesian Network Modelling Society
26th – 30th November 2012 12
Distance Weighted Influence (DWI)
I(Q,T ) = I(path)path
å ,
where the summation is over all unblocked paths
between Q and T, and
I(path) =wlength(path)
![Page 13: Evaluating a Bayesian Network using - abnms.org...ABNMS 2012 4th Annual Conference of the Australasian Bayesian Network Modelling Society 26th th– 30 November 2012 7 Sensitivity](https://reader033.vdocuments.net/reader033/viewer/2022051901/5ff0b9c0ebb2c77b790effa8/html5/thumbnails/13.jpg)
ABNMS 2012 4th Annual Conference of the Australasian Bayesian Network Modelling Society
26th – 30th November 2012 13
Clinic Example
Node DWI W=0.5
Lung Cancer 1 1
Tuberculosis or Cancer
w 0.5
Xray Result w2 0.25
Smoking w 0.5
Dyspnea w2+w3 0.375
Bronchitis w2 0.25
Visit to Asia 0 0
Tuberculosis 0 0
Visit to Asia
Tuberculosis
Tuberculosis or Cancer
Lung Cancer
Bronchitis
Smoking
Dyspnea Xray
Result
![Page 14: Evaluating a Bayesian Network using - abnms.org...ABNMS 2012 4th Annual Conference of the Australasian Bayesian Network Modelling Society 26th th– 30 November 2012 7 Sensitivity](https://reader033.vdocuments.net/reader033/viewer/2022051901/5ff0b9c0ebb2c77b790effa8/html5/thumbnails/14.jpg)
ABNMS 2012 4th Annual Conference of the Australasian Bayesian Network Modelling Society
26th – 30th November 2012 14
Clinic Example with Determistic Nodes
Node DWI W=0.5
Lung Cancer 1 1
Tuberculosis or Cancer
1 1
Xray Result w 0.5
Smoking w 0.5
Dyspnea w+w3 0.625
Bronchitis w2 0.25
Visit to Asia 0 0
Tuberculosis 0 0
Visit to Asia
Tuberculosis
Tuberculosis or Cancer
Lung Cancer
Bronchitis
Smoking
Dyspnea Xray
Result
![Page 15: Evaluating a Bayesian Network using - abnms.org...ABNMS 2012 4th Annual Conference of the Australasian Bayesian Network Modelling Society 26th th– 30 November 2012 7 Sensitivity](https://reader033.vdocuments.net/reader033/viewer/2022051901/5ff0b9c0ebb2c77b790effa8/html5/thumbnails/15.jpg)
ABNMS 2012 4th Annual Conference of the Australasian Bayesian Network Modelling Society
26th – 30th November 2012 15
Clinic Example with Evidence
Node DWI W=0.5
Lung Cancer 1 1
Tuberculosis w2 0.25
Smoking w 0.5
Bronchitis w3 0.125
Visit to Asia w3 0.125
Dyspnea w3 0.125
Xray Result 0 0
Tuberculosis or Cancer
0 0
Visit to Asia
Tuberculosis
Tuberculosis or Cancer
Lung Cancer
Bronchitis
Smoking
Dyspnea Xray
Result
![Page 16: Evaluating a Bayesian Network using - abnms.org...ABNMS 2012 4th Annual Conference of the Australasian Bayesian Network Modelling Society 26th th– 30 November 2012 7 Sensitivity](https://reader033.vdocuments.net/reader033/viewer/2022051901/5ff0b9c0ebb2c77b790effa8/html5/thumbnails/16.jpg)
ABNMS 2012 4th Annual Conference of the Australasian Bayesian Network Modelling Society
26th – 30th November 2012 16
Clinic Example with Evidence and Determisitic Node
Node DWI W=0.5
Lung Cancer 1 1
Tuberculosis w 0.5
Smoking w 0.5
Bronchitis w2 0.25
Visit to Asia w2 0.25
Dyspnea w3 0.125
Xray Result 0 0
Tuberculosis or Cancer
0 0
Visit to Asia
Tuberculosis
Tuberculosis or Cancer
Lung Cancer
Bronchitis
Smoking
Dyspnea Xray
Result
![Page 17: Evaluating a Bayesian Network using - abnms.org...ABNMS 2012 4th Annual Conference of the Australasian Bayesian Network Modelling Society 26th th– 30 November 2012 7 Sensitivity](https://reader033.vdocuments.net/reader033/viewer/2022051901/5ff0b9c0ebb2c77b790effa8/html5/thumbnails/17.jpg)
ABNMS 2012 4th Annual Conference of the Australasian Bayesian Network Modelling Society
26th – 30th November 2012 17
DWI
MI
![Page 18: Evaluating a Bayesian Network using - abnms.org...ABNMS 2012 4th Annual Conference of the Australasian Bayesian Network Modelling Society 26th th– 30 November 2012 7 Sensitivity](https://reader033.vdocuments.net/reader033/viewer/2022051901/5ff0b9c0ebb2c77b790effa8/html5/thumbnails/18.jpg)
ABNMS 2012 4th Annual Conference of the Australasian Bayesian Network Modelling Society
26th – 30th November 2012 18
Conclusions
• DWI gives a measure of the sensitivity of the structure.
• DWI can be calculated before the CPTs are filled in.
• Differences between MI and DWI suggest nodes to be investigated.
![Page 19: Evaluating a Bayesian Network using - abnms.org...ABNMS 2012 4th Annual Conference of the Australasian Bayesian Network Modelling Society 26th th– 30 November 2012 7 Sensitivity](https://reader033.vdocuments.net/reader033/viewer/2022051901/5ff0b9c0ebb2c77b790effa8/html5/thumbnails/19.jpg)
ABNMS 2012 4th Annual Conference of the Australasian Bayesian Network Modelling Society
26th – 30th November 2012 19
Future Work
• Improve the algorithm for calculating DWI.
• Investigated alternative metrics.
• Look more closely at theoretical foundations of these metrics.
• Investigate methods to visual the ordering.
• Investigate methods to visual the differences between MI and these metrics.