A study of MEBN Learning based on Relational Model

Cheol Young Park, Kathryn Blackmond Laskey, Paulo Costa, Shou Matsumoto

1

History of Statistical Relational Learning (SRL)

  Statistical Relational Learning (SRL) is a new branch of machine learning for representing and learning a joint distribution over relational data

  Traditionally, there are two groups of machine learning in data perspective   First group focuses on uncertainty

–  e.g. Incomplete data, Inaccurate data   Second group focuses on complex data structure

–  e.g. First Order Logic, Relation Database

  Both groups have their own learning methods and representing models

2

History of Statistical Relational Learning (SRL)

  Both groups realize that the real world data has the both properties   Uncertainty   Complex data structure

  To deal with the both properties of data, Statistical Relational Learning (SRL) is invented   Statistical: uncertainty   Relational: complex data structure   Learning: generating process from data to model

3

List of Statistical Relational Learning (SRL)

  Bayesian logic programs   BLOG models   Logic programs with annotated disjunctions   Markov logic networks   Multi-entity Bayesian networks (our model)   Probabilistic relational models   Recursive random fields   Relational Bayesian networks   Relational dependency networks   Relational Markov networks   Join Bayes Net

4

Concept of induction and deduction of SRL

Dataset from University A

Dataset from University B

Dataset from University C

Induction Deduction

BN from university A BN from university B BN from university C

University MEBN

5

Survey of SRLs

[A Survey on Statistical Relational Learning. 2010] 6

The goal of initial research of MEBN Learning

  Providing a basic learning method 1.  MEBN-RM Model

–  A bridge between MEBN and RM (Relational Model)

2.  A basic structure learning algorithm for MEBN –  A ground induction method

  Limitation   The basic learning method only focuses on

–  Discrete variable, complete data, relational model, no aggregating and cycle assumption

7

THE BASIC LEARNING METHOD

1.  MEBN-RM Model 2.  A basic structure learning algorithm for

MEBN

8

1.1. MEBN-RM Model: Relational Model

9

1.2. MEBN-RM Model: MEBN Model

10

Multi-entity Bayesian networks Model

Bayesian Networks Model

Deduction or reasoning

1.3. MEBN-RM Model

MEBN-RM Model 11

Relational Model MEBN Model

1.3. MEBN-RM Model: Example

12

Relational Model MEBN Model

MEBN-RM Model

2. Basic structure learning algorithm for MEBN

13

CASE STUDY   To evaluate the method, we used a dataset which came

from the PROGNOS (Probabilistic OntoloGies for Net-centric Operation Systems: The purpose of the system is to provide higher-level knowledge representation, fusion, and reasoning in the maritime domain.)

  The PROGNOS’ simulation provides the ground truth information sampled by a single entity Bayesian Network

  85000 persons, 10000 ships, and 1000 organizations with various values of attributes sampled were used for test dataset

14

Test process

15

Single entity Bayesian Network Relational Model/Database

Generated SSBN of PROGNOS MTheory Learned PROGNOS MTheory

1. Sampling

4. Comparing two models 2. Learning

3. Reasoning

Test Result: Learned PROGNOS MTheory

16

Test Result: Generated situation-specific Bayesian Network (SSBN)

17

Test Result: Comparing two models

18

ROC of Learned MTheory and Single Entity Network

AUC of Learned MTheory and Single Entity Network

DISCUSSION AND FUTURE WORK

  In this paper   As a bridge between MEBN and RM, MEBN-RM Model was

introduced   For induction, the Basic Structure Learning for MEBN was

suggested

  Future work   MEBN learning based on ontology   Incomplete and continuous data   Solving aggregating and cycle problem

19

References [1] Hassan Khosravi, Bahareh Bina. A Survey on Statistical Relational Learning. In Proceedings of Canadian Conference on AI'2010. pp.256~268 [2] Getoor, L., Tasker, B.: Introduction to statistical relational learning. MIT Press, Cambridge, 2007 [3] Domingos, P., Richardson, M.: Markov logic: A unifying framework for statistical relational learning. In: Introduction to Statistical Relational Learning, ch. 12, pp. 339–367, 2007 [4] Nevile, J., Jensen, D.: Relational dependency networks. In: An Introduction to Statistical Relational Learning

[5] Kersting, K., de Raedt, L.: Bayesian logic programming: Theory and tool. In: Introduction to Statistical Relational Learning [6] Oliver Schulte, Hassan Khosravi, Flavia Moser, and Martin Ester. Join bayes nets: A new type of bayes net for relational data. Technical Report 2008-17, Simon Fraser University, 2008. also in CS-Learning Preprint Archive. [7] Laskey, K. B.,MEBN: A Language for First-Order Bayesian Knowledge Bases. Artificial Intelligence, 172(2-3), 2008 [8] Pearl, J.: Probabilistic Reasoning in Intelligent Systems.Morgan Kaufmann, San Francisco, 1988 [9] Jensen, D., Neville, J.: Linkage and autocorrelation cause feature selection bias in relational learning. In: Proceedings of the 19th International Conference on Machine Learning, 2002 [10] S. Natarajan, P. Tadepalli, T. G Dietterich, and A. Fern. Learning first-order probabilistic models with combining rules. Special Issue on Probabilistic Relational Learning, AMAI, 2009. [11] Paulo C. G Costa, Bayesian Semantics for the Semantic Web. PhD Dissertation, George Mason University, July 2005. Brazilian Air Force. [12] Rommel N. Carvalho, Probabilistic Ontology: Representation and Modeling Methodology, PhD Dissertation, George Mason University, July 2011. [13] Codd, E.F. "A Relational Model of Data for Large Shared Data Banks". Communications of the ACM, 1970 [14] Codd, E.F. "Derivability, Redundancy, and Consistency of Relations Stored in Large Data Banks", IBM Research Report, 1969 [15] Suh, N. P., The Principles of Design, Oxford University Press, New York, 1990 [16] Shannon, C.E., "A Mathematical Theory of Communication", Bell System Technical Journal, 27, pp. 379–423 & 623–656, July & October, 1948 [17] Daphne Koller and Nir Friedman. Probabilistic Graphical Models: Principles and Techniques. The MIT Press, 1 edition, August 2009. [18] D. Heckerman. A tutorial on learning with Bayesian networks. In M. I. Jordan, editor, Learning in Graphical Models. MIT Press, Cambridge, MA, 1998. [19] Thomas M. Cover, Joy A. Thomas, Elements of Information Theory, 2006 [20] D.M. Chickering, Optimal Structure Identification with Greedy Search, 2002. [21] D.M. Chickering, Learning Equivalence Classes of Bayesian-Network Structures, 2002.

20