icml-tutorial, banff, canada, 2004 kristian kersting university of freiburg germany „application...
TRANSCRIPT
ICM
L-T
uto
ria
l, B
anff,
Ca
nad
a, 2
004
Kristian KerstingUniversity of FreiburgGermany
„Application of Probabilistic ILP II“, FP6-508861 www.aprill.org
James CussensUniversity of YorkUK
Probabilistic Logic Learning
Probability
Logic Learning
al and Relational
ICM
L-T
uto
ria
l, B
anff,
Ca
nad
a, 2
004
Special thanks to the APrIL II consortium
• „Application of Probabilistic ILP“
• 3 years EU project
• 5 institutes
• www.aprill.org
Heikki Mannila
Stephen Muggleton,Mike Sternberg
Subcontractor: James Cussens
Luc De RaedtSubcontractor: Manfred Jaeger
Paolo Frasconi
François Fages
ICM
L-T
uto
ria
l, B
anff,
Ca
nad
a, 2
004
... special thanks ...
... for discussions, materials, and collaborations to
Alexandru Cocura, Uwe Dick, Pedro Domingos, Peter Flach, Thomas Gaertner, Lise Getoor, Martin Guetlein,
Bernd Gutmann, Tapani Raiko, Reimund Renner, Richard Schmidt, Ingo Thon, ...
ICM
L-T
uto
ria
l, B
anff,
Ca
nad
a, 2
004
Tutorial´s Aims
• Introductory survey
• Identification of important probabilistic, relational/logical and learning concepts
ICM
L-T
uto
ria
l, B
anff,
Ca
nad
a, 2
004
The integration of probabilistic reasoning with
Objectives
One of the key open questions of AI concerns
Probabilistic Logic Learning:
machine learning.
first order / relational logic representations and
Probabilitiy
LearningLogic
ICM
L-T
uto
ria
l, B
anff,
Ca
nad
a, 2
004Diagnosis
Prediction
Classification
Decision-making
Description
Medicine
Computational Biology
Robotics
Web Mining
PLMs
Economic
Text Classification
Computer troubleshooting
Why do we need PLL?
Let‘s look at an example
ICM
L-T
uto
ria
l, B
anff,
Ca
nad
a, 2
004
Web Mining / Linked Bibliographic Data / Recommendation Systems / …
book
author
publisher
Real World
[illustration inspired by Lise Getoor]
publisher
book
book
book
author
ICM
L-T
uto
ria
l, B
anff,
Ca
nad
a, 2
004
Web Mining / Linked Bibliographic Data / Recommendation Systems / …
B1
B2
B3
B4P1
books
authors
publishers
series
author-ofpublisher-of
Real World
Fantasy ScienceFiction
P2
A2
A1
ICM
L-T
uto
ria
l, B
anff,
Ca
nad
a, 2
004
Why do we need PLL?R
eal W
orld
App
licat
ions
Let‘s look at some more examples
StructuredDomains
Not flat but structured representations:Multi-relational, heterogeneous and semi-structured
Uncertainty
Dealing with noisy data, missing data and hidden variables
MachineLearning
Knowledge Acquisition Bottleneck,Data cheap
ICM
L-T
uto
ria
l, B
anff,
Ca
nad
a, 2
004
Blood Type / Genetics/ Breeding
• 2 Alleles: A and a
• Probability of Genotypes AA, Aa, aa ?
CEPH Genotype DB,http://www.cephb.fr/
0
0.2
0.4
0.6
0.8
1AA AA
AA
AA Aa
AA Aa0
0.2
0.4
0.6
0.8
1
Aa aa
Aa aa0
0.2
0.4
0.6
0.8
1
Aa Aa
AA Aa0
0.2
0.4
0.6
0.8
1
Aa aa
aa aa
aa 0
0.2
0.4
0.6
0.8
1AA aa
Aa0
0.2
0.4
0.6
0.8
1
0
0.2
0.4
0.6
AA
Aa
aa
Prior for founders
Father Mother
Offspring
ICM
L-T
uto
ria
l, B
anff,
Ca
nad
a, 2
004
Others
Protein Secondary Structure
Metabolic PathwaysPhylogenetic Trees
Scene interpretation
Social Networks
Data Cleaning
?
ICM
L-T
uto
ria
l, B
anff,
Ca
nad
a, 2
004
Why do we need PLL ?R
eal W
orld
App
licat
ions
Uncertainty
MachineLearning
StructuredDomains
Statistical Learning (SL)Probabilistic Logics
Inductive Logic Programming (ILP)
Multi-Relational Data Mining (MRDM)
- attribute-value representations: some learning problems cannot (elegantly) be described using attribute value representations
+ soft reasoning, learning
- no learning: to expensive to handcraft models
+ soft reasoning, expressivity
- crisp reasoning: some learning problems cannot (elegantly) be described without explicit handling of uncertainty
+ expressivity, learning
ICM
L-T
uto
ria
l, B
anff,
Ca
nad
a, 2
004
Why do we need PLL?
• Rich Probabilistic Models• Comprehensibility• Generalization (similar situations/individuals)• Knowledge sharing• Parameter Reduction / Compression• Learning
– Reuse of experience (training one RV might improve prediction at other RV)
– More robust– Speed-up
ICM
L-T
uto
ria
l, B
anff,
Ca
nad
a, 2
004
When to apply PLL ?
• When it is impossible to elegantly represent your problem in attribute value form– variable number of ‘objects’ in examples– relations among objects are important
• Background knowledge can be defined intensionally :– define ‘benzene rings’ as view predicates
ICM
L-T
uto
ria
l, B
anff,
Ca
nad
a, 2
004
Overview
1. Introduction to PLL2. Foundations of PLL
– Logic Programming, Bayesian Networks, Hidden Markov Models, Stochastic Grammars
3. Frameworks of PLL– Independent Choice Logic,Stochastic Logic
Programs, PRISM,– Bayesian Logic Programs, Probabilistic Logic
Programs,Probabilistic Relational Models – Logical Hidden Markov Models
4. Applications