ilp for mathematical discovery
DESCRIPTION
ILP for Mathematical Discovery. Simon Colton & Stephen Muggleton Computational Bioinformatics Laboratory Imperial College. The Automation of Reasoning. Aims for the talk Discuss a new ILP algorithm (ATF) and its implementation in the HR system Promote maths as a domain for ILP research. - PowerPoint PPT PresentationTRANSCRIPT
ILP for Mathematical DiscoveryILP for Mathematical Discovery
Simon Colton & Stephen Muggleton
Computational Bioinformatics Laboratory
Imperial College
The Automation of ReasoningThe Automation of Reasoning
Aims for the talk– Discuss a new ILP algorithm (ATF)
• and its implementation in the HR system
– Promote maths as a domain for ILP research
AutomatedTheoremProving
MachineLearning
Maths
Bioinformatics
AutomatedReasoning
From Prediction to DescriptionFrom Prediction to Description
Predictivetasks
Descriptivetasks
Supervisedlearning
Unsupervisedlearning
Know what you’re looking for
Don’t knowyou’re even
looking
Don’t know what you’re looking for
A Partial Characterisation A Partial Characterisation of Learning Tasksof Learning Tasks
Concept learningOutlier/anomaly detectionClusteringConcept formationConjecture making
Theory formation
The HR Program in OverviewThe HR Program in Overview Embodies a novel ILP algorithm– We call this “Automated Theory Formation” (ATF)– Designed for descriptive tasks (in maths)
• But has had applications to concept learning tasks
– Incrementally builds a theory • Containing association and classification rules
HR has numerous tools for the user – To extract information from the theory generated
• Which is relevant to the task at hand
ATF OverviewATF Overview
Invent new conceptsDerive classification rule from conceptInduce hypotheses relating the conceptsProve/disprove the relationships– Deductively• Using state of the art ATP/model generators
Extract association rules – From the hypotheses
Input to HRInput to HR
Five inputs to HR– Objects of interest (graphs, groups, etc.)– A labelling of the objects
• If the task at hand is predictive…
– Background predicates (Prolog style)– Axioms relating predicates (ATP style)– Termination conditions
• HR works as an any time algorithm
User can supply– numerous different combinations of these
} SeePaper
Representation of Theory Contents Representation of Theory Contents Three types of frames
– All have a clausal definition slot Example frame Concept frame
– Slot 1: range-restricted program clause – Slot 2: success set– Slot 3: classification rule afforded by definition– Other slots: measures of value
Hypothesis frame– Slot 1: clauses (association rules)– Slot 2: proof/counterexample– Other slots: details of the concepts related
Cut Down Algorithm DescriptionCut Down Algorithm Description Build new concept definition from old
• Using one of 12 generic production rules [PR] (see paper)
Find the success set, S, of new concept If S is empty, derive non-existence hypothesis, H
• Extract association rules from H, try to prove/disprove
If S is a repeat, derive equivalence hypothesis, H• Extract association rules from H, try to prove/disprove
If S is new– Add new concept to theory– Derive classification rule– Derive implication & near-equivalence hypotheses
• Extract association rules, try to prove/disprove
Measure concepts in theory
Concept Space SearchedConcept Space Searched
Space determined by PRs, not language bias Clausal definition is:– range-restricted, fully typed program clauses
Definition: n-connectedness – Every variable appears in a body predicate with head
variable n, or with a n-connected variable Example:– c(X,Y) :- p(X), q(Y), p(Z), r(X,Y), s(Y,Z) is 1-connected– c(X,Y) :- r(Y), s(X,Z) is not 1-connected
HR’s definitions are all 1-connected
Deriving Classification RulesDeriving Classification Rules
Given definition D– Arity = n, head predicate = p, success set = S – Classifying function over constants, o, is:
Classification, C, afforded by D:– Put two objects in the same class if f(o1) = f(o2)
Theorem: – If a definition D is not 1-connected, then a literal can be
removed without changing the classifiction afforded by D– So, HR’s search space is non-redundant with respect to C
Illustrative ExampleIllustrative Example concept17(X,Y) :-
integer(X), integer(Y), divisor(X,Y), ¬ divisor(Y,2).
S17 = {(1,1), (2,1), (3,1), (3,3), (4,1), (5,1), (5,5), (6,1), (6,3)}
Classifying function:– f17(1)={(1)} f17(2)={(1)} f17(3)={(1),(3)} f17(4)={(1)}
– f17(5)={(1),(5)} f17(6)={(1),(3)}
Classification afforded by concept 17:– [ [1,2,4] [3,6] [5] ]
Mathematics ApplicationsMathematics Applications
Two applications given here– Both from external research groups– Data sets available online
See paper for details – Of two more applications
FindingFindingDiscriminantsDiscriminants
Finding discriminants of residue classes Work with Sorge and Meier Overall goal: classify algebraic structures– Bottleneck: showing non-isomorphism
Learning task:– Given two multiplication tables– Find a property true of only one
• Which doesn’t refer to individual elements
Data set: 817 pairs of tables (size 5, 6, 10)
ResultsResults
HR given 500 steps per task (~22 secs)– Worked with four production rules
Found discriminants – For 791 out of 817 pairs (~97%)– Average of 20 discriminants per pair– 517 distinct discriminants in total
Example above:– Idempotent element (a*a=a)
• Appearing once on diagonal
– Only one of two discriminants found for pair
Reformulation of CSPsReformulation of CSPs
Work with Miguel and Walsh Constraint satisfaction solving– Very powerful general purpose technique– Specifying a problem is still highly skilled
Learning task:– Given solutions to small problems
• Find concepts to specialise the problem specification• Find implied constraints to increase efficiency
Data set: QG-quasigroups (5 types)– Multiplication tables up to size 6
ResultsResults
HR ran for an hour for each problem class– Produced on average 150 association rules– And 10 specialisation concepts
In each case, a better reformulation was derived (with human interpretation)– Up to 10 times speed up in some cases
Nice example: QG3: (a*b)*(b*a)=a– These are Anti-abelian, i.e., a*b=b*a a=b– Symmetry relation: a*b=b b*a=a
Some Other ApplicationsSome Other Applications
Concept learning tasks:– Extrapolation of integer sequences: ICML’00
– Mutagenesis regression unfriendly Anomaly detection task:
– Analysis of Bach Chorale melodies (current MSc.) Conjecture making tasks:
– Generating TPTP library theorems: CADE’02 (& paper)– Finding links in the Gene Ontology (current MSc.)– Making Graffiti-style conjectures (current MSc.)
Theory formation task:– Invention of integer sequences: AAAI’00, JIS’01 (& paper)
ConclusionsConclusions
Presented the ATF algorithm– Involves induction & deduction– Presented for first time in ILP terminology– Characterised the concept search space
Presented two learning tasks– In mathematics–More in paper (and in previous work)
Shown that HR can make discoveries
Future WorkFuture Work
Apply HR to bioinformatics– Needs more efficient implementation
Look into the conglomeration of– Creative reasoning techniques
Relate HR to other descriptive programs– CLAUDIEN and WARMR
Can these programs – Do better than HR in maths applications?
A Drosophilia for A Drosophilia for Descriptive Induction?Descriptive Induction?
“Something from (nearly) nothing”Can give HR only 1 concept–Multiplication in number theory
Invents the concept of refactorable nums– Number of divisors is itself a divisor– 1, 2, 8, 9, 12, …
A nice hypothesis it produces is:– Odd refactorable numbers are square