arguments from experience: the padua protocol maya wardeh, trevor bench-capon and frans coenen...
TRANSCRIPT
Arguments From Experience: The PADUA Protocol
Maya Wardeh, Trevor Bench-Capon and Frans Coenen
Department of Computer ScienceUniversity of Liverpool
A Strange Beast• It is warm blooded, has hair, lays eggs, does not suckle
its young. Is it a mammal?
– Albert: It can’ be a mammal because it lays eggs– Bruce: I’ve seen mammals that lay eggs. And it does have
hair– Albert: But mammals with hair also suckle their young– Bruce: All hairy, warm blooded animals are mammals– Albert: So I suppose we must say it is
• Albert and Bruce argue on the basis of mammals they have seen
• Their experiences have been very different• There may be no right answer – yet!
PADUA
• PADUA – Protocol for Argumentation Dialogue Using Association Rules
• A dialogue game to argue about classification
• Arguments are taken directly from a data base of past examples using data mining techniques
Arguing from Experience
• Most dialogue systems are based on belief bases– Participants use facts and rules to construct
their arguments
• PADUA uses examples directly– Participants have a data base containing
collections of instances representing their past experience
• Resembles case based rather than rule based reasoning
Experiences May Differ
• Participants will have seen different samples.– Geographical: all swans are white in
the Northern hemisphere– Exceptions may be only rarely
encountered: insufficient support in some DBs
– Sample may be abnormal: in law, only hard cases seen at highest level of appeal
Advantages
• No knowledge representation bottleneck
• No advance commitment to a theory
• Can deal with gaps and conflicts• Pools experience
Dialogue Moves
• Participants point to features of the current case which are reasons why it should (or should not) be classified in a certain way
• Participants respond by citing other features which provide reasons to challenge the classification
Moves Based On Belief Bases• Claim P: P is the head of some rule• Why P: Seeks the body of rule for
which P is head• Concede P: agrees that P is true• Retract P: denies that P is true• P since S: A rule with P head and S
body
Persuasion in Belief Base Systems
• A Has a Rule with P as head, but one literal Q in the body is unknown: B shows that Q is true.
• B gives A a rule with P as head and body S. A already believes S
• A is shown to have an inconsistency: retraction enables P to be shown
Moves in Case Based Reasoning
• Citing a case: – A past case which shares features with the current case
and had the desired outcome
• Distinguishing a case: – features in the past case missing from the current case– Features in the current case missing from the past case
• Counter Example– A past case which shares features but had a different
outcome
• Arguments from Experience have many similarities
Moves in Argument from Experience• Citing reasons:
– Features in the current case which are typically associated with the desired classification, C
• Distinguishing– An additional feature which typically identifies an
exception– A feature which Cs typically have but which is not
present– A feature which increases confidence in the classification
• Counter Example– Features in the current case which are typically
associated with a different classification, not C
PADUA Scenario
A1 A2
A1: P suggests Q
A2: P’ suggests Q’
Instance Case
Class C1 Class C2 (C1)
PADUA Protocol - Basics
PADUA Protocol - BasicsPADUA Scenario
PADUA Moves1
6 2
3
4
5
1: Propose Rule
2: Distinguish
3: unwanted consequences
4: Counter Rule
5: Increase Confidence
6: Withdraw unwanted consequences
PADUA Protocol
P is a reason for C
it would be more a C if it were RCs are not Q
It need not be S
P’’ is a reason for not C
P and q is a reason for not C
Strategies
• The protocol offers a lot of scope for choice:
• Which move to make? – Introduce a new association or refine
an existing one?
• Which association to propose? – Best or just a good one? In terms of
confidence or support?
Strategies
• We have different strategies according to:– Aim: establish a rule or critique opponents
(build versus critique)– Persistence: concede when reasonable or only
when no argument left (agreeable versus disagreeable)
• Different strategies give rise to different flavours of dialogue:– Build + Disagreeable more like persuasion– Critique + Agreeable more like deliberation
Experiments
• We have experimented with a number of Data Sets:
• Poisonous Fungi• US Senators voting records
(ESQUARU 2007)• Welfare Benefits (COMMA 2008)
Welfare benefits
• Large numbers of cases • Lay adjudication: many (often
inexperienced) adjudicators• A high (often 20-30%) error rate is typical• Particular clerks and offices may make
systematic misinterpretations• PADUA can act as a moderation meeting,
allowing debate over classifications drawn from different adjudication sources
Conditions for Benefit
• Age condition: “Age appropriate to retirement” is interpreted as pensionable age: 60+ for women and 65+ for men.
• Income condition:“Available income” is interpreted as net disposable income, rather than gross income, and means that housing costs should exceed one fifth of candidates’ available income to qualify for the benefit.
• Capital condition: “Capital is inadequate” is interpreted as below the threshold for another benefit.
• Residence condition: “Resident in this country” is interpreted as having a UK address.
• Residence exception: “Service to the Nation” is interpreted as a member of the armed forces.
• Contribution condition: “Established connection with the UK labour force" is interpreted as having paid contributions in 3 of the last 5 years.
Results
• Given two databases, each containing a significant proportion of wrongly decided cases based on different systematic errors, correct classifications can be reached
• While this is true for errors concerning most attributes, success is markedly less when mistakes relate to the contribution condition
Intermediate Predicates
• These are legal concepts, which must be satisfied for the law to apply
• Need to be defined in terms of observable facts• Some can be unfolded into observable facts• Others need to applied on the balance of
consideration of a number of factors• These last present particular problems
– E.g. the contribution condition in the example– Also true for other machine learning and data mining
systems (e.g. Mozina et al)
Nesting Dialogues
• If we are aware of intermediate predicates which do not unfold appropriately into sufficient conditions, we can nest a dialogue to decide this issue within the main dialogue
• This handles the contribution problem• Confirms other work in AI and Law in
which issue based classification is more accurate than holistic approaches
Summary
• PADUA offers a novel kind of persuasion dialogue, based on examples rather than a belief base. The result has more in common with case based than rule based reasoning
• It avoids the need for knowledge representation effort
• The databases are not shared, enabling distinctive features of particular DBs to be identified and maintaining some level of privacy
• Where issues can be identified and resolved in preliminary dialogues, accuracy can be improved
The Talk Is Finished