on bayesian measures 27 may 2005 v. crup i vincenzo crupi department of cognitive and education...

On Bayesian Measures27 May 2005

V. Crupi

Vincenzo Crupi

Department of Cognitive and Education SciencesLaboratory of Cognitive Sciences

University of Trento

27 May 2005

On Bayesian Measures of Evidential Support:

Normative and Descriptive Considerations


V. Crupi

Core Bayesianism…

(CB) confirmation is represented by a function X depending only on probabilistic information about evidence (premise) e and hypothesis (conclusion) h and identifying confirmation with an increase in the probability of h provided by the piece of information e, so that:

X(e,h)

> 0 iff p(h|e) > p(h)

= 0 iff p(h|e) = p(h)

< 0 iff p(h|e) < p(h)


V. Crupi

…and the plurality of Bayesian measures


V. Crupi

if p(h) = 0,5

D R L

continues


V. Crupi

symmetries and asymmetries: an example

Eells, E. & Fitelson, B. “Symmetries and Asymmetries in Evidential Support”, Philosophical Studies, 107 (2002), pp. 129-142

“evidence symmetry”:

X(e,h) = –X(¬e,h)

main thesis: the most adequate Bayesian measure(s) may be selected by testing competitors against intuitively compelling symmetries and asymmetries

e.g.: “does a piece of evidence e support a hypothesis h equally well as e’s negation (¬e) undermines the same hypothesis h?” (p. 129)

?


V. Crupi

symmetries and asymmetries: a unified account

a symmetry s is a function mapping an argument (e,h) onto a different argument by one or more of the following steps:

• negate e• negate h• invert premise and conclusion


V. Crupi

continues


“evidence symmetry” (ES):

ES(e,h) = (¬e,h)


V. Crupi


“hypothesis symmetry” (HS):

HS(e,h) = (e,¬h)

continues


V. Crupi


“inverse symmetry” (IS) such that:

IS(e,h) = (h,e)

continues


V. Crupi


“total symmetry” (TS):

TS(e,h) = (¬e,¬h)

continues


V. Crupi


“inverse evidence symmetry” (IES):

IES(e,h) = (h,¬e)

continues


V. Crupi

(e,h)

evidence (ES): (¬e,h) hypothesis (HS): (e,¬h) total (TS): (¬e,¬h) inverse (IS): (h,e)

inverse evidence (IES): (h,¬e)

inverse hypothesis (IHS): (¬h,e)

inverse total (ITS): (¬h,¬e) convergent

a symmetry s is: • convergent iff (e,h) and s(e,h) have the same direction (both confirmations or both disconfirmations)

• divergent iff (e,h) and s(e,h) have opposite directions

a convergent symmetry s holds iff X(e,h) = X[s(e,h)]

a divergent symmetry s holds iff X(e,h) = –X[s(e,h)]

continues


V. Crupi

Eells & Fitelson: an adequate measure of evidential support should violate “evidence symmetry” for at least some choice of e and h

e.g.:

• X(Jack,face) >> –X(not-Jack,face)

• X(ace, face) << –X(not-ace,face)

“the extremeness of logical implication [refutation] of (or conferring probability 1 [0] on) [the hypothesis] is not what is crucial to the examples for the purposes of evaluating […] (ES)” (p. 134)

continues

E & F suggest to extrapolate from extreme to non-extreme cases by simply assuming that the relevant premises describe reports of “very reliable, but fallible, assistants” (ibid.)


V. Crupi

V(e,h)

= +1 iff e implies h

= –1 iff e refutes h (i.e., implies not-h)

= 0 otherwise

principle of extrapolation (from the deductive to the inductive domain):

(PE) any symmetry s holds for an adequate measure of evidential support iff it demonstrably holds for V

continues


V. Crupi

• (PE) implies that (IS) should not hold generally

counterexample: X(Jack,face) >> X(face,Jack)

BUT (PE) also implies that (IS) should hold generally for pairs of disconfirmatory arguments

X(e,h)– = X(h,e)– because e refutes h iff h refutes e

• (PE) implies that (ES) should not hold generally

counterexamples involving deductive arguments

• (PE) implies that (HS) should hold generally

X(e,h) = –X(e,¬h) because e implies h iff e refutes ¬h

continues


V. Crupi

the whole set of consequences of (PE):

continues


V. Crupi

continues

any Bayesian measure of evidential support fulfils the consequences of (PE) concerning (HS) and (IS)

iff it fulfils all the consequences of (PE) concerning

the other symmetries

theorem:


V. Crupi

the whole set of consequences of PE:

continues


V. Crupi

the need for yet another Bayesian measure of confirmation

Eells & Fitelson (2002) suggest that measures D and L are to be preferred to other measures because both:

• fulfil (HS)• violate (ES)• violate (TS)• violate (IS) [but E&F disregard the disconfirmation case…]

however:


V. Crupi

continues

none of the currently available Bayesian measures of evidential support satisfies both (CB) and (PE)

is it possible to define a measure of evidential support satisfying both (CB) and (PE) (and therefore the whole set of desirable symmetries and asymmetries)?

existence proof:


V. Crupi

continues

0,5 p(h|e) 0,2 p(h|e) 0,8 p(h|e)

measure Z


V. Crupi

continues

Th 1. Z satisfies (HS)

Proof: Z(e,h)+ = 2/π • arcsin{1 – [p(¬h|e)/p(¬h)]}

= –2/π • arcsin{[p(¬h|e)/p(¬h)] – 1} = – Z(e,¬h)– since h = ¬(¬h), this is equivalent to (HS)

Th 2. Z satisfies (IS) for pairs of disconfirmatory arguments

Proof: Z(e,h)– = 2/π • arcsin{[p(h|e)/p(h)] – 1}

= 2/π • arcsin{[p(e|h)/p(e)] – 1} (by Bayes theorem) = Z(h,e)–

Th 3. Z violates (IS) for some pair of confirmatory arguments

Proof:

suppose that: p(h & e) = 49/10 p(h & ¬e) = 41/10 p(¬h & e) = 1/100 p(¬h & ¬e) = 9/100

then: Z(e,h)+ = 2/π • arcsin(4/5) > 2/π • arcsin(4/45) = Z(h,e)+


V. Crupi

empirical comparison of competing measures

are the most normatively justified confirmation measures among the most psychologically descriptive?

Tentori, K., Crupi, V., Bonini, N. & Osherson, D., “Comparison of Confirmation Measures”, 2005


V. Crupi

participants, materials and procedure:

• 26 students (Milan, Trento; mean age 24)

• 2 urns: (A) 30 black balls + 10 white balls (B) 15 black balls + 25 white balls• random selection of one urn (outcome hidden)• ten random extractions without replacement• after each extraction:


V. Crupi

continues

results: average correlations between judged evidential impact and confirmation measures

(evidential impact computed from objective probabilities)

each number is the average of 26 correlations (one per participant)

for each correlation n = 10

p(A[B]|e) denotes p(A|e) or p(B|e) as appropriate.

* = reliably greater than the average for p(A[B]|e) by paired t-test (p < 0,02)


V. Crupi

continues

comparison of Z with other confirmation measures (confirmation computed from objective probabilities)

each cell reports a paired t-test between the correlations obtained with the confirmation measures in the associated row and column

for each t-test, n = 26 (corresponding to the 26 participants)

the correlations each involve 10 observations

the last row of each cell shows the number of participants (out of 26) for whom Z predicted better than the rival measure at the top of the column


V. Crupi

conclusive remarks

• limitations: a strictly probabilistic setting

• the results suggest a remarkable convergence between the normative and the descriptive dimension in the study of evidential support (compare with probability judgments!…)

• most symmetries and asymmetries were not involved in the experiment (e.g., IS); so IF it turned out that the consequences of (PE) are in fact reflected in naïve subjects’ judgments of evidential impact, THEN: – none of the available Bayesian measures could fully account for human inferential processes (not even in purely probabilistic settings) – Z (or similar measures) could have an even greater advantage against competing alternatives than the one detected in our study

• suggestions for further studies: – more direct comparisons – direct test of various symmetries and asymmetries – extension to non-probabilistic settings


V. Crupi

thank you

on bayesian measures 27 may 2005 v. crup i vincenzo crupi department of cognitive and education...

Documents

h inverse

probability of h

h invert premise

hypothesis conclusion

h evidence es

h hypothesis hs

h total ts

e convergent