Judea PearlComputer Science Department

UCLAwww.cs.ucla.edu/~judea

ROBUSTNESS OF CAUSAL CLAIMS

ROBUSTNESS:MOTIVATION

The effect of smoking on cancer is, in general, non-identifiable (from observational studies).

Smokingx y

Genetic Factors (unobserved)

Cancer

u

In linear systems: y = x + is non-identifiable.

ROBUSTNESS:MOTIVATION

Z – Instrumental variable; cov(z,u) = 0

Smokingy

Genetic Factors (unobserved)

Cancer

u

x

ZPrice ofCigarettes

xz

yz

xz

yzRR

RR

is identifiable

ROBUSTNESS:MOTIVATION

Problem with Instrumental Variables:The model may be wrong!

xz

yzyz R

RR

Smoking

ZPrice ofCigarettes

x y

Genetic Factors (unobserved)

Cancer

u

Smoking

ROBUSTNESS:MOTIVATION

Z1

Price ofCigarettes

Solution: Invoke several instruments

Surprise: 1 = 2 model is likely correct2

22

1

11

xz

yz

xz

yzRR

RR

x y

Genetic Factors (unobserved)

Cancer

u

PeerPressure

Z2

ROBUSTNESS:MOTIVATION

Z1

Price ofCigarettes

x y

Genetic Factors (unobserved)

Cancer

u

PeerPressure

Z2

Smoking

Greater surprise: 1 = 2 = 3….= n = qClaim = q is highly likely to be correct

Z3

Zn

Anti-smoking Legislation

ROBUSTNESS:MOTIVATION

x y

Genetic Factors (unobserved)

Cancer

u

Smoking

Symptoms do not act as instruments

remains non-identifiable

s

Symptom

Why? Taking a noisy measurement (s) of an observed variable (y) cannot add new information

ROBUSTNESS:MOTIVATION

x

Genetic Factors (unobserved)

Cancer

u

Smoking

Adding many symptoms does not help.

remains non-identifiable

ySymptom

S1

S2

Sn

ROBUSTNESS:MOTIVATION

Find if can evoke an equality surprise1 = 2 = …n

associated with several independent estimands of

x y

Given a parameter in a general graph

Formulate: Surprise, over-identification, independenceRobustness: The degree to which is robust to violations

of model assumptions

ROBUSTNESS:FORMULATION

Bad attempt: Parameter is robust (over identifies)

f1, f2: Two distinct functions

)()( 21 ff

distinct. are

then constraint induces model if

)]([)]([)()]([)(

,0)(

21

gtgtfgtf

g

i

if:

ROBUSTNESS:FORMULATION

ex ey ez

x y zb c

x = ex

y = bx + ey

z = cy + ez

Ryx = bRzx = bcRzy = c

zyyxzx

yxzxzy

zyzxyx

RRR

RRcRc

RRbRb

/

/

constraint:

(b)

(c)

y → z irrelvant to derivation of b

RELEVANCE:FORMULATION

Definition 8 Let A be an assumption embodied in model M, and p a parameter in M. A is said to be relevant to p if and only if there exists a set of assumptions S in M such that S and A sustain the identification of p but S alone does not sustain such identification.

Theorem 2 An assumption A is relevant to p if and only if A is a member of a minimal set of assumptions sufficient for identifying p.

ROBUSTNESS:FORMULATION

Definition 5 (Degree of over-identification)A parameter p (of model M) is identified to degree k (read: k-identified) if there are k minimal sets of assumptions each yielding a distinct estimand of p.

ROBUSTNESS:FORMULATION

x yb

zc

Minimal assumption sets for c.

x y zc x y zc

G3G2

x y zc

G1

Minimal assumption sets for b. x yb

z

FROM MINIMAL ASSUMPTION SETS TO MAXIMAL EDGE SUPERGRAPHS

FROM PARAMETERS TO CLAIMS

DefinitionA claim C is identified to degree k in model M (graph G), if there are k edge supergraphs of G that permit the identification of C, each yielding a distinct estimand.

TE(x,z) = Rzx TE(x,z) = Rzx Rzy ·x

x y zx y z

e.g., Claim: (Total effect) TE(x,z) = q x y z

CONCLUSIONS

1. Formal definition to ROBUSTNESS of causal claims: “A claim is robust when it is insensitive to

violations of some of the model assumptions”

2. Graphical criteria and algorithms for computing the degree of robustness of a given causal claim.