judea pearl computer science department ucla cs.ucla/~judea
DESCRIPTION
Judea Pearl Computer Science Department UCLA www.cs.ucla.edu/~judea. DIRECT AND INDIRECT EFFECTS 2005. QUESTIONS ASKED. Why decompose effects? What are the semantics of direct and indirect effects (in nonlinear and nonparametric models)? - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Judea Pearl Computer Science Department UCLA cs.ucla/~judea](https://reader035.vdocuments.net/reader035/viewer/2022062800/5681415b550346895dad3632/html5/thumbnails/1.jpg)
Judea Pearl
Computer Science Department
UCLA
www.cs.ucla.edu/~judea
DIRECT AND INDIRECT EFFECTS
2005
![Page 2: Judea Pearl Computer Science Department UCLA cs.ucla/~judea](https://reader035.vdocuments.net/reader035/viewer/2022062800/5681415b550346895dad3632/html5/thumbnails/2.jpg)
QUESTIONS ASKED
• Why decompose effects?• What are the semantics of direct and indirect
effects (in nonlinear and nonparametric models)?• What are the policy implications of direct and
indirect effects?• When can direct and indirect effects be estimated
consistently from experimental or nonexperimental data?
• Can these conditions be verified from accessible causal knowledge, i.e., graphs?
![Page 3: Judea Pearl Computer Science Department UCLA cs.ucla/~judea](https://reader035.vdocuments.net/reader035/viewer/2022062800/5681415b550346895dad3632/html5/thumbnails/3.jpg)
1. Direct (or indirect) effect may be more transportable.2. Indirect effects may be prevented or controlled.
3. Direct (or indirect) effect may be forbidden
WHY DECOMPOSEEFFECTS?
Pill
Thrombosis
Pregnancy
+
+
Gender
Hiring
Qualification
![Page 4: Judea Pearl Computer Science Department UCLA cs.ucla/~judea](https://reader035.vdocuments.net/reader035/viewer/2022062800/5681415b550346895dad3632/html5/thumbnails/4.jpg)
EFFECT-DECOMPOSITIONIN LINEAR MODELS
X Z
Y
ca
b
effect Indirect effect Direct effect Total
a bc
Definition: ))(),(|( zxYEx
a do do
![Page 5: Judea Pearl Computer Science Department UCLA cs.ucla/~judea](https://reader035.vdocuments.net/reader035/viewer/2022062800/5681415b550346895dad3632/html5/thumbnails/5.jpg)
CAUSAL MODELS AND COUNTERFACTUALS
Definition: A causal model is a 3-tupleM = V,U,F
(i) V = {V1…,Vn} endogenous variables,(ii) U = {U1,…,Um} background variables (unit)(iii) F = set of n functions,
The sentence: “Y would be y (in unit u), had X been x,”denoted Yx(u) = y, is the solution for Y in a mutilated model Mx, with the equations for X replaced by X = x. (“unit-based potential outcome”)
),( uvfv ii
![Page 6: Judea Pearl Computer Science Department UCLA cs.ucla/~judea](https://reader035.vdocuments.net/reader035/viewer/2022062800/5681415b550346895dad3632/html5/thumbnails/6.jpg)
COUNTERFACTUALS:STRUCTURAL SEMANTICS
Notation: Yx(u) = y Y has the value y in the solution to a mutilated system of equations, where the equation for X is replaced by a constant X=x.
u
Yx(u)=y
Z
W
X=x
u
Y
Z
W
X
Probability of Counterfactuals:
FunctionalBayes Net
))(|()())(()( xdoyPu
uPyuxYPyxYP
)(uM,P
![Page 7: Judea Pearl Computer Science Department UCLA cs.ucla/~judea](https://reader035.vdocuments.net/reader035/viewer/2022062800/5681415b550346895dad3632/html5/thumbnails/7.jpg)
tindependen- ))(),(|(
))(|(
DETEIE
ZzdoxdoYEx
DE
xdoYEx
TE
TOTAL, DIRECT, AND INDIRECT EFFECTS HAVE CONTROLLED-BASED
SEMANTICS IN LINEAR MODELS
X Z
Y
ca
b z = bx + 1
y = ax + cz + 2
a + bc
bc
a
![Page 8: Judea Pearl Computer Science Department UCLA cs.ucla/~judea](https://reader035.vdocuments.net/reader035/viewer/2022062800/5681415b550346895dad3632/html5/thumbnails/8.jpg)
z = f (x, 1)y = g (x, z, 2)
????
))(),(|(
))(|(
IE
zdoxdoYEx
DE
xdoYEx
TE
X Z
Y
CONTROLLED-BASED SEMANTICS NONTRIVIAL IN NONLINEAR MODELS(even when the model is completely specified)
Dependent on z?
Void of operational meaning?
![Page 9: Judea Pearl Computer Science Department UCLA cs.ucla/~judea](https://reader035.vdocuments.net/reader035/viewer/2022062800/5681415b550346895dad3632/html5/thumbnails/9.jpg)
``The central question in any employment-discrimination case is whether the employer would have taken the same action had the employee been of different race (age, sex, religion, national origin etc.) and everything else had been the same’’
[Carson versus Bethlehem Steel Corp. (70 FEP Cases 921, 7th Cir. (1996))]
x = male, x = femaley = hire, y = not hirez = applicant’s qualifications
LEGAL DEFINITIONOF DIRECT EFFECT
(FORMALIZING DISCRIMINATION)
NO DIRECT EFFECT
',' ' xxxx YYYYxZxZ
![Page 10: Judea Pearl Computer Science Department UCLA cs.ucla/~judea](https://reader035.vdocuments.net/reader035/viewer/2022062800/5681415b550346895dad3632/html5/thumbnails/10.jpg)
z = f (x, u)y = g (x, z, u)
X Z
Y
NATURAL SEMANTICS OFAVERAGE DIRECT EFFECTS
Average Direct Effect of X on Y:The expected change in Y, when we change X from x0 to x1 and, for each u, we keep Z constant at whatever value it attained before the change.
In linear models, DE = Controlled Direct Effect
][001 xZx YYE
x
);,( 10 YxxDE
Robins and Greenland (1992) – “Pure”
![Page 11: Judea Pearl Computer Science Department UCLA cs.ucla/~judea](https://reader035.vdocuments.net/reader035/viewer/2022062800/5681415b550346895dad3632/html5/thumbnails/11.jpg)
POLICY IMPLICATIONS(Who cares?)
f
GENDER QUALIFICATION
HIRING
What is the direct effect of X on Y?
Is employer guilty of sex-discrimination given data on (X,Y,Z)?
X Z
Y
CAN WE IGNORE THIS LINK?
tYYE xZxx
][0
01
![Page 12: Judea Pearl Computer Science Department UCLA cs.ucla/~judea](https://reader035.vdocuments.net/reader035/viewer/2022062800/5681415b550346895dad3632/html5/thumbnails/12.jpg)
SEMANTICS AND IDENTIFICATION OF NESTED COUNTERFACTUALS
Consider the quantity
Given M, P(u), Q is well defined
Given u, Zx*(u) is the solution for Z in Mx*, call it z
is the solution for Y in Mxz
Can Q be estimated from data?
Experimental: nest-free expressionNonexperimental: subscript-free expression
)]([ )(*uYEQ uxZxu
entalnonexperim
alexperiment
)()(*uY uxZx
![Page 13: Judea Pearl Computer Science Department UCLA cs.ucla/~judea](https://reader035.vdocuments.net/reader035/viewer/2022062800/5681415b550346895dad3632/html5/thumbnails/13.jpg)
z = f (x, u)y = g (x, z, u)
X Z
Y
NATURAL SEMANTICS OFINDIRECT EFFECTS
Indirect Effect of X on Y:The expected change in Y when we keep X constant, say at x0, and let Z change to whatever value it would have attained had X changed to x1.
In linear models, IE = TE - DE
][010 xZx YYE
x
);,( 10 YxxIE
![Page 14: Judea Pearl Computer Science Department UCLA cs.ucla/~judea](https://reader035.vdocuments.net/reader035/viewer/2022062800/5681415b550346895dad3632/html5/thumbnails/14.jpg)
POLICY IMPLICATIONS(Who cares?)
f
GENDER QUALIFICATION
HIRING
What is the indirect effect of X on Y?
The effect of Gender on Hiring if sex discriminationis eliminated.
X Z
Y
IGNORE
![Page 15: Judea Pearl Computer Science Department UCLA cs.ucla/~judea](https://reader035.vdocuments.net/reader035/viewer/2022062800/5681415b550346895dad3632/html5/thumbnails/15.jpg)
Theorem 5: The total, direct and indirect effects obeyThe following equality
In words, the total effect (on Y) associated with the transition from x* to x is equal to the difference between the direct effect associated with this transition and the indirect effect associated with the reverse transition, from x to x*.
RELATIONS BETWEEN TOTAL, DIRECT, AND INDIRECT EFFECTS
);*,()*;,()*;,( YxxIEYxxDEYxxTE
![Page 16: Judea Pearl Computer Science Department UCLA cs.ucla/~judea](https://reader035.vdocuments.net/reader035/viewer/2022062800/5681415b550346895dad3632/html5/thumbnails/16.jpg)
Is identifiable from experimental data and is given by
Theorem: If there exists a set W such that
EXPERIMENTAL IDENTIFICATION OF AVERAGE DIRECT EFFECTS
zw
xzxxz wPwzZPwYEwYEYxxDE,
** )()|()|()|()*;,(
Then the average direct effect
)(,*;, ** xZx YEYEYxxDEx
xzWZY xxz and all for |*
![Page 17: Judea Pearl Computer Science Department UCLA cs.ucla/~judea](https://reader035.vdocuments.net/reader035/viewer/2022062800/5681415b550346895dad3632/html5/thumbnails/17.jpg)
HOW THE PROOF GOES?
)(,*;, ** xZx YEYEYxxNDEx
xzWZYW xxz and all for If |*
Proof:
)()|(
),|()(
*
*, *
wWPwWzZP
wWzZYEYE
x
w zxxzZx x
)()|(
)|()(
*
, *
wWPwWzZP
wWYYEYE
x
w zxzZx x
Each factor is identifiable by experimentation.
![Page 18: Judea Pearl Computer Science Department UCLA cs.ucla/~judea](https://reader035.vdocuments.net/reader035/viewer/2022062800/5681415b550346895dad3632/html5/thumbnails/18.jpg)
Example:
Theorem: If there exists a set W such that
GRAPHICAL CONDITION FOR EXPERIMENTAL IDENTIFICATION
OF DIRECT EFFECTS
zw
xzxxz wPwzZPwYEwYEYxxDE,
** )()|()|()|()*;,(
)()|( ZXNDWWZYXZG and
then,
![Page 19: Judea Pearl Computer Science Department UCLA cs.ucla/~judea](https://reader035.vdocuments.net/reader035/viewer/2022062800/5681415b550346895dad3632/html5/thumbnails/19.jpg)
GRAPHICAL CONDITION FOR NONEXPERIMENTAL IDENTIFICATION
OF AVERAGE NATURAL DIRECT EFFECTS
Identification conditions1. There exists a W such that (Y Z | W)GXZ
2. There exist additional covariates that render all counterfactual terms identifiable.
zw
xzxxz wPwzZPwYEwYEYxxDE
,** )()|()|()|(
)*;,(
![Page 20: Judea Pearl Computer Science Department UCLA cs.ucla/~judea](https://reader035.vdocuments.net/reader035/viewer/2022062800/5681415b550346895dad3632/html5/thumbnails/20.jpg)
Corollary 3:The average direct effect in Markovian models is identifiable from nonexperimental data, and it is given by
where S stands for any sufficient set of covariates.
IDENTIFICATION INMARKOVIAN MODELS
X ZExample:S =
Y
s z
sPsxzPzxYEzxYEYxxDE )()*,|()*,|(),|()*;,(
z
xzPzxyEzxYEYxxDE *)|()*,|(),|()*;,(
![Page 21: Judea Pearl Computer Science Department UCLA cs.ucla/~judea](https://reader035.vdocuments.net/reader035/viewer/2022062800/5681415b550346895dad3632/html5/thumbnails/21.jpg)
Y
Z
X
W
x*
z* = Zx* (u)
Nonidentifiable even in Markovian models
GENERAL PATH-SPECIFICEFFECTS (Def.)
)),(*),(();,(* ugpagpafgupaf iiiii
*);,();,( **gMMg YxxTEYxxE
Y
Z
X
W
Form a new model, , specific to active subgraph g*gM
Definition: g-specific effect
![Page 22: Judea Pearl Computer Science Department UCLA cs.ucla/~judea](https://reader035.vdocuments.net/reader035/viewer/2022062800/5681415b550346895dad3632/html5/thumbnails/22.jpg)
SUMMARY OF RESULTS
1. Formal semantics of path-specific effects, based on signal blocking, instead of value fixing.
2. Path-analytic techniques extended to nonlinear and nonparametric models.
3. Meaningful (graphical) conditions for estimating direct and indirect effects from experimental and nonexperimental data.
4. Estimability conditions hold in Markovian models.5. Graphical techniques of inferring effects of
policies involving signal blocking.