causal inference · natural science 40 valuable conclusions about archaeology aredrawn from...
TRANSCRIPT
![Page 1: Causal Inference · Natural Science 40 Valuable conclusions about Archaeology aredrawn from observational data using causal inference techniques Result published in KDD 2018 BiweiHuang,](https://reader030.vdocuments.net/reader030/viewer/2022041013/5ec45862ea0a94064f6de6cf/html5/thumbnails/1.jpg)
Causal InferenceSLIDES BY:YI XIE, Nathan Yan, JIANNAN WANG
1
![Page 2: Causal Inference · Natural Science 40 Valuable conclusions about Archaeology aredrawn from observational data using causal inference techniques Result published in KDD 2018 BiweiHuang,](https://reader030.vdocuments.net/reader030/viewer/2022041013/5ec45862ea0a94064f6de6cf/html5/thumbnails/2.jpg)
Outline
2
Why Should Data Scientists Care?
Basic Concepts
Causal Inference
![Page 3: Causal Inference · Natural Science 40 Valuable conclusions about Archaeology aredrawn from observational data using causal inference techniques Result published in KDD 2018 BiweiHuang,](https://reader030.vdocuments.net/reader030/viewer/2022041013/5ec45862ea0a94064f6de6cf/html5/thumbnails/3.jpg)
Questions Data Scientists Can Answer
3
Is This A or B? ClassificationHow much or How Many? RegressionIs This Weird? Anomaly Detection
How Is This Organized? ClusteringWhat if? Causal Inference
![Page 4: Causal Inference · Natural Science 40 Valuable conclusions about Archaeology aredrawn from observational data using causal inference techniques Result published in KDD 2018 BiweiHuang,](https://reader030.vdocuments.net/reader030/viewer/2022041013/5ec45862ea0a94064f6de6cf/html5/thumbnails/4.jpg)
From Prediction to Causation
Predicting user activity for Xbox● Y = logins in next month● X = logins in past month, number of friends,...
What if we increase the number of friends?● Would it increase user activity?
4
![Page 5: Causal Inference · Natural Science 40 Valuable conclusions about Archaeology aredrawn from observational data using causal inference techniques Result published in KDD 2018 BiweiHuang,](https://reader030.vdocuments.net/reader030/viewer/2022041013/5ec45862ea0a94064f6de6cf/html5/thumbnails/5.jpg)
Maybe or May be not (!)
A causes B
B causes A
C causes A and B
5
Number of Friends
Xbox Activity
Xbox Activity
Number of Friends
User interest in gaming
Number of Friends
Xbox Activity
![Page 6: Causal Inference · Natural Science 40 Valuable conclusions about Archaeology aredrawn from observational data using causal inference techniques Result published in KDD 2018 BiweiHuang,](https://reader030.vdocuments.net/reader030/viewer/2022041013/5ec45862ea0a94064f6de6cf/html5/thumbnails/6.jpg)
A/B Testing Helps!Treatment Group● A random sample of users● Launch a campaign to increase friends● Average activity next month
Control Group● A random sample of users● Not Launch a campaign to increase friends● Average activity next month
6
HypothesisTesting
![Page 7: Causal Inference · Natural Science 40 Valuable conclusions about Archaeology aredrawn from observational data using causal inference techniques Result published in KDD 2018 BiweiHuang,](https://reader030.vdocuments.net/reader030/viewer/2022041013/5ec45862ea0a94064f6de6cf/html5/thumbnails/7.jpg)
A/B Testing Does Not Work
● It is infeasible to do A/B testing○ What if you went to UBC rather than SFU,
would it help you land a better job?
● It is unethical to do A/B testing○ What if subscription price is set to $69 rather
than $99, would it increase revenue?
7
![Page 8: Causal Inference · Natural Science 40 Valuable conclusions about Archaeology aredrawn from observational data using causal inference techniques Result published in KDD 2018 BiweiHuang,](https://reader030.vdocuments.net/reader030/viewer/2022041013/5ec45862ea0a94064f6de6cf/html5/thumbnails/8.jpg)
Example 1: Causality ≠ Correlation
8
When a team was investigating the city death rate data, theyfound that reported cases of teenage drowning death increasewhile the sales of ice cream also increase.
![Page 9: Causal Inference · Natural Science 40 Valuable conclusions about Archaeology aredrawn from observational data using causal inference techniques Result published in KDD 2018 BiweiHuang,](https://reader030.vdocuments.net/reader030/viewer/2022041013/5ec45862ea0a94064f6de6cf/html5/thumbnails/9.jpg)
Example 1: Causality ≠ Correlation
9
![Page 10: Causal Inference · Natural Science 40 Valuable conclusions about Archaeology aredrawn from observational data using causal inference techniques Result published in KDD 2018 BiweiHuang,](https://reader030.vdocuments.net/reader030/viewer/2022041013/5ec45862ea0a94064f6de6cf/html5/thumbnails/10.jpg)
Example 2: Causality ≠ Correlation
10
Is Berkeley gender biased?
Simpson's paradox
![Page 11: Causal Inference · Natural Science 40 Valuable conclusions about Archaeology aredrawn from observational data using causal inference techniques Result published in KDD 2018 BiweiHuang,](https://reader030.vdocuments.net/reader030/viewer/2022041013/5ec45862ea0a94064f6de6cf/html5/thumbnails/11.jpg)
Example 2: Causality ≠ Correlation
11
Department
Admit RateGender
![Page 12: Causal Inference · Natural Science 40 Valuable conclusions about Archaeology aredrawn from observational data using causal inference techniques Result published in KDD 2018 BiweiHuang,](https://reader030.vdocuments.net/reader030/viewer/2022041013/5ec45862ea0a94064f6de6cf/html5/thumbnails/12.jpg)
Example 3: Causality ≠ Correlation
12
● In city areas with nearby trees andnatural landscapes, there is lessdomestic violence.
● Apartment complexes with manytrees had 52% fewer crimes.
● On tree-lined streets, people drivemore slowly, reducing accident risk.
![Page 13: Causal Inference · Natural Science 40 Valuable conclusions about Archaeology aredrawn from observational data using causal inference techniques Result published in KDD 2018 BiweiHuang,](https://reader030.vdocuments.net/reader030/viewer/2022041013/5ec45862ea0a94064f6de6cf/html5/thumbnails/13.jpg)
Outline
13
Why Should Data Scientists Care?Basic Concepts● Outcome / Treatment Variables● Intervention and Do Operation● Counterfactual● Causal Graph
Causal Inference
![Page 14: Causal Inference · Natural Science 40 Valuable conclusions about Archaeology aredrawn from observational data using causal inference techniques Result published in KDD 2018 BiweiHuang,](https://reader030.vdocuments.net/reader030/viewer/2022041013/5ec45862ea0a94064f6de6cf/html5/thumbnails/14.jpg)
Outcome / Treatment Variables
14
Student Gender Class Study Program Grade
Jacky male 1 0 78
Terry male 1 1 82
Mary female 1 0 86
Sarah female 2 1 83
What if participating Study Program, would it improve Grade?
Treatment Outcome
![Page 15: Causal Inference · Natural Science 40 Valuable conclusions about Archaeology aredrawn from observational data using causal inference techniques Result published in KDD 2018 BiweiHuang,](https://reader030.vdocuments.net/reader030/viewer/2022041013/5ec45862ea0a94064f6de6cf/html5/thumbnails/15.jpg)
Intervention and Do Operation❖ Do operator will signal the experimental
intervention (invented by Judea Pearl)
represents the distribution of grad if the person is enrolled in study program
15
P(grade|do(enrolled in study program))
![Page 16: Causal Inference · Natural Science 40 Valuable conclusions about Archaeology aredrawn from observational data using causal inference techniques Result published in KDD 2018 BiweiHuang,](https://reader030.vdocuments.net/reader030/viewer/2022041013/5ec45862ea0a94064f6de6cf/html5/thumbnails/16.jpg)
Intervention and Do Operation❖ P(A | B = b): probability of A being true given
that B is observed as B = b
❖ P(A | do(B) = b): probability of A being true given an intervention that sets B to b
16
![Page 17: Causal Inference · Natural Science 40 Valuable conclusions about Archaeology aredrawn from observational data using causal inference techniques Result published in KDD 2018 BiweiHuang,](https://reader030.vdocuments.net/reader030/viewer/2022041013/5ec45862ea0a94064f6de6cf/html5/thumbnails/17.jpg)
Counterfactual❖ What would have happened if I had changed
“Treatment Variable”
17
Student Gender Class Study Program Actual Grade
Jacky male 1 0 78
Terry male 1 1 82
Mary female 1 0 86
Sarah female 2 1 83
82
82
90
85
(You cannot get it in reality)Counterfactual Grade
![Page 18: Causal Inference · Natural Science 40 Valuable conclusions about Archaeology aredrawn from observational data using causal inference techniques Result published in KDD 2018 BiweiHuang,](https://reader030.vdocuments.net/reader030/viewer/2022041013/5ec45862ea0a94064f6de6cf/html5/thumbnails/18.jpg)
Causality
Causality Definition● The difference between actual outcome and
counterfactual outcome
The Fundamental Problem of Causal Inference● We cannot observe the counterfactual outcome
18
![Page 19: Causal Inference · Natural Science 40 Valuable conclusions about Archaeology aredrawn from observational data using causal inference techniques Result published in KDD 2018 BiweiHuang,](https://reader030.vdocuments.net/reader030/viewer/2022041013/5ec45862ea0a94064f6de6cf/html5/thumbnails/19.jpg)
Causal Graph
19
Causal graph is a directed graph● Nodes: variables● Directed Edges: X affects Y
![Page 20: Causal Inference · Natural Science 40 Valuable conclusions about Archaeology aredrawn from observational data using causal inference techniques Result published in KDD 2018 BiweiHuang,](https://reader030.vdocuments.net/reader030/viewer/2022041013/5ec45862ea0a94064f6de6cf/html5/thumbnails/20.jpg)
Causal Graph
20
❖ Confounding variables: common cause of treatment and outcome
GradeStudy Program
Unobserved Confounders (e.g., study habits, grade history)
Why Causal Graph?● Helpful to identify which
variables to control for● Make assumptions explicit
![Page 21: Causal Inference · Natural Science 40 Valuable conclusions about Archaeology aredrawn from observational data using causal inference techniques Result published in KDD 2018 BiweiHuang,](https://reader030.vdocuments.net/reader030/viewer/2022041013/5ec45862ea0a94064f6de6cf/html5/thumbnails/21.jpg)
Outline
21
Why Should Data Scientists Care?Basic ConceptsCausal Inference● Statistical Inference vs. Causal Inference● Average Treatment Effect (ATE) Estimation ● Causal Inference in Practice
![Page 22: Causal Inference · Natural Science 40 Valuable conclusions about Archaeology aredrawn from observational data using causal inference techniques Result published in KDD 2018 BiweiHuang,](https://reader030.vdocuments.net/reader030/viewer/2022041013/5ec45862ea0a94064f6de6cf/html5/thumbnails/22.jpg)
Statistical vs. Causal Inferences
Statistical inference ● Data is just a sample● Your goal is to infer a population● Think about how to go “backwards” from sample to population
Causal inference ● Derive a treatment group from Data ● Derive a control group (i.e., without treatment) from Data● Think about how to infer the actual effect of treatment from the
derived treatment and control groups
22
![Page 23: Causal Inference · Natural Science 40 Valuable conclusions about Archaeology aredrawn from observational data using causal inference techniques Result published in KDD 2018 BiweiHuang,](https://reader030.vdocuments.net/reader030/viewer/2022041013/5ec45862ea0a94064f6de6cf/html5/thumbnails/23.jpg)
Outline
23
Why Should Data Scientists Care?Basic ConceptsCausal Inference● Statistical Inference vs. Causal Inference● Average Treatment Effect (ATE) Estimation● Causal Inference in Practice
![Page 24: Causal Inference · Natural Science 40 Valuable conclusions about Archaeology aredrawn from observational data using causal inference techniques Result published in KDD 2018 BiweiHuang,](https://reader030.vdocuments.net/reader030/viewer/2022041013/5ec45862ea0a94064f6de6cf/html5/thumbnails/24.jpg)
Individual Treatment Effect
What is the grade difference between enrolling and not enrolling in the study program?
24
Student Gender Class Enroll in Study Program
Not Enroll in Study Program
Jacky male 1 82 78
Terry male 1 82 82
Mary female 1 90 86
Sarah female 2 83 85
4
0
4
-2
![Page 25: Causal Inference · Natural Science 40 Valuable conclusions about Archaeology aredrawn from observational data using causal inference techniques Result published in KDD 2018 BiweiHuang,](https://reader030.vdocuments.net/reader030/viewer/2022041013/5ec45862ea0a94064f6de6cf/html5/thumbnails/25.jpg)
Average Treatment Effect (ATE)
The average of all values for individual treatment effects
25
Student Gender Class Enroll in Study Program
Not Enroll in Study Program
Jacky male 1 82 78
Terry male 1 82 82
Mary female 1 90 86
Sarah female 2 83 85
4
0
4
-2
ATE = (4 + 0 + 4 + -2) / 4 = 1.5
![Page 26: Causal Inference · Natural Science 40 Valuable conclusions about Archaeology aredrawn from observational data using causal inference techniques Result published in KDD 2018 BiweiHuang,](https://reader030.vdocuments.net/reader030/viewer/2022041013/5ec45862ea0a94064f6de6cf/html5/thumbnails/26.jpg)
ATE Estimation
ATE estimation methods❖ Matching based:
➢ Perfect matching➢ Nearest neighbor matching ➢ Propensity score matching
❖ ML based:➢ Regression method➢ Representation learning
26
![Page 27: Causal Inference · Natural Science 40 Valuable conclusions about Archaeology aredrawn from observational data using causal inference techniques Result published in KDD 2018 BiweiHuang,](https://reader030.vdocuments.net/reader030/viewer/2022041013/5ec45862ea0a94064f6de6cf/html5/thumbnails/27.jpg)
Perfect Matching
27
student gender class Study program grade
Terry male 1 1 82
Sarah female 2 1 83
Jacky male 1 0 78
Mary female 1 0 86
Find the “perfect matching in counterfactual world”
![Page 28: Causal Inference · Natural Science 40 Valuable conclusions about Archaeology aredrawn from observational data using causal inference techniques Result published in KDD 2018 BiweiHuang,](https://reader030.vdocuments.net/reader030/viewer/2022041013/5ec45862ea0a94064f6de6cf/html5/thumbnails/28.jpg)
Perfect Matching
28
student gender class Study program grade
Terry male 1 1 82
Sarah female 2 1 83
Jacky male 1 0 78
Mary female 1 0 86
Find the “perfect matching in counterfactual world”
4
![Page 29: Causal Inference · Natural Science 40 Valuable conclusions about Archaeology aredrawn from observational data using causal inference techniques Result published in KDD 2018 BiweiHuang,](https://reader030.vdocuments.net/reader030/viewer/2022041013/5ec45862ea0a94064f6de6cf/html5/thumbnails/29.jpg)
Perfect Matching
29
Student Gender Class Study Program Grade
Terry male 1 1 82
Sarah female 2 1 83
Jacky male 1 0 78
Mary female 1 0 86
By perfect matching, we can’t find ATE over the population, because for tuple Sarah, we can’t find the perfect match in control group
![Page 30: Causal Inference · Natural Science 40 Valuable conclusions about Archaeology aredrawn from observational data using causal inference techniques Result published in KDD 2018 BiweiHuang,](https://reader030.vdocuments.net/reader030/viewer/2022041013/5ec45862ea0a94064f6de6cf/html5/thumbnails/30.jpg)
Nearest Neighbor Matching
30
Find the “nearest matching in counterfactual world”
Student Gender Class Study Program Grade
Terry male 1 1 82
Sarah female 2 1 83
Jacky male 1 0 78
Mary female 1 0 86
3
![Page 31: Causal Inference · Natural Science 40 Valuable conclusions about Archaeology aredrawn from observational data using causal inference techniques Result published in KDD 2018 BiweiHuang,](https://reader030.vdocuments.net/reader030/viewer/2022041013/5ec45862ea0a94064f6de6cf/html5/thumbnails/31.jpg)
Nearest Neighbor Matching
31
ATE = ½ [3 + 4] = 3.5
4
3
Student Gender Class Study Program Grade
Terry male 1 1 82
Sarah female 2 1 83
Jacky male 1 0 78
Mary female 1 0 86
![Page 32: Causal Inference · Natural Science 40 Valuable conclusions about Archaeology aredrawn from observational data using causal inference techniques Result published in KDD 2018 BiweiHuang,](https://reader030.vdocuments.net/reader030/viewer/2022041013/5ec45862ea0a94064f6de6cf/html5/thumbnails/32.jpg)
Propensity Score Matching
32
Steps❖ Using logistic regression to infer 𝑒(𝑥)=𝑃𝑟[𝑇=1|𝑋=𝑥]
❖ Match users with treatment 0 with users with treatment 1 based on propensity score, using matching methods
![Page 33: Causal Inference · Natural Science 40 Valuable conclusions about Archaeology aredrawn from observational data using causal inference techniques Result published in KDD 2018 BiweiHuang,](https://reader030.vdocuments.net/reader030/viewer/2022041013/5ec45862ea0a94064f6de6cf/html5/thumbnails/33.jpg)
Propensity Score Matching
33
Based on PSM derived by logistic regression, the match we’ll find for Terry is Jacky, and the match we’ll find for Sarah is Mary
ATE = 3.5, computation is similar with nearest neighbor matching
Student Gender Class Study Program Grade
Terry male 1 1 82
Sarah female 2 1 83
Jacky male 1 0 78
Mary female 1 0 86
PSE
0.7
0.6
0.7
0.55
![Page 34: Causal Inference · Natural Science 40 Valuable conclusions about Archaeology aredrawn from observational data using causal inference techniques Result published in KDD 2018 BiweiHuang,](https://reader030.vdocuments.net/reader030/viewer/2022041013/5ec45862ea0a94064f6de6cf/html5/thumbnails/34.jpg)
Regression Method
34
Intuition: the distribution of Y given X is different when treatment is different
Train two separate regression models under Treatment = 0 or Treatment = 1, infer p(Y | T = 0, X) and p(Y | T = 1, X)
![Page 35: Causal Inference · Natural Science 40 Valuable conclusions about Archaeology aredrawn from observational data using causal inference techniques Result published in KDD 2018 BiweiHuang,](https://reader030.vdocuments.net/reader030/viewer/2022041013/5ec45862ea0a94064f6de6cf/html5/thumbnails/35.jpg)
Regression Method
35
Treat one regression model 1 on [Terry, Sarah] where Study Program = 1
treat another regression model 2 on [Jacky, Mary] where Study Program = 0
Using model 1 to compute counterfactual outcome of [Jacky, Mary], same for model 2
Student Gender Class Study Program Grade PSE
Terry male 1 1 82 0.7
Sarah female 2 1 83 0.6
Jacky male 1 0 78 0.7
Mary female 1 0 86 0.55
![Page 36: Causal Inference · Natural Science 40 Valuable conclusions about Archaeology aredrawn from observational data using causal inference techniques Result published in KDD 2018 BiweiHuang,](https://reader030.vdocuments.net/reader030/viewer/2022041013/5ec45862ea0a94064f6de6cf/html5/thumbnails/36.jpg)
Other ML-based Methods
36
❖ Representation learning
❖ Intuition: transform dataset into a space where treatment assignment is more evenly distributed
❖ For other more techniques, please check latest publications
![Page 37: Causal Inference · Natural Science 40 Valuable conclusions about Archaeology aredrawn from observational data using causal inference techniques Result published in KDD 2018 BiweiHuang,](https://reader030.vdocuments.net/reader030/viewer/2022041013/5ec45862ea0a94064f6de6cf/html5/thumbnails/37.jpg)
Outline
37
Why Should Data Scientists Care?Basic ConceptsCausal Inference● Statistical Inference vs. Causal Inference● Average Treatment Effect (ATE) Estimation ● Causal Inference in Practice
![Page 38: Causal Inference · Natural Science 40 Valuable conclusions about Archaeology aredrawn from observational data using causal inference techniques Result published in KDD 2018 BiweiHuang,](https://reader030.vdocuments.net/reader030/viewer/2022041013/5ec45862ea0a94064f6de6cf/html5/thumbnails/38.jpg)
Allocating Policy for Homelessness
38
❖ Researchers allocate different interventions (like emergency shelter, rapid rehousing) for homelessness based on causal inference
❖ Report published in AAAI 2019
Amanda Kube, Sanmay Das, Patrick J. Fowler: Allocating Interventions Based on Predicted Outcomes: A Case Study on Homelessness Services. AAAI 2019: 622-629
![Page 39: Causal Inference · Natural Science 40 Valuable conclusions about Archaeology aredrawn from observational data using causal inference techniques Result published in KDD 2018 BiweiHuang,](https://reader030.vdocuments.net/reader030/viewer/2022041013/5ec45862ea0a94064f6de6cf/html5/thumbnails/39.jpg)
Social Media
39
❖ For Twitter, the impact of race, gender and closeness on persuasion is studied using causal inference
❖ Result published in NeurIPS 2019
https://cpb-us-w2.wpmucdn.com/sites.coecis.cornell.edu/dist/a/238/files/2019/12/Id_104_final.pdf
![Page 40: Causal Inference · Natural Science 40 Valuable conclusions about Archaeology aredrawn from observational data using causal inference techniques Result published in KDD 2018 BiweiHuang,](https://reader030.vdocuments.net/reader030/viewer/2022041013/5ec45862ea0a94064f6de6cf/html5/thumbnails/40.jpg)
Natural Science
40
❖ Valuable conclusions about Archaeology are drawn from observational data using causal inference techniques
❖ Result published in KDD 2018
Biwei Huang, Kun Zhang, Yizhu Lin, Bernhard Schölkopf, Clark Glymour: Generalized Score Functions for Causal Discovery. KDD 2018: 1551-1560
![Page 41: Causal Inference · Natural Science 40 Valuable conclusions about Archaeology aredrawn from observational data using causal inference techniques Result published in KDD 2018 BiweiHuang,](https://reader030.vdocuments.net/reader030/viewer/2022041013/5ec45862ea0a94064f6de6cf/html5/thumbnails/41.jpg)
DOWhy Python Library
41
❖ DoWhy is a Python library for causal inference that supports explicit modeling and testing of causal assumptions, developed by Microsoft.
❖ DoWhy is based on a unified language for causal inference, combining causal graphical models and potential outcomes frameworks.
https://github.com/microsoft/dowhy
![Page 42: Causal Inference · Natural Science 40 Valuable conclusions about Archaeology aredrawn from observational data using causal inference techniques Result published in KDD 2018 BiweiHuang,](https://reader030.vdocuments.net/reader030/viewer/2022041013/5ec45862ea0a94064f6de6cf/html5/thumbnails/42.jpg)
Summary
42
Why Should Data Scientists Care?● “What if” Question ?● Why not A/B testing?● Causality ≠ Correlation
Basic Concepts● Outcome / Treatment Variables● Intervention and Do Operation● Counterfactual● Causal Graph
Causal Inference● Statistical Inference vs. Causal Inference● Average Treatment Effect (ATE) Estimation ● Causal Inference in Practice