fairness, privacy, and social norms omer reingold, msr-svc “fairness through awareness” with...
TRANSCRIPT
![Page 1: Fairness, Privacy, and Social Norms Omer Reingold, MSR-SVC “Fairness through awareness” with Cynthia Dwork, Moritz Hardt, Toni Pitassi, Rich Zemel + Musings](https://reader035.vdocuments.net/reader035/viewer/2022062304/56649eb35503460f94bba388/html5/thumbnails/1.jpg)
Fairness, Privacy, and Social Norms
Omer Reingold, MSR-SVC“Fairness through awareness” with Cynthia
Dwork, Moritz Hardt, Toni Pitassi, Rich Zemel+ Musings with Cynthia Dwork, Guy Rothblum
and Salil Vadhan
![Page 2: Fairness, Privacy, and Social Norms Omer Reingold, MSR-SVC “Fairness through awareness” with Cynthia Dwork, Moritz Hardt, Toni Pitassi, Rich Zemel + Musings](https://reader035.vdocuments.net/reader035/viewer/2022062304/56649eb35503460f94bba388/html5/thumbnails/2.jpg)
In This Talk• Fairness in Classification (individual-based
notion)– Connection between Fairness and Privacy– DP beyond Hamming Distance
• A notion of privacy beyond the DB setting.• Empowering society to make choices on
privacy.
![Page 3: Fairness, Privacy, and Social Norms Omer Reingold, MSR-SVC “Fairness through awareness” with Cynthia Dwork, Moritz Hardt, Toni Pitassi, Rich Zemel + Musings](https://reader035.vdocuments.net/reader035/viewer/2022062304/56649eb35503460f94bba388/html5/thumbnails/3.jpg)
Fairness in Classification
paper
acceptance
Schooling
Advertising
Banking
Health Care
Financial aid
Taxation
![Page 4: Fairness, Privacy, and Social Norms Omer Reingold, MSR-SVC “Fairness through awareness” with Cynthia Dwork, Moritz Hardt, Toni Pitassi, Rich Zemel + Musings](https://reader035.vdocuments.net/reader035/viewer/2022062304/56649eb35503460f94bba388/html5/thumbnails/4.jpg)
Concern: Discrimination• Population includes minorities– Ethnic, religious, medical, geographic– Protected by law, policy, ethics
• A catalog of evils: redlining, reverse tokenism, self fulfilling prophecy, … discrimination may be subtle!
![Page 5: Fairness, Privacy, and Social Norms Omer Reingold, MSR-SVC “Fairness through awareness” with Cynthia Dwork, Moritz Hardt, Toni Pitassi, Rich Zemel + Musings](https://reader035.vdocuments.net/reader035/viewer/2022062304/56649eb35503460f94bba388/html5/thumbnails/5.jpg)
Credit Application (WSJ 8/4/10)
User visits capitalone.comCapital One uses tracking information provided by the tracking network [x+1] to personalize offersConcern: Steering minorities into higher rates (illegal)*
![Page 6: Fairness, Privacy, and Social Norms Omer Reingold, MSR-SVC “Fairness through awareness” with Cynthia Dwork, Moritz Hardt, Toni Pitassi, Rich Zemel + Musings](https://reader035.vdocuments.net/reader035/viewer/2022062304/56649eb35503460f94bba388/html5/thumbnails/6.jpg)
Here: A CS Perspective• An individual based notion of fairness – fairness
through awareness• Versatile framework for obtaining and
understanding fairness• Lots of open problems/directions– Fairness vs. Privacy
![Page 7: Fairness, Privacy, and Social Norms Omer Reingold, MSR-SVC “Fairness through awareness” with Cynthia Dwork, Moritz Hardt, Toni Pitassi, Rich Zemel + Musings](https://reader035.vdocuments.net/reader035/viewer/2022062304/56649eb35503460f94bba388/html5/thumbnails/7.jpg)
Other notions of “fairness” in CS• Fair scheduling• Distributed computing• Envy-freeness• Cake cutting• Stable matching
• More closely related notions outside of CS (Economics, Political Studies, …)– Rawls, Roemer, Fleurbaey, Young, Calsamiglia
![Page 8: Fairness, Privacy, and Social Norms Omer Reingold, MSR-SVC “Fairness through awareness” with Cynthia Dwork, Moritz Hardt, Toni Pitassi, Rich Zemel + Musings](https://reader035.vdocuments.net/reader035/viewer/2022062304/56649eb35503460f94bba388/html5/thumbnails/8.jpg)
Fairness and Privacy (1)• [Dwork & Mulligan 2012] objections to online
behavioral targeting often expressed in terms of privacy. In many cases the underlying concern is better described in terms of fairness (e.g., price discrimination, being mistreated).– Other major concern: feeling of “ickiness” [Tene]
• Privacy does not imply fairness – Definitions and techniques useful. – Can Fairness Imply Privacy (beyond DB setting)?
![Page 9: Fairness, Privacy, and Social Norms Omer Reingold, MSR-SVC “Fairness through awareness” with Cynthia Dwork, Moritz Hardt, Toni Pitassi, Rich Zemel + Musings](https://reader035.vdocuments.net/reader035/viewer/2022062304/56649eb35503460f94bba388/html5/thumbnails/9.jpg)
V: Individuals O: outcomes
Ad network(x+1)
x M(x)
Vendor(capital one)
A: actions
![Page 10: Fairness, Privacy, and Social Norms Omer Reingold, MSR-SVC “Fairness through awareness” with Cynthia Dwork, Moritz Hardt, Toni Pitassi, Rich Zemel + Musings](https://reader035.vdocuments.net/reader035/viewer/2022062304/56649eb35503460f94bba388/html5/thumbnails/10.jpg)
V: Individuals O: outcomes
x M(x)
Our goal: Achieve Fairness in the first step (mapping)
Assume
unknown, untrusted, un-auditable vendor
![Page 11: Fairness, Privacy, and Social Norms Omer Reingold, MSR-SVC “Fairness through awareness” with Cynthia Dwork, Moritz Hardt, Toni Pitassi, Rich Zemel + Musings](https://reader035.vdocuments.net/reader035/viewer/2022062304/56649eb35503460f94bba388/html5/thumbnails/11.jpg)
First attempt…
![Page 12: Fairness, Privacy, and Social Norms Omer Reingold, MSR-SVC “Fairness through awareness” with Cynthia Dwork, Moritz Hardt, Toni Pitassi, Rich Zemel + Musings](https://reader035.vdocuments.net/reader035/viewer/2022062304/56649eb35503460f94bba388/html5/thumbnails/12.jpg)
Fairness through Blindness
![Page 13: Fairness, Privacy, and Social Norms Omer Reingold, MSR-SVC “Fairness through awareness” with Cynthia Dwork, Moritz Hardt, Toni Pitassi, Rich Zemel + Musings](https://reader035.vdocuments.net/reader035/viewer/2022062304/56649eb35503460f94bba388/html5/thumbnails/13.jpg)
Fairness through Blindness
• Ignore all irrelevant/protected attributes– e.g., Facebook “sex” & “interested in men/women”
• Point of failure: Redundant encodings– Machine learning: You don’t need to see the label to
be able to predict it– E.g., redlining
![Page 14: Fairness, Privacy, and Social Norms Omer Reingold, MSR-SVC “Fairness through awareness” with Cynthia Dwork, Moritz Hardt, Toni Pitassi, Rich Zemel + Musings](https://reader035.vdocuments.net/reader035/viewer/2022062304/56649eb35503460f94bba388/html5/thumbnails/14.jpg)
Second attempt…
![Page 15: Fairness, Privacy, and Social Norms Omer Reingold, MSR-SVC “Fairness through awareness” with Cynthia Dwork, Moritz Hardt, Toni Pitassi, Rich Zemel + Musings](https://reader035.vdocuments.net/reader035/viewer/2022062304/56649eb35503460f94bba388/html5/thumbnails/15.jpg)
Group Fairness (Statistical Parity)
• Equalize minority S with general population T at the level of outcomes– Pr[outcome o | S] = Pr[outcome o | T]
• Insufficient as a notion of fairness– Has some merit, but can be abused– Example: Advertise burger joint to carnivores in T
and vegans in S.– Example: Self fulfilling prophecy– Example: Multiculturalism …
![Page 16: Fairness, Privacy, and Social Norms Omer Reingold, MSR-SVC “Fairness through awareness” with Cynthia Dwork, Moritz Hardt, Toni Pitassi, Rich Zemel + Musings](https://reader035.vdocuments.net/reader035/viewer/2022062304/56649eb35503460f94bba388/html5/thumbnails/16.jpg)
Lesson: Fairness is task-specific
• Fairness requires understanding of classification task (this is where utility and fairness are in accord)–Cultural understanding of protected groups–Awareness!
![Page 17: Fairness, Privacy, and Social Norms Omer Reingold, MSR-SVC “Fairness through awareness” with Cynthia Dwork, Moritz Hardt, Toni Pitassi, Rich Zemel + Musings](https://reader035.vdocuments.net/reader035/viewer/2022062304/56649eb35503460f94bba388/html5/thumbnails/17.jpg)
Our approach…
![Page 18: Fairness, Privacy, and Social Norms Omer Reingold, MSR-SVC “Fairness through awareness” with Cynthia Dwork, Moritz Hardt, Toni Pitassi, Rich Zemel + Musings](https://reader035.vdocuments.net/reader035/viewer/2022062304/56649eb35503460f94bba388/html5/thumbnails/18.jpg)
Individual Fairness
Treat similar individuals similarly
Similar for the purpose of(fairness in) the classification task
Similar distributionover outcomes
![Page 19: Fairness, Privacy, and Social Norms Omer Reingold, MSR-SVC “Fairness through awareness” with Cynthia Dwork, Moritz Hardt, Toni Pitassi, Rich Zemel + Musings](https://reader035.vdocuments.net/reader035/viewer/2022062304/56649eb35503460f94bba388/html5/thumbnails/19.jpg)
• Assume task-specific similarity metric– Extent to which two individuals are similar w.r.t. the
classification task at hand• Possibly captures some ground truth or society’s
best approximation– Or instead: society’s norms
• Open to public discussion, refinement• Our framework is agnostic to the choice of
metric• User control?
Metric – Who Decides?
![Page 20: Fairness, Privacy, and Social Norms Omer Reingold, MSR-SVC “Fairness through awareness” with Cynthia Dwork, Moritz Hardt, Toni Pitassi, Rich Zemel + Musings](https://reader035.vdocuments.net/reader035/viewer/2022062304/56649eb35503460f94bba388/html5/thumbnails/20.jpg)
• Financial/insurance risk metrics– Already widely used (though secret)
• IBM’s AALIM health care metric– health metric for treating similar patients similarly
• Roemer’s relative effort metric– Well-known approach in Economics/Political
theory• Machine Learning
Maybe not so much science fiction after all…
Metric - Starting Points
![Page 21: Fairness, Privacy, and Social Norms Omer Reingold, MSR-SVC “Fairness through awareness” with Cynthia Dwork, Moritz Hardt, Toni Pitassi, Rich Zemel + Musings](https://reader035.vdocuments.net/reader035/viewer/2022062304/56649eb35503460f94bba388/html5/thumbnails/21.jpg)
Randomized Mapping
V: Individuals O: outcomes
Classification
xM(x)
![Page 22: Fairness, Privacy, and Social Norms Omer Reingold, MSR-SVC “Fairness through awareness” with Cynthia Dwork, Moritz Hardt, Toni Pitassi, Rich Zemel + Musings](https://reader035.vdocuments.net/reader035/viewer/2022062304/56649eb35503460f94bba388/html5/thumbnails/22.jpg)
V: Individuals O: outcomes
Close individuals according to Metric d: V V R
M(x)
yM(y)
x
Towards Formal DefinitionMapped to close distributions
![Page 23: Fairness, Privacy, and Social Norms Omer Reingold, MSR-SVC “Fairness through awareness” with Cynthia Dwork, Moritz Hardt, Toni Pitassi, Rich Zemel + Musings](https://reader035.vdocuments.net/reader035/viewer/2022062304/56649eb35503460f94bba388/html5/thumbnails/23.jpg)
V: Individuals O: outcomes
Close individuals according to Metric d: V V R
M(x)
yM(y)
x
Fairness and D-Privacy (2)Mapped to close distributions
Close databases according to Hamming d: V V R
V: databases O: sanitizations
![Page 24: Fairness, Privacy, and Social Norms Omer Reingold, MSR-SVC “Fairness through awareness” with Cynthia Dwork, Moritz Hardt, Toni Pitassi, Rich Zemel + Musings](https://reader035.vdocuments.net/reader035/viewer/2022062304/56649eb35503460f94bba388/html5/thumbnails/24.jpg)
Key elements of our approach…
![Page 25: Fairness, Privacy, and Social Norms Omer Reingold, MSR-SVC “Fairness through awareness” with Cynthia Dwork, Moritz Hardt, Toni Pitassi, Rich Zemel + Musings](https://reader035.vdocuments.net/reader035/viewer/2022062304/56649eb35503460f94bba388/html5/thumbnails/25.jpg)
Efficiency (with utility maximization)
Efficient Procedure
Metric d: V V R
V: Individuals O: outcomes
x M(x)
d-fair mapping M
lossfunctionL: V O R
Minimize vendor’s expected losssubject to fairness condition
![Page 26: Fairness, Privacy, and Social Norms Omer Reingold, MSR-SVC “Fairness through awareness” with Cynthia Dwork, Moritz Hardt, Toni Pitassi, Rich Zemel + Musings](https://reader035.vdocuments.net/reader035/viewer/2022062304/56649eb35503460f94bba388/html5/thumbnails/26.jpg)
More Specific Question we Address• How to efficiently construct the mapping
M: V -> (O) • When does individual fairness imply group
fairness (statistical parity)?– For a specific metric, which sub-communities are
treated similarly?• Framework for achieving “fair affirmative
action” (ensuring minimal violation of fairness condition)
![Page 27: Fairness, Privacy, and Social Norms Omer Reingold, MSR-SVC “Fairness through awareness” with Cynthia Dwork, Moritz Hardt, Toni Pitassi, Rich Zemel + Musings](https://reader035.vdocuments.net/reader035/viewer/2022062304/56649eb35503460f94bba388/html5/thumbnails/27.jpg)
Fairness vs. Privacy• Privacy does not imply fairness. • Can (our definition of) fairness imply privacy?• Differential Privacy [Dwork-McSherry-Nissim-
Smith’06], privacy for individuals whose information is part of a database:
![Page 28: Fairness, Privacy, and Social Norms Omer Reingold, MSR-SVC “Fairness through awareness” with Cynthia Dwork, Moritz Hardt, Toni Pitassi, Rich Zemel + Musings](https://reader035.vdocuments.net/reader035/viewer/2022062304/56649eb35503460f94bba388/html5/thumbnails/28.jpg)
Privacy on the Web?
• No longer protected by the data of others – my traces can be used directly to compromise my privacy.
• Can fairness be viewed as a measure of privacy?– Can fairness “blend me in with the (surrounding) crowd”?
![Page 29: Fairness, Privacy, and Social Norms Omer Reingold, MSR-SVC “Fairness through awareness” with Cynthia Dwork, Moritz Hardt, Toni Pitassi, Rich Zemel + Musings](https://reader035.vdocuments.net/reader035/viewer/2022062304/56649eb35503460f94bba388/html5/thumbnails/29.jpg)
Relation to K-Anonymity• Critique of k-anonymity: Blending with others
that have the same sensitive property X is a small consolation.
• “Our” notion of privacy is as good as the metric! • If your surrounding is “normative” may imply
meaningful protection (and substantiate, currently unjustified, sense of security of users).
![Page 30: Fairness, Privacy, and Social Norms Omer Reingold, MSR-SVC “Fairness through awareness” with Cynthia Dwork, Moritz Hardt, Toni Pitassi, Rich Zemel + Musings](https://reader035.vdocuments.net/reader035/viewer/2022062304/56649eb35503460f94bba388/html5/thumbnails/30.jpg)
Simple Observation:Who Are You Mr. Reingold?
• If all new information on me obeys our fairness definition with metrics where the two possible Omers are very close then your confidence won’t increase by much …
??
![Page 31: Fairness, Privacy, and Social Norms Omer Reingold, MSR-SVC “Fairness through awareness” with Cynthia Dwork, Moritz Hardt, Toni Pitassi, Rich Zemel + Musings](https://reader035.vdocuments.net/reader035/viewer/2022062304/56649eb35503460f94bba388/html5/thumbnails/31.jpg)
Do We Like It?Challenge – Accumulated Leakage:• Different applications require different metrics.• Less of an issue for fairness …
![Page 32: Fairness, Privacy, and Social Norms Omer Reingold, MSR-SVC “Fairness through awareness” with Cynthia Dwork, Moritz Hardt, Toni Pitassi, Rich Zemel + Musings](https://reader035.vdocuments.net/reader035/viewer/2022062304/56649eb35503460f94bba388/html5/thumbnails/32.jpg)
DPrivacy with Other Metrics• This work gives additional motivation to study
differential privacy beyond Hamming distance.• Well motivated even in the context of database
privacy (there since the original paper).• Example: Privacy of social networks [Kifer-
Machanavajjhala SIGMOD ‘11]– Privacy depends on context
• Privacy is a matter of social norms. • Our burden: give tools to decision makers.
![Page 33: Fairness, Privacy, and Social Norms Omer Reingold, MSR-SVC “Fairness through awareness” with Cynthia Dwork, Moritz Hardt, Toni Pitassi, Rich Zemel + Musings](https://reader035.vdocuments.net/reader035/viewer/2022062304/56649eb35503460f94bba388/html5/thumbnails/33.jpg)
What is the Privacy in DP?• Original motivation mainly given in terms of opt-
out/opt-in incentives. Worry about an individual deciding if to participate.
• A different point of view: a committee that needs to approve a proposed study in the first place.–Does the study incur only tolerable amount of
privacy loss for any particular individual?
![Page 34: Fairness, Privacy, and Social Norms Omer Reingold, MSR-SVC “Fairness through awareness” with Cynthia Dwork, Moritz Hardt, Toni Pitassi, Rich Zemel + Musings](https://reader035.vdocuments.net/reader035/viewer/2022062304/56649eb35503460f94bba388/html5/thumbnails/34.jpg)
On Correlations and Priors• Assume that rows are selected independently,
and no prior information on the database:– DP protects the privacy of each individual.
• But at the presence of prior information, privacy can be grossly violated [Dwork-Naor ‘10]
• Pufferfish [Kifer- Machanavajjhala] A Semantic Approach to the Privacy of Correlated Data• Protect privacy at the presence of pre-specified adversaries
•Interesting case may be when there is a conflict between privacy and utility
![Page 35: Fairness, Privacy, and Social Norms Omer Reingold, MSR-SVC “Fairness through awareness” with Cynthia Dwork, Moritz Hardt, Toni Pitassi, Rich Zemel + Musings](https://reader035.vdocuments.net/reader035/viewer/2022062304/56649eb35503460f94bba388/html5/thumbnails/35.jpg)
Individual-Oriented Sanitization• Assume you only care about the privacy of Alice.• Further assume that the data of Alice is correlated
to the data of at most 10 others.• Enough to erase these 11 rows from the database.• Even if correlated to more, expunging more that
11 rows may exceed the (society defined) legitimate expectation of privacy (e.g., in a health study).
• Differential privacy simultaneously gives “comparable” level of privacy to everyone.
![Page 36: Fairness, Privacy, and Social Norms Omer Reingold, MSR-SVC “Fairness through awareness” with Cynthia Dwork, Moritz Hardt, Toni Pitassi, Rich Zemel + Musings](https://reader035.vdocuments.net/reader035/viewer/2022062304/56649eb35503460f94bba388/html5/thumbnails/36.jpg)
Other variants of DP • Suggests and interprets other variants of DP –
defined by the sanitization we allow individuals.• For example: in social networks, what is the
reasonable expectation of privacy for an individual:– Erase your neighborhood?– Erase information originating from you?
• Another variant: change a few entries in each column.
![Page 37: Fairness, Privacy, and Social Norms Omer Reingold, MSR-SVC “Fairness through awareness” with Cynthia Dwork, Moritz Hardt, Toni Pitassi, Rich Zemel + Musings](https://reader035.vdocuments.net/reader035/viewer/2022062304/56649eb35503460f94bba388/html5/thumbnails/37.jpg)
Objections• Adam Smith: this informal interpretation may
lose too much. For example, the distance in the definition of DP is subtle
• Jonathan Katz: How do you set up epsilon? • Omer Reingold: How do you incorporate input
from machine learning into the decision process of policy makers?
![Page 38: Fairness, Privacy, and Social Norms Omer Reingold, MSR-SVC “Fairness through awareness” with Cynthia Dwork, Moritz Hardt, Toni Pitassi, Rich Zemel + Musings](https://reader035.vdocuments.net/reader035/viewer/2022062304/56649eb35503460f94bba388/html5/thumbnails/38.jpg)
Lots of open problems/directions• Metric– Social aspects, who will define them?– How to generate metric (semi-)automatically,
metric oracle?• Connection to Econ literature/problems– Rawls, Roemer, Fleurbaey, Young, Calsamiglia– Local vs global distributive fairness? Composition?
• Case Study (e.g., in health care)– Start from AALIM?
• Quantitative trade-offs in concrete settings
![Page 39: Fairness, Privacy, and Social Norms Omer Reingold, MSR-SVC “Fairness through awareness” with Cynthia Dwork, Moritz Hardt, Toni Pitassi, Rich Zemel + Musings](https://reader035.vdocuments.net/reader035/viewer/2022062304/56649eb35503460f94bba388/html5/thumbnails/39.jpg)
Lots of open problems/directions• Further explore connection and implications
to privacy.• Additional study of DP with other metrics.• Completely different definitions of privacy? • …
![Page 40: Fairness, Privacy, and Social Norms Omer Reingold, MSR-SVC “Fairness through awareness” with Cynthia Dwork, Moritz Hardt, Toni Pitassi, Rich Zemel + Musings](https://reader035.vdocuments.net/reader035/viewer/2022062304/56649eb35503460f94bba388/html5/thumbnails/40.jpg)
Thank you.
Questions?