fairness via separation of powers - dash home

Fairness via Separation of PowersThe Harvard community has made this

article openly available. Please share howthis access benefits you. Your story matters

Citation Cowan, Ethan Andrew. 2020. Fairness via Separation of Powers.Master's thesis, Harvard Extension School.

Citable link https://nrs.harvard.edu/URN-3:HUL.INSTREPOS:37365039

Terms of Use This article was downloaded from Harvard University’s DASHrepository, and is made available under the terms and conditionsapplicable to Other Posted Material, as set forth at http://nrs.harvard.edu/urn-3:HUL.InstRepos:dash.current.terms-of-use#LAA

http://osc.hul.harvard.edu/dash/open-access-feedback?handle=&title=Fairness%20via%20Separation%20of%20Powers&community=1/14557738&collection=1/14557739&owningCollection1/14557739&harvardAuthors=76d3757ff69d9651125e994a43d98e52&departmentSoftware%20Engineering

https://nrs.harvard.edu/URN-3:HUL.INSTREPOS:37365039

http://nrs.harvard.edu/urn-3:HUL.InstRepos:dash.current.terms-of-use#LAA



Fairness via Separation of Powers

Ethan Cowan

A Thesis in the Field of Software Engineering

for the Degree of Master of Liberal Arts in Extension Studies

Harvard University

May 2020

Abstract

We propose a method of separate actors with mixed incentives for the cre-

ation of models which conform to various fairness conditions. Instead of viewing Fair

Machine Learning as an optimization problem under fairness constraints, we divide

the responsibility into three independently acting groups: a Fairness Condition Cre-

ator (Fairness Maximizer), a Model Trainer (Accuracy Maximizer), and an Overseer

(Justice Maximizer). By following a workflow inspired by theories of democratic gov-

ernance and mixed incentive structures, these three groups can converge onto fair and

accurate models according to context-specific definitions.

Acknowledgements

I would firstly like to thank Professor David Parkes for his guidance during

the creation of this thesis. What began as a nebulously defined interest became a

fully fledged research project only with his support and expertise. This thesis would

not have been possible without his brilliance for navigating the connections between

seemingly disparate domains.

I owe a special deal of thanks to Elizabeth Langdon-Gray and the staff of the

Harvard Data Science Initiative (HDSI), for organizing the conference which piqued

my interest in the topic of algorithmic fairness and ethics in machine learning. The

majority of this work was written in the HDSI office, which provided a congenial work

atmosphere and support network.

This research comes at the end of several years of study in the Harvard Exten-

sion School, which has been an invaluable intellectual resource to me since 2016. It

meant a great deal to me that so many classes were opened to students from around

the world, and many of the skills and topics that I learned were central to the creation

of this research.

This thesis is dedicated to my parents and grandparents.

ii

Contents

Table of Contents iii

List of Figures vi

1 Introduction 1

1.1 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

1.2 Democratic Parallels . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

1.3 Incentives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

1.4 Fairness . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

1.5 Liberty . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

1.6 Related Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

1.7 Contribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

1.8 Outline . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

2 Fairness via Separation of Powers 15

2.1 Key Terms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

2.2 Scenario . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

2.3 Organizational Structure . . . . . . . . . . . . . . . . . . . . . . . . . 17

2.4 Constitution Creation . . . . . . . . . . . . . . . . . . . . . . . . . . 18

2.5 Mixed Incentives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21

2.6 Appeals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

iii

3 Transformations and Metrics 26

3.1 Metrics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27

3.1.1 Metrics on Statistical Distributions . . . . . . . . . . . . . . . 29

3.2 Transformations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30

3.2.1 Statistical Transformations . . . . . . . . . . . . . . . . . . . . 31

4 Simulations 33

4.1 Algorithms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33

4.2 Iris Simulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34

4.2.1 Constitution . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37

4.2.2 Fairness Conditions . . . . . . . . . . . . . . . . . . . . . . . . 40

4.2.3 Tenet 1: Similar Means . . . . . . . . . . . . . . . . . . . . . . 41

4.2.4 Tenet 2: Class Balance . . . . . . . . . . . . . . . . . . . . . . 42

4.2.5 Tenet 3: Closeness Between Minima and Maxima . . . . . . . 42

4.2.6 Experimental Setup . . . . . . . . . . . . . . . . . . . . . . . . 43

4.2.7 Data Transformation Steps . . . . . . . . . . . . . . . . . . . . 44

4.2.8 Transformed Data . . . . . . . . . . . . . . . . . . . . . . . . 49

4.3 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50

5 COMPAS Simulation 53

5.1 Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53

5.2 Constitution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56

5.3 The Importance of Hyper Parameters . . . . . . . . . . . . . . . . . . 60

5.4 Penalties and Disparate Outcomes . . . . . . . . . . . . . . . . . . . . 63

5.5 Experimental Setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64

5.6 Fairness Conditions . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67

5.6.1 Tenet 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67

iv

5.6.2 Tenet 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68

5.6.3 Tenet 3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69

5.7 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70

5.7.1 Step Count Estimation . . . . . . . . . . . . . . . . . . . . . . 73

6 Conclusion 76

A Data Transformation Paths 79

A.1 Transformation Paths and Accuracy (Iris Data) . . . . . . . . . . . . 80

A.2 Transformation Paths (COMPAS Data) . . . . . . . . . . . . . . . . . 83

References 85

v

List of Figures

2.1 Separation of Powers. . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

2.2 Constitutional Convention (Sequential Voting) . . . . . . . . . . . . . 20

2.3 Constitutional Convention (k-Subset Voting) . . . . . . . . . . . . . . 21

2.4 Bonus Incentive Matrix . . . . . . . . . . . . . . . . . . . . . . . . . . 22

2.5 Negative Incentive Matrix . . . . . . . . . . . . . . . . . . . . . . . . 23

3.1 Distance comparisons for data sampled from N (0, 1) . . . . . . . . . 28

3.2 The relationship between DM and DB . . . . . . . . . . . . . . . . . 29

3.3 Distance and Similarity for data sampled from N (0, 1) . . . . . . . . 30

4.1 Iris Data with 2 Classes . . . . . . . . . . . . . . . . . . . . . . . . . 35

4.2 Summary statistics for the Iris Data Set . . . . . . . . . . . . . . . . 36

4.3 Summary statistics for Iris Setosa . . . . . . . . . . . . . . . . . . . . 36

4.4 Summary statistics for other species (Iris Data) . . . . . . . . . . . . 37

4.5 Tenet 1 (Iris Data) . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38

4.6 Tenet 2 (Iris Data) . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38

4.7 Tenet 3 (Iris Data) . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39

4.8 Iris Data Set Constitution . . . . . . . . . . . . . . . . . . . . . . . . 40

4.9 Transformation 1 (Iris Data) . . . . . . . . . . . . . . . . . . . . . . . 41


vi



4.13 Before the transformations (Iris Data) . . . . . . . . . . . . . . . . . 45

4.14 After Step 1 (Iris Data) . . . . . . . . . . . . . . . . . . . . . . . . . 46



4.17 Summary statistics for the Iris Data Set after preprocessing (Iris Data) 49

4.18 Summary statistics for Iris Setosa after preprocessing . . . . . . . . . 50

4.19 Summary statistics for other species after preprocessing . . . . . . . . 50

4.20 Model accuracy distribution (Iris Data) . . . . . . . . . . . . . . . . . 51

5.1 Risk Score Demographic Comparisons . . . . . . . . . . . . . . . . . . 55

5.2 Tenet 1: Recidivism Balance . . . . . . . . . . . . . . . . . . . . . . . 56

5.3 Racial Demographics (COMPAS Data) . . . . . . . . . . . . . . . . . 57

5.4 Tenet 2: Racial Demographic Parity . . . . . . . . . . . . . . . . . . . 57

5.5 Individuals counted by sex (COMPAS Data) . . . . . . . . . . . . . . 58

5.6 Tenet 3: Sexual Demographic Parity . . . . . . . . . . . . . . . . . . 58

5.7 Age distribution (COMPAS Data) . . . . . . . . . . . . . . . . . . . . 59

5.8 COMPAS Data Set Constitution . . . . . . . . . . . . . . . . . . . . 60

5.9 Summary statistics by race (COMPAS Data) . . . . . . . . . . . . . . 61

5.10 Trade off between mean error and disparate impact by race (COMPAS

Data) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62

5.11 Minimal change in mean scores for different penalty types (COMPAS

Data) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64

5.12 Simulated Annealing of AM/FM . . . . . . . . . . . . . . . . . . . . . 65

5.13 fairstep: Fair State Step . . . . . . . . . . . . . . . . . . . . . . . . . 66

vii

5.14 accstep: Accuracy State Step . . . . . . . . . . . . . . . . . . . . . . 67

5.15 Transformation 1 (COMPAS Data) . . . . . . . . . . . . . . . . . . . 67






5.21 Hyperparameter search (COMPAS Data) . . . . . . . . . . . . . . . . 71

5.22 Accuracy Changes (COMPAS Data) . . . . . . . . . . . . . . . . . . 72

5.23 Accuracy difference per step (COMPAS Data) . . . . . . . . . . . . . 73

5.24 Step count summary statistics (COMPAS Data) . . . . . . . . . . . . 74

5.25 Steps to reach a maximum bonus (COMPAS Data) . . . . . . . . . . 75

A.1 Paths with significant accuracy loss . . . . . . . . . . . . . . . . . . . 80

A.2 Paths with significant accuracy loss (cont.) . . . . . . . . . . . . . . . 81

A.3 Paths with no accuracy loss . . . . . . . . . . . . . . . . . . . . . . . 82

A.4 Sample of paths with bonus b/2 . . . . . . . . . . . . . . . . . . . . . 83

A.5 Paths with maximum bonus b . . . . . . . . . . . . . . . . . . . . . . 84

viii

Chapter 1: Introduction

1.1. Background

The accelerating role of technology in daily life has outpaced both the regula-

tory power of governments and the ability of individuals to make informed decisions.

In the period between 2010 and 2020, smartphone usage in the United States increased

by 435.5% (Statista, 2019) along with a commensurate increase in data collection by

smartphone applications. Many of these applications are making seemingly-innocuous

decisions based on this information, such as tailoring the app experience to each user,

or targeting advertisements. Previous research has shown that even the choice of

targeted advertising can have racial biases, leading to different user experiences and

harmful outcomes. In particular, minority businesspeople have lamented the disparity

in search results on a variety of platforms, noting that veteran neighborhood busi-

nesses are often placed in much lower positions than newer, corporate-affiliated stores

(Noble, 2018).

Even more serious consequences can be found in the legal system, where emerg-

ing technologies are being used to make decisions about freedom. The Correctional

Offender Management Profiling for Alternative Sanctions (COMPAS) system was de-

veloped by Equivant Software as a risk estimation tool when making sentencing and

parole decisions. COMPAS uses a variety of data points about a defendant, includ-

ing age of first arrest, and number of prior convictions. The score, between 1 and

1

10, is supposed to inform judges and parole officers about the perceived risk of the

individual being arrested in the future for committing another crime. Prior analysis

by ProPublica argued that these scores show a racial bias, falsely labeling black de-

fendants as future criminals at “almost twice the rate of white defendants” (Angwin,

Larson, Mattu, & Kirchner, 2016). This work led to a discussion about the trade offs

which come about when developing models with potentially high impacts on the lives

of individuals.

The marketing of COMPAS focused on its ability to supposedly make a more

impartial decision than a human actor. Equivant argued that a human is likely to

harbor biases, both conscious and otherwise, that often cloud bail and sentencing de-

cisions. However, the ProPublica analysis argued that the data on which COMPAS

relies is itself biased due to the over-policing and criminalization of communities of

color, and therefore perpetuated the very biases that Equivant claimed to be mini-

mizing.

Since the publication of Dwork et al.’s seminal work (Dwork, Hardt, Pitassi,

Reingold, & Zemel, 2012), there has been a proliferation of interest in the theory

and applications of fair machine learning models. A key early result of this research

demonstrated that neither enforcing demographic parity, nor removing sensitive at-

tributes from training data, are sufficient to ensure a fair outcome. In many cases,

other attributes will correlate strongly with the removed sensitive attribute, leading

the model to continue having a harmful impact. Enforcing demographic parity can

also violate individual fairness, as some individuals in one population may receive a

less favorable outcome in order to balance the demographics of the group.

Common to many previous approaches is the assumption that one individual

or entity is selecting the fairness definition(s), choosing the algorithm, training the

model, and performing the optimization and deploying the final result into the world.

2

This situation supposes that this one individual is acting benevolently, with no intent

to harm any other individual or group. In practice, decision making authority is often

distributed across an organization. Given the seriousness of the issues previously

discussed, it seems unlikely that one individual would be given the power to decide

what will be considered a fair outcome.

1.2. Democratic Parallels

Many democratic societies have adopted a government with a “separation of

powers” arrangement, precisely to prevent abuses of power by one individual or group

with unchecked power. In many countries, this means a government with separate

branches for executive, legislative, and judicial functions, each of which can exercise

certain checks on the power of the others. The ideas leading to this structure can be

traced to Europe before 1700, in which the liberty of subjects and concepts of justice

relied heavily on the whim of an absolute monarch or ruling group.

Following this example from the Enlightenment, a modified form of separation

of powers is proposed for the development of fair machine learning models. The

three entities comprising this system will each have rights and responsibilities in the

process, as well as the ability to check the actions of the other entities should they

fail to act in a way commensurate with their role. By distributing authority and

responsibilities across different parties, the risk of abuses of power by one group is

minimized. It is central to such a system to have a constitution which clearly details

the rights and responsibilities of each party in the system.

The first step of the model building process will be a “constitutional conven-

tion,” during which representatives from the three groups propose principles and cast

votes to create a foundational document against which future work will be held. This

convention serves several purposes. First, it allows individuals to highlight domain-

3

specific challenges in the subject matter, as well as potential biases and problems

which may be encountered in any training data. Second, it serves as a community

meeting during which the group can discuss hopes and concerns for the project, as

well as ways to formally encode these ideals into the model building process. It is

also a time for reservations and concerns to be raised with managers and the rest of

the team. Third, it provides the group with a structured format for voting on tenets,

drafting the constitution, and formally approving it before proceeding with the rest

of the work.

Previous research has focused heavily on the trade-off between fairness and

accuracy. This seems natural, as modifying the training data or subjecting the model

to post-processing, in the service of fairness, will often lead to a decrease in accuracy

relative to the testing data. There is therefore a question of conflicting incentives. If

one fairness condition can be satisfied at minimal expense to overall accuracy, is the

outcome satisfactory? It depends on the situation. By placing this decision making

power in one set of hands, we lose the conversation that could be happening between

stakeholders with different perspectives and definitions of success.

Proposals for the use of democratic structures within a technology company

are not without precedent. In 2018, Facebook proposed an oversight committee of 40

members, to have veto power over the CEO (Read, 2018; “Why Mark Zuckerbergs

Oversight Board May Kill His Political Ad Policy”, n.d.). This decision came as the

company faced criticism over issues of content moderation, particularly the screening

and removal of hate speech. Facebook committed $130 dollars to the new committee,

but had not named any members one and a half years after the initial announcement

(Culliford, 2019).

The model building process will be conducted by three separate actors: the

Justice Maximizer (JM), Accuracy Maximizer (AM), and Fairness Maximizer (FM).

4

The Justice Maximizer acts as an arbiter between the other two parties, approving

the constitution against which they will work, and handling all communication. The

JM also sets the stopping conditions, and has the authority to declare the model

development process over. During the constitutional convention, the JM also has the

authority to approve the final draft of the constitution. The Accuracy Maximizer

has a more traditional role: they perform training and testing of the model using

pre-processed data from the Fairness Maximizer, and attempt to make the model

as performant as possible relative to the given metric. The Fairness Maximizer is

tasked with creating a set of computational conditions (boolean statements) and pre-

processing routines for the training data which will make the fairness conditions true.

For example, if the condition is “a maximum of 25% of the data points can be from

a given zip code,” then the pre-processing can be accomplished via oversampling or

data removal. The Fairness Maximizer is not involved in the model creation process

directly, and has no direct communication with the Accuracy Maximizer. The incen-

tives are structured such that the AM prioritizes conforming to the fairness conditions

first, and only then tries to optimize accuracy for an additional bonus.

1.3. Incentives

In a democratic system, elections are supposed to act as an incentive system

to encourage political actors to work for the good of their constituents, and the

overall well-being of the country. The executive and legislative branches are subject

to regular elections as the public’s check on this political power. The judicial branch

has different levels of courts so that the decision of one level may be overturned by the

court of a higher level. Additionally, some judges are subject to elections as checks

on their decisions.

Since the system being proposed is not subject to the will of voters, an incen-

5

tive structure must be constructed. In the absence of these incentives, the Fairness

Maximizer could write any conditions, the Accuracy Maximizer could ignore them,

and the Justice Maximizer could never approve a constitution or manage communica-

tions between the other two parties. Incentives are constructed to recognize that each

group must act to fulfill their task while also pursuing the common goal. The Fairness

Maximizer only receives a bonus if the Accuracy Maximizer satisfies the given accu-

racy condition laid out in the constitution (e.g. “accuracy can only decrease by 5%

maximum”). Conversely, the Accuracy Maximizer only gets a bonus if a minimum

fraction of the fairness conditions are satisfied, as laid out in the constitution. The

Justice Maximizer receives a bonus only if the constitution is constructed in a way

that the other two parties are able to work effectively and achieve their goals. An

initial bonus is also given to the Justice Maximizer for successfully completing and

approving a constitution within a given window of time. Additional requirements

may be placed on the Justice Maximizer to ensure prompt communication.

The bonus is assumed to be financial or material in nature. Each group has

the opportunity to receive a pay bonus following the completion of the model devel-

opment. The amount depends on context, but must be high enough relative to their

base compensation to motivate the groups to take the process seriously. For amounts

that are viewed by the participants as too small, the fairness maximization process

could be viewed as voluntary or non-essential.

1.4. Fairness

Previous work has focused on two key formulations of fairness, namely Individ-

ual Fairness, and Group Fairness(Dwork et al., 2012), and comparing and contrasting

the two definitions and presenting conditions under which they are and are not sat-

isfactory.

6

Let V denote the set of individuals in question, A the set of outcomes that

these individuals may experience, and ∆(A) the set of probability distributions over

these outcomes. For a distribution S, µS will denote the population mean.

Definition 1. Individual Fairness (Lipschitz Mapping) (Dwork et al., 2012)

The tenet that similar individuals should have similar outcomes relative to a given

measure. Formally, a mapping M : V −→ ∆(A) such that ∀v1, v2 ∈ V ,

D(Mv1,Mv2) ≤ d(v1, v2), (1.1)

where d is a metric describing the similarity of individuals and D is a measure of

similarity between distributions.

Definition 2. Group Fairness (Statistical Parity up to bias ε)(Dwork et al.,

2012) In the context of classification, the tenet that the likelihood of receiving a certain

label does not depend on one’s class membership. Formally, for distributions S and

T , a mapping M : V −→ ∆(A) satisfies group fairness relative to a probability metric

D provided that, for two distributions S, T ∈ ∆A, the distribution means µS and µT

differ by at most ε:

D(µS, µT ) ≤ ε (1.2)

Definition 3. Equalized Odds A classifier Y with α possible labels has equalized

odds relative to an attribute A with β possible values and outcome Y with γ possible

values provided that “Y and A are independent conditional on Y.”(Hardt, Price, &

Srebro, 2016) Formally, for

l ∈ {0, ..., α− 1}, i, j ∈ {0, ..., β − 1}, y ∈ {0, ..., γ − 1} :

P (Y = l|A = i, Y = y) = P (Y = l|A = j, Y = y) (1.3)

7

1.5. Liberty

Questions of liberty are recorded in the Western tradition as far back as Aris-

totle’s Politics, in which the concept was explained both in terms of equal political

representation and unfettered personal action (Aristotle, 2016). The later Hobbesian

formulation is now often referred to as negative liberty, and means the lack of exter-

nal obstacles on one’s actions. For the remainder of this paper, we will use Hobbes’

definition, and will use the term “liberty” to refer to “negative liberty”.

Definition 4. Liberty (or Negative Liberty) - Exemption from external con-

straints, exemption from interference by others. (Pettit, 1989)

Historically, the pursuit of liberty and push against tyranny led to the popular

demand for power to be more widely distributed throughout the political system. In

systems with absolute authority granted to a single group or individual, the rights of

individuals and minority groups were rarely protected and the concept of justice only

existed relative to the whims of those in power.

Montesquieu wrote the modern formulation of separation of powers, including

the prototypical example of separation of powers, as a mechanism for the defense of

liberty. In particular, his writing highlighted the need for an independent judiciary

to judge civil and criminal matters, writing “there is no liberty, if the judiciary power

be not separated from the legislative and executive.”(Montesquieu, 1989)

Definition 5. Separation of Powers - Structure in which authority and responsi-

bilities are uniquely distributed to components of a system. For example, a system of

government in which each branch has powers and limitations (checks and balances)

on those powers imposed by the other branches. The uniqueness of responsibilities is

necessary to prevent different branches from conflicting and competing.

8

1.6. Related Work

Previous work on fairness can broadly be categorized into theoretical research,

data pre-processing methods, applied algorithms for fairness, and interdisciplinary

research, often with connections to the legal literature.

Foundational work in the theory of fair classification largely began with Dwork

et. al.’s work which introduced the concept of “treating similar individual’s similarly”

(Dwork et al., 2012). There has followed a variety of theoretical research focusing

on fairness definitions and the role of sensitive attributes; that is, abstractions for

characteristics such as race and sex which are often the basis of discrimination (Hardt

et al., 2016). Other work has synthesized the ideas of Rawls into a framework for

minimizing disparate impact (Joseph, Kearns, Morgenstern, Neel, & Roth, 2016) and

developed theories for fair representation in order to satisfy individual and group

fairness requirements (Zemel, Wu, Swersky, Pitassi, & Dwork, 2013).

More recent research has outlined methods by which a human “fairness ar-

biter” can answer a series of queries in order to approximate a metric for individual

fairness (Ilvento, 2019). Focus on classifiers has led to work describing unfairness from

a causal perspective, either as an unfair path in a causal Bayesian network (Chiappa

& Isaac, 2019) or relative to other constructed causal models (Khademi, Lee, Foley,

& Honavar, 2019; Kilbertus et al., 2017). Other work has examined unfairness as a

violation of a “contrastive fairness” condition (Chakraborti, Patra, & Noble, 2019),

while others have considered the development of meta-algorithms for ensuring fairness

in classification (Celis, Huang, Keswani, & Vishnoi, 2019).

Fairness conditions have also been considered as part of the optimization pro-

cess (Donini, Oneto, Ben-David, Shawe-Taylor, & Pontil, 2018) by defining a general-

ized loss function which takes group fairness into account. Many of these approaches

9

admit that the academic debate about fairness definitions is ongoing, and must choose

just one definition or else construct a composite out of several existing definitions.

Many applied studies have employed existing ML algorithms in the pursuit of

fair outcomes. For example, studies have focused on the use of adversarial learning

to encourage fair outcomes (Wadsworth, Vera, & Piech, 2018), with a particular eye

toward the COMPAS data set and recidivism prediction. Similarly, work has been

published that uses the k nearest neighbors (kNN) algorithm to both discover and

prevent discrimination (Luong, Ruggieri, & Turini, 2011). This work synthesizes the

legal concept of situation testing with kNN clustering in order to uncover biases in

the classification process.

Goals of individual or group fairness are often pursued via data preprocessing

techniques. As the behavior of a model is heavily guided by the training data it is

given, there are a variety of ways to influence the model outcome, including oversam-

pling, undersampling, and data removal. Early work on the topic of preprocessing

examined the ways in which discrimination on the basis sensitive attributes can be

minimized (Kamiran & Calders, 2012). This work divided the possible methods into

four groups: suppression (removal of attributes), massaging (relabeling the data),

reweighing (assigning different weights to data points to achieve a fair result), and

sampling (for example, changing the class balances in the training data set). More re-

cent work has used the reweighing approach to impose fairness constraints on models

built with biased data (Blum & Stangl, 2019). Work has also been done to examine

what data should be collected at all, recommending the use of third parties to hold

sensitive attributes during the model creation process. In one variation of this system,

the third party acts as a data preprocessor, training the data to “protect anonymity

and preserve bias” (Veale & Binns, 2017).

Many of the data transformations used to demonstrate the framework pro-

10

posed in this thesis are built upon the work of Kamiran and Calders. The approaches

toward fair model development are broadly divided into three categories: modifying

a pretrained classifier, enforcing fairness during the training step, and modifying the

data representation (Donini et al., 2018). This thesis is concerned mainly with the

third approach - modifying representations in the training data, and then performing

an unmodified optimization process.

The legal and ethical literature has focused on the problem of fairness in ML

from several perspectives. While some authors have focused on the compatibility of

fairness definitions between the legal and technological sectors (Xiang & Raji, 2019),

others have used popular writing on the topic to address current issues. In partic-

ular, Chander’s review of The Black Box Algorithm raises questions about contem-

porary algorithmic usage, including Facebook’s patent to determine creditworthiness

via one’s friend graph (Chander, 2017). Chander also addresses the case of Wisconsin

v. Loomis (Wisconsin v. Loomis, No. 2015AP157CR, 2016), in which an individual

convicted of a crime and sentenced using COMPAS challenged the sentencing as a vi-

olation of due process. The appeal challenged the usage of COMPAS on the grounds

that the COMPAS system is closed source and also that it takes race and gender into

account. The appeal went to the Supreme Court of the United States, which declined

to hear the case.

The interaction between transparency and accountability in algorithmic deci-

sion making has been addressed by Kroll et al., in an article that argues that trans-

parency is both infeasible and insufficient to guarantee fairness in many cases. From

an intellectual property and privacy perspective, revealing the inner workings of an

algorithm may be either undesirable or illegal, meaning that accountability cannot

be based on the assumption of transparency. The authors argue that, when designing

technical systems, “accountability must be part of the system’s design from the start”

11

(Kroll et al., 2017).

1.7. Contribution

In this thesis, I introduce a mechanism by which organizations can incentivize

the creation of fair machine learning models. This mechanism contains instructions

for the creation of a project-specific constitution and an avenue for the creation of

fairness conditions which satisfy it. Drawing inspiration from behavioral economics,

I contribute an incentive structure for multiple actors to achieve different aspects of

the same end goal.

The main contributions of this thesis come from identifying parallels between

theories of liberty and democratic governance and the technology creation process.

From this basis, I provide a framework for examining the actions of a ML model. The

actions of the ML model can be thought of in terms of potential violations of liberty,

and separation of powers in the model creation process can minimize this risk. Once

the process is understood as a political process, I propose the separation of powers

mechanism, which incentivizes different groups with different responsibilities within

an organization to work toward each other’s success.

1.8. Outline

Chapter 2 lays the theoretical foundation for the separation of powers mech-

anism. This begins with a discussion of the political history that led to the formu-

lation of separation of powers, as well as distinguishing definitions between positive

and negative liberty. I define key terms which will be used throughout the remainder

of this thesis. I also outline the scenario that necessitates this mechanism, includ-

ing the weaknesses with the current model in which one group or individual makes

12

the key decisions in the model creation process. This is followed by a discussion of

the three main actors in the mechanism: the Fairness Maximizer (FM), the Accu-

racy Maximizer (AM), and the Justice Maximizer (JM), as well as their rights and

responsibilities.

The processes that constitute the mechanism are described as algorithms, to

provide a clear and unambiguous description of each part of the model creation pro-

cess. This includes the constitution creation process, during which the different groups

come together to agree on the fundamental tenets that will be used. This is followed

by a description of the incentive structure, including the mixed incentive matrix,

which ties the success of each group to the success of the other. The final step in the

process, known as the appeals process, is then defined and justified.

Chapter 3 provides several foundational definitions related to measuring the

difference between two data sets. This is central to this work, as many of the fairness

goals are tied to transformations of the training data. The distance between the

original data set and its transformed counterpart will give us a sense of how far the

final, fairness-condition satisfying models will be from a model trained on the original

data with no fairness conditions considered.

Chapter 4 details the algorithms used for simulating the separation of powers

mechanism. Without experiments with organizations, I must perform computational

simulations, where each actor is modelled as a semi-autonomous agent which can

decide which fairness conditions to implement in order to maximize an objective

function. In this case, the system has a constitution composed of a variety of compu-

tational conditions, and data preprocessing functions are constructed to make those

fairness conditions true within the training data. The system then iterates over dif-

ferent permutations of the fairness condition preprocessors in order to satisfy the

maximal number while simultaneously maximizing accuracy.

13

In Section 4.2, I document the results of the first study on a real data set.

Using the Iris Data Set, I construct a scenario in which one outcome is considered

less ideal than the other, and run the data through various preprocessors before

training the model. The results contain accuracy loss relative to the original training

data and also relative to the transformed training data. This data was chosen for its

familiarity, as it is a classic data set that has been previously used for educational

and demonstrative methods.

In Section 5 I apply the separation of powers mechanism to a contemporary

data set: the COMPAS recidivism prediction data set. This simulation shows results

for a data set which has been thoroughly studied and is not as statistically clean

as the Iris Data Set. This is also a data set from a tool that has been accused of

disparate treatment on the basis of race, sex, and age.

Chapter 6 summarizes the separation of powers mechanism, including the the-

oretical foundation from political science, as well as the findings from the simulations.

14

Chapter 2: Fairness via Separation of Powers

2.1. Key Terms

We will be working with the following definitions:

Definition 6. Condition A function ξ : E −→ Y where E is a set of events and

elements in Y are countable and have a weak, total ordering reflecting their prefer-

ability. A mechanism for relating the preferability of outcomes for different events,

where ξ(x1) > ξ(x0) means that x1 is a preferable outcome to x0 relative to some

external reward.

e.g. For an exam, the scores can be ordered in order of preference:

ξ(x) =

0 x ≤ 60

0.5 60 ≤ x ≤ 80

1 x ≥ 80

Throughout this study, many of the conditions will be boolean, with one of

either true or false being a preferable outcome.

Definition 7. Computational Condition - Building upon Definition 6, a com-

putational condition is a tuple (C, τ0, ..., τn) where C is a Condition (boolean function)

C : X −→ {0, 1} and each of τ0, ..., τn is a data transformation such that C(τi(X)) = 1

∀i ∈ {0, ..., n}.

15

Definition 8. Action - Something done by a model which results in consequences,

such as classification or prediction. The Action is the verb performed by the model

which may cause disparate impact or other harmful result. The goal of the model

building process will be to make the Action performant subject to a measure while also

satisfying as many fairness conditions as possible.

For classification, the Action of a classifier M with k possible labels L =

{l0, ...lk−1} and input data ~x ∈ RN is the act of mapping M : RN −→ L which assigns

a label to each point in RN .

Definition 9. Societal Values - A set of conditions S = {C0, ..., CN} , possibly

related or even contradictory, that demarcate the set of preferable characteristics or

judgements over a set of events X.

Definition 10. Tenet - A function T : A −→ [0, 1] where A is a subset of possible

actions, and T(a) represents the acceptability value of the action a ∈ A. For example,

if an action a endangers the safety of others, or otherwise violates a central right, then

T (a) = 0, whereas a socially endorsed action (charity, volunteering) may be assigned

T (a) = 1. The set of tenets is drawn from a universe of tenets U = {T : A −→ [0, 1]}

where each tenet maps the set of possible actions onto an acceptability value.

Definition 11. Constitution - A set C = {T1, ...., TN} of tenets, possibly related,

that demarcate the rights and liberties which are most highly valued by a society.

2.2. Scenario

Suppose an organization is attempting to train and use a machine learning

model, and wants it to be fair in the eyes of both cultural norms and internal stan-

dards. For example, the organization wants the model to respect individual fairness.

These conditions are not assumed to be written in the law.

16

The model creation process can then be recast as one of minimizing the loss

function while simultaneously satisfying the maximal number of tenets within a con-

stitution. However, following historical precedent, placing this much decision making

power into the hands of one individual or one group acting autonomously can lead to

abuses of power. In the current paradigm, the highest priority will likely be speed of

development and accuracy. Without strong incentives and a restructured workflow,

the concept of fairness is likely to be deprioritized and treated as an afterthought. For

now we will suppose that a complete constitution has been received and the task is to

create a model that satisfies fairness relative to this constitution. (Further discussion

in Section 2.4)

2.3. Organizational Structure

The model creation process is split into three semi-autonomous groups: the

Accuracy Maximizer (AM), Fairness Maximizer (FM), and the Justice Maximizer

(JM). Each of thee groups has distinct rights, responsibilities, and authorities in the

process. By separating the power into three groups, we minimize the possibility of

abuse while still maintaining a clear recipe for optimization. These groups are loosely

based on the branches of government found in the trias politica model: executive

(AM), legislative (FM), and judicial (JM).

The FM will interpret the constitution in order to construct a set of compu-

tational conditions that the model must follow, as well as recipes for modifying the

training data to satisfy the conditions. These conditions are expressed as boolean

test conditions, along with transformation functions which modify the training data

to make the conditions true.

The FM is expected to have knowledge of the data set and problem domain

in order to construct these conditions. The AM will similarly use their knowledge to

17

FM AM

JM

Training Data

Fair Conditions

Status Report

Status Report

Model Accuracy / Params

Figure 2.1: Separation of Powers.

train the most accurate model they can, in order to serve as a baseline, without yet

taking fairness conditions in to account. The JM serves as the communication between

the two groups; they send the rules from the FM to the AM, and send information

about the model (e.g. method used, etc.) back from the AM to the FM. The JM

is also ultimately tasked with signing off when an optimal model is believed to have

been found. At each step of the process, the FM will send modified training data

to the AM, along with a vector of binary values specifying which fairness conditions

were used.

2.4. Constitution Creation

For each new project, the organization must hold a “constitutional conven-

tion,” in which representatives from each working group discuss and vote on which

18

tenets will be used in the model creation process. This will be highly context-specific,

and will involve background research on the subject matter and potential ethical and

privacy issues. The drafters will need to depend on legal and ethical standards for the

particular domain being modelled. For example, a model related to housing would

be expected to depend heavily on the Fair Housing Act.

Formally, a constitutional convention is a process where a constitution C

with N tenets {T1, ...., TN} is agreed upon by a committee with P voting members

{I1, ..., IP}. The specifics of the voting process can be left to the organization, which

can choose a consensus or majoritarian model. Regardless of the method, the process

vote(Ti, I) will return 1 if the tenet is accepted, and 0 otherwise. The proposed tenets

can also be submitted anonymously, to prevent personal biases or power dynamics

from interfering with the process. Additionally, the voting process can be carried out

sequentially, sampling each tenet randomly without replacement, or alternatively a

subset vote can be made, in which all tenets are reviewed as a whole by each individ-

ual and a group of top tenets are voted on by each individual. For this approach, a

parameter k must be set for the number of tenets that will be adopted.

In the following algorithm, the function propose(Ii) is a stand in for the ith

individual proposing a tenet to the group, Tprop is an array of the proposed tenets

that have not yet been accepted, and C is the final constitution, containing the tenets

that have been accepted.

19

Input: I = {I0, ..., IN−1}

Result: C = {T0, ..., TM−1 }

j = 0

Tprop = []

C = []

for i← 0 to N − 1 do

Tprop[i] = propose(Ii)

end

foreach T ∼ U(Tprop) do

if vote(Tprop[i], I) == 1 then

C[j] = Tprop[i]

j + +

end

end

return C

Figure 2.2: Constitutional Convention (Sequential Voting)

20

Input: k ∈ Z, I = {I0, ..., IN−1}

Result: C = {T0, ..., TM−1 }

Tprop = []

for i← 0 to N − 1 do

Tprop[i] = propose(Ii)

end

// Return an array of size k*N, the top k tenet choices for each individual

v =choose-top-k(I)

vsort = distinct(countsort(v))

C = {vsort0 , ..., vsortk−1}

return C

Figure 2.3: Constitutional Convention (k-Subset Voting)

2.5. Mixed Incentives

To incentivize these groups to find common success, we propose a bonus struc-

ture based on criteria satisfaction. In a traditional structure, the individual or group

receives a bonus if the model exceeds a given metric, such as accuracy. This is in-

sufficient for the proposed method, as there are two metrics to be optimized against,

which in many cases counteract each other.

Without carefully considering incentives, there are trivially optimal solutions

that can be achieved; for example a model that satisfies all FM rules, but has very

low accuracy. Such outcomes must be disincentivized. To counteract these cases, the

system requires an incentive matrix that aligns the AM and FM to work towards the

21

success of each other. In particular, AM will receive a larger bonus when all FM rules

are satisfied, and FM will receive a larger bonus when all accuracy conditions are

satisfied.

For example, suppose that the traditional bonus for exceeding a minimum ac-

curacy is b, in a regime where there are no fairness considerations. The bonus will

now be expressed as a matrix (Figure 2.1), which incentivizes each actor to design

their piece of the algorithm in order to maximize, to their knowledge, the likelihood

of the other actor achieving their goal. Each element of B is a tuple, where the first

value is the reward for the FM and the second value is the reward for the AM. The

first row of the matrix corresponds to a scenario in which the fairness conditions are

not satisfied, and the second row means that the fairness conditions were satisfied.

Similarly, the first column refers to the accuracy conditions not being satisfied, while

the second column means that the accuracy conditions were satisfied. The JM is

given a bonus for signing off and delivering a completed model in fewer than N steps,

as well as an additional bonus in the case of B11. Formally,

B00 =⇒ No conditions satisfied

B01 =⇒ Fairness not satisfied, accuracy satisfied

B10 =⇒ Fairness satisfied, accuracy not satisfied

B11 =⇒ Both conditions satisfied

B(k) =

(0, 0) ( bk, 0)

(0, bk) (b, b)

(2.1)

Figure 2.4: Bonus Incentive Matrix

22

The matrix B forms a symmetric game over the actions of the two players

AM and FM, where both players have a dominant strategy: the FM should produce

accuracy-preserving transformations, and the AM should implement the minimum

number of required transformations.

With no prior information about the likelihood of either player succeeding in

their respective task, the expected payoff for each player is

E(B(0)) = (0 +b

k) ∗ 1

2=

b

2k(2.2)

E(B(1)) = (0 + b) ∗ 1

2=b

2(2.3)

so that the expected improvement in payoff is

E(B(1)) = k ∗ E(B(0)) (2.4)

This game can also be modified so that each player is negatively incentivized.

Consider the following incentive matrix:

B′(k) =

(0, 0) ( bk,− b

k)

(− bk, bk) (b, b)

(2.5)

Figure 2.5: Negative Incentive Matrix

In this form of the game, the FM loses bk

if accuracy is not satisfied, and the AM

loses bk

if the minimum number of fairness conditions are not satisfied. The expected

bonuses are then:

23

E(B′(0)) = (0 +b

k) ∗ 1

2=

b

2k(2.6)

E(B′(1)) = (− bk

+ b) ∗ 1

2=

(k − 1)b

2(2.7)

Then the expected improvement to the bonus is:

E(B′(1)) = k(k − 1) ∗ E(B′(0)) (2.8)

Algorithm 2.1 Reward Calculation

Input: Xtrain,Xtest,ytrain,ytest,baselineAccuracy,minAccuracyLoss,minFairnessResult: (rfair, racc)model = model.fit(Xtrain,ytrain)X′train,y

′train = resample(Xtrain,ytrain)

fairModel = model.fit(X′train, y′train)conditionsMet = fairModel.computeFairnessConditions()ypred = model.predict(Xtest)acc = computeAccuracy(ytest,predictions)if |acc− baselineAccuracy| < minAccuracyLoss then

if conditionsMet >= minFairness thenrfair = B

elserfair = B

2

end

endelse if conditionsMet >= minFairness then

if |acc− baselineAccuracy| < minAccuracyLoss thenracc = B

elseracc = B

2

end

endelse

racc = rfair = 0end

24

2.6. Appeals

There are two points in the process during which the one group can challenge

the other group’s model and take their bonus. When the AM has converged onto a

model that they believe to be optimal subject to both fairness and accuracy concerns,

then the FM gets one opportunity to make the model more accurate while satisfying

more fairness conditions. If the FM succeeds, then the AM gets a chance to further

outperform them. If the AM cannot improve upon the FM’s improved model, then

the FM receives the full bonus B and the AM receives nothing.

This step in the procedure disincentivizes any actor from simply satisfying

the other actor’s conditions while neglecting their own, in order to reap an easier

reward. For example, the AM could construct a model that satisfies enough fairness

conditions to receive a reward B/2 (Eq. 2.1, above) while depriving the other actor

of any reward. The appeals mechanism gives the FM the chance to create a model

with superior performance, and receive a total reward of B.

25

Chapter 3: Transformations and Metrics

As data transformations will be a key part of this process, it is important to

highlight the transformation methods that can be used, as well as the metrics that

are needed to formalize the notion of distance between a data set and its transformed

version. Supposing that our training data X contains m rows and n columns, we can

say that it is isomorphic to Rm×n, and use a generalization of the Euclidean metric

as a starting off point. Data transformations can be approached from two separate

perspectives. The data itself can be transformed, using computational rules and/or

transformation matrices. Alternatively, the probability distribution which generated

the data can be inferred, and then mapped to a new distribution. This transformed

distribution can then be sampled from to create a new training set. In either case,

there are many metrics to choose from, with a large range of possible outcomes. This

chapter begin with a brief discussion of the notion of distance between elements in

matrices, and then generalize these concepts.

26

3.1. Metrics

For two matrices A,B ∈ Rm×n there are a variety of possible metrics which

provide a notion of distance between them.1 We focus first on the Frobenius metric,

which generalizes the familiar concept of distance in Euclidean space. As the quantity

m ∗ n increases, any change in just one element of A or B will lead to vanishingly

small changes in the distance.

Definition 12. Frobenius (Euclidean) p-metric

For m× n matrices A = (aij) and B = (bij), dp(A,B) = p

√n∑

i=1

n∑j=1

(aij − bij)p

Given two data sets X1 and X2, it may not always be possible or desirable

to require that they have the same features columns appearing in the same order.

For the purposes of comparing a new data set with previously evaluated data sets,

there can be an advantage to finding patterns and structural similarities with data

sets from other fields. This need motivates the following definition, which generalizes

the notion of distance to data sets from different sources, by examining the closest

distances between the individual columns in each data set.

Definition 13. Sorted Distance For two data sets X1, X2 ∈ Rm×n and a distance

metric d : Rm × Rm −→ R, the Sorted Distance first maps X1 −→ X ′1 and X2 −→

X ′2, where X ′1 and X ′2 have the same values as X1 and X2, with columns sorted in

descending order. The value of the distance is then d(X ′1, X′2)

Definition 14. Sorted and Normed Distance For two data sets X1, X2 ∈ Rm×n

and a distance metric d : Rm × Rm −→ R, the Sorted and Normed Distance first

1Until now, we have considered the training data to be a subset of Rm×n, which is only guaranteedto be true after feature selection and processing. For example, a column of data with enumeratedtype and options [A, B, C] will become a column of type integer, with options [1,2,3]. For theremainder of this paper, we will assume that training data is already in such a processed form.

27

normalizes each column X i1 ←− X i

1/‖X i1‖ and X i

2 ←− X i2/‖X i

2‖ ∀i < n, and then sorts

the columns and returns the distance, as in Definition 13.

Definition 15. Minimum Column Distance

For two data sets X1, X2 ∈ Rm×n and a distance metric d : Rm × Rm −→ R, the

Minimum Column Distance is the sum ‖X1 − X2‖ =∑i,j∈S

d(X i1, X

j2) where S is the

set of index pairs (i, j) that minimize the column distance: d(X i1, X

j2) < d(X i

1, Xk2 )

∀k ∈ {0, ..., n}, k 6= j

These definitions can lead to different values of distance over the same pair

of data sets. Consider X1 and X2 where each column is a series of samples from

N (0, 1). According to the Frobenius Metric, dp(X1, X2) = 141.18, since this measure

only captures the agreement in value of each pair of elements in the data sets.

Metric Sorted Normed Distance

Frobenius N N 141.18

Min. Column N N 137.30

Frobenius Y N 8.95

Min. Column Y N 5.14

Frobenius Y Y 0.09

Min. Column Y Y 0.05

Figure 3.1: Distance comparisons for data sampled from N (0, 1)

Sorting and normalizing the data sets before comparing them results in dis-

tance values that more closely reflect the underlying statistics of the data, rather than

the similarity of individual elements. In cases where there is a need to calculate the

similarity of the overall properties of the data sets, there can be an advantage in fitting

distributions to the data, and measuring the distance relative to the distributions.

28

3.1.1 Metrics on Statistical Distributions

An alternative approach is to consider each column as a series of samples

from a statistical distribution with parameters Θ0, ...,Θn−1. Provided that a suitable

distribution can be fit for each column, then the problem of computing distance be-

comes one of distances between statistical distributions. The Bhattacharyya Distance

provides a notion of similarity between distributions, while the Matusita Distance

provides a measure of difference.

Definition 16. Bhattacharyya Distance For statistical distributions F1, F2 with

probability distribution functions p1 and p2, the Bhattacharyya Measure is DB(F1, F2) =∫ √p1(x)

√p2(x)dx

Definition 17. Matusita Distance

For statistical distributions F1, F2 with probability distribution functions p1 and p2,

the Matusita Measure is DM(F1, F2) =∫

(√p1(x)−

√p2(x))2dx

The Matusita Measure is related to the Bhattacharyya Measure via the equa-

tion

DM(F1, F2) = 2− 2DB(F1, F2) (3.1)

Figure 3.2: The relationship between DM and DB

(Matusita, 1955)

This relation shows that minimizingDM(F1, F2) is equivalent to maximizingDB(F1, F2).

While the Matusita Measures quantifies how “far” the distributions are from one an-

other, the Bhattacharyya Measure compares the closeness (or affinity) of the two

29

distributions. Returning to the example data set from Figure 3.1, we have p1 = p2 =

N (0, 1), and both of these measures simplify:

DB(X1, X2) =

∫ √N (0, 1)

√N (0, 1)dx =

∫N (0, 1)dx = 1 (3.2)

DM(X1, X2) = 2− 2DB(X1, X2) = 0 (3.3)

Figure 3.3: Distance and Similarity for data sampled from N (0, 1)

As expected, the measures DB and DM give inverse indicators of the same relation-

ship: X1 and X2 are perfectly similar by statistics (DB) and have metric distance 0

(DM).

3.2. Transformations

Definition 18. Transformation

A Transformation τ : Rm×n −→ Rm×n maps a data set into another data set of the

same dimension. For a distance metric d, the distance from X ∈ Rm×n to τ(X) is

given by d(X, τ(X)) and formalizes the notion of difference between the original data

set and its transformed counterpart.

Under the Bonus Incentive Matrix, the FM will seek to construct transfor-

mations such that the quantity d(X, τ(X)) is minimized, in order to maximize the

possibility of preserving the accuracy score relative to the original data. The FM

will need to decide which transformations to construct, and which metrics to use to

measure distance between the original and transformed data sets.

Regardless of the metric used, transformations will not generally commute.

This means that the data transformations will need to be performed in a variety of

30

orders to test the effect on the final model. For example, oversampling one class in

the data and then trimming extreme values will not, in general, give the same result

as trimming the extreme values and then oversampling. Further discussion of this

non-commutativity is found in 4.3.

3.2.1 Statistical Transformations

Suppose that each column in a data set X is sampled independently, so that

the ith column of X ∈ Rm×n is drawn from a distribution Pi(θi0, ..., θ

ik−1). Then

X ∼ (P0(θ00, ..., θ

0k−1), ..., Pn−1(θ

n−10 , ..., θn−1k−1)) and the initial parameter matrix Θ can

be written:

Θ0 =

θ00 · · · θ0n−1

.... . .

...

θ0k−1 · · · θn−1k−1

(3.4)

Let Θideal denote the matrix of parameters such that d(X, t(X)) is minimized

and C(t(X)) = 1 for all transformations t associated with the Condition C. The ith

column of Θideal is the parameters for the distribution that best fits the data in the

ith column of X. Given the current parameters Θ and the ideal parameters Θideal, we

can transform the parameters via

τ : Rk×n −→ Rk×n (3.5)

which is a lower dimensional transformation than one operating on X directly. Then

for a series of computational conditions (C, τ0, ..., τn) which impose restrictions on

the statistical properties of one or multiple columns, each τi must only operate on

the lower dimensional space Rk×n and the transformed data X ′ can be reconstructed

from the final parameter matrix Θf = τin(τin−1(· · · τ0(Θ0))), where each column is

31

sampled:

X ′i ∼ Pi(Θi0, ...,Θ

ik−1) (3.6)

32

Chapter 4: Simulations

In the absence of human trials, simulations are necessary to show this approach

converging onto ideal solutions. A key part of the simulation is selecting which fairness

conditions and which hyperparameters to use for each step. The algorithms for each

are outlined in the following section. These simulations are not necessarily designed

to mimic the thoughts and actions of a collection of human actors; rather, they serve

to illustrate the interaction between the incentives between the different groups.

4.1. Algorithms

The fairness vector vfair tells the model which fairness conditions to test for,

and which to ignore. At each step in the simulation, a new vfair is drawn that is

near the previous vfair relative to some metric. For modeling purposes, we have been

using the Hamming Distance, to guarantee that each step only changes the value of

one dimension of vfair.

Algorithm 4.1 Fairness Vector GenerationInput: vfair, MResult: vfair ∈ {0,1}dB = {x|x ∈ {0,1}d,d(x,vfair) = 1,

∑k−1i=0 xi ≥M}

vfair ∼ U(B)

The action space for AM is more recognizable from a traditional grid search.

Each dimension in vacc corresponds to a parameter, mapped to an integer value. The

33

AM receives training data which has already been modified according to the values

present in vfair. We then proceed to iterate over the space like so:

Algorithm 4.2 Accuracy Vector Generation (Hyperparameters)

Input: vacc

Result: vacc ∈ Zk

B = {x|x ∈ Zk,d(x,vacc) = 1}vacc ∼ U(B)

The reward calculation process involves comparing the accuracy of the model

trained on unmodified training data (the baseline accuracy) with the accuracy after

applying the fairness conditions. The AM tries to minimize the accuracy loss while

maximizing the number of fairness conditions satisfied.

4.2. Iris Simulation

The Iris Data Set was first described by Fisher in the illustration of taxonomic

problems (Fisher, 1936). It has since become a standard data example in educa-

tional materials when highlighting a particular algorithm or data analysis technique.

Partially for this reason, we have chosen this data set for the first simulation of the

separation of powers mechanism.

The Iris Data Set (N = 150) has 3 classes with 50 samples each, corresponding

to 3 species of Iris flowers: Iris Setosa, Iris Virginica, and Iris Versicolor. There are

4 features: sepal length, sepal width, petal length, and petal width. In several of the

dimensions, Iris Setosa clusters distinctly from the other two species. For the sake of

this simulation, we will combine Iris Virginica and Iris Versicolor into one class, and

consider Iris Setosa to be a less ideal label for the purposes of fairness definitions.

34

Figure 4.1: Iris Data with 2 Classes

As can be seen in the pair plot, there are several dimensions in which the Iris

Setosa species forms a distinct cluster from the rest of the data. This is especially

highlighted in the bottom row, where the y axis represents petal width.

35

sepal length sepal width petal length petal width

count 150 150 150 150

mean 5.843333 3.057333 3.758000 1.199333

std 0.828066 0.435866 1.765298 0.762238

min 4.300000 2.000000 1.000000 0.100000

25% 5.100000 2.800000 1.600000 0.300000

50% 5.800000 3.000000 4.350000 1.300000

75% 6.400000 3.300000 5.100000 1.800000

max 7.900000 4.400000 6.900000 2.500000

Figure 4.2: Summary statistics for the Iris Data Set


count 50 50 50 50

mean 5.00600 3.428000 1.462000 0.246000

std 0.35249 0.379064 0.173664 0.105386

min 4.30000 2.300000 1.000000 0.100000

25% 4.80000 3.200000 1.400000 0.200000

50% 5.00000 3.400000 1.500000 0.200000

75% 5.20000 3.675000 1.575000 0.300000

max 5.80000 4.400000 1.900000 0.600000

Figure 4.3: Summary statistics for Iris Setosa

36


count 100 100 100 100

mean 6.262000 2.872000 4.906000 1.676000

std 0.662834 0.332751 0.825578 0.424769

min 4.900000 2.000000 3.000000 1.000000

25% 5.800000 2.700000 4.375000 1.300000

50% 6.300000 2.900000 4.900000 1.600000

75% 6.700000 3.025000 5.525000 2.000000

max 7.900000 3.800000 6.900000 2.500000

Figure 4.4: Summary statistics for other species (Iris Data)

4.2.1 Constitution

In this hypothetical scenario, we imagine a group of individuals representing

the different parts of the organization, coming together to decide on which tenets will

be written into the constitution. Since we have specified that the label Iris Setosa is

considered less desirable, we must construct tenets that will protect individuals from

being erroneously labeled as such. For this “constitutional convention”, the following

notation is used:

• X - The set of all potential training data

• Xsetosa - The subset of the training data labeled Iris Setosa

• Xother - The subset of the training data labeled other

• µdimlabel - The mean value of column dim for data with label label

Assuming we have prior research demonstrating that the statistical distribu-

tion of Iris Setosa plants is wider than our sample, someone proposes that we should

37

have a global condition specifying how far apart the two subsets in the data set can

be. They propose a tenet that the mean sepal widths (sw) and mean petal lengths

(pl) must be “close together”, which they define as

|µswsetosa − µsw

others| ≤ ε (4.1)

|µplsetosa − µ

plothers| ≤ ε (4.2)

Figure 4.5: Tenet 1 (Iris Data)

Someone else raises the issue of class balance: they claim that in the wild,

Iris Setosa is actually closer to 50% of the observed Iris flowers in the area. They

argue that, knowing this, the training data should reflect the reality of a 50-50 split

between Iris Setosa and the other species. They then propose a tenet mandating

class balance:

|Xsetosa| − |Xothers| ≤ ε (4.3)


Finally, someone raises the issue of individual outliers. Since the petal width

in the Iris Setosa sample appears to be much smaller than the reality, they describe a

tenet which guarantees that the widest Iris Setosa be close in width to the narrowest

sample from the rest of the data. This tenet guarantees closeness of the extreme

38

values, but does not transform the underlying statistics as dramatically as Tenet 1

(Figure 4.5).

|max(Xpwsetosa)−min(Xpw

others)| ≤ ε (4.4)


Following the proposal of the three tenets, the group votes to confirm each of

them, and produces the following constitution. Note that the revisions clause was

not explicitly mentioned during the tenet voting process, as it is a central feature of

the document and allows for revisions on-the-fly as problems are revealed that impact

the majority of the organization.

39

Preamble: Recognizing that our training data does not repre-

sent ground truth, and that the action of labelling may lead to

disparate impact, we are laying out the following tenets which

will govern our model creation process. For all definitions of

“closeness”, we will use a parameter ε, to be defined before

implementation

Tenet 1: Class means must be close together.

Tenet 2: Classes must be equally balanced.

Tenet 3: The widest Iris Setosa sample must be close to the

smallest sample from the rest of the data

Revisions: Should an oversight or flaw be found in these

tenets during the model creation process, any member of the

organization may solicit support from others to call a vote to

amend. A simple majority is needed to call the vote to order,

and a 2/3 majority is needed to accept the change.

Figure 4.8: Iris Data Set Constitution

4.2.2 Fairness Conditions

For the purposes of this study, we have constructed a constitution based on

our knowledge that the sampling of the Iris Setosa was not representative, and in

fact the two species are more closely related than the data shows. The task of the FM

is then to define a set of conditions and associated transformations that will make

those conditions true in the training data. The AM will then have the choice of which

combination of transformations to use in order to honor each of the tenets which were

previously accepted.

40

4.2.3 Tenet 1: Similar Means

Recall that Tenet 1 requires |µswsetosa − µsw

others| ≤ ε and |µplsetosa − µ

plothers| ≤ ε.

The FM then proposes Transformation 1, which modifies the Iris Setosa data, and

Transformation 2, which modifies the rest of the data. In the following algorithms,

ε is the value agreed upon during adoption of the constitution to define the concept of

“closeness”, and α is a scaling factor which is used to bring the data into agreement.

Input: Xsetosa, α, ε

for x in Xsetosa do

xpl = xpl + |ε− α ∗ µplothers|

xsw = xsw − |ε− α ∗ µswothers|

end

Figure 4.9: Transformation 1 (Iris Data)

Input: Xothers, α, ε

for x in Xothers do

xpl = xpl − |ε− α ∗ µplsetosa|

xsw = xsw + |ε− α ∗ µswsetosa|

end


41

4.2.4 Tenet 2: Class Balance

To oversample the training data with equal class sizes, we give the option to

use one of two approaches. The first method is a Random Oversampler, which is a

more naive approach. This method samples with replacement from the minority class,

leading to duplicate data points. This method is computationally straightforward,

but does not capture information about the statistical distribution of the data apart

from the points which are already in the sample, so sampling biases and the effects

of missing will be magnified.

The second method is the Synthetic Minority Oversampling Technique (SMOTE)

(Chawla, Bowyer, Hall, & Kegelmeyer, 2002), which chooses random points from the

minority class and generates new data between those points and their nearest neigh-

bors. SMOTE is a commonly used method for achieving class balance in previously

unbalanced data sets, and has also formed the foundation for a variety of additional

oversampling methods (Douzas, Bacao, & Last, 2018). For this study, an unmodified

version of SMOTE was used, as detailed in the original publication of the algorithm.

4.2.5 Tenet 3: Closeness Between Minima and Maxima

This tenet is only concerned with the difference in value between the widest

instance of Iris Setosa and the thinnest instance of another species. Since this condi-

tion is local in nature, it only requires a transform to modify one data point in order

to be satisfied. We propose two transforms which satisfy this condition; one trans-

formation adds a wider Iris Setosa, while the other transformation adds a thinner

sample from the other species.

42

Input: X, ε

i = arg minXpwother

j = arg maxXpwsetosa

Xpwother[i] = Xpw

setosa[j] + ε


Input: X, ε

i = arg minXpwother

j = arg maxXpwsetosa

Xpwsetosa[j] = Xpw

other[i]− ε


We have now defined 3 tenets, each of which have 2 possible transformations

to be used. Assuming we want the final process to satisfy all 3 tenets, we must pick

one transformation for each of the 3 tenets and perform them in an order so as to

maximize the accuracy.

4.2.6 Experimental Setup

For the Iris Data Set, we performed an exhaustive search over the possible data

transformations, in order to highlight the potential changes in accuracy depending

on the order of the transforms. We begin with the data structures and terminology

that were used for this part of the simulation.

43

We will track the order of transformations using the following notation. For

a series of Computational Conditions (Definition 7), each step s = (Ci, τj) in the

process is defined as performing the jth transformation τj of Ci in order to satisfy a

condition Ci. The transformation history is then a list

S = [(Ci0 , τj0), ..., (Cj0 , τjn)] (4.5)

denoting which transformation was performed to satisfy each of the conditions C0, ..., Cn.

Our goal is then to find a sequence S = [(Ci0 , τj0), ..., (Cj0 , τjn)] such that accuracy is

minimally impacted.

4.2.7 Data Transformation Steps

For the sake of example we will choose the transformation path [(2, 1), (3, 1), (1, 1)].

Before performing any transformations, the data appears like so:

44

Figure 4.13: Before the transformations (Iris Data)

The team takes the first step: Condition 2, Transform 1, which is a SMOTE

oversample. The shape of the data has not changed, though the Iris Setosa points

are now more dense.

45

Figure 4.14: After Step 1 (Iris Data)

The team then takes the second step: Condition 3, Transform 1, which adds

points so that the widest Iris Setosa data point is close to the narrowest data point

from the other class.

46


Finally, the team takes the last transform: Condition 1, Transformation 1.

These transformation shifts the mean of the Iris Setosa data to be close to the mean

of the other data.

47


At this point, the team has a training data set that satisfies all three tenets

of their constitution: the means of the two classes are close, the classes are balanced,

and the minima and maxima of the two classes are close. While it appears from these

two dimensions that the data is no longer clearly separable, a full pairs plot of the

other dimensions shows that there are still clean separations between the classes, and

in fact the accuracy has not been harmed at all in the process.

48

4.2.8 Transformed Data

The following tables show the summary statistics for the Iris Data Set after

undergoing the transformations specified in the previous sections. Note that, in the

transformed data, N = 201 due to resampling to satisfy Tenet 2 and the addition of

one data point to satisfy Tenet 3.


count 201.000000 201.000000 201.000000 201.000000

mean 5.626980 2.822254 4.856254 0.960158

std 0.818997 0.346194 0.593738 0.779370

min 4.300000 1.624724 3.000000 0.100000

25% 5.000000 2.524724 4.640343 0.200000

50% 5.485952 2.800000 4.840343 1.000000

75% 6.300000 3.024724 5.001836 1.600000

max 7.900000 3.800000 6.900000 2.500000

Figure 4.17: Summary statistics for the Iris Data Set after preprocessing (Iris Data)

49


count 101.000000 101.000000 101.000000 101.000000

mean 4.998247 2.773000 4.807000 0.251403

std 0.322563 0.353778 0.159252 0.128155

min 4.300000 1.624724 4.340343 0.100000

25% 4.784979 2.524724 4.740343 0.200000

50% 5.000000 2.724724 4.840343 0.200000

75% 5.179357 3.024724 4.914403 0.300000

max 5.800000 3.724724 5.240343 1.000000

Figure 4.18: Summary statistics for Iris Setosa after preprocessing


count 100.000000 100.000000 100.000000 100.000000

mean 6.262000 2.872000 4.906000 1.676000

std 0.662834 0.332751 0.825578 0.424769

min 4.900000 2.000000 3.000000 1.000000

25% 5.800000 2.700000 4.375000 1.300000

50% 6.300000 2.900000 4.900000 1.600000

75% 6.700000 3.025000 5.525000 2.000000

max 7.900000 3.800000 6.900000 2.500000

Figure 4.19: Summary statistics for other species after preprocessing

4.3. Results

Of the 48 possible transformation histories that satisfied tenets 1, 2, and 3,

there was a wide variety of accuracy impact. The majority of the transformation

50

combinations yielded a drop in model accuracy of more than 20%, and 16 out of 48

combinations led to a decrease in accuracy of less than 10%. For the full results, see

A.1.

Figure 4.20: Model accuracy distribution (Iris Data)

One of the challenges with this exhaustive search is the size of the potential

solution space. In this example, we have 3 conditions, each of which have 2 trans-

formations to choose from. This means that there are 3 ∗ 2 = 6 potential values of

(Ci0 , tj0), (3 − 1) ∗ 2 = 4 potential values for (Ci1 , tj1), and (3 − 2) ∗ 2 = 2 poten-

tial values for (Ci2 , tj2). In total, there are 6 ∗ 4 ∗ 2 = 48 possible transformation

histories which satisfy all 3 tenets. As the number of conditions c and transforma-

tions per condition t grows, this scales according to c! ∗ t, which is quickly dominated

51

by the factorial term. In practice, 5! ∗ 2 = 240 combinations may be tenable, but

10! ∗ 2 = 7257600 may not be. This may serve as an effective limit on the number of

adopted tenets, and should be kept in mind during the constitutional voting process.

In this case, there were multiple combinations of transformations which yielded

less than 10% decrease in accuracy, and these combinations were testable within a

matter of minutes. This was achievable mainly due to the separable nature of the data

- in several dimensions, the data cleanly separates into two distinct classes. Even after

oversampling and transforming the data, the classes still remained clearly separable.

Although the Iris Data Set serves as a useful illustrative example, in general, this will

not be the case.

From the point of view of the AM/FM paradigm, the model building team

would have multiple avenues to achieve full fairness compliance without sacrificing

accuracy, and both teams would receive the maximum bonus.

52

Chapter 5: COMPAS Simulation

We now turn our attention to a real world data set, in order to illustrate the

AM/FM mechanism in use on a domain which has received scrutiny in the past several

years for issues of bias and discrimination. This data was first made available to the

public as part of ProPublica’s investigative report into the use of risk assessment tools

during sentencing and parole decisions. (Angwin et al., 2016)

5.1. Data

The training data consists of 10291 rows, each representing an individual who

has been assigned a risk score by the COMPAS system. Pre-processing was per-

formed to remove duplicate rows, and encode the severity of the charge degrees, with

the following values: {F1, F2, F3,M1,M2} where F means felony and M means mis-

demeanour. The number corresponds to the seriousness of the crime, where a lower

number means more severe than a higher number. For example, F3 is more severe

than M1, and M1 is in turn more severe than M2.

The attributes are as follows:

53

Attribute Name Data Type Example

Sex enum M, F

Age int 24

Race enum African-American, Caucasian

Charge Degree enum F3, M1

Juvenile Felony Count int 0

Juvenile Misdemeanour Count int 0

Juvenile Other Count int 0

Priors Count int 0

Recidivated binary T, F

Violent Recidivated binary T, F

Decile Score float 2.4

Table 5.1: COMPAS data schema

54

Figure 5.1: Risk Score Demographic Comparisons

55

5.2. Constitution

We imagine representatives from across an organization meeting to build a con-

stitution before proceeding to build a model for recidivism prediction. The represen-

tatives are tasked with agreeing upon several tenets and signing off on a constitution

that will guide the development of the model.

The discussion begins after everyone has examined the test data. Someone

brings a topic to the table, that the recidivism rate differs by race. The individual is

concerned that this could reflect over-policing and racially disparate outcomes in the

legal system, rather than a difference in rate of crimes committed. This point brings

objections from others, who feel that the data is reflecting reality, and modifying it

would amount to changing the fundamental behavior of the model. Another member

of the group explains that recidivism is defined as being rearrested and re-convicted

of a crime, but does not guarantee that the crime was actually committed. Con-

versely, committing another crime does not guarantee being arrested and convicted,

so operating solely on this feature would amount to predicting the very biases of the

existing law enforcement system. After a long discussion, the original objector agrees

to support a tenet that brings more racial balance to the recidivism rate, to account

for individuals who were never rearrested, and those who convicted though innocent.

The group agrees to propose Tenet 1, which says that the recidivism rate between

racial groups must be different by no more than ε.

|Xrecidothers| ≤ |Xrecid

african−american|+ ε (5.1)

Figure 5.2: Tenet 1: Recidivism Balance

56

African-American 5138Caucasian 3562Hispanic 939Other 570Asian 51Native American 31

Figure 5.3: Racial Demographics (COMPAS Data)

During the discussion about Tenet 1, someone in the group looks at the data

and notices that the racial distribution in the data is not representative of the general

population. This seems like a problem to them - what if the model learns to expect

one racial group more than others, despite the true distribution in the population?

The group looks at the numbers, and recognizes that they are not in line with the

Census:

Tenet 2 is then presented to the group, stating that the racial makeup of the

training data must match the national numbers according to the U.S. Census, up to

a tolerance ε.

|Xafrican−american| ≤ |X|+ ε (5.2)

Figure 5.4: Tenet 2: Racial Demographic Parity

A third objection is raised, based on the previous discussion about demo-

graphic parity. Someone who notices that sex is also unbalanced in the training set.

There are almost 4 times as many males in the sample as females, instead of an equal

number, as expected.

57

Male 8187

Female 2104

Figure 5.5: Individuals counted by sex (COMPAS Data)

A discussion ensues, during which many argue that such an imbalance by sex

could lead to the model being more accurate for males, or potentially discounting

female individuals. There is broad support for the concept of demographic balance

in terms of sex, though not for a particular implementation. Some argue that exactly

50% of the test data needs to be female, while others argue for a tolerance of several

percent. In the end, the group settles on a requirement that the sex balance must be

within ε = 0.01:

|Xmale| ≤ |Xfemale|+ ε (5.3)

Figure 5.6: Tenet 3: Sexual Demographic Parity

Another consideration is raised, about age discrimination. Someone in the

group recently read a paper about the disparate treatment of people under 25 in

recidivism prediction, and is concerned that the model will perpetuate this bias.

Summary statistics are provided to the group:

58

min age max age count

17 33 5679

34 49 3050

50 65 1389

66 80 170

81 96 3

Figure 5.7: Age distribution (COMPAS Data)

Many others object to this proposition, arguing with research that younger

people can be more impulsive than older adults. There are also concerns that adding

more than three tenets will make the model building process untenable. A vote shows

that there is insufficient support for this tenet, and it is dropped from consideration.

After agreeing on three tenets and rejecting a fourth, the constitution is

adopted with the following wording:

59

Preamble: Recognizing the sensitivity of recidivism predic-

tion and the biases present in data from the criminal justice

system, we are laying out the following tenets which will gov-

ern our model creation process. For all definitions of “close-

ness”, we will use a parameter ε, to be defined before imple-

mentation

Tenet 1: The recidivism rates across racial groups must be

close

Tenet 2: The racial demographics in the data must be close

to the numbers from the U.S. Census

Tenet 3: The count of females must be close to the count of

males

Revisions: Should an oversight or flaw be found in these

tenets during the model creation process, any member of the

organization may solicit support from others to call a vote to

amend. A simple majority is needed to call the vote to order,

and a 2/3 majority is needed to accept the change.

Figure 5.8: COMPAS Data Set Constitution

5.3. The Importance of Hyper Parameters

The following is a comparison of disparate outcomes by race for a Support

Vector Regressor (SVR) trained on COMPAS data with Radial Basis Function (RBF)

kernel with degree=3, ε=0.1. These are the default values used by scikit-learn, and are

used to illustrate the outcome when naively implementing a model without specifying

parameters. In this table, AA means African-American, NCC means Non-Caucasian,

60

and CC means Caucasian.

The free parameter is regularization C which is inversely related to the size of

the hyperplane defined in the SVR. For small values of C we expect more generaliz-

ability at the cost of misclassifying more points, while a larger value of C will result

in a smaller hyperplane and a higher risk of overfitting.

C µAA σAA µNCC σNCC µCC σCC Mean

Err

Stdev

Err

0.0005 3.2555 0.2642 3.2339 0.2722 3.1350 0.2864 -0.5059 2.3321

0.0010 3.2463 0.4924 3.2100 0.5054 3.0087 0.5499 -0.7661 2.3056

0.0050 3.4345 1.1819 3.3075 1.2120 2.8458 1.2760 -0.6223 1.9734

0.0100 3.5301 1.3895 3.3793 1.4264 2.9754 1.4297 -0.6242 1.9404

0.1000 3.8984 1.8008 3.6240 1.8370 2.9659 1.6894 -0.4692 1.8367

0.5000 4.0009 1.9160 3.6672 1.9715 2.8369 1.7899 -0.4211 1.7892

0.7500 4.0849 1.9543 3.7327 1.9996 2.8396 1.7776 -0.4651 1.8105

1.5000 4.0833 1.9652 3.7439 2.0245 2.8243 1.7452 -0.3672 1.7643

5.0000 4.2230 1.9901 3.8052 2.0458 2.8179 1.7980 -0.3135 1.7699

10.0000 4.0674 2.0459 3.7244 2.0755 2.7829 1.7956 -0.3502 1.7845

Figure 5.9: Summary statistics by race (COMPAS Data)

We can see that the absolute mean error is minimized for C = 5.0, however

this leads to a mean risk score difference of µAA − µC = 4.2230 − 2.8179 = 1.4051,

which is not optimal. If we were to only use µAA−µC as our minimization objective,

we would find that the value C = 0.0005 leads to a minimum value of 0.1205; in other

words the disparate treatment by race is more than 11 times higher based solely on

the choice of C.

61

The mean error and mean score difference are equalized near C = e−4 = 0.018

however this leads to a model with mean error approximately 2 times higher than the

minimum, and mean score difference approximately 5 times higher than the demon-

strated minimum. It follows that hyper parameter manipulation subject to a mixed

objective function will not result in a model that is ideal in both regards.

Figure 5.10: Trade off between mean error and disparate impact by race (COMPAS

Data)

The main action of the AM will be parameter manipulation with the end

goal of simultaneously satisfying fairness conditions and accuracy minimums. As

demonstrated, the choice of hyper parameters can have an enormous impact on the

behavior of the final model, and naively using default values can cause significant

62

harm when the model is making recommendations about limiting an individual’s

freedom.

5.4. Penalties and Disparate Outcomes

The penalty (loss) function is one of the fundamental characteristics of a model.

The optimization process is performed in order to minimize the value of this func-

tion. The penalty name is sometimes even present in the algorithm name, taking for

example “Least Squares Regression” which minimizes the squared residuals of the

function.

While we would expect the choice of penalty to strongly affect the outcome

of the model, we find weaker results when the results are averaged over multiple

trials. For Stochastic Gradient Descent Regressor (SGDR) models trained on the

COMPAS data, with potential penalties l1, l2, and elasticnet, we find little change in

the difference in average score between African American and Caucasian individuals.

63

Figure 5.11: Minimal change in mean scores for different penalty types (COMPAS

Data)

5.5. Experimental Setup

To simulate the creation of a COMPAS model in the AM/FM paradigm, we

reformulate the situation as a classification problem and use simulated annealing over

the choice of fairness transforms and hyperparameters. The idea is that simulated

annealing “cools off” over time and becomes less likely to make changes in the values,

similar to the team learning more about the data and domain and becoming more

confident in their choice of transforms and hyperparameters. For the purposes of the

simulation, we predict a label of 0, 1 for each individual, where 0 means the individual

will not be convicted of another crime, and 1 means that they will.

64

Input: X, model, nsteps, conditions, params

X ′ ←− X

Xtrain, ytrain, Xtest, ytest = split(X ′, y′)

model = model(params).f it(Xtrain, ytrain)

accbaseline = accuracyscore(model.predict(Xtest), ytest)

costold = 0

for i in nsteps do

for j in conditions.size do

X ′ = conditions[j].transform(X ′)

end

y′ = X ′.pop(isrecid)

X ′train, y′train, X

′test, y

′test = split(X ′, y′)

model = model(params).f it(X ′train, y′train)

accnew = accuracyscore(model.predict(X ′test), y′test)

costnew = accnew − accbaseline

u ∼ U(0, 1)

if exp (costold − costnew)/t > u then

fairstep(conditions)

accstep(params)

costold ←− costnew

end

end

return

Figure 5.12: Simulated Annealing of AM/FM

65

Input: fvec

op ∼ U(0, 1)

if op == 0 then

{Swap order of operations}

indices ∼ U(0, fvec.size, 2)

a = fvec[indices[0]]

b = fvec[indices[1]]

fvec[indices[0]] = b

fvec[indices[1]] = a

end

else

{Change transform on condition}

index ∼ U(0, fvec.size)

val = fvec[index]

val[1] + +

val[1] = val[1] mod 2

fvec[index] = val

end

Figure 5.13: fairstep: Fair State Step

66

Input: params

k ∼ U(params)

{Have a small but finite chance to not step the parameter}

step ∼ P ([−1, 0, 1]), P (−1) = P (1) and P (0) << P (1)

params[k] = params[k] + step

Figure 5.14: accstep: Accuracy State Step

5.6. Fairness Conditions

5.6.1 Tenet 1

In order to make the recidivism rates closer across different racial groups, the

following two transformations are made available to the model building team (AM).

Input: Xother, α, ε

raa = |Xrecidaa |/|Xaa|

rother = |Xrecidother|/|Xother|

u = raa ∗ ‖Xother‖ − ‖Xrecidother‖

rows[u] ∼ U(Xrecidother)

for i in rows do

Xrecidother[i][recid] = True

end

Xother ←− Xrecidother + (Xother −Xrecid

other)

Figure 5.15: Transformation 1 (COMPAS Data)

67

Input: Xaa, α, ε

raa = |Xrecidaa |/|Xaa|

rother = |Xrecidother|/|Xother|

u = rother ∗ ‖Xaa‖ − ‖Xrecidaa ‖

rows[u] ∼ U(Xrecidaa )

for i in rows do

Xrecidaa [i][recid] = False

end

Xaa ←− Xrecidaa + (Xaa −Xrecid

aa )


5.6.2 Tenet 2

Input: X

y = X.pop(race)

{Oversample to balance racial groups}

X ′, y′ = smote(X, y)

X ′[race] = y′

return X′


68

Input: X

{Demographic balance should be about 17% of the population}

rows = ‖Xaa‖/0.17

Xnew ∼ U(Xother, rows)

X ′ = X +Xnew

return X′


5.6.3 Tenet 3

Input: X

y = X.pop(sex)

X ′, y′ = smote(X, y)

X ′[sex] = y′

return X′


69

Input: X

rows = ‖Xfemale‖

Xnew ∼ U(Xmale, rows)

X ′ = Xfemale +Xnew

return X′


5.7. Results

The simulation was performed with 3 fairness conditions, each of which have

2 potential transformations, and a support vector classier (SVC) with a radial basis

function kernel and model parameter C. A maximum accuracy loss of 2% was allowed.

After 200 iterations, the simulation found 18 configurations that satisfied both the AM

and FM conditions, representing 9% of the configurations. These 18 configurations

lead to both the AM and FM receiving the maximum bonus b (Figure 2.4).

70

bonus

b/2 182

b 18

Figure 5.21: Hyperparameter search (COMPAS Data)

71

Figure 5.22: Accuracy Changes (COMPAS Data)

72

Figure 5.23: Accuracy difference per step (COMPAS Data)

5.7.1 Step Count Estimation

A core concern will be the number of steps required to find an optimal solution,,

such that all fairness and accuracy conditions are satisfied and the bonus is maximized

for all parties. To simulate this, 100 trials were run on a sample of the COMPAS data

(N = 6000) with a maximum of 50 steps per trial. Each trial began with a randomly

selected sequence of data transformations and hyperparameters C ∼ N (2, 2). The

hyperparameters were then iterated over with a step size of ∆C = 2.5. The number of

steps needed to achieve the maximum bonus was tracked for each trial, and is plotted

below. Trials that did not find an optimal solution within 50 steps were marked as

73

failures. In total, 79 of the trials found a maximum bonus point within the first 50

iterations, and 21 did not. Of the 100 trials, 46 required fewer than 5 steps to find

a maximum bonus, and 40 achieved a maximum bonus on the first step. On a 2019

Macbook Pro with a 2.3 GHz processor and 8 GB of RAM, this simulation took 2.04

hours.

steps

count 79.000000

mean 9.822785

std 13.921722

min 0.000000

25% 0.000000

50% 1.000000

75% 16.500000

max 46.000000

Figure 5.24: Step count summary statistics (COMPAS Data)

74

Figure 5.25: Steps to reach a maximum bonus (COMPAS Data)

75

Chapter 6: Conclusion

This thesis has described a novel mechanism for the creation of fair machine

learning models, using structures from the theory of democratic governance. In par-

ticular, a separation of powers approach was taken to distribute the power and re-

sponsibility into the hands of more groups acting semi-autonomously. Separation of

powers is an idea with roots in the Enlightenment, during which time there was ex-

tensive thinking about how to guarantee natural rights and ensure that single actors

or groups no longer had supreme power over others.

The first step in the mechanism is a constitutional convention, during which

representatives from across the organization convene to propose and vote on a series

of tenets to be honored by the model. This step minimizes the likelihood of one

individual or faction imposing their will or beliefs onto the rest of the organization.

Several voting systems were proposed, including a simple majority system, as well as

a ranked choice voting system. Once these tenets are agreed upon and formalized into

a constitution, the FM team translates the tenets into computational conditions: a

boolean function that tests if the condition is true, and a series of data transformations

which make the condition true. The AM can select transformations as needed in any

combination in order to maximize the model’s accuracy score.

An incentive system was constructed to provide each party with a bonus incen-

tive to perform their role. Crucially, the bonus of each party was tied to the success

of the other party, to ensure that the two parties viewed their success as part of a

76

larger whole. The incentives were constructed so that the AM always implements

the required fairness transformations, and the FM constructs the transformations to

maximize the accuracy potential of the model.

A theoretical framework for transformations of training data was described.

This framework examines data transformations from two perspectives: as an opera-

tion on a matrix, and as an operation on the parameters of the statistical distribution

that generates the data. These approaches both have pros and cons, including compu-

tational complexity and feasibility depending on the structure of the data in question.

A new incentive structure was also proposed and implemented, which ensures

that the different groups perform their responsibilities with the success of the other

groups as a priority. The group that creates fairness conditions (FM) is incentivized

to create data transformations that maximize the accuracy potential of the model

being developed, while the model creation team (AM) is incentivized to implement

a maximal number of fairness conditions during the model training process. The

overseer (JM) is incentivized similarly, and receives the maximum bonus only if both

the FM and AM are successful in their goals.

A computational study was performed on the Iris data set, in order to demon-

strate the constitution creation and data transformation process. This demonstration

highlighted the power of the AM/FM approach when finding optimal transformation

paths that satisfy fairness conditions. This study also discussed the computational

scaling issues which are encountered when performing an exhaustive search over the

parameter space, and the need for actors to make informed decisions on transforma-

tions and hyperparameters based on prior domain knowledge.

This thesis concluded with the results of a simulation of the AM/FM system

on data from the COMPAS risk assessment tool. This simulation modeled the tenet

recommendation and voting process, as well as highlighting examples of potential

77

disagreements during the constitutional convention. From 200 pairs of data transfor-

mations and hyperparameters, the simulation found 18 which satisfied both fairness

and accuracy conditions and led to both parties receiving the maximum bonus. These

optimal points in the search space were found in fewer than 10 steps on average.

78

Appendix A: Data Transformation Paths

A.1. Transformation Paths and Accuracy (Iris Data)

transforms acc orig acc trans

((1, 0), (2, 0), (3, 0)) 0.62 1.000000

((3, 0), (1, 0), (2, 0)) 0.62 1.000000

((2, 0), (1, 0), (3, 1)) 0.62 1.000000

((2, 0), (1, 0), (3, 0)) 0.62 1.000000

((3, 0), (1, 0), (2, 1)) 0.62 1.000000

((3, 1), (1, 0), (2, 0)) 0.62 1.000000

((1, 0), (3, 1), (2, 1)) 0.62 1.000000

((2, 1), (1, 0), (3, 0)) 0.62 1.000000

((1, 0), (3, 0), (2, 1)) 0.62 1.000000

((1, 0), (3, 1), (2, 0)) 0.62 0.985075

((3, 1), (1, 0), (2, 1)) 0.62 1.000000

((1, 0), (2, 1), (3, 1)) 0.62 1.000000

((1, 0), (2, 1), (3, 0)) 0.62 1.000000

((1, 0), (2, 0), (3, 1)) 0.62 1.000000

((1, 0), (3, 0), (2, 0)) 0.62 1.000000

((2, 1), (1, 0), (3, 1)) 0.62 1.000000

Figure A.1: Paths with significant accuracy loss

80


((2, 1), (1, 1), (3, 0)) 0.74 1.000000

((3, 0), (1, 1), (2, 1)) 0.78 1.000000

((3, 1), (1, 1), (2, 1)) 0.78 1.000000

((1, 1), (2, 1), (3, 0)) 0.82 1.000000

((3, 1), (1, 1), (2, 0)) 0.84 1.000000

((3, 0), (1, 1), (2, 0)) 0.84 1.000000

((2, 1), (1, 1), (3, 1)) 0.84 1.000000

((1, 1), (3, 0), (2, 1)) 0.84 1.000000

((1, 1), (2, 0), (3, 0)) 0.84 1.000000

((1, 1), (2, 0), (3, 1)) 0.84 1.000000

((2, 0), (1, 1), (3, 0)) 0.84 1.000000

((1, 1), (3, 1), (2, 0)) 0.84 1.000000

((2, 0), (1, 1), (3, 1)) 0.84 1.000000

((1, 1), (3, 0), (2, 0)) 0.84 1.000000

((1, 1), (2, 1), (3, 1)) 0.84 1.000000

((1, 1), (3, 1), (2, 1)) 0.88 1.000000

Figure A.2: Paths with significant accuracy loss (cont.)

81


((3, 1), (2, 0), (1, 1)) 1.00 1.000000

((3, 1), (2, 0), (1, 0)) 1.00 1.000000

((3, 0), (2, 1), (1, 1)) 1.00 1.000000

((3, 0), (2, 1), (1, 0)) 1.00 1.000000

((3, 0), (2, 0), (1, 1)) 1.00 1.000000

((3, 0), (2, 0), (1, 0)) 1.00 1.000000

((3, 1), (2, 1), (1, 0)) 1.00 1.000000

((2, 1), (3, 1), (1, 0)) 1.00 1.000000

((2, 1), (3, 0), (1, 1)) 1.00 1.000000

((2, 1), (3, 0), (1, 0)) 1.00 1.000000

((2, 0), (3, 1), (1, 1)) 1.00 1.000000

((2, 0), (3, 1), (1, 0)) 1.00 1.000000

((2, 0), (3, 0), (1, 1)) 1.00 1.000000

((2, 0), (3, 0), (1, 0)) 1.00 1.000000

((2, 1), (3, 1), (1, 1)) 1.00 1.000000

((3, 1), (2, 1), (1, 1)) 1.00 1.000000

Figure A.3: Paths with no accuracy loss

82

A.2. Transformation Paths (COMPAS Data)

C path cost

17.500001 ((1, 1), (3, 1), (2, 1)) -0.308000

12.500001 ((3, 1), (1, 1), (2, 1)) -0.277333

17.500001 ((3, 1), (1, 1), (2, 1)) -0.256000

12.500001 ((1, 1), (3, 1), (2, 1)) -0.224000

7.500001 ((3, 1), (1, 0), (2, 0)) -0.221333

7.500001 ((2, 1), (3, 0), (1, 0)) -0.220000

10.000001 ((2, 1), (1, 1), (3, 0)) -0.214667

10.000001 ((3, 0), (1, 0), (2, 1)) -0.192000

10.000001 ((3, 1), (1, 1), (2, 1)) -0.189333

10.000001 ((2, 1), (1, 0), (3, 1)) -0.189333

10.000001 ((3, 1), (1, 0), (2, 1)) -0.178667

7.500001 ((2, 1), (1, 1), (3, 0)) -0.174667

5.000001 ((2, 1), (3, 0), (1, 0)) -0.168000

7.500001 ((2, 0), (1, 1), (3, 1)) -0.144000

12.500001 ((1, 1), (2, 1), (3, 1)) -0.141333

10.000001 ((3, 0), (1, 1), (2, 1)) -0.134667

7.500001 ((1, 0), (3, 1), (2, 0)) -0.133333

5.000001 ((2, 1), (3, 0), (1, 1)) -0.132000

20.000001 ((2, 0), (3, 0), (1, 1)) -0.130667

15.000001 ((1, 1), (2, 1), (3, 1)) -0.129333

10.000001 ((2, 0), (3, 1), (1, 1)) -0.129333

Figure A.4: Sample of paths with bonus b/2

83

C path cost

5.000001 ((3, 1), (2, 1), (1, 0)) -0.018667

12.500001 ((1, 1), (3, 0), (2, 0)) -0.018667

0.614844 ((2, 0), (3, 0), (1, 1)) -0.018667

7.500001 ((2, 0), (1, 0), (3, 0)) -0.017333

5.000001 ((2, 0), (3, 1), (1, 1)) -0.014667

5.000001 ((2, 0), (3, 0), (1, 1)) -0.014667

2.500001 ((3, 1), (2, 0), (1, 1)) -0.014667

2.500001 ((3, 0), (1, 1), (2, 1)) -0.014667

10.000001 ((3, 0), (2, 1), (1, 1)) -0.013333

0.614844 ((1, 1), (3, 0), (2, 0)) -0.010667

0.614844 ((2, 1), (3, 0), (1, 1)) -0.010667

2.500001 ((3, 0), (1, 0), (2, 0)) -0.010667

2.500001 ((3, 1), (2, 0), (1, 1)) -0.009333

2.500001 ((2, 0), (3, 0), (1, 0)) -0.009333

0.614844 ((2, 1), (1, 1), (3, 0)) -0.006667

2.500001 ((3, 0), (1, 1), (2, 1)) -0.004000

2.500001 ((2, 0), (1, 0), (3, 1)) -0.004000

2.500001 ((1, 1), (2, 0), (3, 0)) 0.000000

Figure A.5: Paths with maximum bonus b

84

References

Aherne, F., Thacker, N., & Rockett, P. (1998). The bhattacharyya metric as an

absolute similarity measure for frequency coded data. Kybernetika, 34 (4), 363–

368.

Ananny, M. (2016). Toward an ethics of algorithms: Convening, observation, proba-

bility, and timeliness. Science, Technology, & Human Values , 41 (1), 93–117.

Angwin, J., Larson, J., Mattu, S., & Kirchner, L. (2016). Machine bias [Computer

software manual]. Retrieved from https://www.propublica.org/article/

machine-bias-risk-assessments-in-criminal-sentencing

Aristotle. (2016). Aristotle’s politics : writings from the complete works. Princeton,

New Jersey: Princeton University Press.

Baumann, E., & Rumberger, J. L. (2018). State of the art in fair ml: From moral

philosophy and legislation to fair classifiers.

Blum, A., & Stangl, K. (2019). Recovering from biased data: Can fairness constraints

improve accuracy? arXiv.org . Retrieved from http://search.proquest.com/

docview/2321241279/

Celis, L. E., Huang, L., Keswani, V., & Vishnoi, N. K. (2019). Classification with

fairness constraints: A meta-algorithm with provable guarantees. In Proceedings

of the conference on fairness, accountability, and transparency (pp. 319–328).

New York, NY, USA: ACM. Retrieved from http://doi.acm.org/10.1145/

3287560.3287586 doi: 10.1145/3287560.3287586

85

https://www.propublica.org/article/machine-bias-risk-assessments-in-criminal-sentencing

https://www.propublica.org/article/machine-bias-risk-assessments-in-criminal-sentencing

http://search.proquest.com/docview/2321241279/


http://doi.acm.org/10.1145/3287560.3287586

http://doi.acm.org/10.1145/3287560.3287586

Chakraborti, T., Patra, A., & Noble, A. (2019). Contrastive fairness in machine

learning.

Chander, A. (2017). The racist algorithm? Michigan Law Review , 115 , 1023–1213.

Chawla, N., Bowyer, K., Hall, L., & Kegelmeyer, W. (2002). Smote: Synthetic

minority over-sampling technique. Journal Of Artificial Intelligence Research,

16 , 321–357.

Chiappa, S., & Isaac, W. S. (2019). A causal bayesian networks viewpoint on fairness.

In Privacy and identity management. fairness, accountability, and transparency

in the age of big data: 13th ifip wg 9.2, 9.6/11.7, 11.6/sig 9.2.2 international

summer school, vienna, austria, august 20-24, 2018, revised selected papers (pp.

3–20). Cham: Springer International Publishing. Retrieved from https://

doi.org/10.1007/978-3-030-16744-8 1 doi: 10.1007/978-3-030-16744-8 1

Culliford, E. (2019, Dec 12). Facebook pledges $130 million to con-

tent oversight board, delays naming members. Reuters . Retrieved

from https://www.reuters.com/article/us-facebook-oversight/

facebook-pledges-130-million-to-content-oversight-board-delays

-naming-members-idUSKBN1YG1ZG

Donini, M., Oneto, L., Ben-David, S., Shawe-Taylor, J., & Pontil, M. (2018). Em-

pirical risk minimization under fairness constraints. arXiv.org . Retrieved from


Douzas, G., Bacao, F., & Last, F. (2018). Improving imbalanced learning through

a heuristic oversampling method based on k-means and smote. Information

Sciences , 465 , 1–20.

Dwork, C., Hardt, M., Pitassi, T., Reingold, O., & Zemel, R. (2012). Fairness through

awareness. In Proceedings of the 3rd innovations in theoretical computer science

conference (pp. 214–226). New York, NY, USA: ACM. Retrieved from http://

86

https://doi.org/10.1007/978-3-030-16744-8_1

https://doi.org/10.1007/978-3-030-16744-8_1

https://www.reuters.com/article/us-facebook-oversight/facebook-pledges-130-million-to-content-oversight-board-delays-naming-members-idUSKBN1YG1ZG




http://doi.acm.org/10.1145/2090236.2090255

http://doi.acm.org/10.1145/2090236.2090255

doi.acm.org/10.1145/2090236.2090255 doi: 10.1145/2090236.2090255

Fisher, R. A. (1936). The use of multiple measurements in taxonomic problems.

Annals of Eugenics , 7 (2), 179-188. Retrieved from https://onlinelibrary

.wiley.com/doi/abs/10.1111/j.1469-1809.1936.tb02137.x doi: 10.1111/

j.1469-1809.1936.tb02137.x

Hardt, M., Price, E., & Srebro, N. (2016). Equality of opportunity in supervised learn-

ing. In Proceedings of the 30th international conference on neural information

processing systems (p. 33233331). Red Hook, NY, USA: Curran Associates Inc.

Ilvento, C. (2019). Metric learning for individual fairness. CoRR, abs/1906.00250 .

Retrieved from http://arxiv.org/abs/1906.00250

Joseph, M., Kearns, M., Morgenstern, J., Neel, S., & Roth, A. (2016, 10). Rawlsian

fairness for machine learning. ArXiv .

Kamiran, F., & Calders, T. (2009). Classifying without discriminating. In 2009 2nd

international conference on computer, control and communication (pp. 1–6).

IEEE.

Kamiran, F., & Calders, T. (2012, Oct 01). Data preprocessing techniques for clas-

sification without discrimination. Knowledge and Information Systems , 33 (1),

1–33. Retrieved from https://doi.org/10.1007/s10115-011-0463-8 doi:

10.1007/s10115-011-0463-8

Khademi, A., Lee, S., Foley, D., & Honavar, V. (2019). Fairness in algorithmic decision

making: An excursion through the lens of causality. arXiv.org . Retrieved from


Kilbertus, N., Rojas-Carulla, M., Parascandolo, G., Hardt, M., Janzing, D., &

Scholkopf, B. (2017). Avoiding discrimination through causal reasoning. In

Proceedings of the 31st international conference on neural information process-

ing systems (p. 656666). Red Hook, NY, USA: Curran Associates Inc.

87

http://doi.acm.org/10.1145/2090236.2090255

http://doi.acm.org/10.1145/2090236.2090255

https://onlinelibrary.wiley.com/doi/abs/10.1111/j.1469-1809.1936.tb02137.x


http://arxiv.org/abs/1906.00250

https://doi.org/10.1007/s10115-011-0463-8


Kroll, J. A., Huey, J., Barocas, S., Felten, E. W., Reidenberg, J. R., Robinson, D. G.,

& Yu, H. (2017). Accountable algorithms. University of Pennsylvania Law

Review , 165 (3), 633.

Luong, B., Ruggieri, S., & Turini, F. (2011). k-nn as an implementation of situation

testing for discrimination discovery and prevention. In Proceedings of the 17th

acm sigkdd international conference on knowledge discovery and data mining

(pp. 502–510). ACM.

Matusita, K. (1955). Decision rules, based on the distance, for problems of fit, two

samples, and estimation. Ann. Math. Statist., 26 (4), 631–640.

Montesquieu, C. d. S. (1989). The spirit of the laws. Cambridge ; New York:

Cambridge University Press.

Mukhopadhyay, S., & Bindu Jain. (2001, Sep.). Multi-agent markov decision pro-

cesses with limited agent communication. In Proceeding of the 2001 ieee interna-

tional symposium on intelligent control (isic ’01) (cat. no.01ch37206) (p. 7-12).

doi: 10.1109/ISIC.2001.971476

Noble, S. U. (2018). Algorithms of oppression : how search engines reinforce racism.

New York: New York University Press.

Pessach, D., & Shmueli, E. (2020). Algorithmic fairness.

Pettit, P. (1989). A definition of negative liberty. Ratio, 2 (2), 153-168. Re-

trieved from https://onlinelibrary.wiley.com/doi/abs/10.1111/j.1467

-9329.1989.tb00004.x doi: 10.1111/j.1467-9329.1989.tb00004.x

Read, M. (2018). Does facebook need a constitution? New York . Retrieved from


Scarf, H., & Shapley, L. (2016). Games with partial information. In Contributions

to the theory of games (am-39), volume iii (p. 213). Princeton, New Jersey:

Princeton University Press.

88




Statista. (2019). Number of smartphone users in the united states from 2010 to

2023 (in millions). Retrieved from https://www-statista-com.ezp-prod1

.hul.harvard.edu/statistics/201182/forecast-of-smartphone-users

-in-the-us/

Wisconsin v. Loomis, No. 2015AP157CR. (2016).

Veale, M., & Binns, R. (2017). Fairer machine learning in the real world: Mit-

igating discrimination without collecting sensitive data. Big Data & Soci-

ety , 4 (2), 2053951717743530. Retrieved from https://doi.org/10.1177/

2053951717743530 doi: 10.1177/2053951717743530

Wadsworth, C., Vera, F., & Piech, C. (2018). Achieving fairness through adversarial

learning: an application to recidivism prediction. arXiv.org . Retrieved from


Why mark zuckerbergs oversight board may kill his political ad policy. (n.d.). Wired .

Retrieved from https://www.wired.com/story/facebook-oversight-board

-bylaws/

Xiang, A., & Raji, I. D. (2019). On the legal compatibility of fairness definitions.

Zemel, R., Wu, Y., Swersky, K., Pitassi, T., & Dwork, C. (2013, 01). Learning fair

representations. 30th International Conference on Machine Learning, ICML

2013 , 1362-1370.

89

https://www-statista-com.ezp-prod1.hul.harvard.edu/statistics/201182/forecast-of-smartphone-users-in-the-us/



https://doi.org/10.1177/2053951717743530

https://doi.org/10.1177/2053951717743530


https://www.wired.com/story/facebook-oversight-board-bylaws/

https://www.wired.com/story/facebook-oversight-board-bylaws/

fairness via separation of powers - dash home

Documents