The Ethics Of A/B Testing

What is A/B Testing

Also named randomized controlled experiments, online controlled experiments, and split testing.

In marketing, A/B testing is a simple randomized experiment with two variants, A and B.

It sees if different versions of website with little changes make us use more, visit more, click more or buy more.

1

Example

This example for a website that make two version with little changes

2

Facebook feeling A/B test

PNAS magazine published results of an experiment on 689,003 users to see if showing them more positive or negative sentiment posts in the News Feed would affect their happiness levels as deduced by what they posted.

3

The Experiment Had Almost No Effect

Lets take a look on this experiment. Experiment duration : 1 Week

From January 11–18, 2012 People shown fewer positive posts (a

more depressing feed) posted 0.1% fewer positive words in their posts.

People shown fewer negative posts (a happier feed) post 0.07% fewer negative words.

4

Experiment risk and fails

the National Institute Of Mental Health says 9.5% of Americans have mood disorders, which can often lead to depression.

Some people who are at risk of depression were almost surely part of Facebook’s study group.

Status update isn’t the real emotion, may this people just following the trends they see on Facebook.

Another research says that some people feel sad when they see lots of happy status, because they feels that them life isn’t happy as they see on Facebook.

6

Facebook Didn’t Get Ethics Board Approval

Facebook only did an internal review to decide if the study was ethical.

the study would likely fail to meet many of the pre-requisites. But actually Facebook manipulates the News Feed all the time to

test what types of stories and designs makes you spend time, money like ,comment or generally more active.

Facebook technically has consent from all users, as its Data Use Policy people automatically agree to when they sign up says

“we may use the information we receive about you…for…data

analysis, testing, research and service improvement.”

5

Everyone Is A/B Testing

there are lots of A/B Tests but most are pushing for more business-oriented results like increasing usage or clicks or purchases.

A lot of times we don’t use the “real” Facebook. Or Twitter. Or Google, Yahoo, or LinkedIn. We are almost all part of experiments they quietly.

7

Regulation, Or At Least Safeguards

Offering an opt out Not for every test, but ones with some real risk like showing users a more depressing feed.

8

Regulation, Or At Least Safeguards (cont.)

Independent protection against harmful emotional manipulation on a grand scale.

At the very least, the tech companies should educate their data scientists and others designing A/B tests about the ethical research methods associated with having experiments approved by the IRB

9

Till this happen

Till this regulations happen, all of us are Laboratory rats

10

Thank you :D

11