the researcher’s blind spot: 6 cognitive biases we shouldn’t ignore … · 2018-05-07 · the...
TRANSCRIPT
The researcher’s blind spot: 6 cognitive biases we shouldn’t ignore in researchUX New Zealand 2016Ruth Ellison, Principal User Researcher, PwC’s Experience Centre@RuthEllison
@RuthEllison
Hi:)
2
@RuthEllison
Free!
$10
$7
$20
@RuthEllison
$1
$10
$8
$20
@RuthEllison
The Zero Price Effect is a Cognitive Bias
@RuthEllison
What is a cognitive bias?Mental shortcut
@RuthEllison
You can buy a copy of this poster from: https://www.designhacks.co/products/cognitive-bias-
codex-poster
@RuthEllison 8
We are all humanPhoto available under a CC by 2.0 licence https://www.flickr.com/photos/yellowbookltd/14907364551/
@RuthEllison Image source: http://www.businessinsider.com.au/w hat-new-coke-looked-like-in-1985-2013- 3
@RuthEllison
Applying it to design research
PreparingRunning
Analysing
@RuthEllison
Preparing for research
@RuthEllison
@RuthEllison
Selection bias
@RuthEllison
Multiple channels - mix of recruitment companies, social media, trusted networks
Avoid professional respondents
Careful screeners
Behavioural based recruitment
@RuthEllison
Running research sessions
Photo available under a CC by 2.0 licence: https://www.flickr.com/photos/gdsteam/20649386153
@RuthEllison
Observer-Expectancy effect
@RuthEllison 17
Source: https://medium.com/@userfocus/the-7-deadly-sins-of-user-r esearch-22857c5a971b#.xb38ukdtr
@RuthEllison
The way we frame our questions matter
@RuthEllison
Loftus, E. F., & Palmer, J. C. Reconstruction of automobile destruction: An example of the interaction between language and memory. Journal of Verbal Learning and Verbal Behavior, 1974, 13, 585-589.
McLeod, S. A. (2014). Loftus and Palmer. Retrieved from www.simplypsychology.org/loftus-palmer.html
About how fast were the cars going when they smashed into each
other?
@RuthEllison
About how fast were the cars going when they collided into each other?
About how fast were the cars going when they bumped into each other?
About how fast were the cars going when they contacted into each other?
About how fast were the cars going when they smashed into each other?
@RuthEllison
Triangulate research
Use observational methods
Keeping positive-neutral body language, watch the tone of your voice
Avoid leading questions
@RuthEllison 22
I need a couple of volunteers please
23
248
What’s going on?
@RuthEllison
Confirmation bias
@RuthEllison 26
27
Take the opposing view
28
Don’t just confirm your hypothesis, see if you can prove it wrong
Analysis of Competing Hypotheses (ACH)
@RuthEllison
List assumptions
Be skeptical, especially if everyone agrees with you
Remain open
Consider all evidence equally
Multiple user researchers
Leave your ego by the door.
@RuthEllison from PwC’s Digital Services
@RuthEllison
Groupthink & the Bandwagon effect
@RuthEllison 31
@RuthEllison
Triangulate with other research methods (e.g. observational)
Use a mixture of individual and group exercises
Avoid stating preferences and expectations at start
Give someone the devil’s advocate role to question assumptions
@RuthEllison
Anchoring bias
@RuthEllison 34
35https://www.flickr.com/photos/naotakem/17105160150/
@RuthEllison
Consider the order of questions and designs carefully
Open ended questions
Alternate order in which participants are shown concept or design versions
Show version A first1 3 5
2 4 6 Show version B first
@RuthEllison
Analysing
@RuthEllison 38
@RuthEllison
Apophenia
39Source: https://au.pinterest.com/pin/249738741807276910/
@RuthEllison
Clustering illusion & reporting bias
@RuthEllison
Sample sizes – it’s about the WHY
Consider evidence equally – not just the ones that confirm your belief/assumption
Collaborative analysis sessions
Strive for objectivity
@RuthEllison 42
How vulnerable are you to these biases?
@RuthEllison
What percentage of the population will be more biased
than the average person?
@RuthEllison
Are you more biased than the average person?
@RuthEllison
Blind spot bias
@RuthEllison
Listen with an open mind
Become more rational, but less rationalising
Continuous learning
46
Always assess your method, your analysis and yourself for
bias
@RuthEllison 47
We are all human
@RuthEllison Image source: https://en.wikipedia.org/wiki/Twenty_Questions#/media/File:20_questions _1954.JPG
Thank you:)
Say hello @RuthEllison
@RuthEllison
Further reading
List of cognitive biases
You are not so smart: a celebration of self delusion
9 Biases In Usability Testing
50