gamification: how effective is it?
DESCRIPTION
We presented this deck at the ESOMAR Congress 2011 conference in Amsterdam where it was nominated for "Best Methodological Paper". The meat of this deck is a collection of case studies showing the efficacy of gamification in various BUSINESS contexts. It took us ages to contact and collate these various examples, so hopefully having them all in one place will save you time. A big thank you very much to the various folks who helped us put this piece of research together! If you have any questions, comments, requests, or are interested in the original paper that this deck is based on, please feel free to drop us a line :)TRANSCRIPT
Goal?Evaluate the effectiveness of gamification and explore potential applications in business & market research
How?• Literature review• Expert interviews• Experiment• Case studies from
market leaders
Background
Note:This deck only covers gamification in business. It does not touch on serious games, entertainment, education, politics, etc.
Interviewees
Jesse SchellChief Executive Officer and Creative Director of Schell Games
Sebastian DeterdingGamification guru and PhD researcher
Michael WuPrincipal Scientist of Analytics at Lithium Technologies
Danny DayCEO of QCF Design (IGF award-winning developers of Desktop Dungeons)
Kevin Spier and Dan Maier of Bunchball gamification service
Bo NielsenAssociate Director, TNS
Jon PulestonSenior Director of GMI Interactive
Francesco D’OrazioResearch Director and Head of Social Media of Face Group
Phil GromanHead of Innovations for Afroes
Rolfe SwintonDirector of Lumi Mobile
What it’s not
What it is
Examples
How Does It Work?
Where CanIt Go Wrong?
How Effective Is It?
Conclusions
LEVEL 1What it’s not
LEVEL 1: What it’s not
Video games
“Badgification” /“Pointsification”
LEVEL 2What it is
LEVEL 2: What is it?Definition
The integration of the mechanics that make games fun and absorbing into non-game
platforms and experiences in order to improve
engagement and participation
~ The Authors
“
”
LEVEL 3Examples
LEVEL 3: ExamplesFarmville
LEVEL 3: ExamplesFoursquare
LEVEL 3: ExamplesKlout
LEVEL 4: How Does It Work?
LEVEL 4: How Does It Work?Hijacking the brain
Experience systems
Short- and long-term goals
Rapid, frequent feedback
Other people
Rewards for effort
Uncertainty
LEVEL 4: How Does It Work?Example mechanics
Appointment dynamic
Achievement
Community collaboration
LEVEL 4: How Does It Work?Can anything be gamified?
Do people not do something because they are not able to? - then increase ease of use.
Do people not do it because they have no free time? - then work on that.
Only if motivation is the issue can gamification be a [legitimate] way [of influencing behaviour]
~ Sebastian Deterding, researcher
”
“
BOSS BATTLE!Where Can It Go Wrong?
In my experience game mechanics have massive potential in the research industry but low-grade gamification is only
going to distort social interaction and skew research outputs.
~ Francesco "D’Orazio, Research Director for Face Group
“
”
If your idea is to create a bribery system to get [users] to try
something, it can backfire. When the bribes go away, people are less inclined naturally to do the
thing you want, even if it's fun
~ Jesse Schell, CEO Schell Games
“”“
Rewards are not equivalent to achievement
Limited participation bandwidth
Unintended consequences / Gaming the system
Undermining intrinsic values and…
…interfering with social norms
Congratulations!You defeated the Pitfall Boss
BONUS LEVEL: How Effective Is It?
Marketing amplification
Community curation
Market research
Community curation
BONUS LEVEL: How Effective Is It?
Evly experimentBONUS LEVEL: How effective is it?
n=30
1.5
2.3
Non-gamified Gamified
Ave. no. user posts
Non-gamified (control)
83%
17%
Contributors Observers
68%
32%
Contributors Observers
n=25
n=30
User participation
Gamified (experiment)
37%
63%
Comments Answers
3%
97%Comments Answers
n=37
n=68
Community interaction
Non-gamified (control)
Gamified (experiment)
Source: Findlay & Alberts, 2011
Awards Nominations
Questions (2010)
10,000
Answers (2010)
100,000Average response time
3mins (24/7)
Number of employees
16
95% answered in…
60mins (24/7)
Source: Lithium Technologies
The effect of gamification is
pretty astounding and has even surprised us in what it’s able to do
in the case of giffgaff”
~ Michael Wu (Principle Scientist of Analytics at Lithium Technologies)
“
”giffgaffBONUS LEVEL: How effective is it?
9
16
Pre-gamification Post-gamification
Page views (in millions)
2
5Ave. no. monthly visits
Pre-gamification Post-gamification
14
22Ave. time spent on site (mins)
Pre-gamification Post-gamification
Merchandise sales
Pre sales Post sales
+47%
Source: Bunchball
Club PsychBONUS LEVEL: How effective is it?
4
12
Pre-gamification Post-gamification
Actions before logging out
7
15
Pre-gamification Post-gamification
% posting to blog
Source: TechCrunch
Purchases per active user
DevHubBONUS LEVEL: How effective is it?
BONUS LEVEL: How Effective Is It?
Marketing amplification
…of users played more than once
85%
…returned the following month
50%
…increase in revenue via game from one month to next
60%
Source: Bunchball
PlayboyBONUS LEVEL: How effective is it?
Source: Afroes
Champ ChaseBONUS LEVEL: How effective is it?
1.83
2.41
Pre-game Post-game
Ave. awareness score(composite measure)
85%
Identified types of abuse
20%
Identified types of abuse
3%
Identified types of abuse
Pre-game
Post-game
n=20
Pre-game
95%Mentioned Childline Did not
15%
Mentioned Childline Did not
n=20Mentioned Childline
Post-game
BONUS LEVEL: How Effective Is It?
Market research
UK (non-gamified)
99%Completed
30%
Completed
Completion rate
USA (gamified)
3.5
8.0
UK(non-gamified)
USA(gamified)
Ave. happiness (out of 10)
Source: Lumi Mobile
Consumption diaryBONUS LEVEL: How effective is it?
+50%
Enjoyment of Oscars
Non-players Players
Source: Lumi Mobile
Disney & OscarsBONUS LEVEL: How effective is it?
Length watching Oscars
+42%
Non-players Players
Benefits:• Increased engagement• Improved data quality
Unintended consequences:• Drinking game• Returning & asking for more questions, and…• … to continue chatting
Source: TNS
It was like being a part of a community“ ”
Really liked the chat, where people discussed the show“ ”
Noticed more details in the show than normally
”“
Found it funny to rate the TV-show and see the results
”“Fun to be a part of this new kind of test“ ”
EurovisionBONUS LEVEL: How effective is it?
Survey questionSteak au pouivrePesto Pastafish and chipsgarlic chicken
‘Last meal’ game“Scotch broth soup as a starter served with garlic bread. Medium grilled gammon steak with a lightly fried egg on top with chips and side salad. A glass of red wine. A sticky toffee pudding, followed by cheese and biscuits.”
Framing
Source: GMI Interactive
Question designBONUS LEVEL: How effective is it?
Which of these do you have in your room?TV
FishMP3 player
BooksRadio
MagazinesConsoleCamera
SkateboardStereo
DVDsCDs
HampsterClothes
Piggy bankRocket
Which of these do you have in your room?
Visualising
Rapid and frequent feedback
FINAL BOSS BATTLE
FINAL BOSS BATTLE:Market Research Considerations
Market research industry has
been slow in making surveys
more engaging
To make surveys more
engaging:
visual design
reframing language of
questions
rapid & clear feedback
build in novelty/uncertainty
build in status (MROCs)
incorporate social elements
FINAL BOSS BATTLE:Market Research Considerations
Gamification can benefit our industry…
“A double shift in focus and framing:
(1) from usability(reducing friction) to motivation (increasing drive),
(2) from extrinsic motivation (incentives) to intrinsic motivation (competence,
autonomy, relatedness needs).
At best, it is a set of lenses and design patterns to improve intrinsic motivation.”
~ Sebastian Deterding , researcher
“ …if we can step outside our comfort zones
“In order to truly turn something into a game, it
often needs to change so much in order to facilitate player agency that few people are
willing to begin the process”
~ Danny Day, QCF Design (developers of Desktop Dungeons)
“
” ”
FINAL BOSS BATTLE:Conclusions
FINAL BOSS BATTLE:Is it a fad?
“In some ways it is a fad - adding points and badges in tacky ways, looking at
‘gamification’ as an easy way to make boring things seem interesting - that is a fad.
However, the idea of designing business processes so that those who engage in them find them more
intrinsically rewarding - that is a long term trend”.
~ Jesse Schell , CEO Schell Games
“In three years, we will talk about what is at the core of it - design for motivation - not about
the one strategy to get there: getting inspiration from games.”
~ Sebastian Deterding, researcher
“
”
“”
FINAL BOSS BATTLE:Conclusions
Gamification seems to work…
Gamification can benefit research……but it’s no
“magic elixir”…in subtle and fundamental ways……if we keep an
open mindset
CONGRATS!!! You defeated the Final Boss