desperately seeking theory: gamification, theory, and the promise of a data/ai-driven new science of...
TRANSCRIPT
DESPERATELY SEEKING THEORY
Gamification, Theory, and the Promise of a Data/AI-Driven New Science of Design
#GamifIR 2016 Sebastian Deterding @dingstweets
<1> the challenge
we are stuck in a groundhog day
“does gamification work?”
MU
http://www.flickr.com/photos/ewanrayment/1250158647
“does medicine work?”
some better questions
What active substance in what dosage affects what condition under what circumstances? How?
Do 500 mg of orally administered acetylsalicylic acid affect migraine in adult chronic migraine patients? How?
Do publicly displayed achievements linked to publicly known as hard-to-attain goals support goal-setting in achievement-oriented individuals? How?
To ask such questions, we need
theory
what is theory? (a quick refresh)
A theory is a set of statements expressing a nomological network of constructs and their relations. It is tied to observable phenomena through operationalisations.
Whitney, Kite & Adams, 2013; Cronbah & Meehl, 1955
why theory? (a quick refresh)
Whitney, Kite & Adams, 2013
• Allows to describe, explain, predict, and control reality
• Allows to systematically organise and extend knowledge
without theory, no science
how do cognitive states
affectbehaviour
affectdesign features
?
This is a
grand question inviting a new
science of design
the current basic mediation model
design element 2
magic motivation! behaviour
design element 1
design element 3
e.g. Hamari, Koivisto & Sarsa, 2014
the state of gamification theory
our biblical texts
So where is the
issue?
design element 2
magic motivation! behaviour
design element 1
design element 3
the issues with design elements
the state of gamification theory
“… a certain Chinese encyclopaedia entitled ‘Celestial Empire of benevolent Knowledge’. In its remote pages it is written that the animals are divided into: (a) belonging to the emperor, (b) embalmed, (c) tame, (d) sucking pigs (e) sirens, (f ) fabulous, (g) stray dogs, (h) included in the present classification, (i) frenzied, (j) innumerable, (k) drawn with a very fine camelhair brush, (l) et cetera, (m) having just broken the water pitcher, (n) that from a long way off look like flies.”
issue
#1
we have chinese encyclopaedias
badges
points
leader boards
levels
feedbackcl
ear
goal
s
reward
progress
challe
nge
story/theme
when is “a badge” a badge?
issue
#2
all exegesis, no discovery of new species
issue
#3
we’re stuck on low-level design features
issue
#3a
Deterding et al., 2011
where the action is
we need a linné
an empirically grounded, well-operationalised, well-formed taxonomy of design elements
this is a massive undertaking
design element 2
magic motivation! behaviour
design element 1
design element 3
the issues with motivational processes
issue
#1
motives are multiple
johnmarshall reeve
»Motivation’s new paradigm is one in which behavior is energized and directed not by a single grand cause but, instead, by a multitude of multilevel and coacting influences.«
understanding motivation & emotion (2009: 45)
gaming motives Competence Autonomy Relatedness Meaning Curiosity Absorption (»flow«) Control beliefs Goal-setting Recognition, belonging, power Identity/Self Emotions Habituation …
design element 1
design element 2
design element 3
behaviour
competence
autonomy
relatedness
meaning
curiosity
...
Aparicio et al., 2012
Sailer et al., 2013 Zhang, 2008
Zic
herm
ann
&
Cun
ning
ham
, 20
11
none of this is empirically tested
issue
#2
(Another massive undertaking)
Elements are multi-functionalAntin & Churchill 2011
goal-settinginstruction
group identification
status
reputation
issue
#3
design element 1
design element 2
design element 3
behaviour
competence
autonomy
relatedness
meaning
curiosity
...
feedback
appraised as
controllingthwarts
autonomy
motivation
appraised as
informingsupports
competence
+
–
motivational function is an appraisal
issue
#4
Deci & Ryan 2001
design element 1
design element 2
design element 3
behaviour
competence
autonomy
relatedness
meaning
curiosity
...
appraisal
motivation and appraisal are contextual
issue
#5
Reeve, 2006
license to reconfigure & leaveDeterding, 2016
minimised consequenceDeterding, 2016
situation element 1-n
design element 1
design element 2
behaviour
competence
autonomy
relatedness
meaning
curiosity
...
appraisal
situation
different people, different motives
issue
#6
trait levelCroson & Gneezy, 2000
state levelBowman & Tamborini, 2013
situation element 1-n
design element 1
design element 2
behaviour
competence
autonomy
relatedness
meaning
curiosity
...
appraisal
situation
person
http://www.flickr.com/photos/8147452@N05/2913356030/sizes/o/
motivational function is systemic-dynamic
issue
#7
AestheticsMechanics Dynamics
Hunicke, LeBlanc & Zubekmda: a formal approach to game design (2004)
Monopoly
aesthetic
Frustrating end game
mechanic dynamic
Slow poverty gap
+$ !+-$ !-
Diff
icul
ty
Skill/Time
frustration
boredom
function is a dynamic person-situation relation
»flow«
points badges leaderboards
Tracking, Feedback Goals, surprise Competition
we focus on features …
when we should look at how features in relation to dispositions afford motivational functions
Diff
icul
ty
Skill/Time
frustration
boredom
»flow«
e.g. balanced challengeCsikszentmihalyi, 1990
http://www.flickr.com/photos/amrufm/2593920251/sizes/z/in/photostream/
e.g. curiositySilva, 2006
http://www.flickr.com/photos/amrufm/2593920251/sizes/z/in/photostream/
e.g. MeaningfulnessWong, 2012
motivational affordance The complex of necessary and sufficient relations of actor dispositions and environment features that render an action or event functionally significant for a specific motive
Deterding, under review
competence
autonomy
relatedness
meaning
curiosity
...
appraisal
p/s relation affordance 1
p/s relation affordance 2
p/s relation affordance 3
behaviour
<2> the opportunity
Kra
mer
, Gui
llory
& H
anco
ck, 2
015
not that opportunity
jad abumrad
»Facebook has created a laboratory of human behavior the likes of which we’ve never seen.«
the trust engineers (2015)
At any moment, software companies are running
millions of a/B tests to optimise user engagement,
each of which is an experiment in waiting
the scale for a massive undertaking
design is being automated
But to know what to test & how to automate, you need …
theory
»Online hypothesis testing can accelerate both applied and basic Interaction Design Science by making it fast and easy to obtain ecologically-valid measures of the effects of designs on user behavior. Online controlled experiments can help build practical, generalizable, and scientifically-validated theories of how and why designs affect human interactions.«
derek Lomasoptimizing motivation and learning with large-scale game design experiments (2014: 7)
derek Lomasoptimizing motivation and learning with large-scale game design experiments (2014: 7)
Data, suggestions
Designs/tweaks
Theories, formalisations,
indicators
Data, indicators
researcher
designeruser
Adapted content & interface
Experience & behaviour data
data/ai
an example
»flow«
Diff
icul
ty
Skill/time
frustration
boredom
flow (1990)Mihaly Csikszentmihalyi
basic
theory
Difi
icul
ty
Skill/time
frustration
boredom
what’s the optimal curve?Alexander, Sear & Oikonomou, 2013; Sampayo-Vargas et al., 2013; Brazil & Blau, 2014
crowdsourcing
design
application
Diff
icul
ty
Skill/time
crowdsourcing tasks are predetermined
?
?
? ?
?
??
??
?
? ?
?
?
??
?
Diff
icul
ty
Skill/time
?
?
? ?
?
??
??
?
? ?
?
?
??
?
Difficulty is unknown; identification may obsolete crowdsourcing
crowdsourcing tasks are predetermined
Diff
icul
ty
Skill/time
?
?
? ?
?
??
??
?
? ?
?
?
??
?
… hence tasks are often randomly served
crowdsourcing tasks are predetermined
% P
laye
r re
tain
ed
Time/levels
* idealised
tutorial
actual tasks*
hence retention is very poor
Diff
icul
ty
Skill/time
How to order tasks w/o solving?
?
?
?
?
?
??
?
the
question
?
?
?
Diff
icul
ty
Skill/time
?
?
?
?
?
??
?
?
?
?
… which requires skill concepts & indicators
player x Pattern recognition: 7/10 Spatial rotation: 2/10
Diff
icul
ty
Skill/time
?
?
?
?
?
??
?
?
?
? player x Pattern recognition: 7/10 Spatial rotation: 2/10
… which requires difficulty concepts & indicators
task y Pattern recognition: 7/10 Spatial rotation: 2/10
Performance data Improvement suggestions
Task length design Task descriptions
Initial balancing theories Initial indicators of skill
Optimal difficulty curve Indicators for skill, difficulty
researcher
designeruser
Task sequence & task support & feedback
Engagement & performance data
data/ai
<3> summary
to get out of groundhog day …
we need to ask better questions …
to systematically build knowledge …
how do cognitive states
affectbehaviour
affectdesign features
?
… for a grand new science of design.
To ask such questions, we need
theory
and we must abandon scholasticism.
we need better, empirical taxonomies …
competence
autonomy
relatedness
meaning
curiosity
...
appraisal
p/s relation affordance 1
p/s relation affordance 2
p/s relation affordance 3
behaviour
… and better, empirical mechanistic models.
this is a massive undertaking …
with a matching massive opportunity.
we can be at the forefront of design evolution …
… if we loop research into the world’s largest design lab that is running every day.
Data, suggestions
Designs/tweaks
Theories, formalisations,
indicatorsData,
indicators
researcher
designeruser
Adapted content & interface
Experience & behaviour data
data/ai
