the effects of individual game mechanics and web-based testing on cognitive test performance and...

The effects of game mechanics on cognitive test performance and participant enjoymentPrepared for TRACK 3: Quality ImprovementGames for Health Europe, Utrecht, 2016

Jim Lumsdenjim.lumsden@bristol.ac.uk

School of Experimental Psychology,University of Bristol, BS8 1TU, Bristol, UK

AbstractComputerised cognitive assessments are a vital tool in the behavioural sciences, but participants often view them as effortful and unengaging. One potential solution is to add gamelike elements to these tasks in order to make them more intrinsically enjoyable.

In this talk I will present two studies into the effects of individual game mechanics on the data and enjoyment ratings from cognitive tasks designed to measure inhibitory control. In both studies, we tested three variants of the tasks: one in which participants were rewarded with points for performing optimally, one where the task was given an overall theme and graphical upgrade, and a third version which was a non-gamified comparator. I will discuss our findings, and how they might guide the development of future gamified cognitive assessments.

Lumsden J, Skinner A, Woods AT, Lawrence NS, Munafò M. (2016) The effects of gamelike features and test location on cognitive test performance and participant enjoyment, PeerJ

Cognitive Tasks

• Cognitive tasks are a staple tool in psychological research

• Must collect pure measures of cognition and therefore are simplistic and involve lots of repetition

• Cannot produce valid measures if the participant is bored

• They need to be engaging and gamification may be the answer

Research Questions

1. Do different game mechanics effect subjective ratings of engagement?

2. Do different game mechanics effect the cognitive data collected?

3. Do different game mechanics influence attrition rates?B(Study 2 only)

We ran two studies to look at the effects of game mechanics on the data collected by two cognitive tasks

Study 1 Overview (the cowboy study)

• Three variants of a Go-No-Go task (non-game, points and theme)

• 287 participants (lab: 84, online: 203)

Signup through MTurk

Signup in person in

Bristol

Randomly assigned to

variant

Complete task(~ 10min)

Subjective questionnaire

delivered

Study 2 Overview(the attrition study)

• Three variants of a Stop Signal Task (non-game, points and theme)

• Longitudinal (10 days long)

Participants and Procedure(the attrition study)

• 291 participants total (97 in each variant)

• Reimbursed between £4 and £7

Signup through Prolific

Academic

Randomly assigned to

variant

4 consecutive days of test sessions

Reimbursed £4 after all completed

Day 1 Day 2 Day 3 Day 4

6 optional days of test sessions

Reimbursed £0.50 for each completed

Day 5 Day 6 Day 7 Day 8 Day 9 Day 10 Study ends

Cognitive Task Results(the cowboy study)

Go RT:points vs non-game (BF = .157)

No-Go Accuracy: points vs non-game (BF = .253)

Go Accuracy:points vs non-game (BF = .459)

Cognitive Task Results(the attrition study)

Questionnaire Results(the cowboy study)

Questionnaire Results(the attrition study)

Participant Attrition(the attrition study)

Key findings

• Points did not disrupt validity and increased participant enjoyment

• Mixed effects of Theme variant on data and engagement

• Study 2: No difference in participant attrition between task variants

Conclusions• Don’t waste time making graphics or complex games: just pointify

• Gamifying cognitive tests without invalidating the data is possible and can be effective for increasing engagement

• Further research needed to translate improved subjective ratings into objective engagement

• Further research needed to understand the effects of themed graphics

AcknowledgementsTobacco and Alcohol Research Group:Angela Attwood PostdocEmily Crowe PhD StudentKayleigh Easey PhD StudentMeg Fluharty PhD StudentTherese Freuler Research AssistantSuzi Gage PostdocMeryem Grabski PhD StudentGemma Hammerton Postdoc Eleanor Kennedy PhD StudentJasmine Khouja PhD StudentGlenda Lassi PostdocRebecca Lawn PhD StudentJim Lumsden PhD StudentOlivia Maynard PostdocAndy Skinner PostdocAlex Board Administrator Amy Taylor PostdocGemma Taylor PostdocChris Stone Research AssistantDavid Troy PhD StudentMiriam Cohen PhD StudentAndy Eastwood PhD Student

My supervisors:

Dr Jenny BarnettDr David CoyleDr Charlotte HousdenProf Natalia LawrenceProf Marcus Munafò

jim.lumsden@bristol.ac.uk

@jl9937

16

3 May 2023

Any questions?