Playtesting in Video Game Development

DMSP Indie Games Team 1 – Luci Holland

In the development of our indie game Rimo, constant playtesting1 was an essential part of the

process in managing and testing the combination of various assets (environmental and character

design, sound effects, music and programming) and debugging the game to polish and enhance the

overall product. In one DMSP class (20th March 2014) Martin Parker2 made a comment which really

stuck with me: he commented that games need a player to exist, to fulfil their purpose. Without a

person actually loading up the game and interacting with it, the game is a closed world, a collection

of pixels, sounds waiting to be triggered. A video game, no matter how complete and finite, needs

the interaction of a player and the parameter of time to properly exist.

In his talk at GDC3, Naughty Dog Lead Game Designer Richard Lemarchand4 discussed what he

considered integral psychological terms in game playing. He identifies three categories in holding a

player's attention: Beauty (aesthetics, contrast), Story (narrative, social) and Gameplay (ludism,

systems)5. I believe it is also the player's direct involvement with games which is the most

1 Playtesting can refer to game developers testing the progress of a game internally within the development team (not Quality Assurance (QA), in which professional QA testers look for and report specific software bugs to be collated, relayed and then fixed by the development team), but more specifically refers to “the process of exposing a game in development (or some specific parts of it) to its intended audience, to identify potential design flaws and gather feedback.” http://en.wikipedia.org/wiki/Playtest

2 Dr Martin Parker, Course Organiser, Digital Media Studio Project 2013 – 2014 at the University of Edinburgh3 Game Developers Conference, San Francisco, CA 20124 Attention, Not Immersion: Making Your Games Better with Psychology and Playtesting, the Uncharted Way Game

Developers Conference 2012 Session5 R. Lemarchand, Attention, Not Immersion p.70

A playtester takes on 'Rimo' - photo by Luci Holland 20/03/14

important factor of attention. Karen Collins identifies this as the “illusion of control” which leads to

engagement (in relation to video game audio) in “Implications of Interactivity: What Does It Mean

For Sound To Be “Interactive”?”, again defining just how important the role of the player is in a

video game. Collins argues that the definition of interactivity in video games requires the fulfilment

of two elements - control and feedback (control the user or player, feedback the response from the

system, or game)6. Collins discusses how our experience may be more immersive when perceiving

a sense of control (enhanced by interactive audio) as we feel more established within its world7.

Lemarchand's analysis of his three integral categories relates to this immersion8. These analyses are

crucial to understanding the importance and involvement of the player/s in a video game, which is

constantly considered in playtesting.

Playtesting is a crucial and surprisingly complex feature of game development which spans a

range of aspects and variables such as recruiting testers, controlled and alternate tests, data

collection techniques and detailed analysis. The game designers will constantly test the game as

they go, tweaking level designs and programming to better construct the product. In this scenario

the designer already knows what to expect and how the game should operate. However when it

comes to releasing a game, the designers need to know that the game makes sense and plays

smoothly to someone who hasn't been involved with the development – the player. This is the

purpose of a playtest, for which preparation pre-testing and analysis post-testing is integral to

ensuring effective and accurate data collection.

At the before mentioned DMSP class we were able to have an impromptu beta test with some of

our classmates who hadn't yet seen the game. Often during commercial game developments,

depending on the nature and stage of the development, the game itself may yet be missing core

elements or be substantially unfinished. In our case elements that we hadn't considered important or

worthwhile mentioning soon became obviously crucial – for example, we had pink placeholder

marimos in the game which had yet to be animated. However, to the external player who didn't

know that they were placeholders, they assumed the pink marimos to be part of the games visual

environment. We watched players find the last challenge especially difficult, which was extremely

helpful for us in furthering our design of the level. We avoided giving direct feedback to the players

on which direction to head in, however crucial assets were missing (such as sonic cues which

6 K. Collins, “Implications of Interactivity: What Does It Mean For Sound To Be “Interactive”?” in The Oxford Handbook of New Audiovisual Aesthetics p.574

7 K. Collins, “Implications of Interactivity: What Does It Mean For Sound To Be “Interactive”?” in The Oxford Handbook of New Audiovisual Aesthetics p.581

8 In association these categories are also the basis for why he prefers to avoid the terms 'immersion' and 'engagement'. R. Lemarchand, Attention, Not Immersion p.11

indicated correct paths). The game was still in a state of development and the results were

enlightening and helpful in furthering our design. Please see the short video clip included, where

both the developers and new playtesters try Rimo and give feedback to our team on elements such

as controls and camera movement9.

This playtest and our official Team 1 Presentation/Playtest on the 3 rd of April were both of a more

informal nature, in that the playtesters themselves were not in a specifically controlled environment,

and members of our team were on hand to offer advice when requested. Lemarchand discusses

more formal playtesting for commercial game releases, in which playtesters are selected (sometimes

even interviewed) from the general public who have never seen the game before, in a carefully

controlled environment10. The gameplay is screen captured using a DVR box, and information from

the gameplay is electronically stored and collated, which Naughty Dog refer to as 'metric data'.

They also employed clever programming systems to record and visually highlight problem areas

and elements in gameplay11. After the playtest, players were asked to fill in questionnaires on a

range of game assets and elements. The Rimo team also used a short feedback form to gather

information on the game, more centred on the players' reception to the game than anything else.

This is a crucial aspect for our development in terms of taking the game further – but of course we

were operating in a more unique process over a shorter length of time. In Game Design Workshop:

A Playcentric Approach to Creating Innovative Games Tracy Fullerton provides a detailed example

of questions a developer might want to explore with playtesters.12 In contrast to our short and

informal playtests, for Uncharted 3, a professional commercial release, there were twenty-one tests

over a period of six months in the last stage of development13.

Not only is it clear that playtesting is a crucial part of game development, it is also apparent that it

is a complex process which can be unique to particular video games and developments. It includes

innovation in data collation and analysis, controlled situations and tester selection, and can vary

greatly in terms of length and frequency depending on the subject. It can be a surprisingly complex

but critical element of game development which not only brings purpose to the game through

9 Video from impromptu DMSP Indie Games Team 1 playtest at DMSP class on the 20th March 201410 For example, the playtesters are separated from each other by screens, the developers offer no help with gameplay,

and they request that the players avoid talking to each other while playing (attempting to play the game with no external feedback or suggestions). R. Lemarchand, Attention, Not Immersion p.88

11 During development of Uncharted 3, one playtest tracking method Naughty Dog used was to record coordinates to their metrics database whenever a playtester hit the jump button but failed to progress to a climbable ledge (which would have allowed them to continue in the game narrative). These coordinates were exported back into the game after the playtest and marked with a red sphere which highlighted issues in gameplay and enabled a quicker and smoother debugging and development process. They named this their 'bad jumps' system. R. Lemarchand, Attention, Not Immersion p.96

12 T. Fullerton, Game Design Workshop p.26313 R. Lemarchand, Attention, Not Immersion p.87

interaction with the player, but allows the game itself to materialise and bring about its realisation.

Word count: 1044

Sources

Collins, Karen, Chapter 32 “Implications of Interactivity: What Does It Mean For Sound To Be

“Interactive”?” in The Oxford Handbook of New Audiovisual Aesthetics ed. John Richardson,

Claudia Gorbman and Carol Vernallis. New York: Oxford University Press 2013

Fullerton, Tracy. Game Design Workshop: A Playcentric Approach to Creating Innovative Games

Elsevier Inc. 2008

Lemarchand, Richard. Attention, Not Immersion: Making Your Games Better with Psychology and

Playtesting, the Uncharted Way Game Developers Conference 2012

http://gdcvault.com/play/1015464/Attention-Not-Immersion-Making-Your

(accessed 30/03/14)

Vin St. John. Gamasutra Best Practices: Five Tips for Better Playtesting

http://www.gamasutra.com/view/feature/185258/best_practices_five_tips_for_.php?print=1

(accessed 09/04/14)

http://en.wikipedia.org/wiki/Playtest

(accessed 11/04/14)