Max LiebermanETCV 560: Educational Research MethodsSeptember 13, 2009Professor Glenn Kjos, Ph.D.

Literature Review on the Use of Video Games in Humanities Education

Introduction

Interest in the educational applications for video and computer games has risen

markedly in recent years, a trend that is clearly reflected in the quantity of scholarship on

the subject. A search of the Education Resources Information Database for the terms

“video game” and “education” returns a total of 59 documents published between 1990

and 1999. In comparison, 42 results are returned for the same terms in 2008 alone (Fig.

1). Despite this growing focus, a consensus on many fundamental questions is still to be

reached in this nascent field of study. Among these are whether researchers and educators

conducting real-world research have yet identied a set of best practices for using video

games as educational tools, and what measurable benefits video games offer when so

used.

Answering these questions is difficult. The

problem lies partly with approach of academics to

these questions, which has tended more toward the

theoretical than the practical, leaving us with far

more speculative literature than research. Just 16 of

the 42 documents published in 2008, for example,

contained the word “study” in addition to the other

search terms. Perhaps the biggest obstacle to a clear

understanding of how best to use games is precisely

the broad potential of games, which, like the written

word, can be used in myriad ways. 20002001

20022003

20042005

20062007

20080

10

20

30

40

50

516

9 515

2937

5042

Fig. 1: Number of Results Re-turned by a Search for the

Terms “Video Game” and “Ed-ucation” in the Education Re-

sources Information Database, by Year

With an eye to these challenges, this review will examine a portion of the

literature concerning the use of video games in a single context: humanities education,

namely fields such as English/writing/language arts, social studies and history. This

review will focus on research, supplemented with relevant secondary sources, and will

seek to identify a consensus on the best practices for selecting and implementing video

game technology in the context of humanities education, as well as for addressing widely

recognized challenges to such an implementation. This review will conclude with an

analysis of weaknesses in the literature covered, and with suggestions for future research.

Playing Games vs. Making Games

The literature on integrating computer and video games into humanities education

can be divided into two areas of focus: playing games and making games. Studies that

focus on playing games tend to supply arguments about why games are appropriate for

classroom use based on previous theoretical work in this area, such as that by Marc

Prensky, James Paul Gee and Kurt Squire, as well as on positive results from prior

research in the field (Kadakia, 2005; Whelchel, 2007; Charsky and Mims, 2008; Colby

and Colby, 2008). Studies that focus on making games also tend to refer to theoretical

work on the potential of games as educational tools (Robertson and Good, 2006;

Carbonaro et al, 2008; Owston et al, 2009). However, these studies additionally cite

constructivist pedagogical theory to support this approach, arguing both that

constructivism is a good model because “learners are more likely to generate new ideas

when they are actively engaged in developing some type of external artifact” and that

simply playing games is “instructionist” and therefore undesirable, because it reduces the

value of games to “extrinsic motivation for traditionally unpleasant subjects” (Robertson

and Good, 2006; Carbonaro et al, 2008).

Three studies focus on how students can learn by making games. Two are

substantially similar, focusing on the same educational goal (creating interactive

narratives), the similar populations (middle and high school-age students) and the same

game technology (Aurora, the game-building toolset packaged with Neverwinter Nights).

The first of the two studies is an account by Robertson and Good of their work with 10

children ages 12 through 15 over the course of a four-day workshop designed to test the

strengths of the Aurora technology as a tool for young people develop literacy skills by

telling “their own stories in the medium of computer games” (Robertson and Good,

2006). The results of the workshop are assessed using two sets of data: interviews with

the study participants, and the contents of the games themselves, which are evaluated

according to criteria such as available features implemented, number and complexity of

conversations written, and sophistication of written dialogue. Robertson and Good report

that students were highly motivated by the game creation task, and were able to create

complex interactive stories that reflected their personal interests, creativity and literary

sensibilities.

The second study using the Aurora toolset asks a slightly different question:

whether high school students can “successfully construct interactive game stories” using

the Aurora toolset alongside ScriptEase, a programming tool of the researchers’ own

design. ScriptEase is intended to simplify the process of “scripting,” or programming,

complex events in the game engine (Carbonaro et al, 2008). The students’ interactive

stories are evaluated alongside “traditional stories” written by the same students, using a

rubric that allows the researchers to rate the depth and quality of elements including

characters, setting and plot. Most students (73%) were capable of creating interactive

stories that receive a passing grade under this rubric. The researchers conclude that

factors including gender, game playing experience, creative ability and intellectual ability

did not significantly influence students’ success in writing interactive stories (Carbonaro

et al, 2008).

The third study on the impact of making games takes a much different approach.

Sixteen 4th grade classes were taught the same basic literacy, social studies, math and

science curriculum for an hour each week, with half of the classes also conducting an

hour of game development activity related to this curriculum (Owston et al, 2009. As

evaluated through teacher interviews and pre- and post- curriculum standardized tests,

students in this experimental group improved their ability to logically construct sentences

as well as their digital literacy and questioning skills, and retained content better (Owston

et al, 2009).

The two studies that concerned students playing games are far less quantitative in

their approach than those listed above, and could be accurately described as closely

observed pilot studies. The first is an account by Kadakia, a 7th grade Language Arts

teacher, of her experiment in using the computer role-playing game The Elder Scrolls III:

Morrowind (hereafter “Morrowind”) to help her students analyze choices and

consequences. In the case of this study, students did not themselves play the game, but

instead participated by choosing actions that the instructor then performed within the

game in front of the class. Kadakia reports, based on her notes and written assessments

from students, that engagement in classroom activities “escalated” across gender and

racial boundaries (Kadakia, 2005).

The second primary source concerning game play by a student is an account by

Ranker of his efforts to construct a writing curriculum for a single student which drew

both on Ranker’s experience with the traditional “craft approach to teaching writing” and

on his 8-year-old student’s interest in the video game Gauntlet Legends (Ranker, 2006).

This student is “resistant to school-based literacy,” but frequently draws maps and

characters based on his experiences in the action/role-playing game (Ranker, 2006). The

study focuses both on the student’s efforts to append written narrative to these drawings

and on Ranker’s attempts to understand video games as a narrative medium. Ranker is

concerned with finding a way to improve his student’s written creative output, but does

not want to “steer [him] away from working within the video game medium,” a “semiotic

domain” of which Ranker is admittedly ignorant (Ranker, 2006).

Several secondary sources also focus on the value of game playing in humanities

education. These are all concerned with proposing sets of best practices for using video

games, but explore different areas of humanities education. Whelchel discusses previous

research into the use of “civilization simulation” games in history classrooms, and offers

a set of guidelines and pedagogical approaches for instructors interested in adopting any

of three software products for this purpose (Whelchel, 2007). Colby and Colby knit

together decades of theory about the relationship between play and learning in order to

construct a concept of the “classroom as a gamespace” (Colby and Colby, 2008). By

blurring the lines between work and play, they argue, students can find in the Massively

Multiplayer Online Role-Playing Game (MMORPG) World of Warcraft inspiration for

“self-determined, rhetorically focused writing” and research projects which will build

creativity, basic literacy skills and advanced digital literacy skills (Colby and Colby,

2008). Finally, Charsky and Mims draw on a large body of research and theory to

propose a set of general best practices to accomplish the task that is the title of their

paper: “Integrating Commercial Off-the-Shelf Video Games into School Curriculums”

(Charsky and Mims, 2008).

Advantages of Games

The studies on both making and playing games agree on many points, including a

number of advantages that computer and video games hold as teaching tools within the

humanities. The research is perhaps most enthusiastic about the effect games had on

student engagement. With the exception of the Carbonaro study, which seems not to have

been interested in student attitudes, every research article mentions that video games

increased student interest in the subject at hand. Kadakia reports that “engagement was

higher amongst all students” (Kadakia, 2005). Participants in Robertson and Good’s

voluntary workshop sought to skip lunch in order to work on their projects (Robertson

and Good, 2006). Students in the experimental group of the Owston study sought to

pursue game development at home, and asked at least one teacher whether they could

apply the game development curriculum to other educational units, such as mathematics

(Owston et al, 2009). Several researchers speculated that bringing a popular medium like

video games into the classroom allows students to bridge their personal interests with

“school-valued” skills and content (Kadakia, 2005; Ranker, 2006; Robertson and Good,

2006).

Most studies also mentioned the power of games to build nontraditional literacy

skills, such as those related to visual and digital literacy. Several students told Kadakia

how they preferred viewing the game to discussing the same content with no visual aid

(Kadakia, 2005). Both studies which used Neverwinter Nights to investigate interactive

storytelling by students agree that such projects combine elements of computer science

and traditional writing courses (Robertson and Good, 2006; Carbonaro et al, 2008).

Owston emphasizes the importance of digital literacy skills, describing what researchers

perceive as a growing gap between traditional literacy as taught in schools and the “wide

variety of electronic media” that students interact with in countless other contexts

(Owston et al, 2009). The same researchers note that students in their experimental group

synthesized information gathered from such disparate sources as maps, videos, library

books, websites and teacher handouts (Owston et al, 2009). Colby and Colby, in their

proposal for a writing course based around World of Warcraft, implicitly acknowledge

the importance of digital literacy in explaining that their course will encourage students

to write in nontraditional formats and media such as “forms, blogs, websites and various

gamespace guides” (Colby and Colby, 2008). They further contend that factors such as

the statistical framework underlying the game mechanics and the history of academic

interest in the game community will help students learn about “textual evidence,”

“qualitative research,” “basic statistics, the importance of a large sample size, and the

basics of the research method” (Colby and Colby, 2008).

The idea that video game-related coursework allows students to apply non-writing

skills to the creation of narratives was noted as another advantage. These skills may

include drawings, photographs or sculptures, reflecting the importance of graphics to

computer and video games (Ranker, 2006; Robertson and Good, 2006).

However, traditional literacy skills may also improve when games are used.

Teachers whose classes created games in the Owston study reported that students were

careful to spell correctly and write proper sentences “because there was a tangible

outcome—the game—and an authentic audience—their classmates” (Owston, 2009).

Colby and Colby also note the importance of an audience for student writing work. In the

case of their proposed World of Warcraft curriculum, that potential audience extends well

beyond classmates to all players of the game worldwide—a number estimated to be well

north of 10 million as of late 2009 (Colby and Colby, 2008).

Challenges when Using Games

Clear patterns were also visible regarding the challenges that educators may face

when working to integrate games into a humanities curriculum. The primary difficulty

mentioned by several researchers—as well as authors of secondary sources—was the

time and effort required to learn the technology and mechanics of sophisticated games

and game creation tools. Robertson and Good reported that study participants not only

had trouble learning several aspects of the Aurora toolset, but often grew frustrated with

the technology upon discovering its limitations (Robertson and Good, 2006). Whelchel

cites Kurt Squire’s experience with the civilization simulation game Civilization III,

which took students between 15 and 20 hours to learn completely—and 20 hours of class

time is a commodity too dear to part with in many schools (Whelchel, 2007).

Anticipating this dilemma with the highly complex Morrowind, Kadakia circumvented

the problem in two ways. First, by playing the game herself, she removed the need for

students to learn the game’s controls; they needed only to recognize what they were

seeing, a much simpler task. She also used the game's save feature to good effect by

setting up teachable situations at home and saving the game in that state so that it could

be quickly loaded during limited class time (Kadakia, 2005). Charsky and Mims

specifically recommend saved games and teacher-guided scenarios as a solution to the

time and difficulty problem, although not all games are flexible in these regards (Charsky

and Mims, 2008). Ultimately, this problem may also self-correct to some extent. Students

in the Owston study smoothed out the technological learning curve of a web-based game

construction kit by freely sharing knowledge with each other (Owston et al, 2009).

  Interestingly, the other challenges mentioned by authors were more

hypothetical than actual. Several studies mentioned concerns about gender and class

disparities surrounding game content and experience with games (Ranker, 2006;

Robertson and Good, 2006; Carbonaro et al, 2008). However, the results of studies that

examined these potential differences showed that the effects of factors such as class,

gender, intellectual ability and immigrant status were either anecdotally or statistically

insignificant (Kadakia, 2005; Carbonaro et al, 2008; Owston et al, 2009).

The other oft-mentioned spectre was expected cultural opposition, within the

teacher and parent community, to the use of games. Ranker recognizes that his own

ignorance of video games hampered his ability to work with his student, but remains

unsure of “[w]hat weight should be given to new forms of narrative” like games (Ranker,

2006). Colby and Colby consider low cultural regard for games a key obstacle to

integrating computer games into the writing classroom, part and parcel with the hard and

fast demarcation of work from play in academia (Colby and Colby, 2008). They cite as

typical a quote from Edward C. Smith of American University, who said of video game

and educational theorist James Paul Gee that “If you’re going to replace traditional

methods of education with something new, you should replace it with something better. If

this guy [Gee] thinks that playing some goddamn video game is the equivalent of

memorizing a Shakespeare soliloquy, that’s crazy” (Colby and Colby, 2008). Whelchel is

also under the impression that some instructors "may object that... games have no place in

the classroom," but he fails to back this assertion with evidence (Whelchel, 2007).

Charsky and Mims encourage teachers to circumvent opposition to the use of games by

proactively seeking administrative support and the buy-in of parents and colleagues

(Charsky and Mims, 2008). They suggest explaining a single game's educational value

and outlining specific ways it will be used, as well as citing research on games in schools

—and, if available, evidence that similar games have worked well for other classes.

However, they also advise making alternative activities available in case some parents (or

even students) “remain uncomfortable with the idea” (Charsky and Mims, 2008).

Best Practices

While established guidelines for how to implement specific game software in

various classroom contexts are still some ways off, the literature reviewed does point

(directly and indirectly) to a set of more general best practices for games in schools. The

most important of these—the Golden Rule of gaming in the classroom—is that teachers

must themselves become experts in the game that their students will be using. For

Ranker, this manifested itself as a need to learn about not only Gauntlet Legends, but also

about the game genres it represented, and about the medium of games generally (Ranker,

2006). Kadakia, by contrast, was already an accomplished Morrowind player when she

decided to use the game in her classroom; in fact, it was her expertise with the game that

suggested it to her as a teaching aide (Kadakia, 2005). Teachers in the experimental

group of the Owston study participated in a workshop to learn the technology, and

actively shared knowledge amongst themselves during the study (Owston et al, 2009).

Colby and Colby simply assume that the instructor in their hypothetical World of

Warcraft writing course will be an expert on the game, as well as on new media

communication generally. Only a teacher with such content and technology expertise

could provide students with the “collaborative guidance” they identify as “vital” to the

success of the course (Colby and Colby, 2008). Whelchel states explicitly that educators

using civilization simulation games to teach history "will need to be quite familiar with

the software chosen, as they will be expected to answer student questions not only about

the basic mechanics of gameplay but also about the historical issues that underpin" the

games’ content (Whelchel, 2007). Similarly, “[m]astering the game” is the first step in

Charsky and Mims' list of best practices to integrate commercial games into classroom

teaching. They consider this a prerequisite for teachers to formulate a meaningful

curriculum that includes the game, and to provide support to students playing the game

(Charsky and Mims, 2008).

Equally clear, though slightly more difficult to define, is the sense in numerous

studies that gaming works best in humanities classrooms when accompanied by a

democratic, playful or otherwise informal classroom culture. More often than not, this

culture is suggested by anecdotes included by the researchers, rather than stated outright.

For example, participants in Robertson and Good’s study on Neverwinter Nights refused

to engage in a planned training session on creating dialoge, “as they were so engrossed in

the games creation task” (Robertson and Good, 2006). Kadakia describes spontaneous

questioning about Morrowind, and paints a picture of students jostling to stand near the

front of the class to be near the computer screen. One student jumped out of his seat to

exclaim that the game was “cool” (Kadakia, 2005). Ranker’s study would not have come

about at all had he not allowed his student to draw spontaneously during writing group

time, thus discovering the student’s outlet of creative expression based on his favorite

game (Ranker, 2006). Teachers in the Owston study found that students worked best in

self-directed teams of two or three which treated instructors as technical and content

resources rather than as taskmasters (Owston et al, 2009). The vision shared by Colby

and Colby of a course that blurs the lines between schoolwork and play, populated by

students who write “voluntary and self-directed” assignments, is far removed from the

autocratic classroom of old (Colby and Colby, 2009).

One final item that appears in multiple papers is the idea that there is a need to

distinguish content from technology, and to allow students time to learn both elements

contained within a given game. Kadakia accomplishes this by introducing the game to her

students the day before they are to begin using it, in order to ensure “that students would

not be completely distracted by the interface” later (Kadakia, 2005). For Robertson and

Good, operating within a framework where the content was to come from the students

themselves, stories were first written using analog techniques such as pen and paper

drawing, clay modeling and storyboarding, and only then adapted to the game engine

(Robertson and Good, 2006). Charsky and Mims go further, adapting an established

model of experiential learning (Kolb’s, specifically) to describe the ways that game

players first view the game as technology that must be learned, and only later become

able to conceive of its content in abstract terms (Charsky and Mims, 2008).

Weaknesses in the Literature

Despite the apparently high quality of both the quantitative and qualitative

research included in this review, some weaknesses in individual studies are apparent. One

obvious flaw concerns the selection of participants for the Robertson and Good

interactive storytelling study: The fact that participants self-selected to participate in an

extracurricular workshop strongly suggests that the group disproportionately represented

students interested in games and game creation to begin with, which puts their

enthusiasm for the task in a somewhat different light (Robertson and Good, 2006).

The other Neverwinter Nights-centered study is also flawed, in that it lacks a

certain amount of useful context provided by most of the other researchers. Carbonaro’s

group took such a quantitative tack in answering their research questions that they failed

to conduct any interviews with participating students. The study remains a valuable

addition to the literature because it covers ground parallel to Robertson and Good, and

specifically addresses one of the major shortcomings of the Aurora toolset. However,

nowhere do we see the effects of the study on students as people, or on the culture of the

classroom, or on the instructors teaching this new material. This gap is not only

unfortunate for us as readers; it also forces the authors to rely heavily on previous

theoretical scholarship, rather than direct observation, when outlining the strengths of

computer games as an educational tool.

Ranker’s pilot study, while fascinating in its nuanced observation of both teacher

and student, is problematic in that it is written by someone whose knowledge of computer

and video games is so limited that he is incapable of defining elements common to video

game-based storytelling—which one would think would be signicant in a study about

how to help students tell stories based on games.

The results of the Owston study, which seem to show that creating games

alongside a traditionally-structured curriculum allowed 4th-graders to learn more, are

thrown into question by a fundamental issue with the structure of the research: students in

the experimental group spent twice as much time on the study’s “Tracks Across Canada”

curriculum as their control counterparts (Owston et al, 2009). This raises the question of

how a control group which spent equal time on the material would have done. There was

also considerable variance in the game development output of the experimental classes in

the Owston study. Furthermore, one of the nine schools in the study dropped out prior to

completion due to concerns about time management and teacher workload (Owston et al,

2009). This suggests that factors including school and classroom culture, teaching style,

level of teacher comfort with technology and likely others can have a dramatic impact on

the value of a game development curriculum for students, and on its viability for schools.

None of these questions are addressed by the researchers.

Another issue with the Owston study concerns the discrepancy between the

results as perceived by the teachers in the experimental group and the results as

evidenced by the data collected from pre- and post-study standardized tests. If, as the

researchers demonstrate, the experimental group showed “significant differences” from

the control group on only one test (Logical Sentences), then why did teachers in that

group describe game development activities as directly impacting areas such as “content

retention, comparing and contrasting information presented, utilizing a variety of research

materials, editing, and higher level question creation” in a positive way (Owston et al,

2009)? One possibility is that several of these skills were not reflected on the tests

themselves, which focused instead on vocabulary, sentence construction and passage

cognition (Owston et al, 2009). Other possibilities suggested by the researchers

themselves are that time constraints prevented full implementation of the technology

curriculum in some schools, and that the focus was on production of complete games

rather than on the creation by students of questions that tapped higher-order thinking

(Owston et al, 2009).

Suggestions for Future Research

In addition to these weaknesses in specific studies, some notable gaps exist in the

literature on the use of games for humanities education. Suggestions for future research

may be useful at this juncture.

Given that a tentative set of best practices for teaching with games has been

established, comparative studies that isolate specific practices (“learn the game well,”

“create an informal classroom culture”) as independent variables would provide some

much needed hard evidence that these practices truly make a difference. A study that

compared a Colby-style World of Warcraft English class against a more traditional class

examining the game as a cultural artifact using a Carbonaro-like evaluative rubric would

be a tremendous addition to the existing body of research.

Studies focusing on a broader variety of games than the existing literature are also

desirable. As Whelchel’s review makes clear, numerous researchers have focused their

efforts on teaching history through the same few, very similar, game series (Whelchel,

2007). There are dozens of World War II shooter games on the market, some of which

are quite extensively researched. Why not incorporate one or two into established WWII-

era history curricula?

The number of studies focused on the use of games to help students become better

writers is surprising. The comparative lack of focus on asking students to analyze and

critique the storytelling techniques of commercial games is perhaps less surprising, given

the poor quality of much game writing, but it is disappointing nevertheless. Despite the

medium’s reputation, games with well-told stories—and even games that break new

ground in this area—do exist. To offer just one example, Valve Software’s Half-Life 2

could sit alongside 1984 in a literature course on narratives of dystopia. A study

evaluating similar courses, one with this text added to the curriculum and one without,

could demonstrate whether such an inclusion improved students’ digital literacy skills—

or even basic literacy skills, by increasing their understanding of the storytelling

techniques inherent to disparate forms of media.

The explosion of interest in using video games as a tool within humanities

education is encouraging for those who see potential in this unique technology. Good

work is being done by researchers as well as more theoretical authors, and the literature

reviewed displays that ideas are moving back and forth between these groups. Only

through the continued osmosis of ideas between theory and practice can we hope to see

video games establish a meaningful and lasting place in the classroom of the future.

Bibliography

Carbonaro, M., Cutumisu, M., Duff, H., Gillis, S., Onuczko, C., Siegel, J., et al. (2008). Interactive story authoring: A viable form of creative expression for the classroom. Computers & Education. 51 (2), 687-707.

Charsky, D., & Mims, C. (2008). Integrating commercial off-the-shelf video games into school curriculums. TechTrends: Linking Research and Practice to Improve Learning. 52 (5), 38-44.

Colby, R. S., & Colby, R. (2008). A Pedagogy of Play: Integrating computer games into the writing classroom. Computers and Composition. 25 (3), 300-312.

Kadakia, M. (2005). Increasing student engagement by using morrowind to analyze choices and consequences. TechTrends: Linking Research & Practice to Improve Learning. 49 (5), 29-32.

Owston, R., Wideman, H., Ronda, N. S., Brown, C. (2009). Computer game development as a literacy activity. Computers & Education. 53 (3), 977-989.

Ranker, J. (2006). “There’s fire magic, electric magic, ice magic or poison magic”: the world of video games and Adrian’s compositions about Gauntlet Legends. Language Arts. 84 (1), 21-33.

Robertson, J., & Good, J. (2005). Children's narrative development through computer game authoring. TechTrends: Linking Research & Practice to Improve Learning. 49 (5), 43-59.

Whelchel, A. (2007). Using civilization simulation video games in the world history classroom. World History Connected. 4 (2).