mona erfani, beth lameman, hamid maygoil teahyeong...

TeaHyeong Kim

2015-07-21

Korea University

Computer Graphics Lab.

Magy Seif El-Nasr, Bardia Aghabeigi, David MilamMona Erfani, Beth Lameman, Hamid MaygoilSang Mah

CHI 2010: Games and Players

Korea UniversityComputer Graphics Lab.

TeaHyeong Kim | 2015-07-21 | # 2KUCG |

ABSTRACT

• Cooperative design has been an integral part of many games

Many game designers and producers are currently exploring • The addition of cooperative patterns within their games

Very little research • Investigated cooperative patterns

• Methods to evaluate them

We propose Cooperative Performance Metrics (CPM)

INTRODUCTION



INTRODUCTION

• Cooperative games encourage

Participation and collaboration• The goal is not to win as a player but as a team of players

• Discovering effective cooperative game patterns

An elusive and important problem



INTRODUCTION

• Results of our background questionnaire

With 60 6-16 year olds revealed that kids • When asked to choose between cooperative and

competitive games ‗ 55% of them preferred cooperative games

‗ While 77% stated that they would like to play games with both options

∙ The industry realizes this

∙ In the past year alone, several AAA titles, such as Resident Evil 5 (Capcom, 2008) and Left4Dead (Valve, 2008), included an optional cooperative mode



INTRODUCTION

• Cooperative design patterns

Very few research studies discussed or documented

Methods for evaluating them are also in their infancy • User testing groups within game companies evaluate

cooperative games ‗ Using the same methods used to evaluate single player games

Therefore, there is a need a. Understanding current successful cooperative patterns

b. Creating methods to evaluate their effectiveness



INTRODUCTION

• This paper aims to address these issues

within the context of cooperative video games by discussing three contributions1. First, we present a set of cooperative design patterns

‗ We outline a set of Cooperative Performance Metrics (CPMs)

2. Connections between the CPMs and the cooperative design patterns

3. The CPMs themselves constitute



CONTENTS

2. PREVIOUS WORK

Detail the cooperative design patterns

3. IDENTIFIED COOPERATIVE PATTERNS

Explain the CPMs

4. STUDY DESIGN

Outline the study

5. RESULTS

6. CONCLUSION

Discussing the findings and their implications

PREVIOUS WORK



PREVIOUS WORK

• Related works fall within two areas

1. Video games evaluation methods

2. Cooperative design



PREVIOUS WORK

• Methods for Game User Studies

Usability testing is an integral part • Game evaluation methods integrate many of these HCI

methods‗ Extend them to include testing for playability and engagement

∙ [B. Fulton, “Beyond psychological theory: getting data that improve games”, Game Developers Conference 2002]

∙ [K. Isbister and N. Schaffer, “Game Usability”, 2008]



PREVIOUS WORK


Usability testing is an integral part • The game industry realizes importance of developing and

conducting game evaluations; ‗ This is evident by the formation of groups

∙ Such as Microsoft’s User Experience group, Sony’s usability and playtest, Ubisoft’s playtest, and Eidos’ user-research groups

‗ The emergence of several user research companies

∙ Such as Emsense and XEODesign



PREVIOUS WORK


Microsoft developed • Online tracking system called TRUE

‗ Collect and visualize gameplay telemetry data and synchronize them with attitudinal and observational data

∙ Enabled them to “detect issues and understand root causes in the same way usability testing does.”

‗ They validated their system using two games:

∙ Halo 2 and Shadowrun



PREVIOUS WORK


Dracken et al. in cooperation with Eidos interactive developed • Set of metrics to track user behavior in Tomb Raider

‗ Used Geographic Information Systems to visualize spatial gameplay metrics

‗ Developing the Heat Map

∙ Which enabled them to detect level design problems

∙ Such as places where a lot of players died

‗ Similar methods are used by Valve, Bungie and Electronic Arts



PREVIOUS WORK


A few studies concentrated on defining methods • Evaluating engagement or enjoyment in games

‗ Sweetser and Wyeth developed

∙ Concentration, challenge, skills, control, clear goals, feedback, immersion, and social

∙ They validated the model by evaluating two commercial games, and comparing their results to that of expert reviews



PREVIOUS WORK


A few studies concentrated on defining methods • Evaluating engagement or enjoyment in games

‗ Yannakakis and Hallam explored

∙ Development of a quantitative experimental model specifically targeting simple arcade and augmented reality games

∙ They concentrated on challenge as a main aspect of engagement



PREVIOUS WORK


Lazarro et al. • Ran a large study

‗ With 45 participants

‗ With various gaming experience

‗ Who were asked to play 40+ games of different game genres

∙ Including racing, fighting, puzzle solving, and sports



PREVIOUS WORK


Lazarro et al. • Ran a large study

‗ Used observation notes, videotaped interaction, and questionnaires/interviews with friends and family

‗ Based on their study they identified four kinds of fun:

∙ (1) hard fun: motivated by achievement

∙ (2) Easy Fun: motivated by exploration

∙ (3) Altered states: motivated by visceral rewards

∙ (4) Social: motivated by competitive or cooperative play or just to be with friends.

‗ Even though the sample was small and included many games, their work contributed data showing variations in play styles and motivations



PREVIOUS WORK


An alternative approach to playtesting and usability studies • Use game heuristics evaluation

‗ This method is based on the usability heuristic technique developed by Nielsen

∙ [J. Nielsen and R. Molich, Heuristic evaluation of UI, CHI’90]

∙ It does not provide attitudinal, behavior, or play data from actual users



PREVIOUS WORK

• Methods for Game User Studies While previous works present excellent research

• Only work we found targeting this area was Pinelle et al.’s work

‗ While their work and criteria is close to ours

‗ Developed methods to evaluate

∙ Networked games rather than games where participants share the same physical space

• We specifically propose a set of validated CPMs For analyzing and evaluating cooperative play

occurring over the network or within the same space



PREVIOUS WORK

• Cooperative Game Design Patterns Some researchers analyzed a set of cooperative

games to develop cooperative game design patterns• Zagal et al.

‗ Explored cooperative patterns within board games

• Bjork and Holopainen‗ Presented a large number of game design patterns

∙ Which included cooperative and social interaction patterns

• Zagal et al. ‗ Presented an ontology for analyzing game play

• Rocha et al.‗ Presented a framework of several cooperative game design patterns



PREVIOUS WORK

• Cooperative Game Design Patterns

Some researchers analyzed a set of cooperative games to develop cooperative game design patterns• In this paper, we follow in the footsteps of these efforts by

extending Rocha et al.’s model ‗ We chose to extend Rocha et al.’s work,

∙ Because it uses recently published cooperative games

∙ Thus the design patterns and choices are more up to date



PREVIOUS WORK


Rocha et al. identified six cooperative game design patterns• Complementarity: is one of the most commonly used

patterns in co-operative games. It implies that players play different character roles to complement each others’ activities within the game.



PREVIOUS WORK


Rocha et al. identified six cooperative game design patterns• Synergies between abilities: allows one character type to

assist or change the abilities of another. For example, in World of Warcraft (Blizzard, 2001), a Shadow Priest can cause an enemy to become vulnerable to shadow damage, which also results in an increase in the damage that Warlocks (another character type) can cause.



PREVIOUS WORK


Rocha et al. identified six cooperative game design patterns• Abilities that can only be used on another player: an

example can be seen in Team Fortress 2 (Valve, 2007), where Medics can heal other players.



PREVIOUS WORK


Rocha et al. identified six cooperative game design patterns• Shared goals: is a pattern used to force players to work

together, such as in World of Warcraft, where a group of players are given a single quest with a shared goal.



PREVIOUS WORK


Rocha et al. identified six cooperative game design patterns• Synergies between goals: is a pattern that forces players to

co-operate together through synchronized goals. For example, the achievement system developed for the Pyro and Medic character classes within Team Fortress 2 gives Pyros the goal of killing three enemies while ubercharged(being made invulnerable by a Medic). The Medic, on the other hand, has a different goal, which is to ubercharge a Pyro while he/she burns enemies.



PREVIOUS WORK


Rocha et al. identified six cooperative game design patterns• Special rules: denote rules that are used to enforce

cooperation within teams. For example, designers can encode rules to denote specific effects to actions within the game when performed on a friendly player. The idea behind these differences is to promote and facilitate cooperation. A good example is the rule in FPS (First Person Shooter) games that prevents damage when players accidently shoot other players on the same team, known as Friendly Fire modes.

IDENTIFIED COOPERATIVE PATTERNS




• We selected fourteen games Deeper analysis that included cooperative modes

• Left4Dead (Valve, 2008)

• Resident Evil 5 (Capcom, 2009)

• Beautiful Katamari (Namco Bandai)

• Kameo, Lego Star Wars, Wall-E (THQ, 2008)

• Cloning Clyde (Microsoft, 2006)

• Guitar Hero III (Activision, 2007)

• Harry Potter and the Goblet of Fire (Electronic Arts, 2005)

• Kung Fu Panda (Activision, 2008)

• Little Big Planet, Boom Blox (Electronic Arts, 2008)

• Mario Galaxy (Nintendo, 2007)

• Army of Two (Electronic Arts, 2008)




• The analysis process took

Two months to complete

• Two researchers analyzed each game in detail

Using game design theory and previous work on cooperative game design• Identified distinct design techniques, including resource

sharing, controls (user interface), shared goals and puzzles, and reward structures

• Noted visual design characteristics, such as camera settings




• For validation, the patterns were reviewed

By an independent researcher• Who has over 10 years of game industry experience

After his approval• We asked a team of two independent researchers

‗ To play all the identified games and develop their own cooperative game design patterns

We can report a very high agreement• As researchers identified the same patterns

• But have used different terms to denote some of them

• At the end of this process, researchers met and discussed the patterns




• All fourteen games reviewed in our study

Designed for kids to play together • Through a shared screen

‗ In these situations, camera set up emerged as an important design component

• We identify the following additional patterns





Camera Setting• Three design choices for developing a successful camera in

a shared screen co-op games‗ Split screen horizontally or vertically

‗ One character in focus

‗ All characters are in focus (the screen doesn’t move unless all characters are near each other)





Interacting with the same object• Providing interactive objects that can be manipulated by

characters’ abilities‗ In Beautiful Katamari, players share a ball

‗ Similarly, in Little Big Planet, both players can push or grab one object together





Similar to shared goals is Shared Puzzles• This pattern was observed in games such as Lego Star

Wars and Little Big Planet‗ Where both players encounter a shared challenge or obstacle





Special characters targeting lone wolf• This pattern focuses on the design of NPC characters

‗ That target players who are working alone

‗ In Left4Dead, the Hunter and Smoker are good examples of this pattern





Vocalization• Patterns that embed automatic vocal expressions on player

characters ‗ That alert players of different challenging events

‗ Encourages players to play close together and support each other





Limited resources• Concerned with providing a limited number of resources

• Thus encourages players to share or exchange resources to research the same goal

• Resident Evil 5 uses this technique‗ Many examples of this pattern can be seen in board games

STUDY DESIGN



STUDY DESIGN

• We ran a study with a total of 60 participants:

18 females (average age=9.81)

42 male (average age=10.4)

In a total of 25 sessions



STUDY DESIGN

1. Recruit

2. invited participants to come in groups of 2-4 participants

3. First interview (background, habits, gaming experience)

4. Play four games in 10 minute-sessions



STUDY DESIGN

• The selected games

Rock Band 2 (RB)

Lego Star Wars (LSW)

Kameo (K)

Little Big Planet (LBP)

• After each play session

Participants were interviewed individually

• For further analysis

Videotaped all the play sessions front and back



STUDY DESIGN

• Defined several metrics:

Cooperative Performance Metrics (CPMs)• These metrics are associated with observable events within

a play session

• Thus can be used as a basis for video annotation or structured observation of a cooperative play session



STUDY DESIGN

• Created CPMs

Through an iterative process involving expert and team reviews

1. First initial set of metrics was defined based on several play sessions• Researchers played cooperative games and others

observed

2. These metrics were then reviewed and revised• By the team of five researchers involved in this study



STUDY DESIGN

• Created CPMs

3. Metrics were then used to observe and annotate two pilot cooperative play sessions

4. Metrics were also sent in parallel to three industry game designers working at Electronic Arts and Square Enix• Based on their feedback and the results observed from the

two pilot sessions, we revised the metrics

5. Discussed the metrics and approved the final set



STUDY DESIGN

• Final set of CPMs

Laughter or excitement together

Worked out strategies

Helping

Global Strategies

Waited for each other

Got in each others’ way



STUDY DESIGN

• Laughter or excitement together

Laughed at the same time due to a specific game event

Expressed verbally that they are enjoying the game, looking for utterances, such as “sweet”, “it is a lot of fun”, etc.

Shook their heads and showed facial nonverbal behaviors that clearly expressed happiness or excitement



STUDY DESIGN

• Worked out strategies

Talked aloud about solving a shared challenge

Divide a game zone to different parts in order to divide and conquer

Navigated the world while consulting with each other



STUDY DESIGN

• Helping

Talked about controllers, and how one can use the game mechanics

Told each other the correct way of passing a shared obstacle

Saved and rescued the other player while he or she was failing



STUDY DESIGN

• Global Strategies

Players take different roles during gameplay that complement each others’ responsibilities and abilities

• Waited for each other

One player waits for the other to catch up



STUDY DESIGN

• Got in each others’ way

Player leads and the other lags behind

When one player wants to do an action, x, and the other wants to take a different action, y, and whereby taking these actions they will inevitably interfere or hinder each other’s goals

RESULTS



RESULTS

• Used the CPMs to annotate all game play sessions

3000 minutes of video data were reviewed and annotated• One researcher labelled each CPM occurrence

• In this section

We discuss the totals, averages, standard deviation, 95% confidence intervals for all CPMs per game

We also discuss paired t-tests evaluating statistical significance of the results



RESULTS

• For each CPM label within the video analysis

Researcher identified a cause based on the cooperative design patterns, specifically: • Complementarity

• Synergies between abilities

• Shared goals

• Synergies between goals

• Special rules

• Camera styles



RESULTS

• For each CPM label within the video analysis

Researcher identified a cause based on the cooperative design patterns, specifically: • Interacting with the Same Object (ISO)

• Shared Puzzle (SP)

• Shared Character (SC)

• Miscellaneous (PM)‗ Category that includes animations, cut scenes, or special elements

that are specific to one game



RESULTS

• To establish scientific validity

Performed a formal validation process1. Asked two independent researchers to rate two sessions

given the CPMs and the cooperative patterns identified‗ Kameo and Lego Star Wars to analyze

2. Performed inter-rater agreement and calculated kappa values



RESULTS

• Laughter and Excitement Together Events

T-test results were• RB-LSW (extremely significant, sig = .0001), RB-K (not

significant, sig=.9), RB-LBP (significant, sig=.0014), LSW-K (extremely significant, sig=.0003), LSW-LBP (significant, sig=.018), and K-LBP (significant, sig=.009)



RESULTS

• Laughter and Excitement Together Events



RESULTS

• Worked Out Strategies

T-test results were• RB-LSW (extremely significant, sig = .0001), RB-K (extremely

significant, sig=.0001), RB-LBP (extremely significant, sig=.0001), LSW-K (extremly significant, sig=.0001), LSW-LBP (extremely significant, sig=.0001), and K-LBP (not significant, sig=.77)



RESULTS

• Worked Out Strategies



RESULTS

• Helping

T-test results were: • RB-LSW (extremely significant, sig = .0001), RB-K (extremely

significant, sig=.0001), RB-LBP (extremely significant, sig=.0007), LSW-K (very significant, sig=.008), LSW-LBP (significant, sig=.034), and K-LBP (extremely significant, sig=.0001)



RESULTS

• Helping



RESULTS


T-test results were:• RB-LSW (very significant, sig = .017), RB-K (very significant,

sig=.002), RB-LBP (not quite significant, sig=.118), LSW-K (not significant, sig=.246), LSW-LBP (extremely significant, sig=0.0001), and K-LBP (extremely significant, sig=.0001)



RESULTS




RESULTS

• Waited for Each Other

T-test results were: • RB-LSW (extremely significant, sig = .0001), RB-K

(extremely significant, sig=.0001), RB-LBP (significant, sig=.031), LSW-K (not significant, sig=.683), LSW-LBP (very significant, sig=.002), and K-LBP (very significant, sig=.013)



RESULTS

• Waited for Each Other



RESULTS

• Got in Each Others’ Way

T-test results were: • RB-LSW (significant, sig = .034), RB-K (not significant,

sig=.5), RB-LBP (not significant, sig=.5), LSW-K (not significant, sig=.2), LSW-LBP (not significant, sig=.14), and K-LBP (not significant, sig=1)



RESULTS

• Got in Each Others’ Way

CONCLUSION



CONCLUSION

• Global Strategies CPM

Lead due to their use of shared goals, shared puzzles, and complementarity cooperative patterns

• Split screen and camera

Caused Waited for Each Other and Got in Each Others’ Way CPMs



CONCLUSION

• Visual style and animation as well as cut scenes

Caused much of the Laughter and Excitement Together

• Helping CPMs

Occurred when the game was difficult for players



CONCLUSION

• Designing effective cooperative patterns is an important area for the game industry

But still an untapped research area

• We presented several contributions

1. Proposed several cooperative game design patterns

• Extending previous work

2. Proposed a set of Cooperative Performance Metrics

• Used for analysis of the cooperative games

3. Presented results of a study analyzing the experience

• Valuable design lessons



CONCLUSION

• In future research

Extend this work by additional experiments• Different age groups

• Different game types

mona erfani, beth lameman, hamid maygoil teahyeong...

Documents