realistic npcs in video games using different ai...

Realistic NPCs in Video GamesUsing Different AI ApproachesBachelor of Science Thesis in Computer Science and Engineering

Gustav Grund PihlgrenMartin NilssonMikael LarssonOskar OlssonTobias FoughmanVictor Gustafsson

Department of Computer Science and EngineeringCHALMERS UNIVERSITY OF TECHNOLOGYGöteborg, Sweden, June 2016

The Authors grants to Chalmers University of Technology the non-exclusive right topublish the Work electronically and in a non-commercial purpose make it accessibleon the Internet. The Authors warrants that they are the authors to the Work,and warrants that the Work does not contain text, pictures or other material thatviolates copyright law.

The Authors shall, when transferring the rights of the Work to a third party (forexample a publisher or a company), acknowledge the third party about this agree-ment. If the Authors has signed a copyright agreement with a third party regardingthe Work, the Authors warrants hereby that they have obtained any necessary per-mission from this third party to let Chalmers University of Technology store theWork electronically and make it accessible on the Internet.

Realistic NPCs in Video Games Using Different AI Approaches

Gustav Grund PihlgrenMartin NilssonMikael LarssonOskar OlssonTobias FoughmanVictor Gustafsson

© Gustav Grund Pihlgren, June 2016© Martin Nilsson, June 2016© Mikael Larsson, June 2016© Oskar Olsson, June 2016© Tobias Foughman, June 2016© Victor Gustafsson, June 2016

Examiner: Niklas Broberg

Chalmers University of TechnologyDepartment of Computer Science and EngineeringSE-412 96 GöteborgSwedenTelephone + 46 (0)31-772 1000

[Cover: An image from the video game created during this project that has beentransmuted using Deep Dream Generator [1], a website that uses neural networks totransmute images. This image along all other images from the game is made fromseveral other images. Some of these images are made by the project members andthe others can be found correctly accredited in Appendix A]

Department of Computer Science and EngineeringGöteborg, Sweden, June 2016

Abstract

This bachelor thesis describes the process of creating an AI for realistic NPCs. Itcompares different approaches used to implement AI in order to try to convey theiradvantages and disadvantages to contribute towards the creation of more immersiveNPCs in video games.

The work was divided into two main categories: The first to create a video gamein which to implement NPCs and the other to create an AI to control these NPCs.A variety of different implementations were investigated during the course of thisproject and are discussed in this report, and a game was created to test one of them.

Sammandrag

Denna kandidat rapport beskriver processen av att skapa en AI för realistiskaNPCer. Den jämför olika tekniker som används för att implementera AI för attförsöka förmedla deras fördelar och nackdelar för att kontribuera till skapandet avmer inlevelsefulla NPCer i datorspel.

Arbetet var indelat i två huvudkategorier: Den första var att utveckla ett datorspelför att utveckla NPCer i, och den andra var att utveckla en AI för att kontrolleradessa NPCer. En mängd olika implementationer undersöktes under projektet ochdiskuteras i denna rapport, och ett spel utvecklades för att testa en av dessa.

Keywords: AI, programming, NPC, videogame, simulation, realistic.

iii

Dictionary

Character - An object without intelligence that may be controlled by an intelligentsource (i.e. an AI or a player).Artificial Intelligence (AI) - Intelligent control unit for the characters.Non-player character (NPC) - A character which is controlled by the AI.World - The complete collection of items, characters, structures and environmentthat makes up a video game.Model-view-controller (MVC) - A software design pattern used to separate un-derlying data and the visual presentation of that data from each other.Tileset - A collection of images and textures in fixed sizes that are used to modelthe game world, merged into the same image, side by side in a grid.Java - An object-oriented, multi-platform programming language developed by SunMicrosystems, specifically designed to let a program run unaltered on many differentdevices and operating systems.Slick2D - Slick2D is a set of tools and utilities for handling 2D graphics, as well asother things such as music and particles.Sprite - An image or animation in a video game.

v

Preface

This report is a bachelor thesis at the Department of Computer Science and En-gineering, Chalmers University of Technology, performed in the spring of 2016. Itdescribes and discusses the planning and implementation of a realistic AI to be usedin a resource based simulation game.

We want to thank: the institution for technical language, the Chalmers library andDaniel Sjölie, our supervisor.

vi

Contents

1 Introduction 11.1 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.2 Purpose . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31.3 Scope . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

2 Technical Background 52.1 Slick2D . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52.2 Model-View-Controller . . . . . . . . . . . . . . . . . . . . . . . . . . 52.3 Tiled Map Editor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62.4 UML Diagrams . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62.5 Pathfinding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

2.5.1 The A* Search Algorithm . . . . . . . . . . . . . . . . . . . . 72.5.2 A* Algorithm On Two-Dimensional Grids . . . . . . . . . . . 7

2.6 Defining Believable and Realistic NPCs . . . . . . . . . . . . . . . . . 82.7 Artificial Intelligence approaches . . . . . . . . . . . . . . . . . . . . . 9

2.7.1 Finite State Machines as Artificial Intelligence . . . . . . . . . 92.7.2 Structuring FSMs Using Hierarchy . . . . . . . . . . . . . . . 102.7.3 Problem Solving as AI Technique . . . . . . . . . . . . . . . . 112.7.4 Machine Learning . . . . . . . . . . . . . . . . . . . . . . . . . 11

3 Development of the Game 133.1 Software Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143.2 Creating the UML-Diagram . . . . . . . . . . . . . . . . . . . . . . . 143.3 Defining a Human In-Game . . . . . . . . . . . . . . . . . . . . . . . 15

3.3.1 Needs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163.3.2 Traits . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

3.4 Defining the Game World . . . . . . . . . . . . . . . . . . . . . . . . 173.4.1 Resources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 173.4.2 Structures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

3.5 Implementing the Game . . . . . . . . . . . . . . . . . . . . . . . . . 183.5.1 Pathfinding in the Game . . . . . . . . . . . . . . . . . . . . . 183.5.2 Updating the Game World . . . . . . . . . . . . . . . . . . . . 193.5.3 Displaying the Game World . . . . . . . . . . . . . . . . . . . 19

4 Development of the AI 214.1 Preparatory Literary Study . . . . . . . . . . . . . . . . . . . . . . . 21

vii

Contents

4.2 Early Experiments . . . . . . . . . . . . . . . . . . . . . . . . . . . . 214.3 Early stage FSM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 224.4 Development process . . . . . . . . . . . . . . . . . . . . . . . . . . . 224.5 The AI Class: ArtificialBrain . . . . . . . . . . . . . . . . . . . . . . 234.6 Transitions Between States . . . . . . . . . . . . . . . . . . . . . . . . 24

5 Final Implementation of the Game 25

6 Final Implementation of the AI 27

7 Discussion 317.1 Making the game from scratch . . . . . . . . . . . . . . . . . . . . . . 317.2 Scrapped ideas . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32

7.2.1 Social Interaction between characters . . . . . . . . . . . . . . 327.2.2 Relations between characters . . . . . . . . . . . . . . . . . . . 337.2.3 Character skills . . . . . . . . . . . . . . . . . . . . . . . . . . 337.2.4 Long- and short-term goals of Characters . . . . . . . . . . . . 337.2.5 Trading resources between characters . . . . . . . . . . . . . . 337.2.6 Reproduction of characters . . . . . . . . . . . . . . . . . . . . 347.2.7 Lack of Test Data . . . . . . . . . . . . . . . . . . . . . . . . . 34

7.3 Are Our NPCs Realistic? . . . . . . . . . . . . . . . . . . . . . . . . . 347.4 Choosing Between Believable and Realistic . . . . . . . . . . . . . . . 357.5 Ways of Overcoming Obstacles of Realistic NPCs in the Industry . . 367.6 Finite State Machines in Video Games . . . . . . . . . . . . . . . . . 37

7.6.1 Implementing NPCs using FSM . . . . . . . . . . . . . . . . . 377.6.2 Advantages of Finite State Machines . . . . . . . . . . . . . . 377.6.3 Disadvantages of Finite State Machines . . . . . . . . . . . . . 38

7.7 Our Finite-State Machine Implementation . . . . . . . . . . . . . . . 387.8 Problem Solving In Video Games . . . . . . . . . . . . . . . . . . . . 39

7.8.1 Implementing NPCs Using Problem Solvers . . . . . . . . . . 397.8.2 Advantages of Problem Solving . . . . . . . . . . . . . . . . . 407.8.3 Disadvantages of Problem Solving . . . . . . . . . . . . . . . . 41

7.9 Machine Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41

8 Conclusion 43

Bibliography 45

Appendices I

A Images from the Game III

B Software Description V

C States of Early AI Implementation XIII

D States of Final AI Implementation XV

viii

1Introduction

Video games have gone from beginning to gain popularity in the mid-1970s to beinga part of modern culture as well as an economic powerhouse where the global gamemarket is estimated to reach a total of 100 billion USD in revenues in 2016 [2]. Eventhough many high quality games of today have enormous 3D-worlds, with graphicsnear photorealism to show for it, immersion is still lacking when it comes to thebehaviour of the NPCs inhabiting these worlds. “The believability of NPC behavioris crucial for immersion in games and enables seamless interaction between playersand between players and NPCs”[3]. In terms of game immersion, storytelling andAI research (and other research areas as well)[4], a more intelligent behavior of videogame agents opens up a vast array of unseized opportunities.

1.1 Background

The term Artificial Intelligence was first coined by the American computer- andcognitive scientist John McCarthy, at the second Dartmouth Conference in 1956 [5].According to McCarthy "every aspect of learning or any other feature of intelligencecan in principle be so precisely described that a machine can be made to simulate it"[6]. Still, the idea of artificial intelligence can be traced back to philosophy, fictionand imagination as early as in the classical period of ancient Greece [7].

Video games didn’t reach mainstream popularity until the 1970s when arcade gamesand gaming consoles were first introduced to the general public. However, researchon game AI was already being conducted as early as 1948 on a computerized gamecalled “Nim” [8]. Just a few years later, in 1951, Christopher Strachey wrote anAI program for playing checkers and Dietrich Prinz wrote one for chess using theFerranti Mark 1 machine at the University of Manchester. Further advancementsin gaming AI led to an AI (IBM’s Deep Blue) beating the ruling world championof Chess, in 1997 [9]. More recently the world champion of the board game Gowas beaten by AlphaGo, an AI developed by researchers at Google [10]. Go waspreviously the last non-chance combinatorial game with perfect information in whichthe best human players were considered better than their AI counterparts [11].

It is worth mentioning that there are some discussions involving whether or not the

1

1. Introduction

term “game AI” exaggerates the valuation of its notion. AI in video games fill adifferent role from AI in the academic field [12]. In academics there are several defi-nitions of what an AI is, but most agree that an AI automates intelligent behaviouror reasoning [13].

Despite these discussions, there are others who think video games and AI researchgo hand in hand. Looking from an entertainment point of view, advancements in AIresearch can help to increase the demands to add realistic and intelligent behaviourto characters in video games. The other way around, as video game environmentsbecome more complex and realistic, they also open up possibilities for acting as atesting ground for AI research in many different fields [14][4]. Furthermore DemisHassabis (CEO of Google DeepMind) stated in his presentation, The Theory ofEverything, that “Games are the perfect platform for developing and testing AIalgorithms” [15]. An example of this is the use of behaviour trees in AI whichoriginates from its use in video games [16].

There are games out there which have succeeded and have been awarded for theircontribution of influential AI-games. Black & White, currently ranked number oneon the list of the most influential AI games, is a video game developed by Lion-head Studios and was created in 2001; where Demis Hassabis was one of the leadprogrammers. The game incorporates artificial life simulation, in the form of giantanimal-like creatures, combined with strategy. An AI architecture known as belief-desire-intention (BDI) was used in the game engine and the game also had greatsuccess with machine learning, such as neural networks and decision trees [17].

Despite the progress made in AI research over the last few years, the potentialof AI in video games remains unfulfilled. Even in very recent, “triple A” titles,such as Grand Theft Auto or Assassin’s Creed, a player will still encounter numer-ous situations where the AI behaves so unrealistically that it will completely breakimmersion. NPCs in these games representing humans lack some of our most funda-mental cognitive skills, such as memory or the ability to communicate informationto one another. A common trait of the titles mentioned is that a player can commita crime in front of the guards, flee and then return to the very same guard minutes(or even seconds) later. The guard will have no memory of the recent event, norwill he remember the player. [3].

Even games that use AI as a selling point, or have been praised for their AI, oftenwill not provide what one could expect these days in terms of game immersion. In2012, the game Skyrim [18] won the Academy of Interactive Arts & Sciences “Gameof the Year” award [19], as well as numerous other prizes and is considered one of thebest games of all time[20]. Still, the AI of the game is prone to break immersion timeand time again through unrealistic or irrational behavior. A typical example fromSkyrim would be that an NPC could ask you to perform some mundane task thathe or she is not capable of doing themselves, like clearing their house of mudcrabs.However, when a dragon suddenly appears, the NPC will forget that he or she ismerely a weak citizen and will attempt to slay the dragon with their bare hands,showing no signs of fear [3].

2

1. Introduction

1.2 Purpose

The purpose of this bachelor thesis is to describe the process of developing a videogame and the AI that controls the NPCs in this video game. This process also servesas basis for investigating what it takes for an NPC to be realistic and how differentAI approaches can be used to implement realistic NPCs in video games.

1.3 Scope

Considering that the focus of this project lies on creating artificial intelligence, thefoundation of the game (the game without NPC) will be simple in both its visualaspect and its gameplay aspects. The foundation of the game will be designed withfocus on making a world that NPCs can interact with, change, and be challengedby. The game will thus not necessarily be challenging (or fun) for a human player,but rather be an open world to test and challenge the AI.

The game to be created will be a survival game extended with elements from thestrategy genre. The elements from strategy are there to challenge the AI. It forcesthe AI to take different factors into account, such as resource management, in orderfor it to survive. The initial intention was to implement multiple different AIs usingvarious approaches in order to compare them to each other.

3

1. Introduction

4

2Technical Background

These first subsections will briefly describe the tools and techniques used in orderto create the game; since a lot of work is required to get a game running. The lastsubsection of this chapter focuses on giving the reader a brief introduction to wellknown AI approaches.

2.1 Slick2D

Slick2D is an open source Java library, wrapped around LWJGL (Lightweight JavaGraphics Library) [21]. It contains tools to facilitate the use of images, animationsand other graphic components, as well as tools for things like sound and particles.These tools exists to make Java game development easier, as it relieves a lot of theworkload for programmers whose main focus does not comprise of graphics handling.

2.2 Model-View-Controller

Model-View-Controller, MVC for short, is a design pattern for software applications[22][23]. The design is popular as it separates pure logic and the user interface fromone another into three interconnected parts: model, view and controller. The mostprominent benefit of the design is more organized and manageable code, which isan important feature of larger applications. Other benefits of dividing the code inthis way are:

• Reusable code• More extendable code• Allows for concurrent work between developers, responsible for different fields,

without stepping on each others toes.

Some possible drawbacks to Model View Controller:

• Increasing complexity as applications sometimes cover other patterns, in ad-

5

2. Technical Background

dition to the MVC.• If the model is actively changing it can mean excessive update of the corre-

sponding view.• Excessive information passing between the three interconnected parts, if poorly

constructed for the specific purpose.• The three parts have to be synchronized in order to display correct information

in real-time.

How these three parts are interconnected vary significantly between applications,depending on what is most preferable for that specific application. It also comesdown to different opinions and interpretations on what is the overall best way toimplement the design pattern.

2.3 Tiled Map Editor

Tiled is a free map editor program that can be used to create 2D-maps for games[24]. It’s compatible with the Slick2D library and the two are easy to use together.Tiled makes world creation easier as it removes the coding that has to be done inorder to create a world in run time. Instead, one basically paints the world with atile set of one’s own choosing.

2.4 UML Diagrams

UML stands for Unified Model Language and is a general-purpose, modelling lan-guage in the field of software engineering intended to provide a standard way ofvisualizing the design of a system. The notation has evolved from the work ofGrady Booch, James Rumbaugh, Ivar Jacobson, and the Rational Software Corpo-ration to be used for object-oriented design, but has since been extended to cover awider variety of software engineering projects[25][26].

2.5 Pathfinding

In games where NPCs move dynamically around the terrain, as opposed to pre-determined paths, there needs to be some way of moving about the world withregard to solid objects and non-traversable terrain. This way of finding a viablepath is called ‘pathfinding’ and utilizes some kind algorithm in order to to so. Someinstances may require the absolute shortest path from point A to point B, whileothers are content with just finding a path. Many different versions of this kind ofalgorithm have been used throughout the industry [27].

6


A pathfinding algorithm will, given a graph, find a path from one point to another.More advanced algorithms also guarantee that the found path is the shortest pathbetween the two points, at a small reduction in efficiency. There are several way todefine the ‘shortest path’; it can mean simply the least amount of transitions in thepath or the lowest cost of all the transitions in the path.

Initially, when searching a graph, you add the starting node to a list of discoverednodes, called the ‘frontier’. Then the algorithm will check each discovered node andif that node is the final node, the path has been found. If the node was not thefinal one then the that node will be added to a list of considered nodes, to keepthe algorithm from checking a node several times, then adds all connected nodes tothe frontier and selects a new node to search from. If there are no discovered nodesleft to search from then there is no path to be found. In what order the nodes aresearched depends on what pathfinding algorithm is used.

2.5.1 The A* Search Algorithm

One of the most popular pathfinding algorithms is A* (sometimes A-star). A*searches the graph by selecting what node to search from next using a programmer-defined heuristics function. This function is based on what knowledge the program-mer has of the graph and determines what node is most likely to be part of theshortest path to the end node. For the algorithm to be guaranteed to find the short-est path, the heuristic needs to be admissible. This means that, in all cases, the valueof the heuristic is always equal or better than the real value of that path. Becauseof this the performance of A* depends heavily on how good the heuristics functionis at guessing the shortest path. Guessing a bad value will force the algorithm toexplore more nodes than it would need to, resulting in worse performance.

2.5.2 A* Algorithm On Two-Dimensional Grids

In 2D-video games, one particular problem is having NPCs find their way around theworld without walking into solid objects. A solution to this problem is convertingthe map into a 2D grid. A cell in the grid can either be empty or occupied. Emptymeans that the character can walk to or through that cell whereas occupied meansthat the place is unreachable. An example can be found in figure 2.1. To be able topathfind through this grid it is modelled as a graph. Each empty cell becomes a nodewith transitions to all empty adjacent cells. Always selecting the node with shorteststraight line to the end is an efficient heuristic when searching for the shortest paththrough a 2D grid.

7


Figure 2.1: A pathfinding grid on a world of rocks. Crossed out cells are occupied

For searching this grid by using the A* algorithm, an efficient heuristic is to alwaysselect the node with the shortest possible path to the end. This makes A* on 2Dgrids excellent for finding paths around solid objects [28].

2.6 Defining Believable and Realistic NPCs

This section and most of the report will concern NPCs that represent humans orhuman-like characters. For the purpose of this thesis it is important to make adifference between the words believable and realistic. Note that these definitionsare neither standardized in the academic field of AI nor in the video game industry;they are here simply to make the report easier to understand.

According to the book “Artificial Intelligence: A Modern Approach”, the field ofAI can be divided into four main categories [13]. These can be seen as differentapproaches to intelligence. First, the field can be divided into rational and humanAIs. Rational AIs are made to reach a goal or find a solution. This could lead to arational AI taking an unintuitive decision because it calculated that would be thebest decision at the moment. Human AIs are made to make the decision a humanwould. Further these two groups can each be divided into AIs that act and AIs thatthink. When designing an AI that acts one does not care for how that AI worksas long as it acts intelligently. Conversely when designing an AI that thinks theunderlying decision process is important. This gives us the four groups: Thinkingrationally, acting rationally, thinking humanely and acting humanely [13].

In our definition, an NPC is believable when it appears to be human. There areno requirements on the underlying mechanics of the NPC as long as the NPCs’

8


behaviour is apparently human. An example of this could be that an NPC has noneed for drinking, as it cannot become thirsty, but it will purchase a drink anywayto keep up the illusion that it is human. The character of a believable NPC couldbe viewed as a puppet and the AI is tasked with making this puppet seem human.Referring to the categories from the previous paragraph, the AI of a believable NPCwould therefore be of the type that “acts humanely”.

In our definition, an NPC is realistic when it simulates a human. The character canbe seen as a body and the AI as the brain. This type of AI does not simply performtypical human actions for the sake of illusion, but has an actual need to do so orelse it will perish. A good example is The Sims; in The Sims, a character does notdrink because “that is what a human would probably do”, but because it has anunderlying need to do so. The AI in this case would either fall in the category ofthinking rationally or humanely, depending on the design of the AI.

2.7 Artificial Intelligence approaches

There is a multitude of designs available to the programmer when implementingartificial intelligence for video game NPCs. What design to choose should be deter-mined by the desired behaviour of the NPCs in question. The following paragraphswill describe how some of the most popular designs work, as well as what theiradvantages and disadvantages are.

2.7.1 Finite State Machines as Artificial Intelligence

A Finite State Machine, FSM for short, is a system that has a finite number ofstates that an AI can enter. Each state is unique and determines the behaviourof the AI. What state the AI will transition into next is decided by predeterminedconditions. Let us take an example of a basic NPC in the form of a city guard.For simplicity, the following states the AI can enter and transition between willbe patrol, suspicious and attack. If the guard has not spotted the player, it willpatrol the area. Granted that the guard is in patrol state and the player is suddenlyspotted, the guard will transition into the suspicious state and actively search forthe player. When in suspicious state, the guard has 15 seconds to spot the playeragain in order to transition into attack state, chasing and attacking the player. Ifthe condition of spotting the player in the following 15 seconds is not met, the guardwill transition back to patrol state. This FSM is visualized in figure 2.2.

9


Figure 2.2: Visualization of a simple FSM, using three states.

The advantage of an FSM is that it is easy to implement and understand. They areeasy to expand, as long as the number of states is relatively small. Debugging alsobecomes relatively simple, since the actions performed can often be traced back toa specific state [29].

One of the disadvantages of an FSM is that it become increasingly difficult to man-age, maintain and expand as the number of states grow larger. For every statealready defined in the state machine that should be able to transition to the newlycreated state, new transitions and conditions for them have to be defined. FSMs canbe predictable if their states and transitions are defined by overly simplistic rules.They can also get stuck in unrealistic behaviour if the conditions for transitioningare too many, too strict or poorly defined [29].

2.7.2 Structuring FSMs Using Hierarchy

A common problem with FSMs is that often the logic of a state cannot be reusedin a different context. For example, you might want to reuse the attack state of anNPC on several occasions, but the transition conditions may not allow it. Statesare often designed around specific logic that has to be provided in order for thestate to work as it should. If that logic is not provided, which is most likely thecase if the state is used in a context it is not designed for, the FSM will fail. As adeveloper you are faced with two options. Either states have to be complicated andaccount for all contexts the state might be used for, or you have a lot of redundancyand have to make numerous specific transition conditions for a lot of states [30].

10


Hierarchical FSMs allow you to keep your states fairly simple while avoiding a lotof the redundancy [31]. Hierarchical FSMs use something called “super states”. Forexample there is a game with five different weapons, all of which change the wayan NPC attacks and requires its own state. Instead of having five different attackstates that have to be considered in every state that can transition into an attackstate, there can be one attack super state.

The super state contains the logic required to determine which of the five attackstates that should be used, depending on what weapon the NPC has. Now everytime the NPC wants to attack it will first go to the attack super state and then tothe correct sub-state corresponding to the weapon used.

2.7.3 Problem Solving as AI Technique

In the field of AI, problem solving is the study of finding a correct sequence ofactions needed to solve a given problem; the solution is often called the goal of thatproblem. A problem can be modelled as a directed graph where each node representsa possible state of the problem and each transition is an action that changes thestate of the problem. For a problem to be solvable there needs to be at least onesolution (goal state) that is reachable from the starting state [13].

A problem solving AI is a design that, given a certain problem, uses this modellingtechnique to find a solution. To get a sequence following some specific constraints,for example the sequence with the least number of actions, one could use a moreadvanced problem solver.

Commonly a problem solver will not have a complete graph to work with but ratherhave a starting state, the possible actions for that state, conditions for what isregarded as a goal state and knowledge of how the state will change when performingan action. With this information, the problem solver can generate its own graph bycreating the states that each action would transition to; consequently having an AIrepeat the process from the possible actions in each new state until a goal state isfound.

2.7.4 Machine Learning

Machine learning is a field within computer science that deals with constructingalgorithms that look for and recognize patterns in data in order to make predictionsbased on new data. Implementations of these algorithms are trained on a set oftraining data, allowing them to improve themselves. The training data is the samekind of data that is to be supplied to the running program, along with the desiredresult. The program then evaluates its own performance and adapts in order toimprove [32].

11


A subgroup to Machine Learning is the so called Unsupervised Learning. The centralconcept of Unsupervised Learning is that no desired result is supplied. The algorithmgives a result based on all the data supplied by trying to find patterns in it [33].

Machine learning is not a new concept, but has gained a lot of momentum in therecent years through the expanding collection of data, affordable processing powerand inexpensive storage.

12

3Development of the Game

This chapter is going to describe the creation of the game. To be able to implementthe AI in this project a world with distinct rules for the AI was needed. Creatingthis world was done in several steps.

The first step was to define the humans in-game, the world, the objects within itand the interactions between these objects. This was done in a software descriptionwhich consists of two parts, a plain text description of the desired result and thena formal definition using functional requirements. Below is an excerpt of the list offunctional requirements:

1. Characters should have needs.2. The magnitude of the needs should increase over time.3. Characters should have skills.4. The skills should increase when used.5. Characters should have traits.6. . . .

The full software description, including the full reasoning behind these functionalrequirements, can be found in appendix B.

The second step was to structure the process of implementing the game. Firstly howto handle version control. Secondly decide on the overall software design, which wasdecided to be a implementation of the design pattern MVC. And thirdly to createa UML-diagram based on the functional requirements.

The third, and last, step was to implement the game. This step was naturally thelargest, and was done over several iterations. This allowed for the development ofthe AI to start as soon as possible, as only a very basic game was required for testingthe first versions of the AI.

13

3. Development of the Game

3.1 Software Design

It was decided that this project should utilize one of several different possible model-view-controller (MVC for short) implementations. In this section, the way that thedifferent parts of MVC were designed to work and interact in this particular projectwill be explained.

Figure 3.1: Illustration of the implementation of MVC in this project.

The controller is the main thread in the program. It works as a bridge betweenthe model and the view, handling events and passing information between thesepackages. Its main tasks are to update the game at every cycle, listen to the inputfrom the view and run the AI. The controller also sends a list containing whatobjects to render and where to render them. The controller is in theory split intotwo parts; one is the controller that handles the public information and the other isa smaller AI-controller that only has access to the data linked to the characters.

The view in the MVC presents the data stored in the model. It displays everythingthat exists in the game such as characters, structures and resources. The view isthe front-end of the application and works by constantly listening to the input ofthe player, passing on the received information to the controller.

The model is a logic representation of the game. It acts as the memory of the game,keeping track of what objects are active, their position and their state. The modelalso contains the functions for modifying the world, used in the update stage ofthe controller. The model will, when asked by the controller, pass along or updateinformation. The main controller can ask for any public data whilst the AI-partonly can ask for data related to the character it controls. A visualization can beseen in figure 3.1.

3.2 Creating the UML-Diagram

In order to outline how all the classes in the model relate to one another as well asachieving a more sensible structure in the code, a UML-diagram was developed. TheUML-diagram was created by adding a class for every noun found in the functionalrequirements. Then, some were combined into one as they shared most propertieswith each other, and as such could be represented by the same class. Next interfaces

14


and help classes were added where they were considered needed. After a few repe-titions of these two steps, the UML-diagram in figure 3.2 was the result. Althoughsome changes were made during the iterative development of the game, the mostparts of the UML-diagram remained unchanged.

Figure 3.2: The UML-diagram that was used as outline for the development ofthe model.

3.3 Defining a Human In-Game

As the purpose of this game was to implement a realistic AI, it was important toproperly define a human in-game. Several different aspects of the in-game humanswere planned for several different reasons.

In order to make the characters realistic, it was decided to give them needs similarto those of a human. For the character to survive, it has to manage these needs. Itwas also decided that the characters should have traits that determine how they actand how their needs change, forcing them to think differently. These two aspects ofthe characters where the most central and will therefore be covered more thoroughlyin the two following sub chapters.

In order to give the NPCs something to do, other than just satisfying their needs, theconcept of goals was introduced. Two different types of goals were envisioned: shortterm goals as well as a lifelong goal. To further differentiate the NPCs from eachother, the concept of skills was discussed. Skills were supposed to decrease the timeit takes for the character to gather a corresponding resource as well as increasing theyield from that resource, thus allowing the character to gather it more efficiently.

15


3.3.1 Needs

A character was decided to have three different basic needs; hunger, thirst and en-ergy. The three needs were based on the the underlying physiological needs of ahuman being according to Maslow’s hierarchy of needs [34]. Hunger represents theneed to eat, thirst the need to drink and energy the need to sleep. The higher thevalue of the need, the better for the character. These needs must be considered bythe AI, with a high priority, in order to avoid certain death. The main objectivefor the AI is to survive as long as possible. Managing resources is therefore im-portant because by eating, drinking and sleeping, the AI can restore its needs andconsequently survive.

• Hunger: Managed by eating food such as meat, crops or fish, gathered fromplaces like animals, farms and lakes.

• Thirst: Managed by drinking water, gathered from lakes.• Energy: Managed by sleeping in houses (or resting outside if not a house

owner).

The needs should deplete continuously over time in the game, though the depletionrate can vary depending on the traits of the character. If any basic need is fullydepleted the character should die.

There were also plans to implement secondary needs for the characters. Secondaryneeds are non-fatal and will instead affect the character in a negative way if notmanaged properly. There were several secondary needs planned for the game, suchas socializing, intimacy and attention. These were more centered around interactioncompared to the basic needs and were supposed to give the NPCs a more realisticbehaviour around each other.

3.3.2 Traits

The purpose of the traits was to vary a character’s behaviour, and were inspired bythe seven deadly sins. Here follows a description of the way the traits were intendedto work. The character is affected by its traits, which are randomized when thecharacter is initiated. These were implemented to give some uniqueness to eachcharacter. The character is affected by each trait on a scale from 0 to 100. Theeffect of each trait is the following:

• Greed : Will be more likely to amass wealth, in the form of gold.• Gluttony: A character will get hungry faster.• Sloth: A character’s energy will deplete faster.• Lust: Will be more likely to reproduce.• Wrath: Decreases the damage taken from attacks.• Envy: Increases the range of vision for a character.• Pride: More difficult to form relations as well as less likely to accept unfavor-

16


able trades

3.4 Defining the Game World

When the in-game humans (or characters) had been defined, the next step was todefine the world they were to inhabit. To give the characters a way to manage theirneeds, it was decided that the world should contain a number of resources. For theAI to be able to improve their situation and allow for more effective management ofneeds the concept of structures was introduced. The next two sub chapters will infurther detail describe the envisioned resources and needs.

3.4.1 Resources

This chapter will describes the way that resources were envisioned. The resources inthe game are means for the NPC to survive. A character can gather these resourcesand use them for various purposes. Resources are either spawned at random loca-tions at initialization of the world or spread from an already existing source. Theyare divided into three major categories: renewable, finite and infinite resources.

• Renewable resources - these resources reproduce as long as they exist in theworld. For example, new trees will grow in proximity to other trees. If alltrees are chopped down, no more trees will grow.

• Finite resources - these resources spawn in a fixed amount when the world isinitiated. When these resources have been depleted the characters will haveto manage without them. Stone will exist in the world as finite resources.

• Infinite resources - a resource that can be gathered infinitely and will neverdeplete. Water is an example of an infinite resource, as it will remain in thegame no matter how much water is gathered.

Animals are a special case in this project, but can be considered a renewable resource.They will wander the world and may reproduce when in close proximity to oneanother. The characters can hunt these animals for meat, which is why they arementioned in this section.

3.4.2 Structures

This subsection will describe the way that structures were envisioned. One choicethe AI has, in order to replenish needs more conveniently, is to construct differentbuildings. Structures require resources in order to be built, and these have to begathered before beginning construction. There are a few buildings that can beconstructed for different purposes and they are the following:

17


• Houses - accommodate characters. Here they should be able to sleep andrecover energy. A character should preferably have a house, as sleeping in ahouse restores more energy than resting out in the open.

• Stockpiles - a place where characters can store their resources. The stockpileinventory should be larger than a single character’s inventory.

• Windmill - a building that characters can work at to produce crops they canharvest for food. Any character should be able to work at any windmill andharvest the crops growing there.

3.5 Implementing the Game

The implementation of the game was done by dividing the development process intoseveral steps. At each weekly meeting, decisions were made on further improvementsthat had to be done as well as what to implement next. The first step in the processconsisted of implementing the most essential and basic parts of the model. This stepincluded the coding of necessary interfaces as well as setting up the basic skeletoncode to some of the classes in accordance with the UML-diagram.

Next came the development of the view and controller, along with the communica-tion between the three interconnected parts of the MVC. After this step, the onlything remaining before obtaining a simple but working game was to add all theresidual necessities regarding the model, which the AI would later need access to.Following this, was an iterative process of refactoring the code to be more efficientand manageable, as well as adding new features.

The following subsections will describe some of the most central parts of the devel-opment; pathfinding, update function and displaying the game. These parts weredeveloped in several iterations, just as the rest of the game. However, as they werevery central, they did not undergo any major changes in the later iterations.

3.5.1 Pathfinding in the Game

In order for the NPCs to navigate through the world, a pathfinder class was im-plemented. This class has two main uses; the first is to map a grid over the worldcontaining information of where the character can, and cannot, move. The secondone is to calculate a path between the character and some point in the world. Incase there is no way to get to that specific point, the pathfinder will return an emptypath. The path is generated as a list of nodes. The character will visit each node inorder, finally reaching its destination at the final node. By using these nodes, whichare essentially straight lines in the bigger path, the character can move around theworld whilst avoiding solid objects it should not be able to walk through.

To find the shortest path between two points the pathfinder uses a A* algorithm

18


specialized on finding the shortest path on a 2D-grid. This path will be convertedinto a list of nodes that is returned by the pathfinder.

3.5.2 Updating the Game World

In this game it was decided that the update function should normally run 60 timesper second. The update function loops through every dynamic object in the worldand executes a specific update method for each and every one of the different objects.These update functions should do different things depending on what kind of objectit is; trees will spread, crops will grow, characters will move, and so on and so forth.

3.5.3 Displaying the Game World

To make the game interesting as well as understandable, sprites were utilized todisplay the world graphically. The project took imagery from the web that is licensedunder Creative Commons. The images used in the game that were neither made forthis project nor public domain can be found in appendix A. These images were thenapplied to the objects in the world, such as vegetation, structures or characters; sothat it could be seen on the screen. A screenshot of the game can be seen in figure3.3.

Figure 3.3: A screenshot from the game displaying a NPC sowing crops at a farmin a small village. References to the creators of images used in the game can befound in appendix A

19


20

4Development of the AI

This chapter will focus on how the AI was developed and implemented into thegame. It will cover early testing, how an early stage FSM was implemented as wellas how it developed over time. This chapter will only cover the development of theactual implementation. Comparisons and thoughts on how additional approachesmight have been used can be found in chapter 7.

4.1 Preparatory Literary Study

Entering the project all members were more or less inexperienced with ArtificialIntelligence. Learning more about AI in general, as well as the standards used inthe industry, was a top priority. This included reading about AI in popular games,how they were used and how to implement them. Since AI is such a broad field andimplementations vary in so many ways, focus was on the most popular approachesused in the video gaming industry.

4.2 Early Experiments

Early in the development process, before the characters were complete enough tobe controlled by an AI, a prototype problem solver was implemented. The goal ofthe prototype was to see what it would take to implement an actual problem solverAI later on. The prototype could take an abstract model of a character and a listof possible actions the character could perform and then calculate what sequence ofactions would lead to the optimal outcome. Optimal in this case refers to keepingthe character’s needs as satisfied as possible, as well as collecting the maximumamount of resources. Though limited, the prototype worked satisfyingly.

21

4. Development of the AI

4.3 Early stage FSM

Development of the FSM implementation started halfway through the project. Thegame had enough of the basic elements required to test the functionality of the AI,however, it was not complete. From this point in the process the AI and gamefunctionality was developed in parallel. In many cases, when new functionality wasadded, support for that functionality were implemented in the AI as well.

A basic AI was desirable as it could be used to test and confirm that the most triviallogic of the game was working. A primitive FSM was implemented where the AIwould only have a few states for collecting resources. This early implementationwas, as stated, a good start to find bugs in the program and to have something toexpand upon. Only a few resources were available for gathering to satisfy the needsof a character. There were no AI methods for constructing different buildings orsocially interacting with other characters.

Additional functions were added to the AI continuously. As new things like con-struction and more resources were added, states were implemented in the AI thatcould handle these new functions. This kept bugs and other issues to a minimumas they could be fixed before going on to the next feature.

4.4 Development process

The first step was to make a blueprint, seen in figure 4.1, of the states that wereneeded, as well as mapping transitions between these states. State transitions weredesigned hierarchically, meaning that states could only transition from a higher levelin the hierarchy to a lower level, or vice versa, never between states on the samelevel.

Figure 4.1: The state transition blueprint, arrows represent possible transitions,bubbles represent states.

This was the envisioned state machine, what the blueprint looked like before the

22


first line of code was written. At first glance it looks a lot like a behaviour tree,but it differed from a common behaviour tree in the sense that there was very littleplanning at the node level. Additionally, it also had to transition to idle state beforegoing deeper in the tree structure, meaning this implementation can disregard levelsof the tree when transitioning [35].

When the blueprint was laid out, an ArtificialBrain class was made along withclasses that represented each state. The ArtificialBrain is the main class of theAI, it contains all the AI variables along with the functionality to make transitionsbetween and executing each state. At this stage, queues were used to control theflow of states, they were later replaced with stacks. The idle state is at the top ofthe hierarchy and was intended for planning the AI’s next move.

At this point the AI could walk, eat, sleep, drink and gather resources. Althoughsimple, it was advanced enough to test how additional features not essential tosurvival played together with the essential features. A list of the states implementedin the AI at this point in the development process can be found in appendix C.

The AI had perfect information about the world, meaning it knew where everythingwas at all times. There was no functionality for social interaction, reproductionor building houses yet. From this point the AI was developed iteratively as moreelements of the game were finished and could be used by the AI.

4.5 The AI Class: ArtificialBrain

The initial implementation of the ArtificialBrain was simple. It had an update()method that was executed by the controller once every iteration. It was decided thateach state should have its own run() method which determines the AI’s behaviour.This method should be executed from the update() method of ArtificialBrain.

As stated in the previous section, the AI had perfect information about the world.As perfect information is not a realistic representation of a human’s perception ofthe world, the AI should not have this kind of knowledge. For this reason a simpleform of memory was implemented in the AI. This memory was a list of objects thatthe character had walked past in the world. As structures were implemented, thissolution was used to keep track of their position as well.

As new states were implemented and the AI became more advanced, new tools had tobe implemented in the ArtificialBrain to accommodate for the increased complexity.These tools were stacks and lists, intended to store information needed by certainstates, as well as allowing the AI to perform sequences of several states, giving it abasic sense of planning.

To increase performance, it was decided that the supervision of primary needs, thatwas previously in the idle state, was to be moved to the ArtificialBrain. This removed

23


the need for the AI to enter the idle state to gain access to this functionality.

4.6 Transitions Between States

A state transition is moving from one state to another. Without them, the AI wouldnever change state and consequently never do anything.

In the early implementation the transitions were designed to be hierarchical, meaningthat the AI could only transition from a higher to a lower level and back again, in thehierarchy of states. It could not transition between states on the same level. Thisdesign was used to keep the transitions neatly structured but as the FSM grew morecomplex, there was a need for some states to break this structure. The transitionswere kept as hierarchical as possible, giving some states the ability to transitiondirectly to states that would not have been possible in a pure hierarchical designor behaviour tree. Additionally, the idle state was initially always used to performstate transitions.

24

5Final Implementation of the Game

The implementation of the game resulted in a functioning world which the NPCscould exist in and change. The NPCs were able to gather water from the ponds andgather food, either by killing cows, fishing in the ponds or farming crops. They werealso able to construct the three different structures described in subsection 3.4.2.The NPC could also collect various amounts of the raw materials wood and stonein order to construct these structures. Of the original 43 functional requirementsin the software description, all but 8 were successfully implemented in the game.These 8 were:

• 44. Characters should have skills.• 45. The skills should increase when used.• 50. Characters should be born.• 55. The AI should make decisions based on the current goals of the character.• 56. The effect on the needs and wishes of the character, that the traits and• skills have, should affect the AI’s decisions.• 58. The AI should take the actions of other NPCs into account when deciding

on actions for the NPC.• 63. The world should have a basic weather system.• 69. The number of houses in the village should set the max population of the

village.

The majority of the envisioned concepts in chapter 3 were implemented. However,Skills, Goals and three Traits, were not implemented due to time limitations. Thethree unimplemented Traits were Lust, Greed, and Pride.

Most of the classes in the UML-diagram were implemented, however a few wereremoved for various reasons. Some minor changes were also made to the UML-diagram during the course of the project. The final UML-diagram can be seen infigure 5.1.

In the finished game most of the NPCs could successfully live long enough to diefrom old age and before that had time to build a house, farm and stockpile. Ascreenshot of the finished game with explanations can be seen in figure 5.2.

25

5. Final Implementation of the Game

Figure 5.1: The final UML-diagram of the model made in this project.

Figure 5.2: A screenshot from the game with explanatory text explaining thedifferent parts of the game. The screenshot displays three NPC working at a farmin a small village. References to the creators of images used in the game can befound in appendix A.

26

6Final Implementation of the AI

The final implementation of the AI is a “stack based finite state machine”. The AIhas a set of states that determine its behaviour depending on which state it findsitself in, at a given time.

The AI is controlled by the class ArtificialBrain. This class can, as the name sug-gests, be seen as the brain of the character. The brain class is updated and executedevery controller update and contains all the functionality for executing each state.The brain class is also responsible for checking if the character is hungry, thirsty ortired. If so, the AI will enter the state corresponding to the behaviour necessary toaddress this. For instance, if the character is hungry, the AI will stack its currentstate, allowing it return to it later and transition into the hungry state.

Each state is an individual class that has its own unique methods that represents theactions an AI can perform, these methods are then executed in the run() method.The run() method of each state is called by the update() method in the Artificial-Brain. A complete list of the states in the AI can be found in appendix D.

The ArtificialBrain consists of several components that are used to model the be-haviour of the AI, these components are:

• A stack of states - This stack contains all the states that are waiting tobe executed. This is used to control the flow of the states. Some actionsrequire several states and transitions to be performed, for instance reachinga satisfying level of hunger, or building something. By using a stack it canpush all the states required to perform an action and then execute them inorder. It is also used as a means for the AI to be able to remember what itwas doing previously, in case it has to interrupt itself to do something of moreimportance.

• A stack of structures - This stack contains all the structures that are cur-rently due for construction.

• A stack of objects to move to - This stack is used to remember where theNPC was going when returning to a stacked MovingState.

• Two stacks of resources - One stack, the FindResourceStack, contains all

27

6. Final Implementation of the AI

the resources that the AI currently wants to gather, but does not have in mem-ory. The other, the GatherStack, contains all the resources that are necessaryfor a stacked state. In gather state, the AI will gather the first resource on thisstack. If the AI does not know where the first resource on the GatherStack is,it will push that resource onto the FindResourceStack and go look for it.

• A resource memory - This is a list of resources that the AI has walked past.The AI remembers where these resources were so that it can easily find themagain. For instance, where the closest lake that it knows of is.

• A structure memory - This a list of structures that the AI has walked past.The AI remembers where these structures are so that it can easily find themagain. For instance, where the closest farm that it knows of is.

The idle state is the state the AI enters between states or when it has nothing todo. It is also used for popping states from the stack. If it has states on the statestack, the first state is popped once the idle state is executed and the AI transitionsinto that state. The run() method of the popped state is executed, and when thestate reaches the end of run(), the AI will go back into idle state and pop the nextstate from the state stack. If it does not have any states on the stack, the AI willpick another task randomly based on where it is in its lifecycle. For example; if theNPC has not yet built a house, and is lacking the resources to do so, it will chooseto gather these resources unless there is something of more importance to attend to,like managing primary needs.

The AI will always start by building a house for itself. If the AI has a house it willproceed to build a stockpile and a farm, the order in which these structures are builtis random. However, the AI will only build a farm if there is no farm present in itsstructure memory. A farm requires a lot of resources to build, and many characterscan work on the same farm. Therefore it would be unnecessary to build a farm ifone has already been built and it knows where that farm is. If the AI has built astockpile and has a farm in its memory, the AI will either hunt for food, gather arandom resource or work on a farm for crops.

In the beginning of its lifecycle, the AI follows a simple, hardcoded plan. The AIwill start by checking if it has a home, i.e. somewhere to sleep. If it doesn’t, itsfirst task will be to build one if it has the materials required. If it doesn’t have thematerials, it will gather them. It does not make plans in advance or prepare for thefuture. It simply reacts to its current state and does what is necessary to proceed.When the AI is finished with these predetermined actions, it will gather a randomresource.

The AI will only interrupt itself in a task if it recognizes one of its primary needsis under a certain threshold. If so, the ArtificialBrain will push the AI’s currentstate onto the state stack, saving it for later. It will then force the AI into a stateincreasing the corresponding need. When the AI has finished eating, drinking orsleeping, it goes back to do what it did before it got hungry, thirsty or tired.

28


The AI has almost no ability to plan for the future. The AI will only take on a taskif it is what is necessary at that moment in time, and as can be seen earlier in thissection, sometimes it is completely random. The only thing the AI plans for arethe states needed in order to complete a certain task, which could be argued not tobe planning at all. For instance if it wants to gather a certain resource that it hasin memory, the AI knows that it will have to stack the moving state to get there,and the corresponding gather state for that resource in order to gather it once thatresource has been reached.

29


30

7Discussion

The desired behaviours of NPCs in video games differ wildly. It is difficult to saythat there is a best approach to AI for NPCs. With our limitations we have narrowedour scope down to NPCs in survival games. Still the survival genre is too broad afield to raise one AI approach as superior.

In this section we will first discuss alternatives to creating a game from scratch, andfeatures that were left out of our game. This is followed by a discussion of believablecontra realistic NPCs. Finally we will go through three different AI approaches thatone may use to implement an NPC in a video game. We will discuss the advantagesand disadvantages from our own experiences in development, as well as what wehave learned about implementing realistic NPCs through the course of this project.

7.1 Making the game from scratch

We decided to make the game ourselves from scratch to retain control of whatfeatures we could implement. The group was in general interested in creating agame from scratch and saw this project as a good opportunity to explore this area.

Other options were considered, such as Unity or Unreal Engine, which could havebeen very good options since they both have very good AI frameworks that couldhave streamlined the process. Unreal was discarded because we felt like the realpower tools of it are for 3D games, and since we wanted to have our game verysimple, we wanted to make a 2D game. Unity uses C#, which is an object orientedlanguage, but the group had no experience with either C# or Unity. What we hadexperience with though, was Java, and a few members of the group were alreadyexperienced with game development using this language. We realized that makingthe game would take quite some time, and we wanted to hit the ground running tofinish the game as soon as possible. Hence, Java was the language we chose.

As creating a game is no small task, it might have been easier, and quicker, to createa modification (or mod for short) to an already existing game with good supportfor mods. However, the choice of game would limit us to using the tools of thatgame, and the AI to that world. So while it might have been quicker, it would have

31

7. Discussion

been necessary to research the code of said game, it would also have been a limitingfactor to what we could achieve. A benefit of creating a mod would have been thatwe would be able to compare our result with the original NPCs.

7.2 Scrapped ideas

Entering the project there were many ideas concerning how we could make the AImore realistic. We found many of these interesting and we think that they couldhave given a lot of depth to our AI. Unfortunately, many of them had to be discardedfor various reasons. This chapter will cover those features, what they were intendedto accomplish and why they had to be tossed.

7.2.1 Social Interaction between characters

To increase the sense of realism and immersion in the game, we wanted to givethe NPCs the ability to socially interact with each other. Social interaction wassupposed to be a tool for the NPCs to build relations. One NPC talking to anotherwould either increase or decrease the relationship status with that NPC, dependingon how similar their traits were. In this way NPCs could make new friends orenemies. This would have given the AI more depth.

Social interaction was also supposed to be a tool for the AI to gain more informationabout the world, without having to go out and explore everything for itself. Whentwo NPCs would talk to each other, they would exchange information about thelocation of resources and structures.

Socialize was supposed to be a secondary need that, when low, would negativelyaffect the character’s ability to work and possibly lower the amount of energy gainedfrom sleep. Interacting with other characters would replenish this need. The effect ofa low level of the socializing need would be determined by the traits of the character.For instance, if a character had traits making it more of a lone wolf, it would notbe affected negatively by isolation from other characters.

A lot of work went into making the idea of social interaction a reality, it was some-thing we wanted to work because it was a prerequisite for many other features.Among these features were reproduction, relations and trading. There was partialsuccess. We managed to get two NPCs to talk to each other, but as soon as athird one wanted to join the conversation, the game would break. This issue provedcomplicated to solve due to the way that the software architecture was designed. Inthe end, we decided to scrap the idea because time was running out and there werestill many essential features that had not been implemented.

32

7. Discussion

7.2.2 Relations between characters

We wanted the NPCs to be able to build relations with each other. Relationswere supposed to be the base for how an NPC would respond to another in socialinteraction. An NPC was supposed to be helpful towards friends and unhelpfultowards enemies or NPCs that NPC did not particularly like.

Relations were also going to be used for family trees, the AI was supposed to knowwho its mother, father, brothers, sisters, sons and daughters were. Additionallyrelations were intended to allow for reproduction. If a man and a woman got toa specific grade of relation they would get engaged and eventually have children.Since social interaction never made it to the final implementation of the game, therewas no good way of building relations. Thus they were never implemented.

7.2.3 Character skills

We wanted the NPCs to have certain skills that would make them more useful incertain areas than others. When performing a certain task, the NPC’s skill in thattask would increase. It would also decrease over time if the NPC did not perform thetask regularly. Some intended skills were; fishing, farming, trading and exploring.If an NPC was particularly good at fishing, it would net more fish than other NPCswith a lower skill level. However, skills had low priority and were never implemented.

7.2.4 Long- and short-term goals of Characters

Another possible extension of the NPCs were long and short term goals. Achievingthese goals would increase its overall happiness and would give more depth to thegame. Short term goals could be: build a house, make a new friend or somethingalong those lines. Long term goals would require more effort to fulfill and would notnecessarily be something the AI could achieve in its lifetime. Examples of intendedlong term goals were: get married, have three children or become the richest personin the village.

7.2.5 Trading resources between characters

Trading was supposed to be a type of social interaction where one NPC wouldpropose a trade with another, that NPC would then assess if it was a fair trade andchoose to accept or decline it. The idea was that NPCs that had a lot of a particularresource could trade that resource for something else that it needed, removing theneed to always gather everything. This would have played well together with theuse of skills, as proffessions could arise naturally. For example: an NPC skilled

33

7. Discussion

in lumbering could trade wood for fish with an NPC that had a high fishing skill.Trades would have been influenced by relations were an NPC would be more likely toaccept a trade from a friend. Trading was not implemented since social interactionwas discarded.

7.2.6 Reproduction of characters

A vision was that the AI would be smart enough to sustain its society forever. Oncethe game had started running, the NPCs would never go extinct. A prerequisite forthis is of course that the AI is able to reproduce, otherwise extinction is inevitable.Since relations were never implemented in the game, neither was reproduction, ren-dering this vision unachievable.

7.2.7 Lack of Test Data

With the purpose of implementing a realistic AI in a video game, it would have beenhelpful to define tests to verify that the result fulfilled this purpose. These testswould have been designed to prove that the AI we created fulfilled the definition ofa realistic AI to a satisfactory degree. Unfortunately, such tests were never explicitlydefined, and therefore never performed on the final product.

The reason such tests were never defined was simply that in the early stages of theproject, when such tests should have been defined, we were focused on the literarystudy. Therefore, when we realised that such tests should have been defined, it wastoo late to implement or conduct them in any reasonable manner.

7.3 Are Our NPCs Realistic?

We lack the data to be able to confidently state that our NPCs are more believableor realistic than NPCs in games currently on the market. However, several strongpoints can be made as to why our NPC make a good proof of concept for a realisticNPC. In Maslow’s hierarchy of needs the motivators of human behaviour are placedin a hierarchy. The idea is that the lowest unfulfilled step of the hierarchy drivesthe actions of the human. The lowest two steps of the hierarchy Maslow proposesis “physiological needs” followed by “safety” [34].

Our NPCs acts according to the bottom two of this hierarchy. When they havepressing hunger, thirst or energy (physiological needs) they will tend to these needs.If they are satisfied physiologically they will work on security, making sure that theyhave somewhere to live and that they will have food for future use. The furtherintroduction of social interaction and skills could have further led to the inclusion

34

7. Discussion

of two more steps of Maslow’s hierarchy: “Love and belonging” and “Esteem” [34].

One can argue that our NPCs achieve a high level of realism, relative to theirsimplicity (being incapable of doing most things a human can do). However wewould argue that they make a good proof of concept, being realistic in performingthe actions they are capable of. As such, they show that better realism for NPCs isachievable. Worth noting is that the AI in the NPCs is a version of an FSM, whichthen shows that realistic NPCs can be made using approaches already known to theindustry.

7.4 Choosing Between Believable and Realistic

It can be a difficult task to make the decision whether one should have a realisticor believable AI in a video game. One has to take into account what the AI shouldaccomplish and the amount of computational resources it should be allowed to use.An AI with focus on realism will require much more computational power than itsbelievable counterpart; since a realistic AI has to do a lot more planning and decisionmaking. Believable AI does not suffer from this to nearly the same extent, as itsbehaviour is often predetermined.

A well-implemented realistic AI will undoubtedly give the player a deeper sense ofimmersion, since the AI will act and behave as a regular person would. In somegames however this might be an undesirable behaviour. In Role Playing Games(RPG), the player takes on a role of a character in a fictional setting. In thesegames, NPCs play a big part in the player experience and they are essential forthe player’s progression through the game. They give the player missions, serve asstorytelling elements, they purchase and sell items; the list goes on.

If these NPCs were entirely realistic, meaning they would have to eat, drink andsleep, just like human beings, the game could quickly get tedious. Imagine if youneeded a specific quest to progress through the game and the quest giver, a characterwhich hands out missions, is not where you expect him to be. Perhaps he has gonehunting and won’t be back for a couple of hours, or perhaps he has taken a strolldown to the local inn to get something to drink. What if the quest giver dies? Thiswould certainly be a very immersive experience, but it would slow down the paceof the game tremendously and could leave the player stuck with nothing to do forsome time. It might even break the game completely.

In order to have realistic AI in this type of game, a game designer would have toput much bigger thought into the overall design of the game. In these days wheregame developers are often limited by tight budgets and time schedules, realistic AIis rarely a prioritized design choice.

If we take a look at the other side of the coin, what if NPCs are not realistic;which is the case in most games. Here the player will always be able to progress,

35

7. Discussion

as he can always be sure to find the quest giver where he expects him to be. Thegame designer does not have to take into account the difficulties that comes alongwith realistic AI. The AI will always behave as the designers expect it to behave,there is no risk of the AI running into a situation that might not be accounted for(given that the developer has not made a mistake somewhere, of course). However,a stationary quest giver that stands in the same position every second of every daycould arguably be an equally undesirable feature, as it takes away the immersion arealistic AI provides.

As we can see, the choice is anything but easy when it comes to realistic or believable.A lot of games commonly end up somewhere in between the two, with developersusing both realistic and believable elements to shape their AI.

7.5 Ways of Overcoming Obstacles of RealisticNPCs in the Industry

Consider the situation where you have a character that is important to the storyof the game. If the character is realistic it may die before playing its part in thestory leading to one of two outcomes. Either the character is revived, which leadsto disbelief, or the character remains dead and therefore a part of the game, maybean important one, is lost to the player.

Immortal characters can be observed in the game Skyrim in which important NPCswill not die if damaged to zero health. Instead the NPC will collapse and remain stilluntil it regains its health. In this case it not only causes the game to be unrealistic,but you can also use this to your advantage by sending your NPC allies to attackthe enemy without approaching the enemy on your own. Since your allies cannotdie and will regain health after having been knocked out, they will always win thefight eventually.

Another common tactic is counting a death of an essential NPC as a death ofthe player. This will keep the player defending the NPC and if it dies the gamewould usually return to a state before the essential NPC’s, death. Another moreimmersive solution can be found in the game Morrowind, a prequel to Skyrim [36].In Morrowind the essential NPCs are treated just as any other NPC concerningdeath, with one small difference; the game will warn you when a essential NPC havedied (something that rarely happens thanks to the game’s design) and advise youto reload a previous save in which the NPC is still alive. This way the game retainsits realism and gives the player freer hands in how to play, while still keeping theoption to play the game with all of its essential NPCs retained.

Considering this, we would suggest using realistic NPCs when NPC behaviour is acentral part of the game. Games that often fall in this category are Role-PlayingGames (RPG) and simulators. In other cases believable NPCs, imitating human

36

7. Discussion

behaviour should suffice.

7.6 Finite State Machines in Video Games

Finite State Machines is the most popular approach to AIs in video games [37].There are two main reasons for this: First there is tradition. FSMs have beenpopular in the video game industry for a while and as such a lot of the tools used inthe video game industry supports FSMs more than other AI approaches. SecondlyFSM have many qualities appreciated by video game designers a few of which willbe discussed further on in this section.

7.6.1 Implementing NPCs using FSM

How an FSM is used to make AI for NPCs varies from game to game. Sometimesthe FSM is used in cooperation with other AI approaches. However, there are a fewbasic ideas that stick around.

An FSM decides the exact behaviour of a NPC from the states and transitions thathave been programmed. Because of this, one usually implements one unique FSMper NPC type. For example all animals in a game could use the same FSM, if thebehaviour of one species does not differentiate from another, though that same FSMwould probably not suffice for the NPCs representing humans.

7.6.2 Advantages of Finite State Machines

There are good reasons why FSMs have become so popular in the video game in-dustry. Since an FSM is, by its very nature, easy to divide into parts, this makesworkload easy to divide as well. For example one programmer could program thestate transitions while two other program the different states. Through their sim-plicity, FSMs are good for setting up a simple prototype that can be made moreadvanced by adding more states later on. As a lot of time went into making thegame world, the fast development pace FSMs allow for proved very useful to us.

Since the behaviour in each state as well as the transitions follow a strict set ofinstructions, it is easy to implement an NPC with distinct behaviour. The deter-ministic nature of FSMs makes it easy to know how a piece of the program willaffect the behaviour of the NPC, thus changes are easy to make. If one wants tomake tiny changes to a certain part of the NPCs behaviour, that is easily achievedby redesigning the concerned states. For more advanced changes one could simplyadd more states to deal with the specific problem at hand. This aspect made it easyfor us to make changes to the AI as more states were introduced, and existing states

37

7. Discussion

had to be reworked in order to accommodate for changes these new states broughtalong.

7.6.3 Disadvantages of Finite State Machines

As the name suggests, FSMs have finite states. Specifically they only have as manystates as the designer implements. While this approach works great for simple NPCsin simple circumstances, it poses problematic for more complex situations.

Since each state depends on the circumstances of the NPC, the FSM might oftenlack states for the many different situations that can occur in a complex environ-ment. Even if a state is meant to cover a given situation, the state may be toogeneral and not always suit the situation given. You could theoretically expand anFSM until it encapsulates every improbable situation. However, in complex enoughenvironments, this is an almost impossible task.

The other problem that arises is the growing complexity of the FSM. This is easilyobserved by considering the blueprint of an FSM. For each new state created adesigner will have to consider which other states should be accessible from thisone. For an FSM with two states there are two possible moves to be considered,for three states there are six moves, and for four states there are twelve, hence itgrows quadratically. This is not something that ever became a problem for us, aswe countered it by making our FSM hierarchical.

7.7 Our Finite-State Machine Implementation

Our final AI implementation is, as previously stated, an FSM. Though it is not a pureFSM, but rather a Stack-Based FSM (SFSM) which is a more common constructin video game AI [38]. What is referred to as FSM in video game development israrely a pure FSM according to its mathematical description. More often the FSMused is a more complex design. Common designs are Hierarchical FSM or, like inour case, a SFSM [39].

We decided to use an FSM for our AI. There are a few reasons for our choice ofusing FSM rather than any other AI approach we had studied. One reason wastime. Our game had been progressing slower than anticipated and we needed tohave a working AI as fast as possible. We reasoned that FSM was the fastest way toget a working AI up and running. The lack of time also forced us to start workingon the AI before the rest of the game was finished and the expandability of a FSMmade it easy to add more complexity to the AI as the game was being developed.

The use of an SFSM gives the advantages of having an FSM but with the additionalpower of a stack memory. The NPCs could use the stack to push their current state

38

7. Discussion

when they decide to do something else. This way they remember what they weredoing when they complete their new task. For example an AI could go into a buildstate because they need a house. In the build state they could realize that they donot have enough resources to build a house. The natural conclusion is to transitioninto a gather-resource state. In this state the AI gathers the resources it needs andwhen it is finished it needs to transition into a new state. With a normal FSM, theAI could not know why it had gathered resources, with an SFSM it can. The NPC,now finished with the gather state and with no reason to transition to a specificstate, pops the last state pushed onto the stack. This way it returns to the buildstate and can finish building its house.

We chose this implementation over a regular FSM as we deemed it gave us greatercontrol over how the AI behaves. It allowed us to work with sequences of states,rather than the traditional way of letting each state determine what state comesnext. These sequences are good, because we can easily make sure that the AI doesthings in the order that we have intended. An ordinary FSM could give us thisbehaviour as well, but with the use of state sequences we could neglect a lot of thecode that would have been necessary in a regular FSM to prevent it from doingthings it was not supposed to. For example, if each state was to decide what thenext state was, we could never be sure that the next state the AI entered would bethe one we intended. Sequences removed some of the difficulty of having to figureout why the AI did something unexpected.

This implementation could be argued to be bad design, because it comes with a lotmore hard coding and predetermined behaviour, but for the results we sought thiswas satisfactory.

7.8 Problem Solving In Video Games

Problem solving was a popular approach for creating game AI early in the fieldof Artificial Intelligence. Though because of the brute force nature of problemsolvers the purely problem solving approach was soon abandoned for more efficientapproaches [40].

7.8.1 Implementing NPCs Using Problem Solvers

What we know of problem solving comes from a mixture of studying the AI ap-proach in literature, developing an early proof-of-concept AI and an, due to lackof time, unsuccessful attempt to make a full problem solver for the game. Despitethis, or maybe because of it, we feel that we learned a lot of the advantages anddisadvantages of using problem solver to create an AI in a video game.

For an NPC to be able to use problem solving requires a lot of structures in the

39

7. Discussion

game itself. There needs to be a way for the NPC to check what actions are, andwhat actions will become, available (or unavailable) by performing each of theseactions. The NPC needs to know how to value its state and maybe even the stateof the world. This requires structures and standards for what actions there are,what objects are interactable and what a given action would yield. The AI will berequired to simulate the use of each possible action and be able to simulate evenfurther what actions are possible after that one. There need to be a finite types ofactions. For each action type there need to be a way for the AI to tell the outcomeof this action to be able to search further down the tree of possible routes of actions.

In the game we implemented much of the framework required for a problem solvingAI, as well as began work on the problem solver itself. We divided our actions intothree types, not only for the problem solver but for keeping similar interfaces for theFSM as well. The three types are; character bound, item bound, and environmental.The interfaces differed, out of necessity, although it would have been possible to havean AI which interacted in the same way through each action.

A character bound action represents an action a character does with its body onlyinteracting with itself. The example from our game is that a character can at anytime sleep on the ground. It is an action that can be performed simply by decidingto do it.

An item bound action is an action that is bound to some item the character iscarrying. They differ from the character bound in that they require the NPC tohave the given item and in some cases using the item will consume it. An examplefrom the game would in this case be eating a fish. A fish is something you musthave to eat and eating it would hinder the NPC from eating it again.

Lastly an environmental action is a action bound to an object that is part of theenvironment or surroundings. This could be any action that needed or affected apart of the environment. An example from the game would be chopping wood froma tree.

The separation of item- and environmental actions was due to the fact that toperform an environmental action an NPC would need to move to the object theywish to interact with. Although similar actions, interacting with another instance ofthe world, the programming required differed heavily because of the need to movethe character into position.

7.8.2 Advantages of Problem Solving

Problem solving can, given a good interface with the world and a good value function,calculate the best approach to a given situation. It frees the programmer fromthe task of figuring out how the NPC should behave in given situations and focusinstead on the NPCs overall goal. A problem solver can take into account the smalldifferences between slightly different states and take different actions accordingly.

40

7. Discussion

A problem solver can plan far ahead and therefore think of long term goals. Thiscould mean the AI decides to take several unbeneficial actions in a row because itwill allow it to take a more beneficial action later on.

Personality and personal goals can be easily implemented into the NPCs as a changein the AIs value function. Different NPCs value the state of the world differently.For example an NPC with high greed may have a function that considers the value ofmoney or gold higher whereas a gluttonous AI values food more. A more advancedvalue function may consider the state of the other NPCs. If an NPC like anotherthey could value the state where the other AI is satisfied more and thus help thatNPC reach that state. Conversely if an NPC has an enemy they may value a statehigher if it is bad for their enemy.

7.8.3 Disadvantages of Problem Solving

Problem solving requires certain structures in the game to be viable. It is alsodifficult to implement and test a problem solver if the base of the game is not yetfinished. A problem solver can also be rather heavy on processing if it needs toplan too often or if the game has such great complexity that very deep searches areneeded to find satisfactory nodes.

The necessity of a value function is another drawback as it in some cases might bedifficult to know how to value a given state and the sole pursuit of increasing thatvalue might lead to the NPC neglecting everything else. Other problems might ariseif it is difficult to model the changes of the world within the AI. Since it is extremelyineffective to model the entire world within the AI the states of a problem is often amore abstract version of the world. If the conversion of the world into a state in theproblem is done poorly the AI is likely to make some bad decisions. However onecould argue that this is how human works as well; we don’t make perfect decisionsbecause of our imperfect perception of the world.

Problem solving is also a rather difficult AI to predict or make small changes in.Whereas in an FSM you can modify the specific behaviour in a given state or changethe exact transition between states a problem solver is much more limited when itcomes to detailed control. The value function could be slightly changed as well asthe perception of the world but the AI might still not behave satisfactory. This alsomakes the AI less predictable which is important in some games.

7.9 Machine Learning

The possibility of using Machine Learning algorithms in our AI have been discussed,however due to time restrictions we were unable to implement this in the finalproduct. One of the implementations we considered was using an unsupervised

41

7. Discussion

learning method to model the relations between the NPCs. The idea was to use the“k-means” clustering method on the population, represented by their traits, to splitthe population into k different groups. By making this calculation for every NPC,every one of them would get their own idea of what characters belongs to whichgroups. This is due to the fact that the k-means algorithm starts off by randomizingstarting values.

This method can be extended by adding an extra parameter, representing the im-pressions other NPCs have made on the character in question. This value should beable to change over time, as characters should be able to make new impressions. Byusing this parameter, along with the traits when defining clusters, the NPCs willget more differentiating ideas of the different groups. This would also allow for theclusters to change over time, giving the NPCs a dynamic idea of what group theywant to associate themselves with.

This way of thinking can be considered realistic as it depends on what impressionsyou have of a person as well as what their personality traits are. The fact that thecharacters wants to associate with a certain group, rather than single individuals,makes it so that the opinion it has of others is affected by its current group offriends. The advantage of this model is that it is a more complex representation ofrelationships and as such is able to model more complex relations. The computa-tional complexity of this model is naturally higher than that of representing eachrelation with a scalar, but as the relations would not change often this should notbe a issue.

42

8Conclusion

AI is a very broad scope and even in small projects like this it takes a lot of timeto develop an AI that we consider is intelligent. We managed to develop a gameworld as well as an AI inspired by realism. Not only did we get working NPCsbut also a behaviour that somewhat simulates a society in development. The finalimplementation of the game we created contained a hierarchical FSM with a statestack as the AI approach for controlling the NPCs.

We wanted to create the other approaches that are discussed throughout this reportas well, in order to implement more advanced elements for the AI, such as relationsand planning. Our notion is that the problem solver would have proved to be thebetter implementation and would have given a more interesting behaviour, as itallows for better planning and rational, dynamic decision making. However, theFSM implementation is still considered, by the group, to be the right choice as afirst implementation, as this was the easiest to get running and still achieved a goodbase for us to assess the results on. In hindsight, the group agrees on the fact thatdeveloping a game world was something that took more time than we anticipated.

The main focus of this project was the AI, thus we think that using a pre-existingworld or an engine, like Unity or Unreal Engine, where a world can be created fast,would have been a better option. The time saved could have been devoted to AIdevelopment, consequently leading to us being able to explore additional approachesand compare them.

43

8. Conclusion

44

Bibliography

[1] "Login | Deep Dream Generator", Deepdreamgenerator.com, 2016. [Online].Available: http://deepdreamgenerator.com/generator. [Accessed: 16- May-2016].

[2] “Global Games Market Revenues 2016 | Newzoo", Newzoo, 2016. [On-line]. Available: https://newzoo.com/insights/articles/global-games-market-reaches-99-6-billion-2016-mobile-generating-37/. [Accessed: 26- Apr- 2016].

[3] T. Quandt and S. Kröger, Multiplayer: The Social Aspects of Digital Gaming.New York: Routledge, 2014.

[4] M. Buro, "Call for AI research in RTS Games", http://www.aaai.org/,2016. [Online]. Available: http://www.aaai.org/Papers/Workshops/2004/WS-04-04/WS04-04-028.pdf. [Accessed: 31- May- 2016].

[5] J. McCarthy, M. Minsky, N. Rochester and C. Shannon, "A Proposal for theDartmouth Summer Research Project on Artificial Intelligence, August 31,1955", AI Magazine, vol. 27, no. 4, p. 12, 2006.

[6] M. Childs, "John McCarthy: Computer scientist known asthe father of AI", Independent, 2011. [Online]. Available:http://www.independent.co.uk/news/obituaries/john-mccarthy-computer-scientist-known-as-the-father-of-ai-6255307.html. [Accessed: 28- Apr- 2016].

[7] "Machines Who Think", Pamelamc.com, 2016. [Online]. Available:http://www.pamelamc.com/html/machines_who_think.html. [Accessed:16- May- 2016].

[8] R. Redheffer, "A Machine for Playing the Game Nim", The American Mathe-matical Monthly, vol. 55, no. 6, p. 343, 1948.

[9] W. Saletan, "The triumphant teamwork of hu-mans and computers", Slate, 2007. [Online]. Avail-able: http://www.slate.com/articles/health_and_science/ hu-man_nature/2007/05/chess_bump.html [Accessed: 15- May- 2016].

[10] C. Metz, "Google’s AI Wins Fifth And Final Game Against

45

Bibliography

Go Genius Lee Sedol", Wired, 2016 [Online]. Available:http://www.wired.com/2016/03/googles-ai-wins-fifth-final-game-go-genius-lee-sedol/. [Accessed: 13- Apr- 2016].

[11] S. Gibbs, "Google’s AI AlphaGo to take on world No 1 Lee Se-dol in live broadcast", The Guardian, 2016. [Online]. Available:https://www.theguardian.com/technology/2016/feb/05/google-ai-alphago-world-no-1-lee-se-dol-live-broadcast/. [Accessed: 13 Apr 2016].

[12] D. Kehoe, "Designing Artificial Intelligence for Games (Part 1) | In-tel® Developer Zone" ,Software.intel.com, 2009. [Online]. Available:https://software.intel.com/en-us/articles/designing-artificial-intelligence-for-games-part-1/. [Accessed: 22- Apr- 2016].

[13] S. Russell and P. Norvig, Artificial intelligence, 2nd ed. Englewood Cliffs, N.J.:Prentice Hall, 1995.

[14] "Video Games and Artificial Intelligence - Microsoft Re-search", Research.microsoft.com, 2016. [Online]. Available:http://research.microsoft.com/en-us/projects/ijcaiigames/. [Accessed: 15-May- 2016].

[15] Demis Hassabis, CEO, DeepMind Technologies - The The-ory of Everything", YouTube, 2016. [Online]. Available:https://www.youtube.com/watch?v=rbsqaJwpu6A/. [Accessed: 15- May-2016].

[16] M. Olsson, "Behavior Trees for decision-making in Au-tonomous Driving", M. Sc, KTH Royal Institute ofTechnology, 2016. [Online]. Available: http://www.diva-portal.org/smash/record.jsf?pid=diva2%3A907048&dswid=4279/. [Accessed:15- May- 2016].

[17] A. Champandard, "Top 10 Most Influential AI Games |AiGameDev.com", Aigamedev.com, 2016. [Online]. Available:http://aigamedev.com/open/highlights/top-ai-games/. [Accessed: 15-May- 2016].

[18] "The Elder Scrolls Official Site", Elderscrolls.com, 2016. [Online]. Available:http://www.elderscrolls.com/skyrim. [Accessed: 22- Apr- 2016].

[19] Acadamey of Interactive Arts & Sciences, "Game De-veloper Details", Interactive.org, 2016. [Online]. Available:http://www.interactive.org/games/game_developer_details.asp?idAward=2012&idGameDeveloper=47.[Accessed: 31- May- 2016].

[20] "Best PC Video Games of All Time", Metacritic, 2016. [Online]. Available:http://www.metacritic.com/browse/games/score/metascore/all/pc/filtered?sort=desc.

46

Bibliography

[Accessed: 01- Jun- 2016].

[21] "Slick2D Wiki", Slick.ninjacave.com, 2016. [Online]. Available:http://slick.ninjacave.com/wiki/index.php?title=Main_Page. [Accessed:15- May- 2016].

[22] T. Parr, "Enforcing Strict Model-View Separation in Tem-plate Engines", University of San Francisco. [Online]. Available:http://www.cs.usfca.edu/p̃arrt/papers/mvc.templates.pdf/. [Accessed:15- May- 2016].

[23] T. Reenskaug, "The Model-View-Controller (MVC) Its Pastand Present", University of Oslo, 2003. [Online]. Available:heim.ifi.uio.no/t̃rygver/2003/javazone-jaoo/MVC_pattern.pdf/. [Accessed:15- May- 2016].

[24] "Introduction - Tiled", Doc.mapeditor.org. [Online]. Available:http://doc.mapeditor.org/manual/introduction/. [Accessed: 27- Apr-2016].

[25] G. Booch, J. Rumbaugh and I. Jacobson, The unified modeling language userguide. Upper Saddle River, NJ: Addison-Wesley, 2005.

[26] UML Diagrams - Learn What They Are and How to Make Them",Smartdraw.com, 2016. [Online]. Available: https://www.smartdraw.com/uml-diagram/. [Accessed: 31- May- 2016].

[27] A. Botea, B. Bouzy, M. Buro, C. Bauckhage and D. Nau, "Pathfinding inGames", Dagstuhl Follow-Ups, vol. 6, 2013.

[28] "Introduction to A*", Red Blob Games, 2014. [Online]. Available:http://www.redblobgames.com/pathfinding/a-star/introduction.html/. [Ac-cessed: 15- May- 2016].

[29] J. Brownlee, "Background", Ai-depot.com, 2016. [Online]. Available: http://ai-depot.com/FiniteStateMachines/FSM-Background.html. [Accessed: 15- May-2016].

[30] A. Champandard, "On Finite State Machines and Reusability| AiGameDev.com", Aigamedev.com, 2016. [Online]. Available:http://aigamedev.com/open/article/fsm-reusable/. [Accessed: 16- May-2016].

[31] A. Champandard, "The Gist of Hierarchical FSM |AiGameDev.com", Aigamedev.com, 2016. [Online]. Available:http://aigamedev.com/open/article/hfsm-gist/. [Accessed: 16- May- 2016].

[32] K. Murphy, Machine learning. Cambridge, MA: MIT Press, 2012.

47

Bibliography

[33] T. Hastie, R. Tibshirani and J. Friedman, The elements of statistical learning.New York, NY: Springer, 2016.

[34] A. Maslow, "A theory of human motivation.", Psychological Review, vol. 50,no. 4, pp. 370-396, 1943.

[35] I. Millington and J. Funge, Artificial intelligence for games. Burlington, MA:Morgan Kaufmann/Elsevier, 2009.

[36] "The Elder Scrolls Official Site", Elderscrolls.com, 2016. [Online]. Available:http://www.elderscrolls.com/morrowind. [Accessed: 25- Apr- 2016].

[37] P. Sweetser and J. Wiles, "Current AI in Games: A Review", Australian Jour-nal of Intelligent Information Processing Systems, vol. 8, no. 1, pp. 24-42,2002.

[38] S. Rabin, AI game programming wisdom 2. Hingham, Mass.: Charles RiverMedia, 2006.

[39] S. Rabin, AI game programming wisdom. Hingham, Mass.: Charles RiverMedia, 2002.

[40] M. Jones, Artificial intelligence. Sudbury, Mass.: Jones and Bartlett Publish-ers, 2009.

48

Appendices

I

AImages from the Game

Distributed under Creative Commons 3.0 (CC-BY 3.0).Created by Zabin, Hyptosis, and Danial Cook.Available: http://opengameart.org/content/castle-tiles-for-rpgs

Distributed under Creative Commons 1.0 (CC0).Created by Buch.

III

A. Images from the Game

Distributed under Creative Commons 3.0 (CC-BY-SA 3.0).Created by Casper Nilsson, Johann Charlot, Daniel Eddeland, Stephen Challener,Charles Sanchez, Manuel Riecke, Daniel Armstrong, Lanea Zimmerman and SkylerRobert Colladay.

Distributed under Creative Commons 3.0 (CC-BY-SA 3.0).Created by Casper Nilsson, Johann Charlot, Barbara Rivera, Chris Phillips andLanea Zimmerman.

IV

BSoftware Description

Background

We will create a game where the focus lies on creating characters with believableinteraction and behavior and seeing how they affect the world they inhabit. Thegame will be seen from a top-down 2D-view and will be developed for PC. The worldwill be set in a medieval fantasy world and will be generated to contain resourceslike trees, water, stone and gold. When a world has been generated we will placeone, or several, villages in the world, each with their own set of residents who willhave to survive in the world they have been placed. The player will be able toeither stay in the god view or enter the world as a resident, all other residents willbe controlled by the AIs. The AI will decide what the character it controls will dodepending on its current needs. The needs of a character will also depend on thecharacters traits and skills. An example would be that a character is hungry, thenit will need to either buy food to eat, or, if it does not have enough money, gatherthe food itself. The game will also implement a relation system allowing charactersto have a family, friends or enemies.

Characters

The characters will represent the villagers in the game world. They can be seen as asprite (a 2D-animation) and this animation will show the player what the charactersare currently doing. The character is without intelligence, so without a AI it wouldnever act. In the game’s most simple form, the amount of characters in the worldwill decide how fast the population will increase, that is if we don’t implement amore complex reproduction system around the relationships (see Additional goals.).

The game will show the player the needs, traits and skills of characters, this will showthe player why a character is performing a certain task. The needs of a character willrepresent what it requires to be healthy, and staying alive, such as eating, sleepingand socializing. Skills will decide how good a character is at a task, such as fishingor farming, and in this case being better would give more food in less time. Traitswill affect the needs, for example a character might be gluttonous and therefore

V

B. Software Description

requires more food when hungry. These different stats will make up the personalityof the character.

Every character will also have a inventory where it can store items such as resourcesand money. This will enable different characters to trade with one another. Toperform a trade, the characters must be close to each other and both must agree tothe proposed trade.

The characters should have a age as a way for the player to see how long they haveexisted in the world. Characters should also be able to be born, and die of old age.

AI and NPC

The AI will be the ‘brain’ of our character, without it they wouldn’t be able tosurvive, let alone act. Every character will be controlled by a AI and the AI will tella character how to act depending on its current needs and wishes, this union is ourNPC (Non player character). As the traits and skills of the character affect its needsand wishes they will also affect the AI’s decision of what the character should do.The AI will never be seen in the game, it’s purely code running in the backgroundthat makes the NPCs act.

The focus of the project is making this AI capable of making the characters act ina believable and rational way. This means that the characters must make decisionsbased on their environments and decide the best action to take based on this. Thegame will feature a number of different AI and thus the NPCs will perform differentlyin some situations. The NPCs will react on the actions of other NPC, and also affectthem in return, and interact not make any difference between the player’s characterand other NPCs. How this interaction precedes and what it entails will depend onthe traits of the characters.

World

The world will be a top down 2D-view where all the things in the game plays out.The world will be relatively large and therefore the player will be able to zoom inand out on the map to get different levels of detail. The zoom will be implementedby having 3 different views the player can switch between. In the first, the playerwill be able to see the whole world, but smaller things like NPC’s and houses willbe harder to see. The second view will be zoomed out enough to show an entirevillage, where the player can see what’s going on in the entire village and how it’sdeveloping. The third will be the one with highest detail but shows the least of themap. This view will show the player’s character as well as other NPC’s.

The map will also contain mountains, forests and other places where NPC’s cangather resources like farms and mines. Regardless of where the player is the entire

VI


world will be updated equally.

The world will also feature a system for weather and seasons which affects whatresources the NPCs, and player, can find. There will also be a chance for naturaldisasters occurring that would negatively affect the characters in some way or other.There will also be some kind of animal life in the world, that could potentially workas a resource as well as a threat to the characters.

Structures

The most basic structures, houses, will work as a capacity limit for how manyvillagers there can live in a town, there can’t live more people than what the housingallows. If all the houses are occupied more houses must be built in the town forthe population to be able to keep increasing. There will be additional structures forresources, like mills, lumberyard and mining camps.

Resources

Characters will be able to extract resources from nature on the sites in the worldwhere the resource is available. The resources will exist in three categories:

• Unlimited resources – never runs out no matter how much the villagers extractfrom the source.

• Renewable resources – resource will spread while there is still some source thatcan spread it.

• Limited resources – will only exist in a fixed amount and when it runs out theworld will have to manage without it.

Following are some short description of the resources in the game:

• Wood – trees will exist as a renewable resource since trees will grow up nextto other trees so the forest will expand. NPC’s can chop down trees to getwood to sell or work with, hence its categorized as a resource, but if all thetrees in the world are cut down then the resource will not come back.

• Water – an infinite resource which will exists in different forms, such as oceansand rivers. NPC’s can gather water in these locations or build wells which willbe required to sate their thirst. NPC’s can also catch fish to gather food anduse water to improve the agriculture.

• Stone – stone will exist and can be gathered.• Gold – gold will exist and be able to be extracted from mines. It can be used

in trading with other NPC’s.• Food – food will be created in farms or caught from fishing and hunting. It

will be used to keep NPC’s from starving and can also be traded.

VII


Player interaction

The player will be able to either actively interact with the world and the NPC’s bytaking control of a character in the same way that the AI’s do. Or simply view theworld and the NPC’s without being able to influence them or having to worry aboutthe needs of a character. In either mode of playing the player will be able to viewthe world in any of three different views that are listed under the world section.

Additional goals

We hope to implement relations between NPC’s so they can make friends, familyand enemies. The relations will depend on characters traits and somewhat on theskills as well. This is not a requirement since humans can make connections withpeople who have different traits and interests in the real world. If we do implementthis, then the population in the village will depend on how many families living inthe village, also housing will be restricted to family members and so the capacityof houses will not be maximized, a stranger can’t move in just because there’s anextra bed. Another goal is implement conflicts between NPC’s, they might fight overresources or because two family has some kind of rivalry or simply personal issues.Interactions like these can result in death. We are also thinking about spawningmore than one village in the world to see how residents in different villages interactwith and behave to one another.

We also discussed to implement “fog of war”, this will cover the map in darkness if noNPC is currently in that area. If this was the case, NPC’s wouldn’t be all knowingand therefore would have to explore to find resources and check areas where they hadbeen before to know if resources were still there, someone could have taken it whilethe NPC was away. This would make it possible for NPC’s to share informationwith each other to know where resources were most recently collected.

A last goal is to make a tool for the player to create a game world or generate arandom one. Then the player could test scenarios they were interested in or simplyfast get into a new unique world.

Functional Requirements

The following sections will state the planned functionality of the game in the formof functional requirements.

VIII


Characters

1. Characters should have needs.2. The magnitude of the needs should increase over time.3. Characters should have skills.4. The skills should increase when used.5. Characters should have traits.6. The traits should affect the needs.7. The traits should be constant and do not change over time.8. Characters should have an inventory.9. Characters should be born.10. Characters should be able to die.11. Characters should eventually die of old age.

Rationale Characters requires needs and traits so the AI can decide what the charac-ter must do. The needs must decrease over time to continuously present a challengefor the one controlling the character. The traits must somehow affect the needs inorder to have a direct influence on the character. They also need skills to determinehow good they are at different tasks which will affect the best way to fulfill theirneeds. Inventory will be needed to keep track of what resources a character has andthus required for trading. Character must be born and be able to die of age or otherreasons so that the population, and state of the world, will change over time.

AI and NPC

12. All AI should be able to control one character each.13. The AI should make decisions based on the current needs of the character.14. The AI should make decisions based on the current goals of the character.15. The effect on the needs and wishes of the character, that the traits and skills

have, should affect the AI’s decisions.16. The AI should make decisions based on the environment around the character

it controls.17. The AI should take the actions of other NPCs into account when deciding on

actions for the NPC.

Rationale If the AI cannot control a character it has no purpose there will be noNPCs in the game. For the AI to be ‘believable’ it must make decisions based onthe needs, wishes, traits and skills of the controlled character. It must also take intoaccount the environment around the character and the actions of others to appear‘believable’.

IX


World

18. The player should be able to zoom in and out between three different levels.19. The world should contain different resources.20. The world should contain characters.21. The world should contain structures.22. The world should have a basic weather system.23. The world should contain animal life.24. The entire world should be constantly evolve and change regardless of where

the player is.

Rationale The world needs to be possible to view, and as the world should be largedifferent levels of detail are necessary. The world needs characters for the game tobe possible to play. The world must also contain resources so that the characterscan fulfill their needs and survive, less they all die out immediately. The worldshould contain structures as they set the max capacity for the villages population.A basic weather system will be implemented to make the lives of the villagers harderby randomly spawning harsh weather and natural disasters, this is to take some ofthe power of control from the AIs and make them need to adapt to change. Thereshould also be some kind of animal life in the world, again to take some power fromthe AIs in that they may unexpectedly encounter wild animals or gather a resourcethat can move (i.e. hunting). As all characters should have equal importance in theworld, it must progress even when the player is not present.

Structures

25. Characters should be able to construct structures.26. There should be different kinds of structures filling different functions.27. There should be structures that act as houses.28. The number of houses in the village should set the max population of the

village.29. There should be structures that can act as stockpiles for resources.

Rationale For a village to be more than just a collection of characters it needsto have some kind of structures. For the village to behave in a believable way itsmaximum population must depend on the size of the village. The possibility to buildstructures gives the NPCs a possibility to make decisions that have an immediatecost but positive effects in in the long run.

Resources

30. There should be infinite resources in the world.31. There should be renewable resources in the world.

X


32. There should be finite resources in the world.33. Resources should be possible to be collected from set positions in the world.34. The amount of a finite resource at a certain location should decrease if col-

lected.35. The amount of a finite resource at a certain location should be able to be

depleted.36. The amount of a renewable resource at a certain location should decrease if

collected.37. The amount of a renewable resource at a certain location should increase over

time.38. The amount of a renewable resource at a certain location should be able to be

depleted.39. If a renewable resource has been depleted at a certain location, then its amount

at that location should no longer increase.

Rationale For the characters to be able to survive they must be able to collectresources to satisfy their needs. For the collection of resources to present a challengefor the NPCs, or the player, some of them must be able to be depleted, and somelimited so that they NPCs must plan ahead to be successful.

Player interaction

40. The player should be able to control a character .41. The NPC’s should interact with the player controlled character in the same

ways as with each other.42. The player should be able to view the world without controlling a character.43. The player should be able to view the world from the three different views

stated above.

Rationale For the player to be able to test how the NPCs react to different actionstaken by characters it is necessary for the player to be able to control a character,and for the NPCs to interact with this character in the same way as with each other.As controlling a character and taking care of its needs can distract the player fromthe development of the world, it is necessary for the player to be able to view theworld even without controlling a character. As there are several different things onemight want to observe at different times, the world needs to be able to be viewedfrom different views.

Non-Functional Requirements

The NPCs should appear believable as humans. The decisions made by the AIsshould always be relevant to the needs, traits, goals and skills of the controlledcharacter. The game should present information to the player in a clear way. The

XI


calculations made by the AIs should be efficient enough that the addition of NPCsshould not drastically affect the running speed of the game.

XII

CStates of Early AI Implementation

A list of the states that were in the early implementation of the AI.

• Idle - This state was used to plan the AIs next move, as well as popping thenext state from the state stack making it the current state that would then beexecuted in update() of ArtificialBrain. The AI will enter idle state in betweenstates. It was also used to check if the character was hungry, thirsty or tired.This was later moved to update() of ArtificalBrain for performance reasons.

• Hungry - The AI would enter this state if hunger fell below a certain threshold.The AI would first check if the character had food in its inventory. If it did itwould consume it. If it didn’t, the AI would push food onto the gather stackand look for that.

• Thirsty - The AI would enter this state if thirst fell below a certain threshold.The AI would first check if the character had water in its inventory. If it did itwould consume it. If it didn’t, the AI would push water onto the gather stackand look for that.

• LowEnergy - The AI would enter this state when energy fell below a certainthreshold. If the AI had a house it would go home to sleep, if it didn’t it wouldrest where it stood.

• Gather - This state is used to gather resources. The AI would check what thefirst resource on the gather stack was, it would then go look for that resource.If there was no resource on the gather stack, the AI would randomize a resourceto look for.

– Material - Specific state that would be entered once the AI reached acertain resource. This state was used to gather material and put it in theinventory. This was later split up into specific gathering states for eachresource.

– Food - Same functionality as Material, but for food.

– Water - Same functionality as Material, but for water.

XIII

C. States of Early AI Implementation

• Eat - In this state the AI will consume a food item from its inventory toreplenish hunger.

• Drink - In this state the AI will consume water from its inventory to replenishthirst.

• Sleep - In this state the AI will sleep to replenish energy. This state could onlybe executed when inside a house.

• Resting - In this state the AI will rest to replenish energy. This was not aseffective as the sleep state, but could be performed anywhere. It was used asa means to replenish energy before the AI had built a house.

XIV

DStates of Final AI Implementation

A complete list of all the states in the final AI implementation of this project.

• Idle - The AI enters idle state between states. It is used to pop the next statefrom the stack. The idle state contains the functionality to decide what theAI should do next. This is hardcoded, to make sure the AI does things in anappropriate order. It will start by building a house. If it has a house it willproceed to build a farm or a stockpile. If it has built a stockpile and a farmit will either hunt, work on a farm or gather the resource of which it has thelowest amount in its inventory.

• Build - In this state the AI will check what resources it needs in order to builda specific building. It will then push those resources onto the gather stack andfind resource stack (if those resources are not present in the AI’s memory).

• Drink - In this state the AI will consume water from its inventory to replenishthirst.

• Eat - In this state the AI will consume a food item from its inventory toreplenish hunger.

• Sleep - In this state the AI will sleep to replenish energy. This state can onlybe entered when in a house.

• FindResource - In this state the AI will walk in a random direction until itfinds what it is looking for. This state is used to find resources it does notcurrently have in memory.

• Follow - This state is used to follow moving objects around the world; such asanimals. It is used to catch up to an animal the AI is hunting. It was earlierused to catch up to characters as well, but this functionality was removedwhen it was decided not to have social interaction in the game.

• Gather - This state is used to decide what it is to gather if there is nothing onthe gather stack. It is used for transitioning to the specific gather states foreach resource. It is used to determine which resource is the closest, if there isone. If there isn’t, it is used to transition into Gather<Resource>.

• Gather<Resource> - Each resource has its own specific gather state. Thesestates are used to actually gather the resource from the node and put it in itsinventory, once the AI has reached the resource in question. For instance if itis to gather wood and has reached a tree, the AI would enter the GatherWoodstate.

XV

D. States of Final AI Implementation

• Hungry - The AI will enter this state if hunger falls below a certain threshold.The AI will first check if the character had food in its inventory. If it does itwill consume it. If it doesn’t, the AI will push food onto the gather stack andlook for it.

• Thirsty - Same functionality as hunger, but instead of food it will consume orlook for water.

• LowEnergy - The AI will enter this state when energy falls below a certainthreshold. If the AI has a house it will go home to sleep, if it doesn’t it willrest where it stands. Resting - In this state the AI will rest to replenish energy.Resting is not as effective as sleeping, but can be done anywhere.

• Moving - In this state the AI finds and follows a path to a destination in theworld. The destination is set by a previous state which then transitions intothe Moving state to move to that destination.

• Hunting - In this state the AI will hunt for animals. This state contains its ownmovement and works similar to FindResource but for animals. If an animal isspotted the AI goes into the Follow state.

• WorkFarm - In this state the AI will work on a farm. When an AI works ona farm, crops will grow that can be consumed to replenish hunger.

XVI

realistic npcs in video games using different ai...

Documents