giochi e beni culturali

8/16/2019 Giochi e Beni Culturali

1/20

The 10th International Symposium on Virtual Reality, Archaeology and Cultural Heritage VAST - State of the Art Reports (2009)M. Ashley and F. Liarokapis (Editors)

Serious Games in Cultural Heritage

Eike Falk Anderson1, Leigh McLoughlin2, Fotis Liarokapis1, Christopher Peters1, Panagiotis Petridis3, Sara de Freitas3

1Interactive Worlds Applied Research Group (iWARG), Coventry University, United Kingdom2The National Centre for Computer Animation (NCCA), Bournemouth University, United Kingdom

3Serious Games Institute (SGI), Coventry University, United Kingdom

Abstract

Although the widespread use of gaming for leisure purposes has been well documented, the use of games to support cultural heritage purposes, such as historical teaching and learning, or for enhancing museum visits, has been lesswell considered. The state-of-the-art in serious game technology is identical to that of the state-of-the-art in en-tertainment games technology. As a result the field of serious heritage games concerns itself with recent advancesin computer games, real-time computer graphics, virtual and augmented reality and artificial intelligence. On theother hand, the main strengths of serious gaming applications may be generalised as being in the areas of com-munication, visual expression of information, collaboration mechanisms, interactivity and entertainment. In thisreport, we will focus on the state-of-the-art with respect to the theories, methods and technologies used in seriousheritage games. We provide an overview of existing literature of relevance to the domain, discuss the strengths and weaknesses of the described methods and point out unsolved problems and challenges. In addition, several casestudies illustrating the application of methods and technologies used in cultural heritage are presented.

Categories and Subject Descriptors (according to ACM CCS): H.5.1 [Information Interfaces and Presentation]: Mul-timedia Information Systems—Artificial, augmented, and virtual realities I.2.1 [Artificial Intelligence]: Applica-

tions and Expert Systems—Games K.3.1 [Computers and Education]: Computer Uses in Education—Computer-assisted instruction K.8.0 [Personal Computing]: General—Games

1. Introduction

Computer games with complex virtual worlds for entertain-ment are enjoying widespread use and in recent years wehave witnessed the introduction of serious games, includingthe use of games to support cultural heritage purposes, suchas historical teaching and learning, or for enhancing museumvisits. At the same time, game development has been fuelled

by dramatic advances in computer graphics hardware – inturn driven by the success of video games – which have ledto a rise in the quality of real-time computer graphics and in-creased realism in computer games. The successes of gamesthat cross over into educational gaming – or serious gam-ing, such as the popular Civilization (although “abstract andahistorical” [App06]) and Total War series of entertainmentgames, as well as games and virtual worlds that are specif-ically developed for educational purposes, such as Revolu-tion [Fra06] and the Virtual Egyptian Temple [JH05], all of

which exist within a cultural heritage context, reveal the po-tential of these technologies to engage and motivate beyondleisure time activities.

The popularity of video games, especially among youngerpeople, makes them an ideal medium for educational pur-poses [ML87]. As a result there has been a trend towardsthe development of more complex, serious games, which are

informed by both pedagogical and game-like, fun elements.The term ‘serious games’ describes a relatively new concept,computer games that are not limited to the aim of providingentertainment, that allow for collaborative use of 3D spacesthat are used for learning and educational purposes in a num-ber of application domains. Typical examples are game en-gines and online virtual environments that have been used todesign and implement games for non-leisure purposes, e.g.in military and health training [Mac02, Zyd05], as well ascultural heritage (Figure 1).

c 2009. This work is licensed under the creative commons attribution 3.0 Unported Li-cense. creativecommons.org


2/20

Anderson et al. / Serious Games in Cultural Heritage

Figure 1: Experiencing ‘Rome Reborn’ as a game.

This report explores the wider research area of interac-tive games and related applications with a cultural heritagecontext and the technologies used for their creation. Mod-ern games technologies (and related optimisations [CD09])

allow the real-time interactive visualisation/simulation of re-alistic virtual heritage scenarios, such as reconstructions of ancient sites and monuments, while using relatively basicconsumer machines. Our aim is to provide an overview of the methods and techniques used in entertainment gamesthat can potentially be deployed in cultural heritage contexts,as demonstrated by particular games and applications, thusmaking cultural heritage much more accessible.

Serious games can exist in the form of mobile applica-tions, simple web-based solutions, more complex ‘mashup’applications (e.g. combinations of social software applica-tions) or in the shape of ‘grown-up’ computer games, em-ploying modern games technologies to create virtual worlds

for interactive experiences that may include socially basedinteractions, as well as mixed reality games that combinereal and virtual interactions, all of which can be used incultural heritage applications. This state-of-the-art report fo-cusses on the serious games technologies that can be foundin modern computer games.

The report is divided into two main sections:

• The first of these is concerned with the area of culturalheritage and serious games, which integrate the core tech-nologies of computer games with principled pedagogicalmethodologies. This is explored in a range of characteris-tic case studies, which include entertainment games that

can be used for non-leisure purposes as well as virtualmuseums and educationally focused and designed culturalheritage projects.

• The second part investigates those computer games tech-nologies that are potentially useful for the creation of cul-tural heritage games, such as real-time rendering tech-niques, mixed reality technologies and subdomains of (game) artificial intelligence. This literature review in-cludes discussions of strengths and weaknesses of themost prominent methods, indicating potential uses for cul-

tural heritage serious games and illustrating challenges intheir application.

2. The State-of-the-Art in Serious Games

The state-of-the-art in Serious Game technology is identicalto the state-of-the-art in Entertainment Games technology.Both types of computer game share the same infrastructure,or as Zyda notes, “applying games and simulations technol-ogy to non-entertainment domains results in serious games”[Zyd05]. The main strengths of serious gaming applicationsmay be generalised as being in the areas of communication,visual expression of information, collaboration mechanisms,interactivity and entertainment.

Over the past decade there have been tremendous ad-vances in entertainment computing technology, and “today’sgames are exponentially more powerful and sophisticatedthan those of just three or four years ago” [Saw02], which

in turn is leading to very high consumer expectations. Real-time computer graphics can achieve near-photorealism andvirtual game worlds are usually populated with considerableamounts of high quality content, creating a rich user experi-ence. In this respect, Zyda [Zyd05] argues that while peda-gogy is an implicit component of a serious game, it shouldbe secondary to entertainment, meaning that a serious gamethat is not ‘fun’ to play would be useless, independent of itspedagogical content or value. This view is not shared by all,and there exist design methodologies for the developmentof games incorporating pedagogic elements, such as the fourdimensional framework [dFO06], which outlines the central-ity of four elements that can be used as design and evaluationcriteria for the creation of serious games. In any case there is

a need for the game developers and instructional designersto work together to develop engaging and motivating seriousgames for the future.

2.1. Online Virtual Environments

There is a great range of different online virtual world ap-plications – at least 80 virtual world applications existedin 2008 with another 100 planned for 2009. The field isextensive, not just in terms of potential use for educationand training but also in terms of actual usage and uptakeby users, which is amply illustrated by the online platformSecond Life (Linden Labs), which currently has 13 million

registered accounts worldwide. The use of Second Life forsupporting seminar activities, lectures and other educationalpurposes has been documented in a number of recent re-ports and a wide range of examples of Second Life use byUK universities has been documented [Kir08]. Online vir-tual worlds provide excellent capabilities for creating ef-fective distance and online learning opportunities throughthe provision of unique support for distributed groups (on-line chat, the use of avatars, document sharing etc.). Thisbenefit has so far been most exploited in business where

c 2009. This work is licensed under the creative commons attribution 3.0 Unported License. creativecommons.org


3/20


these tools have been used to support distributed or location-independent working groups or communities [Jon05]. On-line virtual worlds in this way facilitate the development of new collaborative models for bringing together subject mat-ter experts and tutors from around the world, and in termsof learning communities are opening up opportunities forlearning in international cohorts where students from morethan one country or location can learn in mixed reality con-texts including classroom and non-classroom based groups(https://lg3d-wonderland.dev.java.net). Online vir-tual worlds also notably offer real opportunities for training,rehearsing and role playing.

2.2. Application to Cultural Heritage: Case Studies

This section provides an overview of some of the mostcharacteristic case studies in cultural heritage. In particularthe case studies have been categorised into three types of computer-game-like applications including: prototypes and

demonstrators; virtual museums; and commercial historicalgames.

2.2.1. Prototypes and Demonstrators

The use of visualisation and virtual reconstruction of ancient historical sites is not new, and a number of projects have used this approach to study crowd modelling[ADG∗08, MHY∗07]. Several projects are using virtual re-constructions in order to train and educate their users. Manyof these systems have, however, never been released to thewider public, and have only been used for academic studies.In the following section the most significant and promisingof these are presented.

Figure 2: ‘Rome Reborn’ Serious Game.

2.2.1.1. Rome Reborn

The Rome Reborn project is the world’s largest digitisa-tion project and has been running for 15 years. The mainaims of the project are to produce a high resolution versionof Rome at 320 AD (Figure 2), a lower resolution modelfor creating a ‘mashup’ application with ‘Google Earth’(http://earth.google.com/rome/), and finally the col-laborative mode of the model for use with virtual world ap-plications and aimed primarily at education [Fri08].

In order to investigate the efficacy of the Rome RebornProject for learning, exploration, re-enactment and researchof cultural and architectural aspects of ancient Rome a seri-ous game is currently under development. In particular, theproject aims at investigating the suitability of using this tech-nology to support the archaeological exploration of histori-cally accurate societal aspects of Rome’s life, with an em-phasis on political, religious and artistic expressions.

To achieve these objectives, the project will integrate fourcutting-edge virtual world technologies with the Rome Re-born model, the most detailed three-dimensional model of Ancient Rome available. These technologies include:

• the Quest3D visualisation engine [God08]• Instinct(maker) artificial life engine (Toulouse University)

[SBLD04]• ATOM Spoken Dialogue System

(http://www.agilingua.com)• High resolution, motion captured characters and objects

from the period (Red Bedlam).The use of the Instinct artificial life engine enables coher-ent crowd animation and therefore the population of the cityof Rome with behaviour driven virtual characters. These vir-tual characters with different behaviours can teach the playerabout different aspects of life in Rome (living conditions,politics, military) [SBLD04]. Agilingua ATOM’s dialoguemanagement algorithm allows determining how the systemwill react: asking questions, making suggestions, and/or con-firming an answer.

This project aims to develop a researchers’ toolkit for al-lowing archaeologists to test past and current hypothesessurrounding architecture, crowd behaviour, social interac-

tions, topography and urban planning and development, us-ing Virtual Rome as a test-bed for reconstructions. By us-ing such game the researches will be able to analyse theimpact of major events. For example, the use of this tech-nique would allow researchers to analyse the impact of major events, such as grain distribution or the influx of peopleinto the city . The experiences of residents and visitors asthey pass through and interact with the ancient city can alsobe explored.

2.2.1.2. Ancient Pompeii

Pompeii was a Roman city, which was destroyed and com-pletely buried in the first recorded eruption of the volcanoMount Vesuvius in 79 AD [PCSa, PCSb]. For this project

a model of ancient Pompeii was constructed and populatedwith avatars in order to simulate life in Pompeii in real-time.The main goal of this project was to simulate a crowd of virtual Romans exhibiting realistic behaviours in a recon-structed district of Pompeii [MHY∗07]. The virtual entitiescan navigate freely in several buildings in the city model andinteract with their environment [ADG∗08].

2.2.1.3. Parthenon Project

The Parthenon Project is a short computer animation that


https://lg3d-wonderland.dev.java.net/https://lg3d-wonderland.dev.java.net/https://lg3d-wonderland.dev.java.net/http://earth.google.com/rome/http://earth.google.com/rome/http://www.agilingua.com/http://www.agilingua.com/http://www.agilingua.com/http://earth.google.com/rome/https://lg3d-wonderland.dev.java.net/


4/20


“visually reunites the Parthenon and its sculptural decora-tions” [Deb05]. The Parthenon itself is an ancient monu-ment, completed in 437 BC, and stands in Athens whilemany of its sculptural decorations reside in the collection of the British Museum, London (UK). The project goals wereto create a virtual version of the Parthenon and its separatedsculptural elements so that they could be reunited in a virtualrepresentation.

The project involved capturing digital representations of the Parthenon structure and the separate sculptures, recom-bining them and then rendering the results. The structure wasscanned using a commercial laser range scanner, while thesculptures were scanned using a custom 3D scanning systemthat the team developed specifically for the project [Tch02].The project made heavy use of image-based lighting tech-niques, so that the structure could be relit under differentillumination conditions within the virtual representation. Aseries of photographs were taken of the structure togetherwith illumination measurements of the scene’s lighting. Aninverse global illumination technique was then applied toeffectively ‘remove’ the lighting. The resulting “lighting-independent model” [DTG∗04] could then be relit using anylighting scheme desired [TSE∗04, DTG∗04].

Although the Parthenon Project was originally an offline-rendered animation, it has since been converted to work inreal-time [SM06, IS06]. The original Parthenon geometryrepresented a large dataset consisting of 90-million polygons(after post-processing), which was reduced to 15-million forthe real-time version and displayed using dynamic level-of-detail techniques. Texture data consisted of 300MB andhad to be actively managed and compressed, while 2.1GBof compressed High-Dynamic-Range (HDR) sky maps werereduced in a pre-processing step. The reduced HDR mapswere used for lighting and the extracted sun position wasused to cast a shadow map.

2.2.2. Virtual Museums

Modern interactive virtual museums using games technolo-gies [JC02, LV04] provide a means for the presentation of digital representations for cultural heritage sites [EHML∗06]that entertain and educate visitors [HCB∗01] in a much moreengaging manner than was possible only a decade ago. Arecent survey paper that examines all the technologies andtools used in museums was recently published [SLKP09].

Here we present several examples of this type of culturalheritage serious game, including some virtual museums thatcan be visited in real-world museums.

2.2.2.1. Virtual Egyptian Temple

This game depicts a hypothetical Virtual Egyptian Temple[JH05], which has no real-world equivalent. The temple em-bodies all of the key features of a typical New Kingdom pe-riod Egyptian temple in a manner that an untrained audiencecan understand. It is divided into four major areas, each one

of which houses an instance of the High Priest, a pedagogi-cal agent. Each area of this virtual environment represents adifferent feature from the architecture of that era.

The objective of the game is to explore the model andgather enough information to answer the questions asked

by the priest (pedagogical agent). The game engine that thissystem is based on is the Unreal Engine 2 (Figure 3) [JL05],existing both as an Unreal Tournament 2004 game modifi-cation [Wal07] for use at home, as well as in the form of aCave Automatic Virtual Environment (CAVE [CNSD∗92])system in a real museum.

Figure 3: New Kingdom Egyptian Temple game.

2.2.2.2. The Ancient Olympic Games

The Foundation of the Hellenic World has produced a num-ber of gaming appliucations associated with the OlympicGames in ancient Greece [GCP04]. For example, in the‘Olympic Pottery Puzzle’ exhibit the user must re-assemblea number of ancient vases putting together pot shards. Theusers are presented with a colour-coded skeleton of the ves-sels with the different colours showing the correct positionof the pieces. They then try to select one piece at a time froma heap and place it in the correct position on the vase. An-other game is the ‘Feidias Workshop’ which is a highly in-teractive virtual experience taking place at the constructionsite of the 15-meter-tall golden ivory statue of Zeus, one of the seven wonders of the ancient world. The visitors enterthe two-storey-high workshop and come into sight of an ac-curate reconstruction of an unfinished version of the famousstatue of Zeus and walk among the sculptor’s tools, scaf-folding, benches, materials, and moulds used to construct it.They take the role of the sculptor’s assistants and activelyhelp finish the creation of the huge statue, by using virtual

tools to apply the necessary materials onto the statue, pro-cess the ivory and gold plates, apply them onto the woodensupporting core and add the finishing touches. Interaction isachieved using the navigation wand of the Virtual Realiyy(VR) system, onto which the various virtual tools are at-tached. Using these tools the user helps finish the work onthe statue, learning about the procedures, materials and tech-niques applied for the creation of these marvellous statues.The last example is the ‘Walk through Ancient Olympia’,where the user, apart from visiting the historical site, learns



5/20


Figure 4: Walk through Ancient Olympia [GCP04].

about the ancient games themselves by interacting with ath-letes in the ancient game of pentathlon (Figure 4). The vis-itors can wonder around and visit the buildings and learn

their history and their function: the Heraion, the oldest mon-umental building of the sanctuary dedicated to the goddessHera, the temple of Zeus, a model of a Doric peripteral tem-ple with magnificent sculpted decoration, the Gymnasium,which was used for the training of javelin throwers, dis-cus throwers and runners, the Palaestra, where the wrestlers, jumpers and boxers trained, the Leonidaion, which waswhere the official guests stayed, the Bouleuterion, where ath-letes, relatives and judges took a vow that they would up-held the rules of the Games, the Treasuries of various cities,where valuable offerings were kept, the Philippeion, whichwas dedicated by Philip II, king of Macedonia, after his vic-tory in the battle of Chaeronea in 338 BC and the Stadium,

where most of the events took place. Instead of just observ-ing the games the visitors take place in them. They can pickup the discus or the javelin and they try their abilities inthrowing them towards the far end of the stadium. Excitedabout the interaction they ask when they will be able to in-teract with the wrestler one on one. A role-playing model of interaction with alternating roles was tried here with prettygood success as the visitors truly immersed in the environ-ment wish they could participate in more games [GCP04].

2.2.2.3. Virtual Priory Undercroft

Located in the heart of Coventry, UK, the Priory Under-crofts are the remains of Coventry’s original Benedictinemonastery, dissolved by Henry VIII. Although archaeolo-

gists revealed the architectural structure of the cathedral, thecurrent site is not easily accessible for the public. Virtual Pri-ory Undercroft offers a virtual exploration of the site in bothonline and offline configurations.

Furthermore, a first version of a serious game (Figure 5)has been developed at Coventry University, using the Object-Oriented Graphics Rendering Engine (OGRE) [WM08].Themotivation is to raise the interest of children in the mu-seum, as well as cultural heritage in general. The aim of

the game is to solve a puzzle by collecting medieval objectsthat used to be located in and around the Priory Undercroft.Each time a new object is found, the user is prompted toanswer a question related to the history of the site. A typ-ical user-interaction might take the form of: “What did St.George slay? – Hint: It is a mythical creature. – Answer: TheDragon”, meaning that the user then has to find the Dragon.

Figure 5: Priory Undercroft – a Serious Game

2.2.3. Commercial Historical Games

Commercial games with a cultural heritage theme are usu-ally of the ‘documentary game’ [Bur05b] genre that depictreal historical events (frequently wars and battles), which thehuman player can then partake in. These are games that wereprimarily created for entertainment, but their historical ac-curracy allows them to be used in educational settings aswell.

2.2.3.1. History Line: 1914-1918

An early representative of this type of game was History

Line: 1914-1918 (Blue Byte, 1992), an early turn-basedstrategy game depicting the events of the First World WarThe game was realised using the technology of the moreprominent game Battle Isle, providing players with a 2D top-down view of the game world, divided into hexagons thatcould be occupied by military units, with the gameplay verymuch resembling traditional board-games.

The game’s historical context was introduced in a long(animated) introduction, depicting the geo-political situationof the period and the events leading up to the outbreak of war in 1914. In between battles the player is provided withadditional information on concurrent events that shaped thecourse of the conflict, which is illustrated with animations

and newspaper clippings from the period.

2.2.3.2. Great Battles of Rome

More recently a similar approach was used by the His-tory Channel’s Great Battles of Rome (Slitherine Strategies,2007), another ‘documentary game’, which mixes interac-tive 3D real-time tactical simulation of actual battles withdocumentary information (Figure 6), including footage orig-inally produced for TV documentaries, that places the battlesin their historical context.



6/20


Figure 6: Great Battles of Rome.

2.2.3.3. Total War

The most successful representatives of this type of histor-ical game are the games of the Creative Assembly’s Total

War series, which provide a gameplay combination of turn-based strategy (for global events) and real-time tactics (forbattles). Here, a historical setting is enriched with informa-tion about important events and developments that occurredduring the timeframe experienced by the player. While thefree-form campaigns allow the game’s players to change thecourse of history, the games also include several independentbattle-scenarios with historical background information thatdepict real events and allow players to partake in momentsof historical significance.

The use of up-to-date games technology for rendering, aswell as the use of highly detailed game assets that are reason-ably true to the historical context, enables a fairly realistic

depiction of history. As a result, games from the Total Warseries have been used to great effect in the visualisation of armed conflicts in historical programmes produced for TV[War07].

Figure 7: Reliving the battle of Brandywine Creek [ McG06 ].

The latest title in the series, Empire Total War (released inMarch 2009), depicting events from the start of the 18th cen-tury to the middle of the 19th century, makes use of some of the latest developments in computer games technology (Fig-ure 7). The game’s renderer is scalable to support different

types of hardware, including older systems, especially oldergraphics cards (supporting the programmable Shader Model2), but the highest visual fidelity is only achieved on re-cent systems (Shader Model 3 graphics hardware) [Gar09].If the hardware allows for this, shadows for added realismin the virtual world are generated using Screen Space Ambi-ent Occlusion [Mit07, BS08], making use of existing depth-buffer information in rendered frames. Furthermore the vir-tual world of the game is provided with realistic vegetationgenerated by the popular middleware system SpeedTree (In-teractive Data Visualization, Inc.), which “features realis-tic tree models and proves to be able to visualise literallythousands of trees in real-time” [FK04]. As a result the hu-man player is immersed in the historical setting, allowing theplayer to re-live history.

3. The Technology of Cultural Heritage Serious Games

Modern interactive virtual environments are usually imple-

mented using game engines, which provide the core technol-ogy for the creation and control of the virtual world. A gameengine is an open, extendable software system on which acomputer game or a similar application can be built. It pro-vides the generic infrastructure for game creation [Zyd05],i.e. I/O (input/output) and resource/asset management facil-ities. The possible components of game engines include, butare not limited to: rendering engine, audio engine, physicsengine, animation engine.

3.1. Virtual World System Infrastructure

The shape that the infrastructure for a virtual environmenttakes is dictated by a number of components, defined by

function rather than organisation, the exact selection of which determines the tasks that the underlying engine is suit-able for. A game engine does not provide data or functionsthat could be associated with any game or other applicationof the game engine [ZDA03]. Furthermore, a game engine isnot just an API (Application Programming Interface), i.e. aset of reusable components that can be transferred betweendifferent games, but also provides a glue layer that connectsits component parts. It is this glue layer that sets a game en-gine apart from an API, making it more than the sum of itscomponents and sub-systems.

Modern game engines constitute complex parallel sys-tems that compete for limited computing resources [Blo04].

They “provide superior platforms for rendering multipleviews and coordinating real and simulated scenes as wellas supporting multiuser interaction” [LJ02], employing ad-vanced graphics techniques to create virtual environments.Anderson et al. [AEMC08] provide a discussion of sev-eral challenges and open problems regarding game engines,which include the precise definition of the role of contentcreation tools in the game development process and as partof game engines, as well as the identification of links be-tween game genres and game engine architecture, both of



7/20


which play a crucial role in the process of selecting an ap-propriate game engine for a given project.

Frequently, the technology used for the development of virtual environments, be they games for entertainment, se-rious games or simulations, is limited by the development

budget. Modern entertainment computer games frequentlyrequire “a multimillion-dollar budget” [Ove04] that cannow rival the budgets of feature film productions, a signif-icant proportion of which will be used for asset creation(such as 3D models and animations). Developers are usu-ally faced with the choice of developing a proprietary in-frastructure, i.e. their own game engine, or to use an ex-isting engine for their virtual world application. Commer-cially developed game engines are usually expensive, andwhile there are affordable solutions, such as the Torque gameengine which is favoured by independent developers andwhich has been successfully used in cultural heritage appli-cations [LWH∗07, MSLV08], these generally provide fewerfeatures, thus potentially limiting their usefulness. If one of the project’s requirements is the use of highly realistic graph-ics with a high degree of visual fidelity, this usually requiresa recent high-end game engine, the most successful of whichusually come at a very high licensing fee.

There are alternatives, however, as several older commer-cially developed engines have been released under OpenSource licences, such as the Quake 3 engine (id Tech 3)[ST08, WM08], making them easily accessible, and whilethey do not provide the features found in more recently pub-lished games, they nevertheless match the feature sets of thecheaper commercial engines. Furthermore, there exist opensource game engines such as the Nebula Device [RM03], orengine components, such as OGRE [RM03, WM08] or ODE

(Open Dynamics Engine) [MW03], which are either com-mercially developed or close to commercial quality, makingthem a viable platform for the development of virtual worlds,although they may lack the content creation tools that arefrequently packaged with larger commercial engines.

Finally, there is the possibility of taking an existing gameand modifying it for one’s own purposes, which many re-cent games allow users to do [Wal07, ST08]. This has thebenefit of small up-front costs, as the only requirement isthe purchase of a copy of the relevant game, combinedwith access to high-spec modern game engines, as wellas the content development tools that they contain. Ex-amples for this are the use of the game Civilization III

for the cultural heritage game The History Game Canada(http://historycanadagame.com) or the use of the Un-real Engine 2 [ST08] for the development of an affordableCAVE [JL05], which has been used successfully in culturalheritage applications [JH05].

3.2. Virtual World User Interfaces

There are different types of interface that allow users to in-teract with virtual worlds. These fall into several different

categories, such as VR and augmented reality (AR), severalof which are especially useful for cultural heritage applica-tions, and which are presented in this section.

3.2.1. Mixed Reality Technologies

In 1994, Milgram [MK94] tried to depict the relationship be-tween VR and AR. To illustrate this he introduced two newterms called Mixed Reality (MR), which is a type of VR buthas a wider concept than AR, [TYK01] and Augmented Vir-tuality (AV). On the left hand side of the Reality-Virtualitycontinuum, there is the representation of the real world andon the right hand side there is the ultimate synthetic envi-ronment. MR stretches out in-between these environmentsand it can be divided into two sub-categories: AR and AV[MK94]. AR expands towards the real world and thus it isless synthetic than AV which expands towards virtual envi-ronments. To address the problem from another perspectivea further distinction has been made. This refers to all the objects that form an AR environment: real objects and virtual

objects. Real objects are these, which always exist no mat-ter what the external conditions may be. On the other hand,a virtual object depends on external factors but mimics objects of reality. Three of the most interesting characteristicsbetween virtual and real objects are illustrated here.

The first and most obvious difference is that a virtual object has to be viewed through a display device after it hasbeen generated and then simulated. On the contrary, real objects, since they exist in essence, can be viewed either di-rectly or through a synthetic device. The second differenceconcentrates on the quality of the viewed image that is gen-erated by using state-of-the-art technologies. More specifi-cally, virtual information cannot be sampled directly but can

be synthesised. Therefore, the quality of the resulting object may look real but this does not guarantee that the objectis real. In addition, the virtual and real information can bedistinguished depending on the luminosity of the locationwhere it appears. Real images have some luminosity at thelocation at which it appears to be located while virtual im-ages do not have any at the location at which it appears. Thisdefinition includes direct viewing of a real object, as wellas the image on the display screen of a non-directly viewedobject. Examples of virtual images include holograms andmirror images [MK94] .

3.2.2. Virtual Reality

Ivan Sutherland originally introduced the first Virtual Re-ality (VR) system in the 1960s [Sut65]. Nowadays VR ismoving from the research laboratories to the working en-vironment by replacing ergonomically limited HMD’s withprojective displays (such as the well known CAVE and Re-sponsive Workbench) as well as online VR communities. Ina typical VR system the user’s natural sensory information iscompletely replaced with digital information. The user’s ex-perience of a computer-simulated environment is called im-mersion. As a result, VR systems can completely immerse a


http://historycanadagame.com/http://historycanadagame.com/http://historycanadagame.com/http://historycanadagame.com/


8/20


user inside a synthetic environment by blocking all the sig-nals of the real world. In addition, a VR simulated worlddoes not always have to obey all laws of nature. In immer-sive VR systems, the most common problems of VR systemsare of emotional and psychological nature including motionsickness, nausea, and other symptoms, which are created bythe high degree of immersiveness of the users.

Moreover, internet technologies have the tremendous po-tential of offering virtual visitors ubiquitous access viathe World Wide Web (WWW) to online virtual envi-ronments. Additionally, the increased efficiency of In-ternet connections (i.e. ADSL/broadband) makes it pos-sible to transmit significant media files relating to theartefacts of virtual museum exhibitions. The most popu-lar technology for WWW visualisation includes Web3Dwhich offers tools such as the Virtual Reality ModelingLanguage (VRML – http://www.web3d.org/x3d/vrml/)and its successor X3D (http://www.web3d.org/x3d/),

which can be used for the creation of an interac-tive virtual museum. Many cultural heritage applica-tions based on VRML have been developed for theweb [Gat00, PEHBP01, SM01]. Another 3D graphics for-mat, is COLLAborative Design Activity (COLLADA –https://collada.org/ which defines an open standardXML schema (http://www.w3.org/XML/Schema) for ex-changing digital assets among various graphics software ap-plications that might otherwise store their assets in incom-patible formats. One of the main advantages of COLLADAis that is includes more advanced physics functionality suchas collision detection and friction (which Web3D does notsupport).

In addition to these, there are more powerfultechnologies that have been used in museum en-vironments, which include the OpenSceneGraph(OSG) high performance 3D graphics toolkit(http://www.openscenegraph.org/projects/osg)and a variety of 3D game engines. OSG is a freely available(open source) multi-platform toolkit, used by museums[CCF∗05, LGSB06] to generate more powerful VR appli-cations, especially in terms of immersion and interactivitysince it supports the integration of text, video, audio and3D scenes into a single 3D environment. An alternativeto OpenSceneGraph, is OpenSG which is an open-sourcescene graph system used to create real-time VR applications

(http://www.opensg.org/) On the other hand, 3D gameengines are also very powerful and they provide superiorvisualisation and physics support. Both technologies (OSGand 3D game engines), compared to VRML and X3D,can provide very realistic and immersive museum envi-ronments but they have two main drawbacks. First, theyrequire advanced programming skills in order to designand implement custom applications. Secondly, they do nothave support for mobile devices such as PDAs and ThirdGeneration mobile phones.

3.2.3. Augmented Reality

The concept of AR is the opposite of the closed world of virtual spaces [TYK99] since users can perceive both vir-tual and real information. Most AR systems use more com-plex software approaches compared to VR systems. The ba-

sic theoretical principle is to superimpose digital informa-tion directly into a user’s sensory perception [Fei02], ratherthan replacing it with a completely synthetic environment asVR systems do. An interesting point is that both technolo-gies may process and display the same digital informationand often they make use of the same dedicated hardware.Although AR systems are influenced by the same factors theamount of influence is much less than in VR since only aportion of the environment is virtual. However, there is stilla lot of research to be done in AR [Azu97] to measure accu-rately its effects on humans.

The requirements related to the development of AR ap-plications in the cultural heritage field have been well doc-

umented [BAEB99, LSM08, SLKP09]. An interactive con-cept is the Meta-Museum visualised guide system based onAR, which tries to establish scenarios and provide a commu-nication environment between the real world and cyberspace[MKN96]. Another AR system that could be used as an au-tomated tour guide in museums is the automated tour guide,which superimposes audio in the world based on the locationof the user [Bed95]. There are many ways where archaeo-logical sources can be used to provide a mobile AR system.Some of the wide range of related applications includes theinitial collection of data to the eventual dissemination of in-formation [Rya00]. MARVINS is an AR assembly, initiallydesigned for mobile applications and can provide orienta-

tion and navigation possibilities in areas, such as sciencemuseums, art museums and other historic or cultural sites.Augmented information like video, audio and text is relayedfrom a server via the transmitter-receiver to a head-mounteddisplay [SCFS00].

In addition, a number of EU projects have been under-taken in the field of virtual heritage. The SHAPE project[HCB∗01] combined AR and archaeology to enhance theinteraction of persons in public places like galleries and mu-seums by educating visitors about artefacts and their history.The 3DMURALE project [CIG∗01] developed 3D multime-dia tools to record, reconstruct, encode and visualise archae-ological ruins in virtual reality using as a test case the an-

cient city of Sagalassos in Turkey. The Ename 974 project[PCKS00] developed a non-intrusive interpretation systemto convert archaeological sites into open-air museums, calledTimeScope-1 based on 3D computer technology originallydeveloped by IBM, called TimeFrame. ARCHEOGUIDE[SDS∗01] provides an interactive AR guide for the visual-isation of archaeological sites based on mobile computing,networking and 3D visualisation providing the users with amulti-modal interaction user interface. A similar project isLIFEPLUS [PPM∗02], which explores the potential of AR


http://www.web3d.org/x3d/vrml/http://www.web3d.org/x3d/vrml/http://www.web3d.org/x3d/http://www.web3d.org/x3d/https://collada.org/http://www.w3.org/XML/Schemahttp://www.w3.org/XML/Schemahttp://www.openscenegraph.org/projects/osghttp://www.openscenegraph.org/projects/osghttp://www.openscenegraph.org/projects/osghttp://www.opensg.org/http://www.opensg.org/http://www.opensg.org/http://www.openscenegraph.org/projects/osghttp://www.w3.org/XML/Schemahttps://collada.org/http://www.web3d.org/x3d/http://www.web3d.org/x3d/vrml/


9/20


so that users can experience a high degree of realistic inter-active immersion by allowing the rendering of realistic 3Dsimulations of virtual flora and fauna (humans, animals andplants) in real-time

Moreover, AR can be applied successfully for gaming in

cultural heritage. One of the earliest examples is the VirtualShowcase [BFSE01] which is an AR display device that hasthe same form factor as a real showcase traditionally used formuseum exhibits and can be used for gaming. The potentialsof AR interfaces in museum environments and other culturalheritage institutions [Lia07] as well as outdoor heritage sites[VIK∗02] have been also briefly explored for potential edu-cational applications. A more specific gaming example is theMAGICand TROC systems [RNBP04] which were based ona study of the tasks of archaeological fieldwork, interviewsand observations in Alexandria. That’s a mobile game andthe players discover archaeological objects while moving.

Another cultural heritage AR application is the serious

game SUA that was part of the BIDAIATZERA project[LCM∗07]. This project takes the form of a play whichrecreates the 1813 battle between the English and the Frenchin San Sebastian. Researchers developed an interactive sys-tem based on AR and VR technologies for recreational andeducational applications with tourist, cultural and socio-economical contents, the prototype for which was presentedat the Museo del Monte Urgull in San Sebastian.

3.3. Advanced Rendering Techniques

One of the most important elements of the creation of in-teractive virtual environments is the visual representationof these environments. Although serious games have de-sign goals that are different from those of pure entertain-ment video games, they can still make use of the wide va-riety of graphical features and effects that have been devel-oped in recent years. The state-of-the-art in this subject areais broad and, at times, it can be difficult to specify exactlywhere the ‘cutting edge’ of the development of an effect lies.A number of the techniques that are currently in use wereoriginally developed for offline applications and have onlyrecently become adopted for use in real-time applicationsthrough improvements in efficiency or hardware. Here, the‘state-of-the-art’ for real-time lags several years behind thatfor offline – good examples of this would be raytracing orglobal illumination, which we shall briefly examine. A num-

ber of effects, however, are developed specifically for imme-diate deployment on current hardware and can make use of specific hardware features – these are often written by hard-ware providers themselves to demonstrate their use or, of course, by game developers. Other real-time graphical fea-tures and effects can be considered to follow a developmentcycle, where initially they are proven in concept demonstra-tions or prototypes, but are too computationally expensive toimplement in a full application or game. These are usuallydeveloped by academics or blue-sky research departments

within games companies. They may then be progressivelyoptimised for speed, or held back until the development of faster hardware.

The primary reason for the proliferation of real-timegraphics effects has been due to advances in low-cost graph-

ics hardware that can be used in standard PCs or gamesconsoles. Modern graphics processing units (GPUs) areextremely powerful parallel processors and the graphicspipeline is becoming increasingly flexible. Through the useof programmable shaders, which are small programs that de-fine and direct part of the rendering process, a wide variety of graphical effects are now possible for inclusion in games andvirtual environments, while there also exist a range of effectsthat are currently possible but still too expensive for practicaluse beyond anything but the display of simple scenes.

The graphics pipeline used by modern graphics hard-ware renders geometry using rasterisation, where an objectis drawn as triangles which undergo viewing transformations

before they are converted directly into pixels. In contrast,ray-tracing generates a pixel by firing a corresponding rayinto the scene and sampling whatever it may hit. While theformer is generally faster, especially using the hardware ac-celeration on modern graphics cards, it is easier to achieveeffects such as reflections using ray-tracing. Although theflexibility of modern GPUs can allow ray-tracing [PBMH02]in real-time [HSHH07, Shi06], as well as fast ray-tracingnow becoming possible on processors used in games con-soles [BWSF06], rasterisation is currently still the standardtechnique for computer games.

Although the modern graphics pipeline is designed andoptimised to rasterise polygonal geometry, it should be noted

that other types of geometry exist. Surfaces may be de-fined using a mathematical representation, while volumesmay be defined using ‘3D textures’ of voxels or, again, us-ing a mathematical formula [EHK∗06]. The visualisation of volumetric ‘objects’, which are usually semi-opaque, is acommon problem that includes features such as smoke, fogand clouds. A wide variety of options exist for renderingvolumes [EHK∗06, CPCP∗05], although these are generallyvery computationally expensive and it is common to emu-late a volumetric effect using simpler methods. This ofteninvolves drawing one or more rectangular polygons to whicha four-channel texture has been applied (where the fourth, al-pha, channel represents transparency) – for example a cloudelement or wisp of smoke. These may be aligned to always

face the viewer as billboards [AMHH08], a common gametechnique with a variety of uses [WP05], or a series of thesemay be used to slice through a full volume at regular in-tervals. An alternative method for rendering full volumes isray-marching, where a volume is sampled at regular inter-vals along a viewing ray, which can now be implementedin a shader [CNLE09], or on processors that are now beingused in games consoles [KJ09].

It is sometimes required to render virtual worlds, or ob-



10/20


jects within worlds, that are so complex or detailed that theycannot fit into the graphics memory, or even the main mem-ory, of the computer – this can be especially true when deal-ing with volume data. Assuming that the hardware cannotbe further upgraded, a number of options exist for such ren-dering problems. If the scene consists of many complex objects at varying distances, it may be possible to adopt a level-of-detail approach [EHK∗08] and use less complex geome-try, or even impostors [AMHH08], to approximate distantobjects [SM06]. Alternatively, if only a small sub-sectionof the world or object is in sight at any one time, it maybe possible to hold only these visible parts in memory and‘stream’ replace them as new parts come into view, whichis usually achieved by applying some form of spatial parti-tioning [CNLE09]. This streaming approach can also be ap-plied to textures that are too large to fit into graphics mem-ory [MG08]. If too much is visible at one time for this tobe possible, a cluster of computers may be used, where theentire scene is often too large for a single computer to hold

in memory but is able to be distributed among the clusterwith the computers’ individual renders being accumulatedand composited together [HHN∗02] or each computer con-trolling part of a multi-screen tile display [YJSZ06].

3.3.1. Post-Processing Effects

One important category of graphical effect stems from theability to render to an off-screen buffer, or even to multiplebuffers simultaneously, which can then be used to form afeedback loop. A polygon may then be drawn (either to ad-ditional buffers or to the visible framebuffer) with the previ-ously rendered texture(s) made available to the shader. Thisshader can then perform a variety of ‘post-processing’ ef-

fects.Modern engines frequently include a selection of such

effects [Fei07], which can include more traditional imageprocessing, such as colour transformations [Bur05a, Bjo04],glow [JO04], or edge-enhancement [ND03], as well as tech-niques that require additional scene information such asdepth of field [Gil07, ZCP07], motion blur [Ros08] and oth-ers which will be mentioned in specific sections later.

The extreme of this type of technique is deferred shad-ing, where the entire lighting calculations are performed asa ‘post-process’. Here, the scene geometry is rendered into aset of intermediate buffers, collectively called the G-buffer,and the final shading process is performed in image-space

using the data from those buffers [Koo08].

3.3.2. Transparency, Reflection and Refraction

The modern real-time graphics pipeline does not deal withthe visual representation of transparency, reflection or re-fraction and their emulation must be dealt with using spe-cial cases or tricks. Traditionally, transparency has beenemulated using alpha blending [AMHH08], a compositingtechnique where a ‘transparent pixel’ is combined with the

framebuffer according to its fourth colour component (al-pha). The primary difficulty with this technique is that theresults are order dependent, which requires the scene ge-ometry to be sorted by depth before it is drawn and trans-parency can also present issues when using deferred shading[FM08]. A number of order-independent transparency tech-niques have been developed, however, such as depth-peeling[Eve01, NK03].

Figure 8: Achieving a mirror effect by rendering the geom-etry twice [ AM07 ].

Mirrored background reflections may be achieved usingan environment map [BN76, WP05], which can be a simplebut effective method of reflecting a static scene. If the sceneis more dynamic, but relatively fast to render, reflections ona flat surface may be achieved by drawing the reflective sur-face as transparent and mirroring the entire scene geometryabout the reflection surface, drawing the mirrored geome-

try behind it (Figure 8) or, for more complex scenes, usingreduced geometry methods such as impostors [TI06]. Alter-natively, six cameras can be used to produce a dynamic en-vironment map [Bly06]. Alternative methods have also beendeveloped to address the lack of parallax, i.e. apparent mo-tion offsets due to objects at different distances, which aremissing in a fixed environment map [YYM05].

Perhaps surprisingly on first note, simple refraction ef-fects can be achieved using very similar techniques to thoseused for reflection. The only differences are that the sampleray direction points inside the object and that it is bent dueto the difference in refractive indices of the two materials,in accordance with Snell’s Law [AMHH08]. Thus, environ-

ment mapping can be used for simple refractions in a staticscene, which may be expanded to include chromatic disper-sion [FK03]. In some cases, refraction may also be achievedas a post-processing effect [Wym07].

3.3.3. Surface Detail

The simplest method of adding apparent detail to a sur-face, without requiring additional geometry, is texture map-ping. The advent of pixel shaders means that textures can



11/20


now be used in more diverse ways to emulate surface detail[Ros06, WP05, AMHH08].

A variety of techniques exist for adding apparent high-resolution bump detail to a low-resolution mesh. In normalmapping [Bli78] the texture map stores surface normals,

which can then be used for lighting calculations. Parallaxmapping [KTI∗01] uses a surface height map and the cam-era direction to determine an offset for texture lookups. Re-lief texture mapping [OBM00, WP05] is a related techniquewhich performs a more robust ray-tracing of the height mapand can provide better quality results at the cost of perfor-mance.

3.3.4. Lighting

The old fixed-function graphics pipeline supported aper-vertex Gouraud lighting model [OSW∗07], but pro-grammable shaders now allow the developer to imple-ment their own lighting model [Ros06, Hof06]. In general,though, the fixed-function lighting equation is split into: adiffuse component, where direct lighting is assumed to bescattered by micro-facets on the surface; a specular com-ponent, which appears as a highlight and is dependent onthe angle between the viewer and the light; and an ambientcomponent, which is an indirect ‘background’ lighting com-ponent due to light that has bounced off other objects in thescene [AMHH08].

3.3.4.1. Shadows

Although the graphics pipeline did not originally supportshadows, it does now provide hardware acceleration for tex-ture samples of a basic shadow map [AMHH08, EHK∗08].

However, this basic method suffers from aliasing issues, istypically low resolution and can only result in hard shadowedges. Except in certain conditions, the majority of shadowsin the real world exhibit a soft penumbra, so there is a desirewithin computer graphics to achieve efficient soft shadows,for which a large number of solutions have been developed[HLHS03, Bav08]. Shadowing complex objects such as vol-umes can also present issues, many of which have also beenaddressed [LV00, HKSB06, RKH08].

3.3.4.2. High-Dynamic-Range Lighting

HDR Lighting is a technique that has become very popularin modern games [She06, EHK∗08]. It stems from the fact

that real world luminance has a very high dynamic range,which means that bright surface patches are several ordersof magnitude brighter than dark surface patches – for exam-ple, the sun at noon “may be 100 million times brighter thanstarlight” [RWPD06]. In general, this means that the 8-bitintegers traditionally used in each component of the RGBtriplet of pixels in the framebuffer, are woefully inadequatefor representing real luminance ranges. Thankfully, modernhardware nowallows a greater precision in data types, so thatcalculations may be performed in 16 or even 32-bit floating-

point format, although it should be noted that a performancepenalty usually occurs when using more precise formats.

One of the most striking visual effects associated withHDR lighting is bloom, where extremely bright patches ap-pear to glow. Practically, this is usually applied as a post-

process effect in a similar way to a glow effect, where brightpatches are drawn into a separate buffer which is blurred andthen combined with the original image [Kaw04, Kaw03].This can also be applied to low-dynamic-range images, tomake them appear HDR [Sou05].

Modern displays still use the traditional 8-bit per colourcomponent format (with a few exceptions [SHS∗04]), sothe HDR floating point results must be converted, which isthe process of tonemapping [RWPD06]. Some tonemappingmethods allow the specification of a brightness, or exposurevalue as taken from a physical camera analogy. In an envi-ronment where the brightness is likely to change dramati-cally this exposure should be automatically adjusted – much

like a real camera does today. Various methods are availableto achieve this, such as by downsampling the entire image toobtain the average brightness [Kaw04], or by asynchronousqueries to build a basic histogram of the brightness level todetermine the required exposure [MGM06, SH07].

3.3.4.3. Indirect Lighting: Global Illumination

Incident light on a surface can originate either directly froma light source, or indirectly from light reflected by anothersurface. Global illumination techniques account for both of these sources of light, although in such methods it is the indi-rect lighting component that is usually of most interest andthe most difficult to achieve. The main difficulty is that in

order to render a surface patch, the light that is reflectedby all other surface patches in the scene must be known.This interdependence can be costly to compute, especiallyfor dynamic scenes, and although indirect lighting accountsfor a high proportion of real world illumination, the com-putational cost of simulating its effects has resulted in verylimited use within real-time applications. [DBB03]

The simplest inclusion of indirect lighting is through pre-computed and baked texture maps, which can store anythingfrom direct shadows or ambient occlusion results to thosefrom radiosity or photon mapping [Mit07]. However, thistechnique is only viable for completely static objects withina static scene. Another simple global illumination technique,

which is commonly associated with HDR lighting, is image-based lighting [RWPD06]. Here, an environment map storesboth direct and indirect illumination as a simple HDR im-age, which is then used to light objects in the scene. The im-age may be captured from a real-world location, drawn by anartist as an art asset or generated in a pre-processing stage bysampling the virtual environment. Multiple samples can thenbe used to light a dynamic character as it moves through the(static) environment [MMG06]. Although the results can bevery effective, image-based lighting cannot deal with fully



12/20


dynamic scenes without having to recomputed the environ-ment maps, which may be costly.

Fully dynamic global illumination techniques generallywork on reduced or abstracted geometry, such as using discsto approximate the geometry around each vertex for am-

bient occlusion [SA07, HJ08b]. It is also possible to per-form some operations as a post-process, such as ambient oc-clusion [Mit07] and even approximations for single-bounceindirect lighting [RGS09]. The general-purpose use of theGPU has also allowed for radiosity at near real-time for verysmall scenes [CH05] and fast, but not yet real-time, photonmapping [PDC∗03]. The latter technique can also be usedto simulate caustics, which are bright patches due to con-vergent rays from a refractive object, in real-time on theGPU [KBW06], although other techniques for specificallyrendering caustics are also possible [WS03], including as animage-space post-process effect [Wym07].

3.4. Artificial Intelligence

Another important aspect of the creation of populated virtualenvironments as used in cultural heritage applications is thecreation of intelligent behaviour for the inhabitants of thevirtual world, which is achieved using artificial intelligence(AI) techniques.

It is important to understand that when we refer to theAI of virtual entities in virtual environments, that which werefer to is not truly AI – at least not in the conventionalsense [McC07] of the term. The techniques applied to vir-tual worlds, such as computer games, are usually a mixtureof AI related methods whose main concern is the creation

of a believable illusion of intelligence [Sco02], i.e. the be-haviour of virtual entities only needs to be believable to con-vey the presence of intelligence and to immerse the humanparticipant in the virtual world.

The main requirement for creating the illusion of intelli-gence is perception management, i.e. the organisation andevaluation of incoming data from the AI entity’s environ-ment. This perception management mostly takes the formof acting upon sensor information but also includes commu-nication between or coordination of AI entities in environ-ments which are inhabited by multiple entities which mayhave to act co-operatively. The tasks which need to be solvedin most modern virtual world applications such as computer

games and to which the intelligent actions of the AI entitiesare usually restricted to (by convention rather than technol-ogy) are [And03]:

• decision making• path finding (planning)• steering (motion control)

The exact range of problems that AI entities within a com-puter game have to solve depends on the context in whichthey exists and the virtual environment in which the game

takes place. Combs and Ardoint [CA04] state that a popu-lar method for the implementation of game AI is the use of an ‘environment-based programming style’, i.e. the creationof the virtual game world followed by the association of AIcode with the game world and the entities that exist in it.This means that the AI entity intelligence is built around andis intrinsically linked to the virtual game environment. Thistype of entity intelligence can be created using ‘traditional’methods for ‘decision making’, ‘path finding’ and ‘steering’.

Of the three common AI tasks named above, ‘decisionmaking’ most strongly implies the use of intelligence. Finitestate machines (FSMs) are the most commonly used tech-nique for implementing decision making in games [FH04].They arrange the behaviour of an AI entity in logical states– defining one state per possible behaviour – of which onlyone, the entity’s behaviour at that point in time, is active atany one time. In game FSMs each state is usually associatedwith a specific behaviour and an entity’s actions are oftenimplemented by linking behaviours with pre-defined anima-tion cycles for the AI entity that allow it to enact the selectedbehaviour [Ork06]. It is relatively simple to program a verystable FSM that may not be very sophisticated but that “willget the job done”. The main drawback of FSMs is that theycan become very complex and hard to maintain, while on theother hand the behaviour resulting from a too simple FSMcan easily become predictable. To overcome this problemsometimes hierarchical FSMs are used that break up com-plex states into a set of smaller ones that can be combined,allowing the creation of larger and more complex FSMs.

In recent years, there has been a move towards performingdecision making using goal-directed techniques to enable thecreation of nondeterministic behaviour. Dybsand describesthis as a technique in which an AI entity “will execute a se-ries of actions ... that attempt to accomplish a specific objec-tive or goal” [Dyb04]. In its simplest form, goal-orientationcan be implemented by determining a goal with an embed-ded action sequence for a given AI entity. This action se-quence, the entity’s plan, will then be executed by the entityto satisfy the goal [Ork04a]. Solutions that allow for morediverse behaviour can improve this by selecting an appropri-ate plan from a pre-computed ‘plan library’ [Eva01] insteadof using a built-in plan. More complex solutions use plansthat are computed dynamically, i.e. ‘on the fly’, as is thecase with Goal-Oriented Action Planning (GOAP) [Ork04a].In GOAP the sequence of actions that the system needs to

perform to reach its end-state or goal is generated in real-time by using a planning heuristic on a set of known valueswhich need to exist within the AI entity’s domain knowl-edge. To achieve this in his implementation of GOAP, Orkin[Ork04b] separates the actions and goals, implicitly integrat-ing preconditions and effects that define the planner’s searchspace, placing the decision making process into the domainof the planner. This can be further improved through aug-menting the representation of the search space by associatingcosts with actions that can satisfy goals, effectively turning



13/20


the AI entity’s knowledge base into a weighted graph. Thisthen allows the use of path planning algorithms that find theshortest path within a graph as the planning algorithm forthe entity’s high-level behaviour [Ork06]. This has the addi-tional benefit of greater code re-use as the planning methodfor high-level decision making, as well as path planning isthe same and can therefore be executed by the same codemodule [Ork04b] if the representations of the search spaceare kept identical. The most popular path planning algorithmused in modern computer games is the A* (A-Star) algo-rithm [Sto00, Mat02, Nar04], a generalisation of Dijkstra’salgorithm [Dij59]. A* is optimal, i.e. proven to find the op-timal path in a weighted graph if an optimal solution exists[DP85], which guarantees that AI entities will find the leastcostly path if such a solution exists within the search space.

Challenges in game AI that are relevant to serious gamesinclude the construction of intelligent interfaces [LC04],such as tutoring systems or virtual guides, and particularlyreal-time strategy game AI, part of which is concerned withthe modelling of great numbers of virtual entities in largescale virtual environments. Challenges there include spa-tial and temoral reasoning [Bur04], which can be addressedthrough the use of potential fields [HJ08a].

3.4.1. Crowd Simulation

The AI techniques described in the previous section are im-portant tools with which more complex systems can be con-structed. A domain of great potential relevance to culturalheritage that is derived from such techniques is the simu-lation of crowds of humanoid characters. If one wishes toreconstruct and visualise places and events from the past,a crowd of real-time virtual characters, if appropriately at-tired and behaving, can add new depths of immersion andrealism to ancient building reconstructions. These charac-ters can feature merely as a backdrop [CUCT04] to add lifeto a reconstruction, or can assume the centre stage in moreactive roles, for example, as virtual tour guides to directthe spectator [DeL99]. Indeed, the type of crowd or char-acter behaviour to be simulated varies greatly with respectto the type of scenario that needs to be modelled. In thisvein, [UT02] model crowd behaviour of worshippers in avirtual mosque, while [MHY∗07] and [RFD05] focus on thecreation of more general pedestrian crowd behaviours, theformer for populating a virtual reconstruction of a city re-sembling ancient Rome.

More general crowd synthesis and evaluation techniquesare also directly applicable to crowd simulation in culturalheritage. A variety of different approaches have been taken,most notably the use of social force models [HM95], pathplanning [LD04], behavioural models incorporating percep-tion and learning [ST05] sociological effects [MT97] andhybrid models [PAB07].

The study of real world corpus has also been used as a ba-sis for synthesising crowd behaviour in approaches that do

not entail the definition of explicit behaviour models. Lerneret al. [LCD07] manually track pedestrians from an inputvideo containing real world behaviour examples. They usethis data to construct a database of pedestrian trajectories fordifferent situations. At runtime, the database is queried forsimilar situations matching those of the simulated pedestri-ans: the closest matching example from the database is se-lected as the resulting trajectory for each pedestrian and theprocess is repeated.

Lee et al. [LCHL07] simulate behaviours based on aerial-view video recordings of crowds in controlled environments.A mixture of manual annotation and semi-automated track-ing provides information from video about individuals’ trajectories. These are provided as inputs to an agent movementmodel that can create crowd behaviours of a similar natureto those observed in the original video.

Human perception of the animation of crowds and char-acters has been increasingly recognised as an important fac-

tor in achieving more realistic simulations. Research hasbeen conducted regarding the perception of animation andmotion of individuals [RP03, MNO07], groups and crowds[PEMO08, EPO08]. For example, [PEMO08] examined theperceptual plausibility of pedestrian orientations and foundthat participants were able to consistently distinguish be-tween those virtual scenes where the character orientationsmatched the orientations of the humans in the correspondingreal scenes and those where the character orientations wereartificially generated, according to a number of different ruletypes.

A key factor of differentiation between crowd controlmethods concerns where knowledge is stored in the system.One approach is to endow knowledge separately to indi-vidual characters, an extreme example of which would cre-ate autonomous agents that have their own artificial percep-tions, reasoning, memories, etc with respect to the environ-ment, as in [LD04]. Another method is to place knowledgeinto the environment itself, to create a shared or partially-shared database accessible to characters. According to thissmart object methodology [PDMNO03], graphical objectsare tagged with behavioural information and may inform,guide or even control characters. Such an approach is ap-plicable also to crowd simulation in urban environments.For example, navigation aids, placed inside the environ-ment description, may be added by the designer during

the construction process. These have been referred to asannotations [DHR98]. The resulting environment descrip-tion [FBT99, TD00, PO09] contains additional geometric,semantic and spatial partitioning information for informingpedestrian behaviour, thus transferring a degree of the be-havioural intelligence into the environment. In [Hos02], forexample, skeletal splines are defined that are aligned withwalkways. These splines, called ribbons, provide explicit in-formation for groups to use, such as the two major directionsof travel on the walkway. In addition to environment anno-



14/20


tation and mark-up, interfaces for managing the definitionof crowd scenarios have also been investigated. Crowdbrush[UdHCT04] provides an intuitive way for designers to addcrowds of characters into an environment using tools anal-ogous to those found in standard 2D painting packages. Itallows designers to paint crowds and apply attributes andcharacteristics using a range of different tools in real-time,obtaining immediate feedback about the results.

3.4.2. Annotated Entities and Environments

A fairly recent method for enabling virtual entities to interactwith one another as well as their surroundings is the use of annotated worlds. The mechanism for this, which we refer tousing the term ‘Annotated Entities’, has been described us-ing various names, such as ‘Smart Terrain’ [Cas02], ‘SmartObjects’ [PDMNO03, Ork06] and ‘Annotated Environment’[Doy02], all of which are generally interchangeable andmostly used with very similar meanings, although slight dif-ferences in their exact interpretation sometimes remain. Acommon aspect to all of the implementations that utilise thismechanism is the indirect approach to the creation of believ-able intelligent entities.

The idea of annotated environments is a computer ap-plication of the theory of affordance (or affordance theory)[COST03] that was originally developed in the fields of psy-chology and visual perception. Affordance theory states thatthe makeup and shape of objects contains suggestions abouttheir usage. Affordance itself is an abstract concept, the im-plementation of which is greatly simplified by annotationsthat work like labels containing instructions which providean explicit interpretation of affordances. Transferred into thecontext of a virtual world, this means that objects in the envi-ronment contain all of the information that an AI controlledentity will need to be able to use them, effectively makingthe environment ‘smart’.

A beneficial side effect of this use of ‘annotated’ objects[Doy99] is that the complexity of the entities is neutral tothe extent of the domain knowledge that is available for theiruse, i.e. the virtual entities themselves can not only be keptrelatively simple, but they do not need to be changed at allto be able to make use of additional knowledge. This al-lows for the rapid development of game scenarios [COST03]and if all annotated objects use the same interface to provideknowledge to the world’s entities then there is no limit to thescalability of the system, i.e. the abilities of AI controlled

entities can practically be extended indefinitely [Ork02] de-spite a very low impact on the system’s overall performance.Furthermore, this method provides an efficient solution tothe ‘anchoring problem’ [CS99] of matching sensor data tothe symbolic representation of the virtual entity’s knowledgeas objects in the world themselves have the knowledge as tohow other virtual entities can interact with them.

Annotations have been employed in several differenttypes of applications in order to achieve different effects.

They have proven popular for the animation of virtual ac-tors in computer animation production, where they facili-tate animation selection [LCL06], i.e. the choice of appro-priate animation sequences that fit the environment. Otheruses of annotations include the storage of tactical informa-tion in the environment for war games and military simula-tions [Dar07], which is implemented as sensory annotationsto direct the virtual entities’ perception of their environment.Probably the most common form of annotations found inreal-time simulated virtual environments affects behaviourselection, usually in combination with animation selection[Ork06], i.e. the virtual entity’s behaviour and its visual rep-resentation (animation) are directed by the annotated objectsthat it uses.

Virtual entities that inhabit these annotated worlds can bebuilt utilising rule-based system based on simple FSMs incombination with a knowledge interface based on a triggersystem that allows the entities to ‘use’ knowledge (instruc-tions) for handling the annotated objects. The interactionprotocol employed to facilitate the communication betweenentity and ‘smart’ object needs to enable the object to ‘ad-vertise’ its features to the entities and then allow them to re-quest from the object relevant instructions (annotations) onits usage [Mac00]. The success of this technique is demon-strated by the best-selling computer game The Sims, where‘Smart Objects’ were used for behaviour selection to greateffect. Forbus and Wright [FW01] state that in The Sims allgame entities, objects as well as virtual characters, are im-plemented as scripts that are executed in their own threadswithin a multitasking virtual machine. A similar approach,based on a scripting language that can represent the be-haviours of virtual entities, as well as the objects that the

can interact with, has been presented more recently by An-derson [And08]. These scripting-language based approachesare most likely to provide solutions for the creation of largescale virtual environments, such as the serious game compo-nent of the Rome Reborn project. This is the automatic gen-eration of AI content [Nar07], which in combination withtechniques such as procedural modelling of urban environ-ments [VAW∗09], will require the integration of the creationof complex annotations with the procedural generation of virtual worlds, automating the anchoring of virtual entitiesinto their environment.

4. Conclusions

The success of computer games, fuelled among other factorsby the great realism that can be attained using modern con-sumer hardware, and the key techniques of games technol-ogy that have resulted from this, have given way to new typesof games, including serious games, and related applicationareas, such as virtual worlds, mixed reality, augmented real-ity and virtual reality. All of these types of application utilisecore games technologies (e.g. 3D environments) as well asnovel techniques derived from computer graphics, human



15/20


computer interaction, computer vision and artificial intelli-gence, such as crowd modelling. Together these technologieshave given rise to new sets of research questions, often fol-lowing technologically driven approaches to increasing lev-els of fidelity, usability and interactivity.

Our aim has been to use this state-of-the-art report todemonstrate the potential of games technology for culturalheritage applications and serious games, to outline key prob-lems and to indicate areas of technology where solutions forremaining challenges may be found. To illustrate that firstwe presented some characteristic case studies illustrating theapplication of methods and technologies used in cultural her-itage. Next, we provided an overview of existing literature of relevance to the domain, discussed the strengths and weak-nesses of the described methods and pointed out unsolvedproblems and challenges. It is our firm belief that we areonly at the beginning of the evolution of games technologyand that there will be further improvements in the qualityand sophistication of computer games, giving rise to seriousheritage games of greater complexity and fidelity than is nowachievable.

5. Acknowledgements

The authors would like to thank the following: The HerbertArt Gallery & Museum (Coventry, UK), Simon Bakkevig,and Lukasz Bogaj. This report includes imagery generatedusing the Virtual Egyptian Temple, which is a product of PublicVR (http://publicvr.org).

References

[ADG∗08] ARNOLD D., DAY A., GLAUERT J., HAEGLER S. ,JENNINGS V., KEVELHAM B. , LAYCOCK R. , MAGNENAT-THALMANN N., MAM J. , MAUPU D., PAPAGIANNAKIS G. ,THALMANN D., YERSIN B., , RODRIGUEZ-E CHAVARRIA K .:Tools for populating cultural heritage environments with interac-tive virtual humans. In Open Digital Cultural Heritage Systems,EPOCH Final Event Rome (2008). 3

[AEMC08] ANDERSON E. F., ENGEL S., MCLOUGHLIN L. ,COMNINOS P.: The case for research in game engine architec-ture. In Future Play ’08: Proceedings of the 2008 Conference onFuture Play (2008), pp. 228–231. 6

[AM07] ANDERSON E. F., MCLOUGHLIN L.: Critters in theclassroom: a 3d computer-game-like tool for teaching program-ming to computer animation students. In SIGGRAPH ’07: ACM SIGGRAPH 2007 educators program (2007), p. 7. 10

[AMHH08] AKENINE-MÖLLER T., HAINES E., HOFFMAN N .: Real-Time Rendering 3rd Edition. A. K. Peters, 2008. 9, 10, 11

[And03] ANDERSON E. F.: Playing Smart - Artificial Intelligencein Computer Games. In Proceedings of zfxCON03 Conference onGame Development (2003). 12

[And08] ANDERSON E. F.: Scripted smarts in an intelligent vir-tual environment: behaviour definition using a simple entity an-notation language. In Future Play ’08: Proceedings of the 2008 Conference on Future Play (2008), pp. 185–188. 14

[App06] APPERLEY T. H.: Virtual unaustralia: Videogames and

australiaŠs colonial history. In UNAUSTRALIA 2006: Proceed-ings of the Cultural Studies Association of AustralasiaŠs AnnualConference (2006). 1

[Azu97] AZUMA R.: A survey of augmented reality. Presence:Teleoperators and Virtual Environments 6 , 4 (1997), 355–385. 8

[BAEB99] BROGNI B ., AVIZZANO C ., EVANGELISTA C. ,

BERGAMASCO M.: Technological approach for cultural her-itage: augmented reality. In RO-MAN ’99: Proceedings of the8th IEEE International Workshop on Robot and Human Interac-tion (1999), pp. 206–212. 8

[Bav08] BAVOIL L.: Advanced soft shadow mapping techniques.Presentation at The Game Developers Conference 2008, 2008.11

[Bed95] BEDERSON B. B.: Audio augmented reality: a prototypeautomated tour guide. In CHI ’95: Conference companion on Human factors in computing systems (1995), pp. 210–211. 8

[BFSE01] BIMBER O., FRÖHLICH B., SCHMALSTIEG D., EN-CARNAÇÃO L. M.: The virtual showcase. IEEE Computer Graphics and Applications 21, 6 (2001), 48–55. 9

[Bjo04] BJORKE K.: Color Controls. In GPU Gems, Fernando

R., (Ed.). Pearson Education, 2004, pp. 363–373. 10[Bli78] BLINN J. F.: Simulation of wrinkled surfaces. SIG-

GRAPH Comput. Graph. 12, 3 (1978), 286–292. 11

[Blo04] BLOW J.: Game development harder than you think. ACM Queue 1, 10 (2004), 28–37. 6

[Bly06] BLYTHE D.: The direct3d 10 system. ACMTrans. Graph.25, 3 (2006), 724–734. 10

[BN76] BLINN J. F., NEWELL M. E.: Texture and reflection incomputer generated images. Commun. ACM 19, 10 (1976), 542–547. 10

[BS08] BAVOIL L., SAINZ M.: Screen space ambient occlusion.NVIDIA developer information: http://developers.nvidia.com,2008. 6

[Bur04] BURO M.: Call for ai research in rts games. In Proceed-

ings of the AAAI-04 Workshop on Challenges in Game AI (2004),pp. 139–142. 13

[Bur05a] BURKERSRODA R.: Colour Grading. In Shader X3: Advanced Rendering with DirectX and OpenGL, Engel W., (Ed.).Charles River Media, 2005, pp. 357–362. 10

[Bur05b] BURTON J.: News-game journalism: History, currentuse and possible futures. Australian Journal of Emerging Tech-nologies and Society 3, 2 (2005), 87–99. 5

[BWSF06] BENTHIN C., WALD I., SCHERBAUM M.,FRIEDRICH H.: Ray tracing on the cell processor. pp. 15–23. 9

[CA04] COMBS N. , ARDOINT J.: Declarative versusImperative Paradigms in Games AI. Available from:http://www.red3d.com/cwr/games/, 2004. 12

[Cas02] CAS S S.: Mind Games. IEEE Spectrum 39, 12 (2002),

40–44. 14[CCF∗05] CALORI L., CAMPORESI C., FORTE M., GUIDAZ-

ZOLI A., PESCARIN S.: Openheritage: Integrated approach toweb 3d publication of virtual landscape. In Proceedings of the IS-PRS Working Group V/4 Workshop 3D-ARCH 2005: Virtual Re-construction and Visualization of Complex Architectures (2005).8

[CD09] CHALMERS A., DEBATTISTA K.: Level of realism forserious games. In VS-Games 2009: Proceedings of the IEEE Vir-tual Worlds for Serious Applications First International Confer-ence (2009), pp. 225–232. 2


http://publicvr.org/http://publicvr.org/http://publicvr.org/http://publicvr.org/


16/20


[CH05] COOMBE G., HARRIS M.: Global Illumination UsingProgressive Refinement Radiosity. In GPU Gems 2, Pharr M.,(Ed.). Pearson Education, 2005, pp. 635–647. 12

[CIG∗01] COSMAS J., ITEGAKI T., GREEN D., GRABCZEWSKIE., WEIMER F., VAN GOOL L., ZALESNY A., VANRINTELD., LEBERL F., GRABNER M., SCHINDLER K., KARNER K .,

GERVAUTZ M., HYNST S., WAELKENS M., POLLEFEYS M.,DEGEEST R., SABLATNIG R., KAMPEL M.: 3d murale: a mul-timedia system for archaeology. In VAST ’01: Proceedings of the 2001 conference on Virtual reality, archeology, and culturalheritage (2001), pp. 297–306. 8

[CNLE09] CRASSIN C., NEYRET F., LEFEBVRE S., EISEMANNE.: Gigavoxels: ray-guided streaming for efficient and detailedvoxel rendering. In I3D ’09: Proceedings of the 2009 symposiumon Interactive 3D graphics and games (2009), pp. 15–22. 9, 10

[CNSD∗92] CRUZ -NEIRA C., SANDIN D. J., DEFANTI T. A.,KENYON R. V., HART J . C .: The cave: audio visual experienceautomatic virtual environment. Commun. ACM 35, 6 (1992), 64–72. 4

[COST03] CORNWELL J., O’BRIEN K., SILVERMAN B., TOTHJ.: Affordance Theory for Improving the Rapid Generation,

Composability, and Reusability of Synthetic Agents and Objects.In BRIMS 2003: Proceedings of the Twelfth Conference on Be-havior Representations in Modeling and Simulation (2003). 14

[CPCP∗05] CEREZO E ., PEREZ-CAZORLA F. , PUEYO X. ,SERON F., SILLION F.: A survey on participating media ren-dering techniques. the Visual Computer (2005). 9

[CS99] CORADESCHI S., SAFFIOTTI A.: Symbolic Object De-scriptions to Sensor Data. Problem Statement. Linköping Elec-tronic Articles in Computer and Information Science 4, 9 (1999).14

[CUCT04] CIECHOMSKI P. D. H., ULICNY B. , CETRE R. ,THALMANN D.: A case study of a virtual audience in a re-construction of an ancient roman odeon in aphrodisias. In The5th International Symposium on Virtual Reality, Archaeology and Cultural Heritage, VAST (2004) (2004). 13

[Dar07] DARKEN C. J.: Level Annotation and Test by Au-tonomous Exploration. In AIIDE 2007: Proceedings of the Third Artificial Intelligence and Interactive Digital Entertainment Con- ference (2007). 14

[DBB03] DUTRÉ P., BEKAERT P., BALA K.: Advanced Global Illumination. A. K. Peters, 2003. 11

[Deb05] DEBEVEC P.: Making "The Parthenon". 6

giochi e beni culturali

Documents