Mark Nelson mjas@itu.dk

Misc topics: Audio and PCG

Fall 2013 www.itu.dk

Other parts of game engines

Audio

Animation

Debugging/toolchain

Procedural components

Etc.

Audio

Traditionally not a big part of engine design

Sound design can impact a game significantly, but was seen as mainly an aesthetic concern

Bigger technical concern more recently with 3d positional audio, lots of simultaneous sounds, etc.

Types of audio in games

Two familiar types:

Music

Sound effects

Music

Typically plays in the background

Significant length, plus looping

Infrequent changes Per-level or per-segment music Can be tied to in-game status

Music

Main technical concern: Keep it playing with a minimum of hassle No skips, low CPU/memory usage

Format depends on hardware: On limited hardware, avoid audio files MIDI driving a wavetable synth Specialized sound hardware, e.g. SNES sound module Otherwise, commonly use Ogg Vorbis

Sound effects

Happen in response to events

Can be near-instantaneous or extended Keycard beep, gunshot, explosion NPC voice clip

Sound effects

Main technical conerns: Integrate into the engine’s event system Trigger with low latency

Simultaneous sounds? Old-school approach has a new sound effect cut off an old one Still common to have a limit, avoid things going haywire from trying to play

dozens of sound effects at once

Environmental audio

Sounds with position in the world

Can be sound effects, but often ongoing Waterfalls, machinery, crackling bonfire, television

In the background like music, but without the disembodied, comes-from-nowhere aspect

Simple environmental audio

Give sound sources an (x,y,z) position E.g. In Unity, attach an AudioSource to a GameObject

Attenuate volume by distance from player

Optimize by having a max audible distance and doing a collision-detection-style search to only play nearby sounds

Sound attenuation Sound energy falls off by 1/d2

Or 1/d if measured in decibels (a logarithmic unit) Doubling or halving distance to a source equals about +/-6 dB

Can use other attenuation curves Exaggerated positional effects Account for un-modeled features of the environment

Sound position Attenuation accounts for distance

Do something else for orientation? Sounds coming from the left, in front, behind, etc.

Sound position Two-channel (stereo) case is easiest

Basic model:

Consider headphones as capturing what comes into each of an avatar’s ears

Shift volume towards left or right based on angle

Sound position Not just level differences, but time differences:

Slight delay in sound reaching the far ear

On the order of hundreds of microseconds, but perceptible through phase differences

Instead of engine-level delay (too precise), can slightly phase-offset the stereo channels

Sound position Additional complex stuff going on

Sound travels through the head to the far ear: ”head-transfer function”

Reverberations: sound echoes, and/or feels like it fills a room rather than point source

Sound position Games commonly support more than 2-channel audio now

as well

Better 3d reproduction, but also more complex Map 3d position to 5.1 speakers 5.1 speakers no longer match 1:1 with ears, so may need to model those

effects separately

Often this is a whole API/subsystem, possibly outsourced

3d audio middleware OpenAL

OpenGL-like interface for 3d audio Open source

FMOD Commercial game-engine audio middleware 3d audio API Low-level sound/event management Used in Unity

Procedural content generation

Algorithmic generation, of what would traditionally be hand-created assets

Terrain, levels, vehicles, trees, rocks, games …

Different design uses of PCG

Tool: Help designers create content Material: A new kind of dynamic material to work with Designer: Automated or semi-automated AI designer Expert: Knows about players and/or aesthetics

R. Khaled, M.J. Nelson, P. Barr, ”Design metaphors for procedural content generation in games”, CHI 2013

Offline v. online

Offline procedural content generation (PCG) External tool generates assets Assets are then used in an engine the normal way Purpose is to reduce costs or allow greater range of content

Online PCG Assets or components of them are generated at runtime Closer integration with the engine Various motivations, from compression to allowing infinite worlds

Online PCG for compression

An early use: Elite (1984) Universe, planet locations, even planet names are algorithmically generated Avoids storing them as data on limited-memory machines

Algorithmic PCG, with hand-coded algorithm: Store a smallish seed and a smallish algorithm Pick a seed and expand out to content on the fly

Algorithmic PCG

Works well if variation either: Has a precise mathematical description (fractals, etc.) Doesn’t have strong constraints, so some weirdness is ok

Fractal terrain generation

Algorithmic PCG user control

Not fully controllable: algorithm does what it does

Two common ways of taking user input:

Algorithm takes parameters that control its behavior

”Seed” is more than a number; instead is a starting point that’s expanded

Algorithmic PCG parameter control

Algorithmic PCG as seed-content-expansion

SpeedTree: the de-facto standard for videogame trees

In this case, also interactive and real-time in a nice editor

Infinite worlds

Common use of algorithmic, online PCG

Rogue dungeon generation

Minecraft terrain generation

Infinite Mario

Infinite worlds: Rogue

Rogue dungeon generation: Start with a grid of rooms Mark a random starting room as ’connected’ If current room has unconnected neighbors, connect to one of them randomly,

then make that room the current room Otherwise, pick a random unconnected room and repeat Stop when all rooms are connected to the dungeon

Entrance in the first room chosen, exit in the last room to be connected

Infinite worlds: Minecraft

3d Perlin noise with some tweaks

Generates based on a 2d world grid, filled/unfilled

When any player gets near enough to an unfilled part of the map, fill in that grid’s terrain

Infinite worlds: Infinite Mario

Grammar-based generation

Extracted common Mario level elements into a grammar specifying how they combine

As player goes right, generates new parts of the level

Intelligence in PCG

Instead of a hand-coded algorithm, generator can do some kind of intelligent evaluation of the content

Ensure constraints like level-playability are respected

Optimize a function encapsulating aesthetic judgment

Intelligence in PCG: SketchaWorld

Intelligence in PCG

Other examples:

Genetic algorithms optimizing aesthetic or ”fun” criteria (lots of work by Julian Togelius et al)

Constraint-solvers finding optimal map according to set of constraints (answer-set-programming-based level generator for Warzone 2100)

Simulation-based PCG

Simulate a ”real” process to produce content Attempt to recreate how the content would’ve actually been generated in real

life (with simplifications)

For example: Produce canyons by simulating millions of years of erosion Simulate city-building using agent models

Simulation-based PCG: Transport Tycoon

Transportation-oriented game, where you build railroads, roads, bus lines, etc. to connect cities

Cities grow or shrink according to a model of how the transportation affects their economies

In-game PCG as a game mechanic Fuzzy line to ”normal” simulation gaming

Simulation-based PCG: Dwarf Fortress

World generated by insane levels of simulation

Generates terrain, simulations millions of years of sedimentation and erosion

Simulates thousands of years of the world’s history, battles, civilizations, etc…

Then game starts

PCG resources

Lots more can be found:

Our new PCG book! http://www.pcgbook.com/

ITU PCG course: https://blog.itu.dk/MPGG-E2013/schedule/

PCG mailing list