Lua Patient Zero: Grim Fandango

Bret MogilefskyFormerly of LucasArts, now SCEA

Setting the stage

• One famous designer: Tim Schafer• One venerable game engine: SCUMM– Huge pedigree of beloved games includedManiac

Mansion, Secret of Monkey Island, Sam and Max, Day of the Tentacle, Full Throttle

• One new game engine: Sith, for Jedi Knight• One naïve coder: Me, fresh out of Cal!– I stole from everyone!

Scripting not a new idea atLucasArts

• SCUMM dated back to 1986Example:

actor guybrush walk-to banana-treewait-for-actoractor guybrush say-line “Mmm, bananas…”

wait-for-actoractor guybrush face-cameraactor guybrush say-line “Wish I had a banana-picker”

• Various attempts to unseat it had failed• Nobody stopped me, though they predicted doom

Breaking with the past

• SCUMM was long in the tooth and newb-hostile– SCUMM the language was inseparable from

SCUMM the engine– I am as guilty of not-invented-here as anyone

• Got sets, characters, lighting, animation, and audio in no time, but no scripting– Jedi Knight put me way ahead, but it wasn’t an

adventure game

Why Lua

• I was lucky!– <Insert pic of Dr. Dobb’s Journal>

• It was small, elegant, and ready to go– The manual was small enough that you could hand

it to someone and expect them to read it– It had no external dependencies– The embedding API was ideal– The code itself was approachable and readable

Lua as I found it

• Started with Lua 2.5– Switched to Lua 3.1 beta before shipping

• Very nice …but still lacked coroutines• Any LEC adventure game required them!

Why coroutines?

• Independent characters and objects get programmed with isolated scripts controlling their actions– Think about what it would be like in C/C++!

• Threads were too heavy for undead pigeons, flaming beavers, driving demons, and Pentium 90MHz processors– True concurrency wasn’t needed; just wanted to

pick up where we left off last time each frame

Lua modifications

• Addition of coroutines– Made it re-entrant– Flattened the Lua stack

• Implemented dump/restore of Lua state

Lua re-entrancy

• Took state out of global variables to make it thread-safe– First attempt was an array of Lua state structures

looped over until a “yield()” was called, using a single thread

• But… DISASTER!

Lua’s recursive stack

• Every Lua call also pushed a stack frame in C• A yield from one Lua instance to resume

another might happen at a different stack depth–Welcome to Crash City

• <Sequence diagram: C->Lua->C->Lua->C->Lua>

Flattening Lua’s stack

• Instead of recursing, return a yield code, push a structure, and start the next function

• <C->Lua->Lua->Lua>• A couple years later, “Stackless Python” solved

the same problem– It still exists, whereas Lua incorporated coroutines

as part of the language• This would not have been possible if Lua

wasn’t nice code to begin with

Ways we used Lua

• Engine extension• Command-line and console• Debugger, including tasking extension• Tool macro language• Patching

Engine extension

• All performance-critical code in C/C++– Renderer, animation, lighting, sound, save/load

• All game-specific code in Lua– Engine just calls “BOOT()”, then loops tasks– Not just adventure gameplay• Controls, menus, dialog trees, puzzle mosaic

• The scripting language is glue!– 120K lines of code

Command-line and console

• Copy and paste code into game on-the-fly– Iterate on a bugfix without restarting– Trigger any condition as needed

• This kind of productivity was unprecedented!• The compiler was small so we left it in

Debugger

• Written by a scripter using the in-language line-hook and call-hook facility– Added the task-hook for our coroutines

• Fancy features like evaluating expressions at each step were trivial due to compiler in engine

Tool macro language

• Developed choreography rehearsal tool alongside game

• Stitched together assets into “costumes” and coordinated timing

• ChoreTool used Lua as a macro language– Example: Tie real game trigger logic to a menu item– Example: Automate insertion of keys at regular intervals

• Some macros became permanent parts of the tool, saving lots of programmer time

Patching

• Blocker bugs in scripts were trivially replaced– Saved enormous amounts of burn time

• Update to shipped game retro-patched save games as well– Probably not an issue if we’d saved differently

It wasn’t all roses

• High memory requirements• Debugger went stale• Heavyweight garbage collection• So much flexibility it was easy to write bugs– Typos in variable names not caught

• Lacked languageconventions – No example code to examine– Savegames steadily grew larger as you progress…

How’d we do?

• Despite huge technical risks, we pulled it off– Great reviews, and the game remains beloved

• Poor timing: We lost to Half-Life in “game of the year” categories over and over

• The engine went on to ship Escape from Monkey Island (with lots of mods)– But really, the end of the adventure genre was

upon us, and that was the end of the line for GrimE

TheLua explosion

• Concurrent success of QuakeC and UnrealScript had made scripting languages HOT!

• GDC99 held a panel on them– Rob Huebner on Java in Vampire: The Masquerade– Kevin Bruner on writing your own for Obi-Wan– Seamus McNally on not using one for Treadmarks

• 200 miserable developers slumped in their chairs• “Or you could just use Lua…”• Furious scribbling followed by sudden influx on the

mailing list

Why Lua endures

• It’s small, simple, and proven– Lua turned “scripting language” into a commodity

feature• You would be crazy to write your own and

expect to do better