Computer Animation in Games

For The Phoenix IEEE SocietyWednesday, November 4, 2009

Ryan AndersonSenior Technology EngineerRainbow Studios

Tonight’s Presentation•Animation Challenges (or, why we couldn’t play Toy Story)

•2D Animations – Past and Present

•3D Animations – Past, Present, and Future

•Making the Game

Games vs. MoviesWhy can’t we play what we see?

It’s a question of resources:• Processing Power• RAM• ROM• Human Resources

Pixar Toy Story• 1995• 2 – 15 hours spent animating each frame• 4 hours rendering each scene• Sun SPARCStation (up to 200MHz, 512MB RAM)

Console Games – 1995 - MGSPlaystation 1• 33MHz• 2MB RAM (1MB VRAM)• Games must run at least 30 fps (33ms frames)

In The Beginning

Work just like a flip book

Low processing power

2D Animations

“Lots” of RAM –for the time

Pacman Frames

3 Frames

Don’t forget rotated frames

Fortunately for us, frame 3 works in both orientations

5 Frames for Pacman•Most Hardware could flip picture for you, no need to animate both directions

•Saves you half the memory!

•More savings from palette swapping ghosts.

How much is this costing us?

Most textures were palette based, cost is determined by number of colors

•256 colors (where available) 1Byte per pixel•16 colors .5 Bpp•4 colors .25 Bpp•2 colors .125 Bpp (or, 1 bit per pixel)

How big was Mario?

32x32 Mario4 colors (2 bits per pixel)256 Bytes per frame

The Map The Palette

0

123

Original Mario was only 16x16 (64 bytes per frame)

And Luigi was free!

•No need to do two sets of animations, just change the palette.•Very common in early computer games to fight “new” monsters and bosses that were palette swaps of old ones.

How is 2D animation still used in 3D games?

Billboards – Camera facing quads with 2D or rendered 3D textures

• Perspective problems: Might have to render from a lot of angles• Can constrain camera angles to help prevent this

One Example of Current Use of 2D

Nintendo DS

• Limited to 2048 triangles per frame

• Pre-rendering characters can save a LOT of polygons

• Fast cart access allows swapping in and out: no need to hold them all in RAM

Billboarded Particles

Particles are billboarded rectangles that change in shape and location, and can play frames of animations at the same time to give off the appearance of additional motion.

Internal pictures removed

Billboarded Crowd (paper people)10,000 crowd members can really add up (even if low poly)

Render from different angles, then display on camera facing polygons

3D Animations

Mesh of triangles Each vertex usually has at least a position, texture coordinate, and a normalModern games can add multiple texture coordinates, tangent vectors, and more!

Morph Target Animations

Blend vertex positions to animate

Keep a mesh per frame

Can start to use a lot of RAM fastCan have linearity issues when blending

Morph Target Blending

Mesh minute hand at noon

Mesh minute hand at 15 after

Blend meshes to get 7 after

Blended positions cause a shortened hand

You can add more frames to minimize this effect, but at additional RAM costs

Morphs Advantages

Allows artists to give exact positions to vertices.

• Great for cloth and facial animations

Fast on older processors (1 vector scale, 1 vector madd) – Used with such formats as the Quake 3 .md3

Morph DisadvantagesBlending artifactsSize gets big fast

• Size of vertex * number of vertices in mesh * number of animation frames

Sometimes difficult to create – often created in tools using skeletal animations anyway.

Difficult (if not impossible) to do IK, ragdoll, etc.

Skeletal Animations

Still uses a mesh of trianglesPretty much how we do things now

Every vertex is associated with a mathematical “bone”

Animation now takes place in 2 parts

• Animate the bone frame

• “Skin” the mesh to the bone

Bone FramesEach bone has exactly 1 parent (up to root node) but can have multiple childrenCreates a hierarchy of bones, each with a local rotation and positions

Root Bone 1 Bone 2

Local bone matrices

Concatenated world matrix

Rotate bone 1 by 90 degrees

Root Bone 1

Bone 2

Local Matrices

Concatenated World Matrices

Skinning to a bone: Associate a vertex with bone 2

Each vertex must be in its associated bones local space

So, apply the inverse of the bone’s world space transform to each vertex

=

V

Now, when a new world transform is calculated for the bone, apply it to the local space position for every associated vertex

Root

=

V

Weighting

In early games, it was not uncommon to associate each vertex with just 1 bone, due to processing cost.

V1

V2

V3

V4

Imagine a flexible mesh over our earlier 2 bone manipulator (kind of like a square-ish thumb), made up of 4 vertices.

Now, if we associate each vertex with a bone, V1 and V2 pretty obviously go with bone 2, but what about V3 and V4?

Bone 1 Bone 2

V2

V3

V4

V3 and V4 if associated with Bone 1

This looks painful!

V3 and V4 if associated with bone 2

V1 V2

V3V4

A bit better, but still very distorted around the joint.

What we need to do is allow V3 and V4 to be “weighted” to both bones 1 and 2

V1 V2

V3

V4

Where the sum of all the weights is equal to 1

A model in its “T-Pose” and the associated bone hierarchy

Internal pictures regretfully removed

Now we only need 1 copy of the mesh in memory, and can store animations as a series of bone rotations and translations.

• This is important, because modern models can be 10,000 – 20,000 polygons with large vertices.

• “Skinning” can still be CPU intensive, but is generally done by the GPU on modern hardware

• The Xbox360 can transform millions of vertices each frame.

Size still matters

Computers have more RAM and processing power these day, but that just means game designers want to pack even more into them.

Often games will even do their cut scenes as animations rather than video.

• Metal Gear Solid 4 for the PS3 had HOURS of animated cut scenes.

Animation Compression

Store rotations as Quaternions and not Matrices (4 numbers instead of 9)

• Makes interpolating rotations much faster

• Not Euler angles!

Store 16 bit numbers instead of 32 bit numbers for rotations

• Quaternions are normalized anyway, so our values don’t get above a 1

• Usually 1/16000 is plenty accurate for animations

Key Frame Reduction

Data Compression is still not enough.

Animations usually sampled at 30 frames per second.

We could cut our animation size in half by simply halving our sample rate and interpolating between them, but this isn’t good enough.

• Loss of quality

• We can do better!

• Sample rate of change per bone over frames

• Drop frames where a tolerance value is met

Frame reduction is very similar to any other sampling problem

Other compression methods also exist

• Wavelets

• Be careful of 180 degree rotations!

Issues with skeletal animationsLabor intensive to create

• Mocap can help, but not always appropriate

Limited motions cause blending artifacts

• Targeting objects (grab, punch, kick, etc.)

• The dreaded foot-slide

Some solutions are …

Procedural Animations… for today

• Ragdoll … Crashes are fun!

• IK

Lots of different kinds• Headlook (simplest kind in our game)

Hands and feet

Spine moving with the bumps

But how would you stand up from this?

• Jersey Flap

Procedural Animations… moving forwards

Define what a motion is, generate an animation for it on the fly

• Actions such as punch, kick, dodge

• Standing up from various fallen positions

• Define a walking path with start and endpoints to avoid foot-slide

Some procedural videos

Creating MX vs. ATVPhysics driven rather than animation driven

• The game state controls the animation state

• Animations are blended together according to a tree traversal of physics based inputs

Animation Based Event Driven Games

Game state based on animation states

Events in games cause state traversal

Faster content creation, scripting possible, less programmer time

Havok Behavior is a recent example of middleware supporting this

Questions, comments, or complaints?