Echo and Bounce

Marc O’Morain

Presentation

• About 15 Minutes of PowerPoint

• 5 Minutes of video and demonstration

• Please ask questions at any time

A Rabbit

A Visual Proxy

A Physical Proxy

An Audio Proxy

• This project will explore the idea of using an audio proxy

• A proxy to generate sound

VisualProxy

PhysicalProxy

AudioProxy

Realistic Movement Realistic SoundRealistic Graphics

Past Research

• Synthesizing Sounds from Rigid-Body Simulations.

– O'Brien, J. F.Shen, C., Gatchalian, (SIGGRAPH 2002)

• Synthesizing Sounds from Physically Based Motion.

– O'Brien, J. F., Cook, P. R., Essl G., (SIGGRAPH 2001)

• FoleyAutomatic: Physically-based Sound Effects for Interactive Simulation and Animation

– K. van den Doel, P. G. Kry and D. K. Pai, (SIGGRAPH 2001)

Why?

• Why is this useful?– No need to record sounds– Information needed is in physics engine

already

• What is new?– Level of detail– Using mass springs – no pre-computation

What is an Audio Proxy

• What is an audio proxy made of?– Masses and Springs– Tetrahedra

• How do we build one?

Mass Spring System

Mass

Spring

Mass Spring System• When a spring is at

rest it has no resultant effect on masses

• When compressed, pushes two masses away from each other

• When extended, pulls two masses toward each other

Natural Length

3D Meshes

• A triangular mesh can approximate any 3D surface

• Commonly used in graphics

• Just a skin:– No Volume– No Density

Tetrahedra

• A Tetrahedron is a 4 sided shape(Triangular based pyramid)

• Made from 4 triangles

• Collection of tetrahedra can approximate any 3D volume

Mass Spring Tetrahedra

• Create tetrahedra from mass-spring system:– Soft body of any

shape can be created

• This is how all objects in the project are represented

Digital Audio

65536

0

16-bit

Time

Am

plitu

de

+

-

44100

1of a second

Sound Generation

Sound Generation

Sound Generation

Apply A Force Resultant Vibration

Sound Generation

For each face:• Find displacement at each

vertex• Find average displacement• Multiply by area

Wave ampltiude =(Average displacement * area)

Vibration to Sound

Time

Up Sampling

• Take samples at quite a low granularity• Fit a hermite curve to the sample data• Re-sample from curve at higher resolution• Sample at CD-quality

1st Major Problem

• The Sound was crackling– Reason: •Hermite curve blends

between 4 values

•Don’t know what is coming in the future

•Assume Zero

•Curve goes below zeroAudio Buffer

Do 5 simulation steps, then send audio to sound card.

(unsigned short) -0.00007f = 65536

A Click @ 33Hz

65536

0

16-bit

Simulation

Simulation

Mass Stability

• Each Mass in the system has a ‘short term memory’ (Last 5 timesteps)

• At each timestep: current movement is added to memory

• If the mass does not remember moving:– It is ‘stable’

• Masses will stay stable until an external force acts on them (Newtons 1st Law)

Audio Proxy Stability

• Simulating a stable mass is free

• If all the masses are stable, the entire proxy is stable

• Simulating a stable proxy is free

A Physics Engine

• This is not a physics engine

• All external forces will come from a physics engine

• When a collision happens, the physics engine will provide:– The Point of collision– A Force

Objects in Model Space

• All objects are kept in model space– (All at the origin)

• All models are independent from each other

• Simulate all models seperately

Levels of Detail

Levels of Detail

Levels of Detail

Because this project uses:– Up sampling– Separate Models

Different timesteps for each model:– Small timestep for close objects– Larger timestep when further away

tt

1

Different Sampling RatesLow Sampling Rate

Higher Sampling Rate

Levels of Detail

• Simulation time is proportional to:– Number of masses in the system– Number of springs in the system

• System with s springs and m masses:– Time t per calculation is:

mst

Levels of Detail

6 Tetrahedra 166 Tetrahedra24 Tetrahedra

Variable Level Of Detail

Listener is far away –

Low level of detail

Variable Level Of Detail

Listener is medium distance -

Higher level of detail

Variable Level Of Detail

Listener is very close -

Highest level of detail

Resonance

A Screenshot

Top View

Front View

3D View

Side View

A Screenshot

Sound Wave

(Left and right channels)

A Screenshot

Field Of Vision

(Blue Cone)

A Screenshot

Level of Detail Selection

A Screenshot

Click to apply a force

Echo and Bounce

Now for some videosand a demonstration