# physically based sound

Post on 13-Jan-2016

47 views

Embed Size (px)

DESCRIPTION

Physically Based Sound. COMP259Nikunj Raghuvanshi. Overview. Background FEM Simulation Modal Synthesis (FoleyAutomatic) Comparison/Conclusions. Motivation. Sounds could in-principle be produced automatically, just like graphics: Sound Rendering - PowerPoint PPT PresentationTRANSCRIPT

Physically Based Sound

COMP259Nikunj Raghuvanshi

OverviewBackground

FEM Simulation

Modal Synthesis (FoleyAutomatic)

Comparison/Conclusions

MotivationSounds could in-principle be produced automatically, just like graphics: Sound RenderingSound Rendering has not received much research effortMain Goal: Automatic generation of non-music, non-dialogue sound

Sound Production TodayMovies: Foley Artistshttp://www.marblehead.net/foley/index.html

Games: Anyone noticed the huge sound directory in Unreal Tournament?

PBS: Sound Production in NatureCollisions/Other interactions lead to surface vibrationsVibrations create pressure waves in airPressure waves sensed by earSurface VibrationPressure WaveEarVibrationPropagationPerception

Main Aims of PBSPhysics simulator gives contact/collision informationAssign material properties for sound, Wood, concrete, metal etc.Sound simulator generates sound using this data (in real time?)

ChallengesSound must be produced at a minimum of ~44,000 HzExtremely High Temporal Resolution (timesteps in the range of 10-6-10-8 s)Stiffness of underlying systems (eg. Metallic sounds. K/m~=108)Stability may require even smaller timesteps

Two ApproachesFEM deformable simulation O'Brien, J. F. et. al., Synthesizing Sounds from Physically Based Motion. SIGGRAPH 2001.

FoleyAutomatic (Modal Synthesis) Kees van den Doel et. Al., FoleyAutomatic: Physically-based Sound Effects for Interactive Simulation and Animation. SIGGRAPH 2001.

Main ideasDeformable Simulation (arguably) much more physically basedFoley Automatic: Additive SynthesisComponent SinusoidsSound Signal

OverviewBackground

FEM Simulation

Modal Synthesis (FoleyAutomatic)

Comparison/Conclusions

Simulation RequirementsTemporal ResolutionSimulate Vibration as well as PropagationVibration Modeling: Deformable Model for ObjectsPropagation Modeling: Explicit Surface RepresentationPhysical/Perceptual Realism

System Structure

Vibration ModellingFEM with Tetrahedral ElementsLinear Basis Functions, greens strainExplicit Time IntegrationTypically #nodes = 500, #elements = 1500, dt = 10-6-10-7 s

Sound Propagation ModellingFluid Dynamic FEM simulation of surrounding air? Very expensive. InsteadEmploy Huygens Principle: Pressure Wave may be seen as sum of pressure waveletsReceiverReceiverPressure WavePressure Wavelets

Acoustic Impedance of AirSurface Vibrations and SoundPressure contribution of a patch,VelocityDensity of AirSound Propagation Speed in AirUnit Normal

Surface Vibrations and SoundApproximate differential elements with surface trianglesApply band pass filters:Low pass: windowed sinc filterHigh pass: DC blocking filterResult: Pressure known for all surface triangles

Putting it all togetherPressure/Signal at ReceiverFiltered Average PressureArea of TriangleVisibility TermApproximation of Beam PatternDistance FalloffReceiverVibration

Propagation DelayAccumulation BufferReceiverd1d2Sourcet=0t1= d1/ct2= d2/c12Receiver Distance from SourceSound Propagation Speed

Results: CapabilitiesGeneral models

Generated sounds are accurate

Stereo Sound

Dopplers Effect

Demo

Results: Accuracy

Results: SpeedScene TimeStep(s) Nodes/Elems Time/Audio TimeBowl 10-6 387/1081 91.3/4.01 minsClamped Bar 10-7 125/265 240.4/1.26 minsVibraphone 10-7 539/1484 1309.7/5.31 mins (~1 day)

Timings on a 350MHz SGI Origin MIPS R12K processor

OverviewBackground

FEM Simulation

Modal Synthesis (FoleyAutomatic)

Comparison/Conclusions

FeaturesModal resonance model of solidsLocation dependent soundsImpact, slide, roll excitation modelsReal-time, low latencyEasy integration with simulation/animationPracticalDo not model propagation of sound from source to receiver

Synthesis MethodForceVibrationEmissionPropagationListenerSpeakersSound SamplesUser

VibrationSurface u(x,t) of body responds to external contact force F(x,t)u(x,t)F(x,t)Strain Functional Speed of SoundUnder suitable boundary conditions, the solution to the PDE is a sum of sinusoids

EmissionSound pressure s(t) linear functional L of surface vibration u(x,t)u(x,t)Ls(t)Note that propagation is not modeled in above

The Modal Synthesis Modelu(x,t)F(p,t)Ls(t)Impulse response/modal modelThe response u(x,t) of an arbitrary solid object to an external force can be described as a weighted sum of damped sinusoidsSince L is linear, it implies at s(t) must be a sum of damped sinusoids too

Example: A 1D string 1st Mode2nd Mode Frequency = f0Higher modes Frequency = f1= 2*f0 Frequency = fk= k*f0Main Idea: Sum contributions of all the modesThe point of impact decides the proportions in which the modes are to be mixed: ak. Therefore, ak is a function of p, the point of impactThe frequencies and damping parameters are a property of the object, and independent of how the object is hit+ +...+a0a1ak

The Modal Synthesis Modelu(x,t)F(p,t)Ls(t)Impulse response,modal modelParameters measured experimentallyKth mode: Gain Factor Point Damping Vibration of impact Term Frequency

Force ModelingImpactSlidingRollingWavetableStochasticAt runtime: Find gain parameters given the location, strength and kind of force. Synthesize sound from previous equation.

Impact Forces

Duration: hardness (T)Magnitude: energy transfer (w)Multiple micro-collisions Example:

Sliding/ScrapingMicro-collisions lead to noisy audio-force

Sliding/ScrapingWavetable approachStore force parametersModulate amplitude with energy transferModulate rate with contact speedSynthesis ApproachFractal noise represents roughnessFilter through reson filterResonance ~ contact speedWidth ~ randomness of surface

RollingNo relative surface motionDifferences with sliding:Smoother: Use low passMore dampingHarder to createLess understoodEssential coupling?

Rolling: Smooth SurfacesPolyhedral objects do not lead to smooth rolling forcesInstead use smooth surfaces directly

Rolling: Contact EvolutionEvolve the contact in Reduced coordinatesq = (u,v,s,t, )c(u,v)d(s,t)

Rolling: Contact EvolutionPiecewise parametric surfaces, loop subdivision surfacesExplicit integration, no stabilizationMultiple contacts and conforming contacts are not handledUsed only when multiple contacts in close spatio-temporal proximity

Demo

Dynamic ForcesContact forceRolling speedSlipping speedImpulsesand locationsPebble-in-Wok Demo

Results 0.1% CPU time per mode Graceful degradation of quality The bell demo is interactive Uses a PHANToM for interaction Authors do not report any real timings State that sound quality is perception-based and has no metric as of now

OverviewBackground

FEM Simulation

Modal Synthesis (FoleyAutomatic)

Comparison/Conclusions

DiscussionFEM: Physically Rigorous and GeneralToo slow for interactive applicationsDoesnt scale wellInappropriate to apply a 30fps technique to 44000fps?Maybe too general for the problem domain?

DiscussionModal model exploits the vibrational natureHigher EfficiencyBut, not rigorously physically basedFinding the parameters requires experimentation and earballingNo rigorous correlation between physical and perceptual parameters

DiscussionFor Realtime: Need for a technique to cover the middle groundExtracting modal parameters in general requires solving PDEsNot possible to do in an automated mannerApproximate modal parameters and then use modal synthesis?

ConclusionPBS involves orders of magnitude smaller temporal and spatial scalesResearch is sparse, problems are denseMain contributions of the two papers besides vibration modeling:FEM: Efficient modeling of sound propagationFoleyAutomatic: Efficient, Approximate models to handle surface properties and contact forces

ReferencesO'Brien, J. F., Cook, P. R., Essl G., "Synthesizing Sounds from Physically Based Motion." The proceedings of ACM SIGGRAPH2001, Los Angeles, California, August 11-17, pp. 529-536.Kees van den Doel, Paul G. Kry and Dinesh K. Pai, FoleyAutomatic: Physically-based Sound Effects for Interactive Simulation and Animation Computer Graphics (ACM SIGGRAPH 01 Conference Proceedings), pp. 537-544, 2001.

AcknowledgementsSome images were taken from the referred papers and the corresponding SIGGRAPH slides

#22/23 - it may be good to mention Pentland's work on modal analysis for simulating deformation #25 - what are c & g? speed of sound and gravity? Surface Properties are modelled using a wavetable which gives the relationship between excitation point, force and the associated gain coefficients. The force model which tries to model the surface properties is stochasticl.Series of microcollisions instead of one single impact for 15 ms.~lower 3 modes taken into account.Surface properties for sliding: Superimpose High frequency noise over low frequency audio force at animation speeds. Essential Coupling: Somehow ball always gives exciation force at frequencies matching the modes of the object: It kn ows about the modal frequencies.