1

Use of Sound in Games

CIS 487/587

Bruce R. Maxim

UM-Dearborn

2

Speech Technology

• Discrete word recognition

• Continuous speech recognition

• Speech store and forward

• Speech generation

3

Discrete Word Recognition

• 90 to 98% reliability for small vocabulary

• Usually requires speaker dependent training

• Most people would rather type than dictate

4

When should you use it?

• Speakers hands are busy

• Mobility required

• Speakers eyes are occupied

• Harsh or cramped conditions prevent use of key board

5

Speech Store and Forward

• Voice mail type technology

• Video games

• Low cost

• Resource intensive

6

When to use computer generated speech?

• Message is simple • Message is short • Message will not be referred to later • Message deals with events in time • Message requires immediate response • Visual communications channels are overloaded • Environment lighting is bad • User must move around • User subjected to high G forces or lack of oxygen

7

Sound

• Sound travels more slowly than light (this makes coordination tricky)

• Sound wave travel at constant speed and can be specified with two parameters:– Amplitude (wave height, volume of air moved)– Frequency (number of complete cycles per

second)

8

The next 4 slides are from Rabin’s book

9

Digital Representationof a Sound Wave

• Most common technique known as sampling– Sampling involves measuring the amplitude of the

analog wave file at discrete intervals– The frequency of sampling is known as sampling

rate– Each sample is typically stored in a value ranging

from 4 to 24 bits in size– The size of the sample value in bits is known as

the ‘bit depth’– Music CDs have a sample rate and bit depth of

44.1 kHz (samples/sec) and 16 bits (sample size)

10

Bit Depth and Signal Noise

• Bit depth of sample data affects signal noise– Signal to noise ratio = number of available bits / 1– For example, 8-bit samples have a 256:1 SNR

(~48 dB), and 16-bit samples have a 65,536:1 SNR (~96 dB)

– Decibel ratio is calculated using 10 x log10 (ratio) or 8.685890 x log e (ratio)

11

Sampling Frequency and Frequency Reproduction

• Sampling frequency affects range and quality of high-frequency reproduction

• Nyquist Limit– Frequencies up to one-half the sampling

rate can be reproduced– Audio quality degrades as frequency

approaches this limit

12

Modern Audio Hardware

• Samples are piped into sound “channels”– Often a hardware pipeline from this point

• Various operations, such as volume, pan, and pitch may be applied

• 3D sounds may apply HRTF algorithms and/or mix the sound into final output buffers.

13

Wave Shapes

• Sine wave (pure sound)• Square wave• Saw tooth wave• Half-rectified sine wave• Most sounds are mixtures of several waves,

their spectrums (frequency distributions) look quite ragged

• To make realistic sounds we need to replicate this spectrum

14

Computer Sound

• Two types of computer generated sounds– Digital (recordings of sound)

• Used for sound effects and people talking

– Synthesized (programmed reproductions of sounds)

• Might only be used for music

15

Digital Sound

• Analog-to-digital• Created by converting the analog sound

vibrating a microphone to bit string that can be written to a disk– Sample rate (frequency) – should be two

times the frequency of original sound (e.g 400 Hz for male voice)

– Amplitude resolution (8 bits for games and 16 bits for professional sounds and music)

16

Synthesized Sound

• Not as good as digital

• People are used to hearing 16 to 32 different tones in a sound’s spectrum (not just one pure note)

• FM synthesizers use feedback to synthesize additional “background” noise in a sound’s spectrum

17

MIDI

• Musical Instrument Digital Interface

• Example (using English):– Turn on channel 1 using an A– Turn on channel 2 using a C#– Turn off channel 1– Turn off all channels

• Good for music, bad for explosions

18

Wave Table Synthesis

• Mix between synthesis and digital recording

• Real sampled sounds are stored in a wave table to be played back by the DSP (digital sound processor)

19

Wave-Guide Synthesis

• Uses DSP chips and special hardware

• Sound synthesizer can generate a mathematical model of a virtual instrument and play it

• Most game companies buy sound libraries for sound effects

20

Getting Sample Sounds

• Sample from real world using a microphone

• Buy a sample sound library (e.g. LaMothe CD)

• Use synthesizer (e.g. Sound Forge) to create them either 22 KHz or 11 KHz using either 8-bit or 16-bit

21

Recording You Own Sounds

• Use 16 bit samples, 22 KHz, mono (two microphones need wide separation to notice stereo effect)

• Cleanup sounds with Sound Forge• Apply effects (frequency shift, echoes,

distortions)• Write your processed sounds using the

same settings as recording

22

The next 10 slides are from Rabin’s book

23

Sound Playback Techniques

• Two basic playback methods:1. Play sample entirely from memory buffer2. Stream data in real-time from storage

medium• Streaming is more memory efficient for very

large audio files, such as music tracks, dialogue, etc

• Streaming systems use either a circular buffer with read-write pointers, or a double-buffering algorithm

24

Sample Playback and Manipulation

• Three basic operations you should know– Panning is the attenuation of left and right

channels of a mixed sound• Results in spatial positioning within the aural stereo field

– Pitch allows the adjustment of a sample’s playback frequency in real-time

– Volume control typically attenuates the volume of a sound

• Amplification is generally never supported

25

Compressed Audio Format

• Compressed audio formats allow sound and music to be stored more compactly– Bit reduction codecs generally are lightweight

• ADPCM compression is implemented in hardware on all the major current video game console systems

– Psycho-acoustic codecs often have better compression

• Require substantially more computational horsepower to decode

26

MP3, Ogg Vorbis,Licensing & Patent Issues• The MP3 format is patented

– Any commercial game is subject to licensing terms as determined by Fraunhofer & Thompson Multimedia, the holders of the patents

• Ogg Vorbis is similar to MP3 in many ways– Open source and patent-free (royalty-free)

• Be aware of patent and license restrictions when using 3rd party software

27

Programming Music Systems

• Two common music systems– MIDI-based systems

• (Musical Instrument Digital Interface)

– Digital audio streaming systems• (CD audio, MP3 playback, etc)

28

Advantages and Disadvantages of MIDI

• Actual music data size is negligible

• Easy to control, alter, and even generate in real-time

• High quality music is more difficult to compose and program

• Only effective if you can guarantee playback of a common instrument set

29

Other MIDI-based technologies to be aware of

• DLS (DownLoadable Sound) Format– A standardized format for instrument

definition files

• iXMF (Interactive eXtensible Music Format)– New proposed standard for a container

format for interactive music

30

Advantages / Disadvantages of Digital Audio Streams

• Superb musical reproduction is guaranteed

• Allows composers to work with any compositional techniques

• Some potential interactivity is sacrificed for expediency and musical quality

• Generally high storage requirements

31

A Conceptual Interactive Music Playback System

• Divide music into small two to eight-bar chunks that we’ll call segments.

• A network of transitions from segment to segment (including loops and branches) is called a theme.

• Playing music is now as simple as choosing a theme to play. The transition map tracks the details.

32

API Choices

• DirectSound (part of DirectX API)– Only available on Windows platforms

• OpenAL– Newer API– Available on multiple platforms

• Proprietary APIs– Typically available on consoles

• 3rd Party Licensable APIs– Can offer broad cross-platform solutions

33

Direct Sound

• IDirectSound object – need one for each installed sound card (emulation possible if sound card is missing)

• IDirectBuffer – use primary and secondary buffers like DirectDraw

• IDirectSoundCapture – used to record sounds and speech recognition

• IDirectSoundNotify – used to send messages in complex systems

34

LaMothe Examples

35

Game_Init( )

// create directsound object and test for error

if (DirectSoundCreate(NULL,&lpds,NULL)!=DD_OK)

return(0);

 

// set cooperation level to normal priority

if (lpds->SetCooperativeLevel(main_window_handle,DSSCL_NORMAL)!=DS_OK)

return(0);

36

Game_Shutdown( )

// release the directsoundobject

if (lpds!=NULL)

lpds->Release();

37

Generating a Sound

// this example does everything: it sets up directsound

// creates a secondary buffer, loads it with a synthesizer

// sine wave and plays it

 

void *audio_ptr_1 = NULL, // used to lock memory

*audio_ptr_2 = NULL;

 

DWORD dsbstatus; // status of sound buffer

 

DWORD audio_length_1 = 0, // length of locked memory

audio_length_2 = 0,

snd_buffer_length = 64000; // working buffer

 

// allocate memory for buffer

UCHAR *snd_buffer_ptr = (UCHAR *)malloc(snd_buffer_length);

38

Generating a Sound// we need some data for the buffer, you could load a .VOC or .WAV

// but as an example, lets synthesize the data

 

// fill buffer with a synthesized 100hz sine wave

for (int index=0; index < (int)snd_buffer_length; index++)

snd_buffer_ptr[index] = 127*sin(6.28*((float)(index%110))/(float)110);

 

// note the math, 127 is the scale or amplitude

// 6.28 is to convert to radians

// (index % 110) read below

// we are playing at 11025 hz or 11025 cycles/sec therefore, in 1 sec

// we want 100 cycles of our synthesized sound, thus 11025/100 is approx.

// 110, thus we want the waveform to repeat each 110 clicks of index, so

// normalize to 110

39

Generating a Sound

// set cooperation level

if (lpds->SetCooperativeLevel(main_window_handle,DSSCL_NORMAL)!=DS_OK)

return(0);

 

// set up the format data structure

memset(&pcmwf, 0, sizeof(WAVEFORMATEX));

 

pcmwf.wFormatTag = WAVE_FORMAT_PCM;

pcmwf.nChannels = 1;

pcmwf.nSamplesPerSec = 11025;

pcmwf.nBlockAlign = 1;

pcmwf.nAvgBytesPerSec = pcmwf.nSamplesPerSec * pcmwf.nBlockAlign;

pcmwf.wBitsPerSample = 8;

pcmwf.cbSize = 0;

40

Generating a Sound

// create the secondary buffer (no need for a primary)

memset(&dsbd,0,sizeof(DSBUFFERDESC));

dsbd.dwSize = sizeof(DSBUFFERDESC);

dsbd.dwFlags = DSBCAPS_CTRLDEFAULT | DSBCAPS_STATIC | DSBCAPS_LOCSOFTWARE;

dsbd.dwBufferBytes = snd_buffer_length+1;

dsbd.lpwfxFormat = &pcmwf;

 

if (lpds->CreateSoundBuffer(&dsbd,&lpdsbsecondary,NULL)!=DS_OK)

return(0);

// copy data into sound buffer

if (lpdsbsecondary->Lock(0, snd_buffer_length,

&audio_ptr_1, &audio_length_1, &audio_ptr_2,

&audio_length_2, DSBLOCK_FROMWRITECURSOR)!=DS_OK)

return(0);

41

Generating a Sound// copy first section of circular bufferCopyMemory(audio_ptr_1, snd_buffer_ptr, audio_length_1); // copy last section of circular bufferCopyMemory(audio_ptr_2,(snd_buffer_ptr+audio_length_1),audio_length_2); // unlock the bufferif (lpdsbsecondary->Unlock(audio_ptr_1, audio_length_1,

audio_ptr_2, audio_length_2)!=DS_OK) return(0); // play the sound in looping modeif (lpdsbsecondary->Play(0,0,DSBPLAY_LOOPING )!=DS_OK)

return(0);

// release the memory since DirectSound has made a copy of itfree(snd_buffer_ptr);

42

Reading from Files

• DirectSound has no support for reading .WAV files from disk (involves 2 steps)

• Reading .WAV files, involves reading a header to get format info (e.g. # channels, bits/channel, playback rate, length of sample sound

• Loading the sound

43

Dsound_Load_WAV

• Open .Wav file and extract header info

• Create and fill DirectSound buffer

• Stores info in open slot in sound.fx[ ] and returns SounfID as an index

• Sound card can be played at any time using

sound_fx[sound_id].dsbuffer->Play(0,0,DSBPLAY_LOOPING);

44

Globals

// this holds a single sound

typedef struct pcm_sound_typ

{

LPDIRECTSOUNDBUFFER dsbuffer; // the ds buffer containing the sound

int state; // state of the sound

int rate; // playback rate

int size; // size of sound

int id; // id number of the sound

} pcm_sound, *pcm_sound_ptr;

45

GlobalsLPDIRECTSOUND lpds; // directsound interface pointerDSBUFFERDESC dsbd; // directsound descriptionDSCAPS dscaps; // directsound capsHRESULT dsresult; // general directsound resultDSBCAPS dsbcaps; // directsound buffer caps LPDIRECTSOUNDBUFFER lpdsbprimary, // you won't need this normally lpdsbsecondary; // the sound buffers WAVEFORMATEX pcmwf; // generic waveformat structure pcm_sound sound_fx[MAX_SOUNDS]; // array of secondary sound buffers HWND freq_hwnd, // window handles for controls volume_hwnd, pan_hwnd;

int sound_id = -1; // id of sound we load for demo

46

Game_Init( )

// create a directsound object

if (DirectSoundCreate(NULL, &lpds, NULL)!=DS_OK )

return(0);

 

// set cooperation level

if (lpds->SetCooperativeLevel(main_window_handle,DSSCL_NORMAL)!=DS_OK)

return(0);

 

// clear array out

memset(sound_fx,0,sizeof(pcm_sound)*MAX_SOUNDS);

47

Game_Init( )// initialize the sound fx arrayfor (int index=0; index<MAX_SOUNDS; index++){ // test if this sound has been loaded if (sound_fx[index].dsbuffer) {

// stop the soundsound_fx[index].dsbuffer->Stop();// release the buffersound_fx[index].dsbuffer->Release();

} // end if // clear the record out memset(&sound_fx[index],0,sizeof(pcm_sound)); // now set up the fields sound_fx[index].state = SOUND_NULL; sound_fx[index].id = index;} // end for index

48

Game_Init( )

// load a wav file in

if ((sound_id = DSound_Load_WAV("FLIGHT.WAV"))!=-1)

{

// start the voc playing in looping mode

sound_fx[sound_id].dsbuffer->Play(0,0,DSBPLAY_LOOPING);

} // end if

49

Game_Shutdown( )

// release the sound buffer

if (sound_fx[sound_id].dsbuffer)

sound_fx[sound_id].dsbuffer->Release();

 

// release the directsoundobject

if (lpds!=NULL)

lpds->Release();

50

DSound_Load_Wav( )

HMMIO hwav; // handle to wave file

MMCKINFO parent, // parent chunk

child; // child chunk

WAVEFORMATEX wfmtx; // wave format structure

 

intsound_id = -1, // id of sound to be loaded

index; // looping variable

 

UCHAR *snd_buffer, // temporary sound buffer to hold voc data

*audio_ptr_1=NULL, // data ptr to first write buffer

*audio_ptr_2=NULL; // data ptr to second write buffer

 

DWORD audio_length_1=0, // length of first write buffer

audio_length_2=0; // length of second write buffer

51

DSound_Load_Wav( )

// step one: are there any open id's ?

for (index=0; index < MAX_SOUNDS; index++)

{

// make sure this sound is unused

if (sound_fx[index].state==SOUND_NULL)

{

sound_id = index;

break;

} // end if

 

} // end for index

 

// did we get a free id?

if (sound_id==-1)

return(-1);

52

DSound_Load_Wav( )// set up chunk info structureparent.ckid = (FOURCC)0;parent.cksize = 0;parent.fccType = (FOURCC)0;parent.dwDataOffset = 0;parent.dwFlags = 0; // copy datachild = parent; // open the WAV fileif ((hwav = mmioOpen(filename, NULL, MMIO_READ | MMIO_ALLOCBUF))==NULL) return(-1); // descend into the RIFF parent.fccType = mmioFOURCC('W', 'A', 'V', 'E');

53

DSound_Load_Wav( )

if (mmioDescend(hwav, &parent, NULL, MMIO_FINDRIFF))

{

// close the file

mmioClose(hwav, 0);

 

// return error, no wave section

return(-1);

} // end if

 

// descend to the WAVEfmt

child.ckid = mmioFOURCC('f', 'm', 't', ' ');

54

DSound_Load_Wav( )if (mmioDescend(hwav, &child, &parent, 0)){ // close the file mmioClose(hwav, 0); // return error, no format section return(-1); } // end if // now read the wave format information from fileif (mmioRead(hwav, (char *)&wfmtx, sizeof(wfmtx)) != sizeof(wfmtx)){ // close file mmioClose(hwav, 0); // return error, no wave format data return(-1);} // end if

55

DSound_Load_Wav( )// make sure that the data format is PCMif (wfmtx.wFormatTag != WAVE_FORMAT_PCM){ // close the file mmioClose(hwav, 0); // return error, not the right data format return(-1); } // end if // now ascend up one level, so we can access data chunkif (mmioAscend(hwav, &child, 0)){ // close file mmioClose(hwav, 0); // return error, couldn't ascend return(-1); } // end if

56

DSound_Load_Wav( )

// descend to the data chunk

child.ckid = mmioFOURCC('d', 'a', 't', 'a');

 

if (mmioDescend(hwav, &child, &parent, MMIO_FINDCHUNK))

{

// close file

mmioClose(hwav, 0);

 

// return error, no data

return(-1);

} // end if

 

// finally!!!! now all we have to do is read the data in and

// set up the directsound buffer

57

DSound_Load_Wav( )

// allocate the memory to load sound data

snd_buffer = (UCHAR *)malloc(child.cksize);

 

// read the wave data

mmioRead(hwav, (char *)snd_buffer, child.cksize);

 

// close the file

mmioClose(hwav, 0);

 

// set rate and size in data structure

sound_fx[sound_id].rate = wfmtx.nSamplesPerSec;

sound_fx[sound_id].size = child.cksize;

sound_fx[sound_id].state = SOUND_LOADED;

58

DSound_Load_Wav( )// set up the format data structurememset(&pcmwf, 0, sizeof(WAVEFORMATEX)); pcmwf.wFormatTag = WAVE_FORMAT_PCM; // pulse code modulationpcmwf.nChannels = 1; // mono pcmwf.nSamplesPerSec = 11025; // always this ratepcmwf.nBlockAlign = 1; pcmwf.nAvgBytesPerSec = pcmwf.nSamplesPerSec * pcmwf.nBlockAlign;pcmwf.wBitsPerSample = 8;pcmwf.cbSize = 0; // prepare to create sounds bufferdsbd.dwSize = sizeof(DSBUFFERDESC);dsbd.dwFlags = control_flags | DSBCAPS_STATIC | DSBCAPS_LOCSOFTWARE;dsbd.dwBufferBytes = child.cksize;dsbd.lpwfxFormat = &pcmwf;

59

DSound_Load_Wav( )

// create the sound buffer

if (FAILED(lpd->CreateSoundBuffer(&dsbd,&sound_fx[sound_id].dsbuffer,NULL)))

{

// release memory

free(snd_buffer); 

// return error

return(-1);

} // end if 

// copy data into sound buffer

if (FAILED(sound_fx[sound_id].dsbuffer->Lock(0, child.cksize,

(void **)&audio_ptr_1, &audio_length_1, (void **)&audio_ptr_2, &audio_length_2, DSBLOCK_FROMWRITECURSOR)))

return(0);

60

DSound_Load_Wav( )

// copy first section of circular buffer

memcpy(audio_ptr_1, snd_buffer, audio_length_1);

// copy last section of circular buffer

memcpy(audio_ptr_2, (snd_buffer+audio_length_1),audio_length_2);

// unlock the buffer

if (FAILED(sound_fx[sound_id].dsbuffer->Unlock(audio_ptr_1,

audio_length_1, audio_ptr_2, audio_length_2)))

return(0);

 

// release the temp buffer

free(snd_buffer);

 

// return id

return(sound_id);

61

Direct Music

• Supports downloadable sounds (e.g. MIDI files)

• Supports on the fly music composition

• Supports unlimited # MIDI channels (not just the standard 16)

• Uses hardware acceleration if available

62

Direct Music Interfaces

• IDirectMusic – main interface• IDirectMusicPerformance – controls and

manipulates playback• IDirectMusicLoader – loads MIDI files• IDirectMusicSegment – chunk of music data• IDirectMusicSegmentState – checks current

state of segment• IDirectMusicPort – destination of streamed

output data

63

Globals

#include <dsound.h> // include dsound, dmusic

#include <dmksctrl.h>

#include <dmusici.h>

#include <dmusicc.h>

#include <dmusicf.h>

// direct music globals

IDirectMusicPerformance *dm_perf = NULL;

IDirectMusicLoader *dm_loader = NULL;

IDirectMusicSegment *dm_segment = NULL;

IDirectMusicSegmentState *dm_segstate = NULL;

64

CreatePerformance( )IDirectMusicPerformance* CreatePerformance(void){// this function creates the performance IDirectMusicPerformance* pPerf;  if (FAILED(CoCreateInstance(CLSID_DirectMusicPerformance, NULL, CLSCTX_INPROC, IID_IDirectMusicPerformance, (void**)&pPerf))) { // return null pPerf = NULL; } // end if  return pPerf;} // end CreatePerformance

65

CreateLoader( )

IDirectMusicLoader* CreateLoader(void)

{

// this function creates the loader

IDirectMusicLoader* pLoader;

if (FAILED(CoCreateInstance(CLSID_DirectMusicLoader,

NULL,CLSCTX_INPROC,IID_IDirectMusicLoader,

(void**)&pLoader)))

{

pLoader = NULL;

} // end if

return pLoader;

} // end CreateLoader

66

FreeDirectMusic( )// If there is any music playing, stop it.dm_perf->Stop( NULL, NULL, 0, 0 );  // Unload instruments this will cause silence.// CloseDown unloads all instruments, so this call is also not // strictly necessary.dm_segment->SetParam(GUID_Unload, -1, 0, 0, (void*)dm_perf); // Release the segment. dm_segment->Release(); // CloseDown and Release the performance object. dm_perf->CloseDown();dm_perf->Release(); // Release the loader object.dm_loader->Release(); // Release COM. CoUninitialize(); return S_OK;

67

LoadMidiSegment( )IDirectMusicSegment* LoadMIDISegment(IDirectMusicLoader* pLoader, WCHAR *wszMidiFileName ){ // this function loads a midi segment off disk DMUS_OBJECTDESC ObjDesc; HRESULT hr; IDirectMusicSegment* pSegment = NULL; // get current working directory char szDir[_MAX_PATH]; WCHAR wszDir[_MAX_PATH]; if(_getcwd( szDir, _MAX_PATH ) == NULL) { return NULL; } // end if

// convert to wide charactersMULTI_TO_WIDE(wszDir, szDir)

68

LoadMidiSegment( )

// set the search directory

hr = pLoader->SetSearchDirectory(GUID_DirectMusicAllTypes,wszDir, FALSE);

 

if (FAILED(hr))

{

return NULL;

} // end if

// setup object description

ZeroMemory(&ObjDesc, sizeof(DMUS_OBJECTDESC));

ObjDesc.dwSize = sizeof(DMUS_OBJECTDESC);

ObjDesc.guidClass = CLSID_DirectMusicSegment;

wcscpy(ObjDesc.wszFileName, wszMidiFileName );

 

ObjDesc.dwValidData = DMUS_OBJ_CLASS | DMUS_OBJ_FILENAME;

69

LoadMidiSegment( )// load the object and query it for the IDirectMusicSegment interface// This is done in a single call to IDirectMusicLoader::GetObject// note that loading the object also initializes the tracks and does // everything else necessary to get the MIDI data ready for playback.

hr = pLoader->GetObject(&ObjDesc,IID_IDirectMusicSegment,(void**)&pSegment);if (FAILED(hr)) return(0); // ensure that the segment plays as a standard MIDI file// you now need to set a parameter on the band track// Use the IDirectMusicSegment::SetParam method and let // DirectMusic find the trackby passing -1 (or 0xFFFFFFFF) // in the dwGroupBits method parameter.

hr = pSegment->SetParam(GUID_StandardMIDIFile,-1, 0, 0, (void*)dm_perf); if (FAILED(hr)) return(0);

70

LoadMidiSegment( )// The next step is to download the instruments. // This is necessary even for playing a simple MIDI file // because the default software synthesizer needs the DLS data // for the General MIDI instrument set// If you skip this step, the MIDI file will play silently.// Again, you call SetParam on the segment, this time specifying the

GUID_Download parameter: hr = pSegment->SetParam(GUID_Download, -1, 0, 0, (void*)dm_perf); if (FAILED(hr)) return(0); // return the pointerreturn pSegment; } // end LoadSegment

71

Game_Init( )// set up directmusicif (FAILED(CoInitialize(NULL))){ // Terminate the application. return(0);} // end if // create the performancedm_perf = CreatePerformance();if (dm_perf == NULL){ return(0);// Failure -- performance not created} // end if // initialize the performanceif (FAILED(dm_perf->Init(NULL, NULL, main_window_handle))){ return(0);// Failure -- performance not initialized} // end if

72

Game_Init( )

// add the port to the performance

if (FAILED(dm_perf->AddPort(NULL)))

{

return(0);// Failure -- port not initialized

} // end if

 

// create the loader to load object(s) such as midi file

dm_loader = CreateLoader();

 

if (dm_loader == NULL)

{

return(0);// Failure -- loader not created

} // end if

73

Game_Init( )// release the old segmentif (dm_segment){ dm_segment->Release(); dm_segment = NULL;} // end if // load the segmentif (dm_loader){ dm_segment = LoadMIDISegment(dm_loader,L"BATTLE.MID");} // end if// start the songif (dm_segment){ dm_perf->PlaySegment(dm_segment, 0, 0, &dm_segstate);} // end if

74

Game_Shutdown( )

int Game_Shutdown(void *parms = NULL, int num_parms = 0)

{

// this is called after the game is exited and the main event

// loop while is exited, do all you cleanup and shutdown here

 

FreeDirectMusic();

 

// return success or failure or your own return code here

return(1);

 

} // end Game_Shutdown

75

Using Both

• You can use DirectSound and DirectMusic in the same application

• You must initialize DirectSound before DirectMusic

76

The next 4 slides are from Rabin’s book

77

Audio Scripting and Engine Integration

• Very little audio programming should be done by general game programmers

• Game Engine should offer robust support for audio triggers and scripts

• Engine should deal with audio scripts, not “sound files”

• Why is this so important?

78

Audio Scripting

• Many situations require much more information than can be embedded in a linear audio file– Sound Variation– Sound Repetition– Complex Sound Looping– Background Ambience

79

Lip-sync Technology

• Lip-sync technology is a blending of audio and visual techniques to create realistic-looking speech by in-game actors. – Simple techniques such as waveform amplitude

measurement has worked previously, but…– In future titles, it will be considered inadequate.– Much work can still be done in this field.

80

Advanced Voice Playback

• Real-time spoken feedback is especially important in sports titles (simulated announcers)

• Game are reaching the limits of what current techniques (canned, prerecorded phrases combined in series) can provide.

• Again, this is an opportunity for future groundbreaking audio work.