business and computing deanery multimedia week 9 sound

Business and Computing Deanery

Multimedia

Week 9 Sound

Multimedia 2

Last weekLecture Media dictionary Bitmaps

Your interface

Supported session Using imported

artwork Look at navigation

Multimedia 3

This weekLecture Your interface and

Flash navigation Sound

Chapter 9 Digital Multimedia, 2nd edition

The nature of sound Digitization Compression File formats Working with sound

Supported session Working with sound Look at navigation

Multimedia 4

Interactivity and Flash An event triggers every action

In Flash no action occurs unless something triggers it

There are two types of events in Flash Movie events User events

All actions in Flash are a form of ActionScript

Multimedia 5

Flash and interaction Behaviors

Ready made scripts You can use behaviors to control movie clips,

video, and sound files. ActionScript

A script can consist of a single command, such as instructing a SWF file to stop playing,

Or a series of commands and statements, such as first evaluating a condition and then performing an action

Multimedia 6

Flash and navigation Frame to frame navigation

Behaviorson (release) {

//Movieclip GotoAndStop Behavior

this.gotoAndStop("50");

//End Behavior

Load another movie ActionScript

Multimedia 7

Events, handlers and actions

Whenever a user clicks the mouse or presses a key, an event is generated.

In a SWF file, frames, buttons, movie clips, and text fields all generate events to which your user can respond

Actions this.gotoAndStop("50"); this.gotoAndPlay("50");

Multimedia 8

Sound in multimedia Consider why you would use sound,

will it add to your multimedia production?

List how it could be used in your multimedia production

What controls would you use? Why?

The Nature of Sound Conversion of energy into vibrations in

the air (or some other elastic medium) Most sound sources vibrate in complex

ways leading to sounds with components at several different frequencies

Frequency spectrum – relative amplitudes of the frequency components

Range of human hearing: roughly 20Hz–20kHz, falling off with age

275–276

Chapman N and Chapman J (2004).

Waveforms Sounds change over time

e.g. musical note has attack and decay, speech changes constantly

Frequency spectrum alters as sound changes

Waveform is a plot of amplitude against time Provides a graphical view of characteristics of

a changing sound Can identify syllables of speech, rhythm of

music, quiet and loud passages, etc

276–280


Multimedia 11

Amplitude Amplitude is perceived as loudness

High amplitude Lower amplitude

Click on the amplitudes to hear the difference.

Multimedia 12

Frequency Frequency is perceived as pitch.

High frequency Low frequency

Click on the frequencies to hear the difference.

Multimedia 13

Sampling and Quantization Why is the sampling rate important? Why is the quantisation level

important?

36

Sampling Quantisation


Multimedia 14

Sampling Theorem If the highest frequency component

of a signal is at fh the signal can be properly reconstructed if it has been sampled at a frequency > 2fh Nyquist rate

Undersamping leads to aliasing Sound distortion, image 'jaggies' or

Moiré patterns, jerky or retrograde motion

40–41


Digitization – Sampling Captures amplitude values at regular

intervals Sampling rate is expressed in kHz (1000

per second) Sampling Theorem implies minimum rate

of 40kHz to reproduce sound up to limit of hearing CD: 44.1kHz

Sub-multiples often used for low bandwidth – e.g. 22.05kHz for Internet audio

DAT: 48kHz (Hence mixing sounds from CD and DAT will require

some resampling, best avoided)

281–282


Multimedia 16

Sampling standards Telephone Quality

11.025 kHz, 8-bit depth, Mono (10kb/sec)

Radio Quality 22.050, 8-bit depth, Mono (21 kb/sec)

CD Quality 44.100 kHz, 16-bit depth, Stereo (172

kb/sec)

Digitization – Quantization 16 bits, 65536 quantization levels, CD

quality 8 bits: audible quantization noise, can

only use if some distortion is acceptable, e.g. voice communication

Dithering – introduce small amount of random noise before sampling Noise causes samples to alternate

rapidly between quantization levels, effectively smoothing sharp transitions

283–285


Undersampling & Dithering

283–284


Multimedia 19

Stereo Stereo sound is composed of TWO

very similar sound waves. Subtle differences in location and

movement Must have a separate sound wave

per channel (twice the size file)

Multimedia 20

Mono Uses only one channel Good way to keep file sizes small As a general rule, use mono where

possible.

Data Size Sampling rate r is the number of

samples per second Sample size s bits Each second of digitized audio

requires rs/8 bytes CD quality: r = 44100, s = 16,

hence each second requires just over 86 kbytes (k=1024), each minute roughly 5Mbytes (mono)

287


Clipping If recording level is

set too high, signal amplitude will exceed maximum that can be recorded, leading to unpleasant distortion

But if level is set too low, dynamic range will be restricted

287–288


Sound Editing Timeline divided into tracks Sound on each track displayed as a

waveform 'Scrub' over part of a track e.g. to

find pauses Cut and paste, drag and drop May combine many tracks from

different recordings (mix-down)

289


Effects and Filters Noise gate Low pass and high pass filters Notch filter De-esser Click repairer Reverb Graphic equalizer Envelope Shaping Pitch alteration and time stretching etc

290–295


Compression In general, lossy methods required

because of complex and unpredictable nature of audio data

CD quality, stereo, 3-minute song requires over 25 Mbytes Data rate exceeds bandwidth of dial-up

Internet connection Difference in the way we perceive

sound and image means different approach from image compression is needed

295


Perceptually-Based Compression Identify and discard data that doesn't

affect the perception of the signal Needs a psycho-acoustical model, since

ear and brain do not respond to sound waves in a simple way

Threshold of hearing – sounds too quiet to hear

Masking – sound obscured by some other sound

298–299


MP3 MPEG Audio, Layer 3 Three layers of audio compression in

MPEG-1 (MPEG-2 essentially identical) Layer 1...Layer 3, encoding proces

increases in complexity, data rate for same quality decreases e.g. Same quality 192kbps at Layer 1,

128kbps at Layer 2, 64kbps at Layer 3 10:1 compression ratio at high quality Variable bit rate coding (VBR)

300–301


AAC Advanced Audio Coding Defined in MPEG-2 standard, extended

and incorporated into MPEG-4 Not backward compatible with earlier

standards Higher compression ratios and lower

bit rates than MP3 Subjectively better quality than MP3 at

the same bit rate

301


Audio Formats Platform-specific file formats

AIFF, WAV, AU Multimedia formats used as

'container formats' for sound compressed with different codecs QuickTime, Windows Media, RealAudio

MP3 has its own file format, but MP3 data can be included as audio tracks in QuickTime movies and SWFs

302


MIDI Musical Instruments Digital Interface Instructions about how to produce music,

which can be interpreted by suitable hardware and/or software cf. vector graphics as drawing instructions

Standard protocol for communicating between electronic instruments (synthesizers, samplers, drum machines)

Allows instruments to be controlled by hardware or software sequencers

303–304


MIDI and Computers MIDI interface allows computer to send

MIDI data to instruments Store MIDI sequences in files, exchange

them between computers, incorporate into multimedia

Computer can synthesize sounds on a sound card, or play back samples from disk in response to MIDI instructions Computer becomes primitive musical

instrument (quality of sound inferior to dedicated instruments)

304


Multimedia 32

Editing sound Use a piece of Sound Editing

Software Cool Edit 2000

Sound manipulation program open, create or record new sound files Edit existing sounds Mix audio or musical parts, touch them

up, and add effects Adobe Audition

Multimedia 33

Sound and Flash Sounds that continuously,

independent of the Timeline Or you can synchronize animation to

a sound track. You can add sounds to buttons to

make them more interactive

Macromedia, Flash MX 2004

Multimedia 34

Event sounds and Stream sounds An Event sound must download

completely before it begins playing, and it continues playing until explicitly stopped

Stream sounds begin playing as soon as enough data for the first few frames has been downloaded; stream sounds are synchronized to the Timeline for playing on a website.


Multimedia 35

Importing sound You can import the following sound file formats

into Flash: WAV (Windows only) AIFF (Macintosh only) MP3 (Windows or Macintosh)

Select File > Import > Import to Library To add a sound to a document from the library,

you assign the sound to a layer and set options in the Sound controls in the Property inspector

It is recommended that you place each sound on a separate layer.

To define the starting point of a sound or to control the volume of the sound as it plays, you use the sound-editing controls in the Property inspector Macromedia, Flash MX 2004

Multimedia 36

Controlling sound You can control sound playback

using sound Behaviors or ActionScript

The most common sound-related task in Flash is starting and stopping sounds at keyframes in synchronization with animation

You can use Flash to compress sound when it is published


Multimedia 37

Sync options Event

The entire sound must download first Can be triggered more than once Will play all the way through

Start Cannot have multiple instances

Stop Stops Event or Start sound

Stream Breaks the sound up into frame-sized pieces

which means it does not all need to be downloaded before the Movie can play

Multimedia 38

Sound strategy Re-use sound from the library

It is only stored once Loop shorter sounds in place of

importing longer sounds Compress as much as possible

without loss of quality Use separate sound-editing program

to simplify and remove extraneous noise Kay M. (2003)

Multimedia 39

A question Scenario: You have been asked by

Hope library to add a sound track to a flash movie on using the library Discuss and critically compare two

methods of incorporating sound within your Flash movie

The MP3 sound track that the library have given you is poor quality, discuss the reasons that may have caused this during the digitisation process

Multimedia 40

Activity Play a CD Copy a portion of a track Experiment with sound effects Add fades Export 30 secs as an mp3 Make a media dictionary entry Add to a Flash movie Insert controls using Behaviours

Multimedia 41

For Next Week Accessibility Publishing and distribution Independent study

Attend the supported session to get to know Flash

Create accessible Flash Content Publishing

Multimedia 42

References Chapman N and Chapman J (2004).

Digital Multimedia, Second Edition. London. Wiley

Flash Help and How Do I Kay M. (2003) The Web Wizard's

Guide to Flash, Pearson Education, USA

business and computing deanery multimedia week 9 sound

Documents

sound slide

navigation slide

multimedia production

digital multimedia

action slide

frequency frequency

gotoandplay50 slide

changing sound