Hearing I: Sound & The Ear

1

• Philosophical Aside: If a tree falls in the forest and no one is there to hear it...

• Qualities of sound energy and sound perception

• Anatomy of the ear

• Auditory brain areas

Overview of Topics

2

Chapter 5 in Chaudhuri

• If a tree falls in the forest and no one’s there, does it make a sound?

• Depends on two definitions of “sound”

• Physical definition - sound is pressure changes in the air or other medium (answer: Yes)

• Perceptual definition - sound is the experience we have when we hear (answer: No)

Sound vs. Sound

3

• Objects make sound by moving back and forth rapidly (20 to 20000 times/second) through a medium (air)

• Example: A speaker produces sound like this:

• The diaphragm of the speaker moves out, pushing air molecules together (compression)

• The diaphragm moves in, pulling the air molecules apart (rarefaction)

• The cycle of this process creates alternating high- and low-pressure regions that travel through the air

Sound Waves

4

• Sound waves are longitudinal waves, meaning that variations in intensity (air density) are parallel to the wave’s direction of travel.

• This is unlike light or water waves, which are transverse waves, meaning that variations are perpendicular to the direction of the wave’s travel.

• Note that it is the sound pressure that moves through the air, and not the air itself that moves. (Think of “the wave” at a sports stadium)

Sound Waves

5

Linear vs. Transverse Waves

6

Transverse Wave (water or light)

Longitudinal Wave (sound)

Movement Makes Sound Waves

Slow(low freq.)

Fast(high freq.)

7

Questions

• Define sound (hint: give two definitions)

• How is a sound wave like “the wave” at a stadium?

• What moves through the air in a sound wave?

8

• Pure tone: The simplest form of sound wave.

• Pressure variations are sinusoidal.

• Can be defined by two qualities:

• Amplitude (µPa) ≈ subjective loudness

• Frequency (Hz) ≈ subjective pitch

• All other sounds are made up of combinations of (usually many) pure tones.

Pure Tones

9

A tuning fork vibrates sinusoidally, thus producing a pure tone

10

11

Two waves with same frequency but different amplitudes

Larger amplitude ≈ subjectively louder

12

Two waves with same amplitude but different frequency

Higher frequency ≈ higher pitch

Amplitude

• Amplitude = difference in sound pressure between high and low peaks of wave

• Sound pressure (P) is typically measured in micropascals (μPa)

• Sound intensity is square of pressure (I = P2) and is more directly related to subjective loudness.

• Usually, we speak in terms of the ratio of sound intensity to some reference (Is / Ir)

13

Amplitude

• Alternatively, we speak in terms of the ratio of squared pressures: Ps2 / Pr2

• Reference pressure (Pr) is typically 20 µPa.

• This is the lowest pressure variation amplitude detectable by an average human at 1000 Hz frequency

• Called “sound pressure level” or SPL

14

Decibels

• The range of pressure ratios that humans can hear covers 7 orders of magnitude (i.e., from 1 to 10000000)

• The range of intensity ratios thus covers 14 orders of magnitude (from 1 to 100000000000000)

• This makes direct use of pressure/intensity measures unwieldy

• We instead take the log (base 10) of intensity ratios, log10(Is / Ir), to get bels

15

Decibels

• Bels make for a somewhat coarse range, so we take tenths of bels to get decibels (dB).

dB = 10 × log10(Is / Ir) dB = 10 x log10(Ps2 / Pr2) dB = 20 x log10(Ps / Pr)

• Most often we use dBSPL (“decibels, sound pressure level”), indicating that Pr is 20 µPa.

16

DecibelsdB = 20 × log10(Ps/Pr)

• Ps is the difference between maximum and minimum pressure and Pr is an arbitrary reference pressure

• Pr is usually taken to be 20 μPa, referred to as SPL or Sound Pressure Level

• Each increase by a decibel yields approximately the same increase in subjective loudness.

• That is, log nature of the decibel scale compensates (roughly) for response compression in the auditory system

17

20200

2 00020 000

2 000 00020 000 000

200 000 0002 000 000 000

Pressure (μPa)

18

Questions• What is log10 of 10? 100? 1000?

• Ambient (reference) pressure is 40 µPa and a pressure wave is hitting your ear with an amplitude of 40000 µPa. How many decibels is that?

db = 20 x log10(40000/40) db = 20 x log10(1000) db = 20 x 3 = 60 db

Is that really loud? Really soft?

19

Example for Self-Test

• Reference pressure is 20 µPa and a pressure wave is hitting your ear with an amplitude of 20 µPa. How many decibels is that?

• Reference pressure is 20 µPa and a pressure wave is hitting your ear with an amplitude of 20000 µPa. How many decibels is that?

20

j.jj dBSPL

fj.j dBSPL

• Frequency - number of amplitude cycles within a given time period

• Measured in Hertz (Hz): 1 Hz is 1 cycle per second

• Perception of pitch is related to frequency (but there’s more to it)

Frequency

21

The Doppler Effect

• When a sound source approaches at high speed, its sound seems to increase in frequency, then decrease as it passes due to compression and expansion of sound waves

22

• Natural sounds do not have a single frequency and/or amplitude.

• Rather, they are made up a complex set of combined pure tones, each of which has its own frequency and amplitude

• The lowest frequency element of a set of tones is called the fundamental frequency

Complex Sounds

23

• Higher frequency elements of a sound are called overtones

• Overtones whose frequency is a whole integer multiple of the fundamental frequency are called harmonics.

• Together, the tones that make up a sound are referred to as its frequency spectrum, which is in part responsible for a sound’s timbre.

• Good musical instruments produce sounds which have mostly harmonics and not many non-harmonic partials. (e.g., Stradivarius violins)

Complex Sounds

24

25

Frequency spectra for 3 instruments playing a tone with a fundamental frequency of 196 Hz (G3).

The coloured lines indicate the frequencies and intensities of the harmonic overtones.

Very different sets of pure tones add up to the same note, but with different timbre.

Attack & Decay

Attack Only

Decay Only

No Attack or Decay

Attack & Decay

Attack Only

Decay Only

No Attack or Decay

• Other qualities that make up timbre:

• Attack of tones - buildup of sound at the beginning of a tone

• Decay of tones - decrease in sound at end of tone

Attack & Decay

26

Questions

• Describe the relationship between pure tones and natural sounds.

• Define the following: Fundamental frequency, overtone, harmonic (or harmonic overtone), timbre, attack, decay.

28

Periodic and Aperiodic Sounds

• Periodic sounds have patterns that repeat across time. E.g., musical notes, vowel sounds

• Aperiodic sounds have no repeating pattern. E.g., hissing/fricative sounds, thumps, etc.

• AKA “noise”. If equal energy at all frequencies, we call it “white noise”

29

Peri

odic

Sou

nds

30

Ape

riod

ic S

ound

31

Fourier Analysis/Synthesis

• Fourier discovered that any function could be mathematically broken down into a series of sine wave elements (Fourier Analysis)

• It is also therefore true that any function can be built up from a series of sine wave elements (Fourier Synthesis).

• Remarkably, your auditory system (and visual system) do something very much like Fourier Analysis with the information they receive.

32

Fourier Synthesis

(a) Pressure changes for a pure tone with frequency of 440 Hz.

(b) The 2nd harmonic overtone of this tone. Frequency = 880 Hz.

(c) The 3rd harmonic overtone. Frequency = 1,320 Hz. (3x440)

(d) The sum of the three harmonics creates the waveform for a complex tone.

33

Five Sound Examples

1) 440 Hz tone

2) 880 Hz tone

3) 440 + 880 Hz tone

4) 1320 Hz tone

5) 440 + 880 + 1320 Hz tone

Note how the combinations sound more “complex”, i.e., closer to real-world sounds.

34

Fourier spectrum for the last tone on the previous slide. The heights of the lines indicate the amplitude of each of the frequencies that make up the tone.

35

Frequency

Am

plitu

de(a)

(b)(c)

Mor

e ex

ampl

es o

f Fou

rier

Syn

thes

is

36

Waveform(time domain)

Spectrum(frequency domain)

Time Frequency

Pres

sure

Am

plitu

de

37

Questions

• What is the basic idea behind Fourier Analysis? Fourier Synthesis?

• What is a Fourier Spectrum?

38

Sound Transmission

• Sound waves must move through a medium (e.g., air or water)

• Speed of sound depends on density and elasticity of medium.

• More dense = slower transmissionMore elastic = faster transmission

• Speed of sound in air ≈ 331 m/sSpeed of sound in water ≈ 1400 m/s

39

Inverse Square Law

• As sound moves out from a point source, its energy is spread over a larger and larger area

• Area is proportional to square of the relative distance

• 2×distance = 1/4 intensity 3×distance = 1/9 intensity

40

Sound & Objects

• Objects create different sounds based on their size, mass, and elasticity

• E.g., the longer and thicker a piano wire, the lower frequency sound it produces

• Objects also vibrate when sound wave hit them. Each object has a resonant frequency at which it vibrates most strongly

• This will be a lower frequency the larger the object is, and higher for more elastic objects.

41

Sound & Objects

42

Sound & Objects

• Objects also absorb, reflect, transmit, or diffract sound based on their physical properties.

• The larger and denser an object is, the more it will absorb sound

• The more elastic an object is, the more it will tend to reflect sound (e.g., large rock walls cause echoes)

43

Sound & Objects

• The shape of an object is also important in how it will absorb or reflect sound.

• Acoustic foam has a shape designed to trap sound waves

44

Sound & Objects

• Diffraction occurs when sound waves encounter objects.

• Sound waves tend to re-form on the other side of small objects (red) but not larger ones (yellow)

45

Acoustic Impedance

• Sound is reflected when it moves from one medium to another that has higher acoustic impedance

• For example, when moving from air to water, 99.9% of sound energy is reflected

• This will become important when we consider the inner ear, which is fluid-filled.

46

Sound Illusion: Shepperd’s Glissando

• Pitch seems to drop forever• But how can this be?

47

Sound/Vision Illusion: The McGurk Effect

48

Questions

• What does it mean for a sound to diffract around an object?

• What different processes can occur to a sound when it encounters an object?

• If the sound intensity is 180 units at distance 10 m, what will it be at 30 m?

49

50

• Outer ear = pinna and auditory canal

• Pinna helps with sound localization (more later)

• Auditory canal ≈ 3 cm long tube. Protects the tympanic membrane at the end of the canal

• Resonance Effect: The resonant frequency of the outer ear amplifies frequencies between 2,000 and 5,000 Hz

Outer Ear

51

Middle Ear

• ≈2 cm3 cavity separating inner from outer ear

• Contains the three ossicles: Malleus, Incus, & Stapes

• Also Eustachian tube, which equalizes pressure

52

• Outer and middle ear are filled with air

• Inner ear filled with fluid that is much denser than air

• Pressure changes in air transmit poorly into the denser medium (-30 dB!)

• Ossicles act in two ways to amplify the vibration for better transmission to the fluid

Function of Ossicles

53

• Condensation Effect:

• Eardrum is larger than stapes footplate

• So force is concentrated down on a smaller area to create higher pressure

• By far the larger of the two effects, at about 25 dB

Function of Ossicles

54

• Lever Effect:

• Ossicles are set up like a lever arm (malleus is longer than incus)

• Allows weaker force of air vibrations to move liquid in cochlea

• A small effect, however (about 2 dB)

Function of Ossicles

55

Force of Air Movement

Resistance of cochlear fluid

• Fluid-filled snail-like structure set into vibration by the stapes

• Divided into three canals by two membranes: Scala vestibuli, (Reissner’s membrane), cochlear duct/scala media (Cochlear partition) and scala tympani.

• Cochlear partition extends from the base (stapes end) to the apex (far end)

• Subsection of the cochlear partition is the Basilar Membrane

The Inner Ear: The Cochlea

56

The Cochlea

57

The cochlea is shown here in its real, coiled, position

Cochlea: Partly Unwrapped

58

The Cochlea: Fully Unwrapped

59

60

• Cochlear partition is narrow at apex and wide at base.

• Basilar membrane is a part of the cochlear partition that is opposite: Wide at apex and narrow at base.

• Spiral lamina make up the rest of the cochlear partition.

61

Questions

• Describe the basic structure of the cochlea

• What is the function of the ossicles? How do they accomplish it?

• Why do we have a pinna? What are the functions of the auditory canal?

62

• Consists of inner and outer hair cells and their supporting structures

• Rests on the Basilar membrane, which vibrates in response to sound stimuli, activating hair cells

• Inner hair cells are the receptors for hearing

• Tectorial membrane extends over the hair cells

• Transduction at the hair cells takes place due to the bending of tectorial and basilar membranes

The Organ of Corti

63

64

Cross-section of the cochlea, showing how the Organ of Corti rests on the basilar membrane.

65

Close-up of the Organ of Corti, showing how hair cells’ stereocilia extend between basilar and tectorial membranes

66

Animation of the Organ of Corti, showing how hair cells’ stereocilia are bent due to different fulcrum locations of basilar and tectorial membranes.

Questions

• Which cells transduce sound into neural signals?

• What two membranes move relative to one-another to stimulate the IHCs?

67

Fundamental Concept: Theoretical Synthesis

• Hegel* suggested that understanding progresses through three stages:

• Thesis: Someone proposes a theory

• Antithesis: A counter-proposal is made, often apparently contradictory or even mutually exclusive of the thesis

• Synthesis: The two are brought together for a more complete model

• Science often progresses this way. Watch out for false dichotomies. There is often potential for synthesis.

* The attribution of this idea to Hegal is debatable...Also, there’s way more to it than this.68

• There are two ways nerve fibres signal frequency

• Which fibres are responding (the Place Theory)

• Hair cells at different points along the OoC fire to different sound frequencies

• How fibres are firing (the Frequency Theory)

• Rate or pattern of firing of nerve impulses

Neural Signals for Sound Frequency

69

Neural Signals for Frequency

Base Apex

70

• Hz of sound is coded by the place on the organ of Corti that has the greatest vibration, and thus greatest stimulation of neurones

• Békésy provided evidence for this in two ways

• Direct observation of basilar membranes from cadavers

• Building a model of the cochlea using the physical properties of the basilar membrane

Place Theory

71

• Physical properties of the basilar membrane

• Base of the membrane (by stapes) is

• 3 to 4 times narrower than at the apex

• 100 times stiffer than at the apex

• Therefore, the resonant frequency of the base is much higher than the apex

• Indeed, resonant frequency of basilar membrane changes systematically from 20 to 20000 Hz from apex to base.

Békésys’ Place Theory

72

Békésy suggested that sounds produce a travelling wave along the basilar membrane.

The peak of this wave occurs at the point where the membrane’s resonant frequency matches that of the sound’s frequency.

73

74

• The peak of the envelope* of the traveling wave indicates the point of maximum displacement of the basilar membrane

• Hair cells at this point are stimulated the most strongly, leading to the nerve fibres firing the most strongly at this location

Békésys’ Place Theory

75

* You may want to review “envelope functions”

76

A single travelling wave at 3 points in time. Dashed line shows envelope over entire wave. P is the point of maximum displacement.

Envelope functions of basilar membrane vibration at various frequencies (Békésy,1960).

Based on measurements of cadavers’ cochleas.

Envelopes much more sharply peaked in living cochleas due to active feedback (about which, more later).

77

• Record activity from single hair cell and measure how intense a sound (dB) is required to activate the cell at each frequency (Hz)

• i.e., measure the absolute threshold for the cell across the frequency spectrum

• Resulting function is the frequency tuning curve

• Hz to which the cell is most sensitive (lowest threshold) is the characteristic frequency (CF)

Evidence for Place Theory: Frequency Tuning of Hair Cells

78

Frequency Tuning Curves of 4 Hair Cells

79

• Characteristic frequencies of hair cells along the Cochlea shows tonotopic map*

• i.e., Cochlea shows an orderly map of frequency response along its length

Evidence for Place Theory: Tonotopic Organization of CFs

80

* You may want to review topological maps

Frequency Theory

• At first, a competing theory with place theory

• Proposed that all hair cells fire to every sound, and their rate of vibration indicates frequency of sound

• Problem: Whole cochlea physically can’t vibrate across the range 20 - 20000 Hz, so this cannot be the whole story. Also, max neurone firing rate = 500 Hz

• However, action potentials from IHCs are found to be phase locked to the sound signals

81

Phase Locking

• Phase is a characteristic of sine waves describing where the wave starts

• The red and blue waves (top right) are ≈90° out of phase with one another

• The upper set of blue waves (bottom right) are in phase while those below are out of phase

82

Phase Locking

• IHCs spike when a pure tone is at highest compression

• But max firing rate is 500 spikes/sec, so how to code for higher frequencies?

83

• No single neurone can signal each pressure maximum for tones above about 500 Hz

• But, by aliasing and firing out of phase with one-another, several neurones can encode high frequencies.

84

Volley Theory

Synthesis

• Both place and volley theories are partly correct

• For sounds below 1000 Hz, only volley theory applies

• For those between 1000 and 5000 Hz, both place and volley theory apply

• For 5000 Hz and above, only place theory applies

85

Questions

• The hair cells are sandwiched between what two membranes?

• Which end of the basilar membrane codes for low frequency sounds?

86

Transduction: Inner Hair Cells

• Hearing requires extremely rapid response.

• To response to a 20000 Hz tone, for instance, cell must be able to encode changes (not fire!) every 50 microseconds!

• GPCRs, used in other sensory systems, are too slow for this

• Instead, a direct physical mechanism is used

87

Inner Hair Cells

• Transduction occurs at the inner hair cells, specifically at the stereocilia

• When stereocilia are bent, tiny filaments between their tips, called “tip links” are pulled

• The tip links directly and mechanically open ionic channels, causing the cell to alternately hyper polarize and depolarize

88

Tip Links

89

Active Response: Outer Hair Cells

• OHCs do not transduce sound, but they play an important role in boosting IHC response

• When sound stimulates the OHCs, they rhythmically contract and expand, like muscle cells

• This amplifies the motion of the basilar and tectorial membranes, which in turn more actively stimulate the IHCs

90

Active Response: Outer Hair Cells

91

Effect of Active Response

92

What Bekésy

saw

Living Chochlea

Questions

• Describe the basic structure of the cochlea

• What is the function of the ossicles? How do they accomplish it?

• Why do we have a pinna? What are the functions of the auditory canal?

93

Amplitude Transduction

• Humans can sense amplitudes between 20 µPa and 20 000 000 µPa (7 orders of magnitude)

• Neurones can vary firing rates from about 1 to 500 spikes per second (≈2.5 orders of magnitude)

• So a single class of neurones cannot code for the whole range

94

Amplitude Transduction• Instead, different hair cells

code for different limited ranges of amplitude

• “A” codes for a low range, starting at ≈10 dB & saturating at ≈50 dB. It has a high resting response rate

• “B” responds only to upper range, starting at ≈50 dB and saturating at ≈100 dB

95

• Fourier analysis - mathematical process that separates complex waveforms into a series of sine waves

• Research on the response of the basilar membrane shows the highest response in auditory nerve fibers with characteristic frequencies that correspond to the sine-wave components of complex tones

• In this sense the basilar membrane does a Fourier Analysis of the incoming sound signal, breaking it down into component pure tones.

The Basilar Membrane & Complex Tones

96

Fourier Analysis

(a) A complex sound wave.

Applying Fourier analysis to this wave indicates that it is made up of the three components in (b), (c), (d).

These can be represented as a Fourier Spectrum, below

97

Fourier Analysis

98

Questions

• What is the active response? What is its function?

• What does it mean to say that the basilar membrane performs Fourier analysis?

• What are two ways that frequency is coded for by cochlear neurones? How is phase-locking involved?

99

Of Sound MindSubcortical and Cortical Auditory Processing Areas

100

• Auditory nerve fibres synapse in a series of subcortical structures (C? SONIC MG is A1!)

• Cochlear nucleus

• Superior olivary nucleus (in the brain stem)

• Inferior colliculus (in the midbrain)

• Medial geniculate nucleus (in the thalamus)

• Auditory receiving area (A1 in the temporal lobe)

Pathway from the Cochlea to the Cortex

101

102

• Has tonotopic organization: Ventral = low Hz, Dorsal = high Hz

• Bushy cells code for different frequencies and inhibit one another to provide sharper frequency tuning.

• Stellate cells fire for duration of stimulus (slow adapting) with rate indicating intensity of sound.

• Octopus cells fire at sound onset/offset (fast adapting), and may provide sound timing information.

Cochlear Nucleus

103

• Has tonotopic organization

• First site of binaural activity.

• First analysis of sound direction (horizontal only), which works via analysis of:

• Timing differences between the 2 ears.

• Intensity differences between the 2 ears.

Superior Olivary Nucleus

104

Inferior Colliculus

• Receives inputs from many higher areas, such as A1

• May be a “switchboard” for regulating auditory attention

• Also an integrating area for multi-modal perceptual responses such as startle reflexes and reflexive looking.

105

Medial Geniculate Nucleus

• Nucleus of the thalamus

• Tonotopically organized, but in a complex fashion

• Processes all aspects of sound. May be the first site of complex pitch (as opposed to simple frequency) perception.

106

Descending Pathways

• Connections do not only go from ear up to cortex (afferent), but also back down (efferent)

• For instance, olivocochlear neurones--going from superior olivary down to cochlea--can turn down the gain on IHCs to allow them to process higher sound volumes

• Descending connections also cause contractions in small muscles attached to ossicles, causing them to be less mobile and thus “turning down the volume” on high amplitude sounds

107

Questions

• What is the sequence of subcortical nuclei that carries auditory information from ear to cortex? (remember the mnemonic?)

• What do the descending (efferent) fibres leading into the cochlea do?

108

• Signals from MGN arrive in A1, primary auditory cortex

• A1 seems to process relatively simple sound information regarding frequency and location

• A2 processes more complex aspects of sound

• Other areas (Wernicke’s, Broca’s) process speech

Auditory Areas in the Cortex

109

Organization of A1

• A1 is organized in isofrequency sheets, which run left-right.

• Neurones within a sheet have the same CF

• Transverse to the sheets are aural dominance columns and suppression/summation columns

110

111

Schematic of A1 Columnar Organization

16kHz 8kHz 4kHz 2kHz 1 kHz

SuppressLeft

Sum

SuppressBinaural

Sum

SuppressRight

Sum

SuppressLeft

Sum

SuppressBinaural

Sum

SuppressRight

Sum112

• Owl monkeys were trained to discriminate between two frequencies near 2,500 Hz

• Trained monkeys showed tonotopic maps with enlarged areas responding to 2,500 Hz compared to untrained monkeys

• Cases of humans with brain damage to this area show perception difficulties with pitch

Effect of Experience on Tonotopic Maps

113

Effect of Training on Tonotopic Maps

114

• It has been proposed that there are separate “what” and “where” processing circuits in auditory cortex

• What stream starts in the anterior portion of temporal lobe and extends to the prefrontal cortex

• It is responsible for identifying sounds

• Where stream starts in the posterior temporal lobe and extends to the parietal and prefrontal cortices

• It is responsible for locating sounds

• A similar division of labour is well-established in vision

What and Where Streams for Hearing

115

What, Where, & Integration

116

Questions

• What is an isofrequency sheet?

• Describe the columnar organization of A1.

• What do we mean by “what and where streams”?

117