CNBH, PDN, University of Cambridge

Roy PattersonCentre for the Neural Basis of Hearing Department of Physiology, Development and Neuroscience University of Cambridge

Part II: Lent Term 2014: ( 1 of 4)

email   rdp1@cam.ac.uk

http://www.pdn.cam.ac.uk/groups/cnbh/teaching/lectures/

Lecture slides, sounds, videos and background papers on

Central Auditory Processing

Lecture slides on CamTools:

CNBH, PDN, University of Cambridge

Act III: the processing of communication sounds: How (auditory signal processing)

Act I: the information in communication sounds (animal calls, speech, musical notes)

The Overture

Act IV: the processing of communication sounds: Where (auditory anatomy, physiology, brain imaging)

[Tune7nCPHtone.mov]

axial axial

Act II: the perception of communication sounds (the robustness of perception)

CNBH, PDN, University of Cambridge

Act IThe form of communication sounds including

speech and musical notes

http://www.pdn.cam.ac.uk/groups/cnbh/teaching/lectures/PSDWshar08.pdf

http://www.pdn.cam.ac.uk/groups/cnbh/teaching/lectures/PGW_SHAR_2010.pdf

CNBH, PDN, University of Cambridge

The resonance provides distinctive information about the shape and size of resonators in the sender’s body.

to declare territories and attract mates, are typically

Pulse

Resonance

Time

Am

plitu

de

Sounds used to communicate at a distance,

Pulse-Resonance Sounds

The pulse marks the start of the communication.

CNBH, PDN, University of Cambridge

source-filter systems : pulse-resonance soundssource filter

The pulses are produced in streams, each with a resonance

The streams have distinctive onsets and offsets, and a stream with its onset and offset is referred to as a syllable.

http://www.pdn.cam.ac.uk/groups/cnbh/teaching/lectures/PSDWshar08.pdf

CNBH, PDN, University of Cambridge

Communication ‘syllables’ of four different animals

Fish

Frog

Human

Macaque

400 ms

Pat

ters

on, S

mith

, van

Din

ther

and

Wal

ters

(20

08).

CNBH, PDN, University of Cambridge

Long vocal tract ~ Man

VC mass determines GPR (voice pitch)

VT shape determines resonance shape (vowel type)

The information in speech sounds:

/a/m

VT length determines resonance rate

CNBH, PDN, University of Cambridge

/a/w

Shorter vocal tract ~ WomanLong vocal tract ~ Man

2/3

The information in speech sounds:

VT length determines resonance rate

/a/m

CNBH, PDN, University of Cambridge

Low

High

Pitch

Long

Short

VT

L

Time Time

Patterson, Smith, van Dinther and Walters (2008).

CNBH, PDN, University of Cambridge

there are three important kinds of information:

• resonance shape the message

• resonance scale resonator size, or body size

• glottal pulse rate pitch

In natural communication sounds, at the syllable level,

What happens as a child grows up into an adult?

http://www.pdn.cam.ac.uk/groups/cnbh/teaching/lectures/PSDWshar08.pdf

CNBH, PDN, University of Cambridge

waveform and spectrum of a child’s /a/

fine structure

pulse rate resonance shape

envelope shape

CNBH, PDN, University of Cambridge

waveform and spectrum of a child’s /a/

Frequency on a logarithmic axis (octaves)

position of the envelope

pulse rate

position of the fine-structure

resonance rate

Ss

Sf

CNBH, PDN, University of Cambridge

The role of Ss and Sf in speaker perception

Dec

reas

ing

VT

L

Increasing GPR

Kawahara and Irino (2004). Principles of speech manipulation system STRAIGHT. In Speech separation by humans and machines, P. Divenyi (Ed.), Kluwer Academic, 167-179.

( Ss )

(1/S

f )

CNBH, PDN, University of Cambridge

Instrument FamilyInstrument Register

Envelope ShapeSs and Sf

The role of Ss and Sf in instrument perception

http://www.pdn.cam.ac.uk/groups/cnbh/teaching/lectures/PGW_SHAR_2010.pdf

CNBH, PDN, University of Cambridge

Instruments with same shape and construction,

sound similar.

Scaling

violinviola

cello

Regist

erS s and S f

Acoustic scale in instrument perception

Envelope Shape

CNBH, PDN, University of Cambridge

The perception of musical notes and instruments

Instrument FamilyInstrument Register

Envelope ShapeSs and Sf

CNBH, PDN, University of Cambridge

Waveforms for trumpet and trombone

resonance

Time

van

Din

ther

and

Pat

ters

on (

2004

)

pulses

CNBH, PDN, University of Cambridge

The effect of Ss and Sf on instrument perception

PitchDec

reas

ing

Siz

eFrench Horn

(1/S

f )

(Ss)

CNBH, PDN, University of Cambridge

The perception of musical notes and instruments

Instrument FamilyInstrument Register

Envelope ShapeSs and Sf

CNBH, PDN, University of Cambridge

The acoustic scale values in communication sounds tell us about the register of an individual or instrument

within a population of people or a family or instruments

The ratio of the acoustic scale values provides information about the quality of an instrument or

the naturalness of an individual’s voice

http://www.pdn.cam.ac.uk/groups/cnbh/teaching/lectures/PGW_SHAR_2010.pdf

CNBH, PDN, University of Cambridge

pi pe ko kuuu ni ne mo muuu

SsSf

CNBH, PDN, University of Cambridge

pi pe ko kuuu ni ne mo muuu

pi pe ko kuuu ni ne mo muuu

pi pe ko kuuu ni ne mo muuu

CNBH, PDN, University of Cambridge

1/V

TL

(1/

Sf )

Position of the fine structure

Pos

itio

n o

f th

e en

velo

pe

CNBH, PDN, University of Cambridge

1/V

TL

(1/S

f )

CNBH, PDN, University of Cambridge

General conclusions:

Partitioning the information in the magnitude spectrum into (a) envelope shape, (b) position of the envelope, Sf, and

(c) position of the fine structure, Ss,

provides us with the basis for a structured description of the perception of communication sounds.

CNBH, PDN, University of Cambridge

What and Who in vocal sounds

The shape of the envelope largely determines vowel type. It determines the “What” aspect of the communication, in the sense of “What is being said”.

I:

II: The acoustic scale variables, Ss and Sf jointly determine the register of the voice, and whether the speaker sounds normal or unusual. They jointly determine much of the “Who” aspect of the communication in the sense of “Who is speaking”.

CNBH, PDN, University of Cambridge

Family and Register in musical sounds

The shape of the envelope/resonance largely determines the sound of the instrument family, or the family aspect of instrument perception.

I:

II: The acoustic scale variables, Ss and Sf, jointly determine the register or size aspect of instrument perception.

III: The relationship between the acoustic scale variables, Ss

and Sf determines whether an instrument sounds good.

CNBH, PDN, University of Cambridge

End of Act IThank you

Patterson, R. D., Gaudrain, E. and Walters, T. C. (2010). “The perception of family and register in musical tones,” In: Music Perception. Jones, M.R., Fay, R.R. and Popper, A.N., (eds).  New York, Springer-Verlag. 13-50.

Patterson, R. D., van Dinther, R. and Irino, T. (2007). “The robustness of bio-acoustic communication and the role of normalization,” Proc. 19th International Congress on Acoustics, Madrid, Sept, ppa-07-011.

Patterson, R. D., Smith, D. R. R., van Dinther, R. and Walters, T. C. (2008). “Size Information in the Production and Perception of Communication Sounds,” In: Auditory Perception of Sound Sources. Yost, W. A., Popper, A. N., and Fay, R. R. (Eds). Springer, LLC, New York, 43-75.

http://www.pdn.cam.ac.uk/groups/cnbh/teaching/lectures/PGW_SHAR_2010.pdf

http://www.pdn.cam.ac.uk/groups/cnbh/teaching/lectures/PSDWshar08.pdf