11/11/2015copyright james d. johnston 20041 a lightning tutorial on spatial hearing james d....
TRANSCRIPT
04/20/23 Copyright James D. Johnston 2004 1
A Lightning Tutorial on
Spatial HearingJames D. Johnston
Neural Audio
Kirkland, Wa, USA
04/20/23 Copyright James D. Johnston 2004 2
The Origins of Audio
Audio, in the way we use it here, is the technology, science,and art of recording or creating something to be played backvia transducers to the human auditory system. The musicalissues are not included, and indeed warrant a separate and
serious discussion by an expert in the subject.
There have been several varieties of recording and/or transmission
methods, starting with none, i.e.:Live Performance
Acoustic Recording of Live PerformanceElectronic Recording of Complete Performance
Studio Recording of Individual SessionsDirect Synthesis of Music
04/20/23 Copyright James D. Johnston 2004 3
What do we hear in each of these formats or methods?
In the original venue, we hear whatever parts ofthe soundfield are present where we sit. As we move,
turn our heads, move our heads, and focus our attention,
we can learn a great deal about the concert hall, theperformers, and everything else perceptible from the
seat we are sitting in.
No method of recording to the present provides us withanything near this experience.
04/20/23 Copyright James D. Johnston 2004 4
WHAT DID I JUST SAY?Well, it’s not all bad news. There are things that both olderand newer recording methods do very well. What are they?
1) Capture, at one or more points, the pressure, velocity, or 3d pressure/velocity of the sound at a given point in the
atmosphere.
2) Store that in a secure, accurate, and easily reproduced and/or
transmitted form.
3) Reproduce/amplify that recorded (set of) pressures, velocities, and such as electrical analogs.
04/20/23 Copyright James D. Johnston 2004 5
What do we do an “ok” job with?
In a word, loudspeakers. While it’s no doubt that loudspeakersare far and away the weakest link in the array of equipment, they
are still not too bad. They can reproduce most of the dynamicrange, over most of the frequency range, and (if you spend yourmoney wisely) at a level of distortion (in the waveform sense)
that is at least sufferable, if not perfect.
What loudspeakers can’t do, however, is replace what was lostat the original acquisition, or replace what was never extant in
therecording to begin with. Unfortunately, loudspeakers are
very often called upon to do exactly that.
04/20/23 Copyright James D. Johnston 2004 6
What are we missing?
In a real venue, the soundfield is very complex, and wesample it as we move and turn our heads. The soundfield in any
reverberant performing venue willchange in less than one wavelength at any given frequency.
That is to say, at 1 kHz, 1 foot, give or take, will yield asubstantially different soundfield. In venues with highlydispersive reverberation (which is an ideal), in fact, the
coherence length of a signal may be substantially less thana wavelength under some important and realistic conditions.
04/20/23 Copyright James D. Johnston 2004 7
In such a diffuse soundfield, there is an enormous amount of analytic information present in the 1 meter
space about one’s head. Speaking purely from an analyticalviewpoint, one would have to spatially sample the soundfieldaccording to the Nyquist criterion, and capture an enormous
number of channels.
For example, one would have to tile an imaginary sphere withmicrophones every .25” or so in order to correctly sample
the surface of such a space at 20kHz. This sort of calculationleads directly to a catastrophic growth of number of
channels and data rate.
04/20/23 Copyright James D. Johnston 2004 8
What kinds of things will such a hypothetical microphonearray pick up?
1) A set of coherent, plane/spherical waves passing throughthe space. (We will call these the perceptually direct parts
of the soundfield for reasons that will be obvious later.)
2) A set of more or less uncorrelated (beyond the coherencelength at any given frequency) waveforms from each of
the microphones. These mostly uncorrelated microphonescapture the details of the soundfield. (For reasons that
will be obvious later, we will call these diffuse orperceptually indirect parts of the soundfield. The two terms
are not quite synonyms.)
04/20/23 Copyright James D. Johnston 2004 9
How do our ears work?
And what can we detect in a
natural soundfield?
04/20/23 Copyright James D. Johnston 2004 10
The Outer EarThe outer ear provides frequency directivity via shadowing,shaping, diffraction, and the like. It is different (by enoughto matter) for different individuals, but can be summarized
by the “Head Related Transfer Functions” (HRTF’s) or “Head Related Impulse Responses”(HRIR’s) mentioned in the
literature, at least on the average or for a given listener.
The HRTF’s or HRIR’s are ways of determining the effect ona sound coming from a given direction to a given ear.
The ear canal inserts a 1 octave or so wide resonance at about1 to 4 kHz depending on the individual.
04/20/23 Copyright James D. Johnston 2004 11
The Middle EarThe middle ear carries out several functions, the most
important of which, for levels and frequencies thatare normally (or wisely) experienced, is matching the
impedence of the air to the fluid in the cochlea.
There are several other functions related to overloadprotection and such, which are not particularly germane
under comfortable conditions.
The primary effect of the middle ear is to provide a 1-zero
high pass function, with a matching pole at approximately
700Hz or so, depending on the individual.
04/20/23 Copyright James D. Johnston 2004 12
The Inner Ear
A complicated subject at best, the inner ear can be thought ofas having two membranes, each a travelling wave filter. Between the
two membranes are two sets of hair cells, the inner hair cells,and the outer hair cells. The inner hair cells are primarily
detectors. They fire when the movement of the two membranes are different. The outer hair cells are primarily a system thatcontrols the exact points of the very steep low pass filters and
high pass filters. The outer hair cells can polarize and depolarize,and change both their length and stiffness. This polarization is
how they affect the relative tunings of the two membranes.
04/20/23 Copyright James D. Johnston 2004 13
Critical Bands and Cochlear Filters
The overall effect of this filter structure is time/frequency analysis of a particular sort, called critical band (Bark
Scale) or effective rectangular bandwidth (ERB) filter functions. Note that this
is not a set of filters, but rather a continuous set of filters,with lower and higher bandwidths varying according to
the center frequency.
Roughly speaking, critical bandwidths are about 100Hz up to700Hz, and 1/3 octave thereafter. ERB’s are usually
a bit narrower, especially at higher frequencies.
04/20/23 Copyright James D. Johnston 2004 14
A discussion of which is right, and which should be used,is, by itself, well beyond the range of a one-hour seminar.
The basic point that must come out of this discussion is thatthe sound arriving in an ear will be analyzed in somethingapproximating 100Hz bandwidth filters at low frequencies,
and at something like 1/3 octave bandwidths at higherfrequencies, and that the system will detect either thesignal waveform itself (below 500Hz) or the signalenvelope (above 4000 Hz), or a bit of both (in the
range between 500Hz and 4000 Hz). Exactly what isdetected is likewise, by itself, well beyond a one hour
seminar, and furthermore, a consensus is yet to emerge.
04/20/23 Copyright James D. Johnston 2004 15
Hey, we have two ears, not one.
Now that we know roughly what and how one ear candetect a signal, we can examine what can be picked up by
two ears.
That would be, at low frequencies, the leading edges of thefiltered signal itself, and at high frequencies, the leading edges
of the signal envelopes.
The shapes of the waveform/envelope can also be compared,to some extent, as well as their onsets.
04/20/23 Copyright James D. Johnston 2004 16
Because we all live with our HRTF’s, that change slowlywhile we are growing, and very little afterwards (hairhas some effect at high frequencies), the brain learns
them, and we take them into account without anyconscious effort.
In short, we know the delay at different frequencies betweenthe two ears, the relative attenuation, and other such
information as part of learning to hear binaurally.
This means that we can do several things.
04/20/23 Copyright James D. Johnston 2004 17
Detection of Direct (planar) Waves
Since we know the time delay at a given frequency, as wellas the relative attentuations to the two ears as a function offrequency and direction, we can place a direct plane wavesound in direction by understanding at a very low level the
time delays and amplitude attenuations at different frequenciesfor wideband signals with quick onsets. For such signals,
having an idea of what the source “sounds like” is very helpful,as the spectral shaping at either ear can disambiguate thetime-delay information, and provide more easily accessed
elevation and front/back information.
04/20/23 Copyright James D. Johnston 2004 18
Listening to Diffuse SoundfieldsDiffuse soundfields will not have correlated envelopes at the
two ears in the relevant frequency ranges. The brain interprets suchwaveforms as “surrounding us” or “omnidirectional”. Note that
these waveforms inside a critical bandwidth may or may not haveleading edges. Note that leading edges can sometimes line up
to provide false or confusing directional cues.A perceptually indirect signal (a term applying only above 2kHz
or so) will have a flat envelope, and thereby provide no informationto correlate. In other words, the flat envelopes will in fact
‘correlate’but the auditory system has no features to lock onto to determine
either direction or diffusion. Such signals are often ignored in the most complex stages of hearing, but will have the same
effect of “surround” or “omnidirectional’ when focused upon.
04/20/23 Copyright James D. Johnston 2004 19
How does this relate to recording and playback?
First, a recording of a plane wave should be reproducedas a plane wave, or as something that the ear’s detection
mechanisms can treat as a plane wave.
Second, diffuse signals should not be recorded at one pointand then reproduced as plane (or spherical) waves, because
the auditory system will detect any “leading edges” in them asfalse directional cues, leading to the classic problem of
being “sucked into” loudspeakers during things like applause, or
in multichannel audio, reverberation from the rear speakers.
04/20/23 Copyright James D. Johnston 2004 20
3 Front Channels - a good start?
With 3 front channels, and using nearly coincident mikingtechniques, it is possible to create a good illusion of the
front soundfield without having depth or elevation problems.
Aside from avoiding the problem of interference, non-coincident 3-channel miking has other good
effects, specifically a wider listening area, measured by subjective test to be 6 to 8 times the
listening area available in the 2-channel presentation.
BUT – You notice we’re still talking direct sound here!
04/20/23 Copyright James D. Johnston 2004 21
How about the Back?
Again, empirical research shows that two back channels canprovide a good sense of reverberation and depth to the rear
of the listener.
The same two channels can create, if they are at the side of thelistener, a good sense of side to side envelopment.
You can’t have both at once, unless you have 4 more channels,of course.
So much for 5.1, then?
04/20/23 Copyright James D. Johnston 2004 22
No, so much for 5.0The usual 5.1 home theatre setup, meaning 5 high-frequencyradiators, and one subwoofer radiating below 90Hz or so, has
additional problems beyond the 5.0 problem.
Specifically, although one can not LOCALIZE signals belowabout 90 Hz, one can detect spatial effects from interaural
phase differences down to about 40Hz.
The AT&T Labs “Perceptual Soundfield Reconstruction”Demo, no longer available, contained a very nice example of these
effects, and how they can change “boomy bass” in the2-radiator case into “bass spread about a room” in the
5-channel case.
(And you notice we’re still talking mostly about direct signals.)
04/20/23 Copyright James D. Johnston 2004 23
Ok, what about Indirect Signals?Well, as mentioned long ago, indirect signals with envelopes
that have peaks or “leading edges” can create false cuesif reproduced by a direct radiator.
What does this mean?
That’s a good question. What I think it means is that agood recording and playback setup will have both direct
and indirect radiation capability in each loudspeaker,and that the two will be driven with appropriate signals
Alternative, interference between speakers in well-recorded multichannel recordings can break up direct-sound patterns.
04/20/23 Copyright James D. Johnston 2004 24
What Can Be Done?
Except for a few parts of the space (Gerzon’s Soundfield Microphone,
various widely separated miking methods, the Neural Mike Array and AT&T Labs (Johnston/Lam AES Preprint) Perceptual
Soundfield Reconstruction), recording methods have not been systematically
or carefully examined, and the results of many promisingmiking and simulation methods are simply unknown.
Effectively, the field is just now becoming feasible.
04/20/23 Copyright James D. Johnston 2004 25
A variety of tutorials can be found at:www.aes.org/sections/pnw
Look at the “slide deck”” tab.