Sound Localization Strategies Sound Localization Strategies in Simple and Complex in Simple and Complex

Environments Environments

Norbert Kopčo

Department of Cybernetics and AI, TU Košice, Slovakia

Hearing Research Center and Center for Computational NeuroscienceBoston University

2

Overview

Intro – Horizontal sound localization

Two studies that examine auditory localization in more complex scenes:

Experiment 1: Shifts in localization due to contextual factors during sequential stimulus presentation

Experiment 2: A priori information and localization of a talker in a multitalker environment

Horizontal Sound localization

Is based on binaural cues (ITD, ILD)

depends on:• stimulus type: spectrum, temporal aspects• environment: anechoic, reverberant• source movement: static, dynamic

But also on:• presence of other stimuli• a priori knowledge / expectations about the

scene

Effect of additional stimuli

The extra sound can act as a:

Masker – localization worse (Gilkey & Good, 1986)

Adaptor – localization biased (Attraction/Repulsion) (Carlile et al., 1986)

Real sound (of which the target is a reflection) – localization worse/suppressed (Litovsky et al., 1999)

Perceptual stream of which the target is or is not a part – localization better or worse (Clifton & Freyman, 1997)

Cue – localization better (Rhodes, 1987; Sach et al., 2000)

Anchor – change localization strategy (Durlach & Braida , 1969)

Current Experiments

Understand auditory localization in a more complex scenes:

Exp 1:- when target is preceded by another identical sound (distractor) from a known location that the listener should ignore- when the task of single target localization is interleaved with the task of localization of a target preceded by a distractor

Exp 2:- when localizing a talker in a multi-talker environment- when we know the location of the distractor talkers

Goal: Show that listeners change how they interpret the binaural cues depending on the context, strategy, and a priori information

Experiment 1: Perceptual and central effects in a

click-sound localization with a preceding distractor

Preceding distractor: Intro

Several preceding studies indicated that preceding stimulus influences localization at SOAs of several hundreds milliseconds (Kopco et al., 2001, Perrott and Pacheco, 1989)

Goal:

- Characterize this influence (bias and std.dev. in responses)

- Determine its cause. Candidates:- short-term adaptation in brainstem representations- reverberation suppression and acoustics- strategy- perceptual organization- attention: focused away from distractor location

Hypotheses

Peripheral factors will have short-term effects

Central factors will influence results at longer separations

Effect of reverberation can be separated by comparing performance in anechoic and echoic rooms

Effect of perceptual organization can be addressed by modifying the stimuli

9

Methods

We measured azimuthal localization performance for a click target stimulus when preceded by another identical click (coming from a known location):

- presented with a short onset asynchrony

- from a different azimuthal location

- in either an anechoic chamber or an ordinary classroom

Performance was compared to that in a control in which there was no preceding distractor click.

Stimuli and setup

Results: Raw data

Complex pattern of biases in responses due to preceding distractor

Perceived target location of frontal sources is “attracted” by lateral distractors

Interactions for SOAs up to 200 ms

Possible mechanisms:

-Precedence effect

-Short-term adaptation

-Grouping of identical target and distractor

Results: Bias due to Distractor

Replacing the single-click distractor by a 5-Hz click-train (such that the target does not fall into the temporal pattern of the distractor) eliminates the “attractive” bias.

Effects not due to acoustics because correct representation is available

Exp 1 – Perceptual organization

Contextual plasticity

There is bias also in the no-distractor responses

The bias is always away from the non-present distractor

Because the runs were interleaved, this bias had to build up anew during each run

Contextual plasticityDifference in no-distractor responses in the frontal and lateral distractor context

-Is independent of azimuth

-Grows over time

-Slightly stronger for the 8-click train context

Contextual plasticity on time scale of minutes

Similar to effects of long-term exposure

Either due to bottom-up factors (distribution of stimuli) or top-down factors (focusing away from distractor)

Follow-up experiments and analysis to explore the contextual effect

Con

text

ual b

ias

Context Exp 1a – Reverberation?

Experimental Design:Compare results from room and anechoic chamber.

Result:Contextual plasticity equally strong in anechoic and reverberant spaces

Context Exp 1b: Effect of timing

Experimental Design:Fix Distractor aheadAdd no-distractor runSystematically vary:- SOA (25 – 400 ms)- distr/no-distr trial occurance ratio

Result:Contextual plasticity- modulated by SOA- modulated by occurance ratio- persists even at longest SOA & lowest dist-trial occuranceOccurance frequency does not matter

Context Exp 1c: Target-Distr. Order

Experimental Design:Use only longest SOAFix ratio at 75% distractorsReverse the Distractor-Target order

Results:Contextual plasticity- persists even for T-D ordering

SOA and D-T T-D order does not matter

Context Exp 1c: Target-Distr. Order

Experimental Design:Use only longest SOAFix ratio at 75% distractorsReverse the Distractor-Target order

Results:Contextual plasticity- persists even for T-D ordering- affects equally dist and no-dist resps- has sharp on- / off- setsBaseline changes over timeIs D used as Anchor?

Context Exp 1d: Sp. Arrangement

Experimental Design:In separate runs, vary which speakers present targets on distractor-trialsNo-distractor targets come from all locations

Result:Contextual plasticity - strongest where distractor-targets are present (+ D location)

- weaker if distr targets on both sides

D-T reference required for contextual plasticity

Context Exp 1d: Abs and Rel Info

Questions:Does distractor provide additional info?Do we combine D-T relative and T-only absolute info?

Analysis:Correlation coefficient between actual location and responses on no-distr trials

Result:Performance consistency improves when distractor- trials are present

Similar results for standard deviations (not shown)

Exp 1 - SummaryPreceding distractor:

-Affects perceived target location at SOAs of up to 200 ms

-Is likely due to combination of - short-term plasticity, and - perceptual-organization factors

Contextual plasticity:

-Is modulated by temporal and spatial characteristics of the inducing task

-Persists even when SOA is 400 ms or when target precedes distractor

-Is likely due to an automatic process that combines the distractor and target localization information, e.g., to provide the optimum target location estimate based on absolute and relative target location

-Results in improvement in localization in terms of correlation coefficient, likely because distractor used as anchor for relative localization (automatic change of strategy?)

Experiment 2: Localizing a speech target

in a multitalker mixture

24

Introduction

Spatial separation of sources enhances speech perception

In complex environments (e.g., with multiple talkers), spatial perception also important for “sorting” acoustic scene into objects and focusing attention on sources of interest (Brungart et al 2001; Freyman et al 1999; Kidd et al 2005; Best et al 2007; Shinn-Cunningham 2008)

Relatively few studies actually measured localization of speech in a multitalker environment (Yost et al., 1996; Hawley et al.1999; Drullman and Bronkhorst 2000; Brungart et al. 2006)

25

Experiment and GoalsStudy horizontal localization of speech in a multitalker environment

Question 1: How does presence of maskers influence localization performance?

Evaluate the effect of maskers on RMS errors in localization responses. Separate effect of detection on localization errors.

Question 2: Is performance affected by a priori knowledge / uncertainty about distribution of masker locations?

Compare performance when masker distribution fixed vs. varied from trial to trial.

Hypotheses: 1. Masker location uncertainty will hurt performance.2. A priori information will eliminate some of the loss, in particular if a simple strategy can be employed to use it.

26

Setup and masker patterns

Pattern 1 Pattern 3 Pattern 5Pattern 4Pattern 2

Room:approx. 3 m x 5 m

Speakers:

only presenting targets

presenting targets andmaskers (see panel b)presenting targets andmaskers (see panel b)

27

MethodsStimuli:

Target: word “two” spoken by a female talkerMaskers: 4 different monosyllabic words,

spoken by 4 male talkers (all longer than target)Target-to-Masker energy ratios: 0 dB or -5 dB

Task: Subjects pointed head to perceived target locationSubjects asked to indicate location only if target heard (5 catch trials with no target per block to monitor

obedience)

Conditions (separate blocks):- Control: No masker- Fixed: Masker pattern fixed across block of trials- Mixed: Masker pattern randomly chosen for each trial

28

Detection

Detection worse at lower TMR, similar in both uncertainty conditions

0 dB -5 dB0

5

10

15

20

25

False Alarm Rate

Pe

rce

nt (

ou

t of 5

ca

tch

tria

ls p

er

blo

ck)

N = 7

TMR0 dB -5 dB

0

5

10

15

20

25

Miss Rate

TMR

Pe

rce

nt

FixedMixed

29

Localization: Control

Good performance with no maskers

All effects of maskers plotted re. control performance on following slides

30

Average across patterns

Detrimental effect of maskers is strong, both for fixed and mixed conditions.

Averaged across patterns and target locations, a priori knowledge helps slightly, by approximately 20%.

31

Average across patterns

When looking only at off-masker locations, a priori knowledge helps dramatically (by 36%)

32

Average across patterns

When looking only at on-masker locations, a priori knowledge has no effect (or hurts performance)

33

Interim SummaryPresence of maskers hurts performance (H1 confirmed), even after accounting for lower detectability.

A priori knowledge of masker locations influences target talker localizability:

- Improving performance at locations from which (the subject knows) no masker can come

- Not affecting (or worsening) performance at locations from which (the subject knows) maskers will come (H2 partially confirmed)

Possible mechanism: - Redistribution of processing resources - “incorrect” strategy: focusing only on off-masker locations

Next, analyze patterns separately to gain more insight into behavior re. H2.

34

Raw Data

Complex effect of target location, masking pattern, uncertainty and TMR

-50 0 500

10

20

Sta

nd

ard

De

via

tion

[°]

-50 0 50

-50 0 50

TMR 0 dB

-50 0 50

-50 0 50

FixedMixed

-50 0 500

10

20

-50 0 50

-50 0 50

TMR -5 dB

Target Azimuth [°]-50 0 50

-50 0 50

35

Analysis of Patterns

A priori information helps for off-masker targets - in almost all patterns- at both TMRs (more at -5 dB)

A priori information can hurt for on-masker targets, mainly for patterns 1 and 2

Overall, effects large for Patts 1 & 2, small for Patt 5.

Complexity of pattern limits use of a priori information.

36

Bias due to Maskers

-50 0 50

-20

0

20

Me

an

Re

spo

nse

Bia

s [°

]

-50 0 50

-50 0 50

TMR 0 dB

-50 0 50

-50 0 50

FixedMixed

-50 0 50

-20

0

20

-50 0 50

-50 0 50

TMR -5 dB

Target Azimuth [°]-50 0 50

-50 0 50

Compression strongest for targets near peripheral maskers

Leftw

ard

R

ight

war

d

37

Masker Uncertainty and Bias

When masker pattern fixed throughout a block, responses biased away from maskers

Bia

s du

e to

Mas

ker

Unc

erta

inty

Left

Rig

ht

-50 0 50

-20

0

20

Me

an

Re

spo

nse

[°]

-50 0 50

-50 0 50

Average across TMR

-50 0 50

-50 0 50

FixedMixed

-50 0 50

-10

-5

0

5

10

m

ea

n [°

]

-50 0 50

-50 0 50

Target Azimuth [°]

Difference between meanFixed

and meanMixed

-50 0 50

-50 0 50

38

Exp 2: Summary1. Mixture has complex effects on localization performance

- compression of mean localization responses near peripheral maskers- it increases localization errors, even after detection errors are eliminated- effect depends on masker pattern, location of target re. maskers, and TMR

2. A priori information about the distribution of speech maskers modulates the effect of masking:

- reducing it (as expected)- but sometimes increasing it (unexpected)

3. These modulatory effects are - likely to be due to change in strategy / assignment of resources:

focusing on off-masker locations in fixed condition- most useful when a priori information can be simply applied (simple patterns)- least useful when a priori information cannot be simply applied (complex

patterns)

4. Can models like Feller and Merima (2004) or Dietz, Ewert and Hohmann (2010) predict these results?

Kopčo N, Best V, Carlile S (2010). Journal of the Acoustical Society of America, 127, 1450-1457

39

Exp 2a: Hearing Impairment

Q: Is the effect of complex masker mixture similar for Hearing-Impaired listeners?

Methods:As in Exp. 2, but only the Mixed condition.

Result:HI listeners performance only affected in mixture (re. normal hearing listeners)

Best, Carlile, Kopčo, van Schaik (2011) J of the Acoust Soc of Am, 129, EL210-EL215

40

Overall Summary

Previous studies showed that the effect of attention and a priori information on sound localization is small in simple scenarios.

Here we show that localization performance does change - even in simple conditions when the listeners can change

their response strategy

- In complex scenarios (e.g., in mixture of 5 talker):- People use a priori information to optimize their

localization performance.- The result is not always an improvement.

Beáta Tomoriová, Ľuboš Hládek, Rudolf AndogaPerception and Cognition Lab, TU Košice, Slovakia

Gin BestHearing Research Center, Boston U & Australian National Acoustics Lab

Barbara Shinn-CunninghamHearing Research Center, Boston University

Simon CarlileUniversity of Sydney

Financial Support: Human Frontiers Science Program, US National Institutes of Health, US National Science Foundation,US National Academy of Sciences, Slovak Science Grant Agency

Collaborators and Support

Current Projects

TU Košice & Boston University:Contextual Plasticity in Sound Localization (US NIH)

Boston University, Harvard Medical School/MGH, UC Riverside, University of Edinburgh:Perceptual and cross-modal learning in auditory distance perception (Marie Curie Project, 7FP EU)

More info: http://pcl.tuke.sk, http://cns.bu.edu/~kopco