Dual-Microphone Speech Dereverberation using a ReferenceSignalCitation for published version (APA):Habets, E. A. P., & Gannot, S. (2007). Dual-Microphone Speech Dereverberation using a Reference Signal. InProceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP 2007)15-20 April 2007, Honolulu, Hawaii, ISA (Vol. 4, pp. 901-904) https://doi.org/10.1109/ICASSP.2007.367216

DOI:10.1109/ICASSP.2007.367216

Document status and date:Published: 01/01/2007

Document Version:Publisher’s PDF, also known as Version of Record (includes final page, issue and volume numbers)

Please check the document version of this publication:

• A submitted manuscript is the version of the article upon submission and before peer-review. There can beimportant differences between the submitted version and the official published version of record. Peopleinterested in the research are advised to contact the author for the final version of the publication, or visit theDOI to the publisher's website.• The final author version and the galley proof are versions of the publication after peer review.• The final published version features the final layout of the paper including the volume, issue and pagenumbers.Link to publication

General rightsCopyright and moral rights for the publications made accessible in the public portal are retained by the authors and/or other copyright ownersand it is a condition of accessing publications that users recognise and abide by the legal requirements associated with these rights.

• Users may download and print one copy of any publication from the public portal for the purpose of private study or research. • You may not further distribute the material or use it for any profit-making activity or commercial gain • You may freely distribute the URL identifying the publication in the public portal.

If the publication is distributed under the terms of Article 25fa of the Dutch Copyright Act, indicated by the “Taverne” license above, pleasefollow below link for the End User Agreement:www.tue.nl/taverne

Take down policyIf you believe that this document breaches copyright please contact us at:openaccess@tue.nlproviding details and we will investigate your claim.

Download date: 09. Mar. 2020

DUAL-MICROPHONE SPEECH DEREVERBERATION USING A REFERENCE SIGNAL

EAR Habets, Student member, IEEE

Department of Electrical EngineeringTechnische Universiteit Eindhoven

Eindhoven, The NetherlandsEmail: e. habets@ieee.org

ABSTRACT

Speech signals recorded with a distant microphone usually containreverberation, which degrades the fidelity and intelligibility of speech,and the recognition performance of automatic speech recognitionsystems. In this paper we propose a speech dereverberation sys-tem which uses two microphones. A Generalized Sidelobe Canceller(GSC) type of structure is used to enhance the desired speech signal.The GSC structure is used to create two signals. The first signal isthe output of a standard delay and sum beamformer, and the secondsignal is a reference signal which is constructed such that the directspeech signal is blocked. We propose to utilize the reverberationwhich is present in the reference signal to enhance the output of thedelay and sum beamformer. The power envelope of the referencesignal and the power envelope of the output of the delay and sumbeamformer are used to estimate the residual reverberation in theoutput of the delay and sum beamformer. The output of the delayand sum beamformer is then enhanced using a spectral enhancementtechnique. The proposed method only requires an estimate of thedirection of arrival of the desired speech source. Experiments us-ing simulated room impulse responses are presented and show sig-nificant reverberation reduction while keeping the speech distortionlow.

Index Terms- Speech dereverberation, speech enhancement.

1. INTRODUCTION

Acoustic signals radiated within a room are linearly distorted by re-flections from walls and other objects. These distortions degradethe fidelity and intelligibility of speech, and the recognition perfor-mance of automatic speech recognition systems. Early reflectionsmainly contribute to coloration, or spectral distortion, while late re-flections, or late reverberation, contribute noise-like perceptions ortails to speech signals [1]. Spectral coloration and late reverbera-tion cause users of hearing aids to complain of being unable to dis-tinguish one voice from another in a crowded room. One of thereasons why reverberation degrades speech intelligibility is the ef-fect of overlap-masking, in which segments of an acoustic signalare affected by reverberation components of previous segments. Inthis paper we have investigated the application of signal processingtechniques to improve the quality of speech recorded in an acousticenvironment.

Dereverberation algorithms can be divided into two classes. Theclassification depends on whether the Room Impulse Responses(RIRs) need to be known or estimated beforehand. Until now blind

Thanks to the Dutch Technology Foundation STW (project EEL 4921),applied science division ofNWO, for funding.

S. Gannot, Senior Member, IEEE

School of EngineeringBar-Ilan UniversityRamat-Gan, Israel

Email: gannot@eng.biu.ac.il

estimation of the RIRs, in a practical scenario, remains an unsolvedbut challenging problem [2]. Even if the RIRs could be estimated,the inversion and tracking would be very difficult. While these tech-niques try to recover the anechoic speech signal we like to sup-press the tail of the RIR by means of spectral enhancement. In thelast decade many speech enhancement solutions have been proposedwhich do not require an estimate ofthe RIR. For example algorithmsbased on processing of the linear prediction (LP) residual signal[3, 4]. Other algorithms are based on spectral enhancement tech-niques and utilize a statistical reverberation model [5, 6, 7]. The lateralgorithms do not require detailed knowledge on the RIR structure,but require some apriori information about room characteristics, forexample the reverberation time.

In this paper we propose a dual-microphone speech dereverber-ation system. A Generalized Sidelobe Canceller (GSC) [8] type ofstructure is used to enhance the desired speech signal. The GSCstructure is used to create two signals. The first signal is the outputof a standard delay and sum beamformer, and the second signal isa reference signal which is constructed such that the direct speechsignal is blocked. We propose a novel method which utilizes the re-verberation present in the reference signal to enhance the output ofthe delay and sum beamformer. The power envelope of the refer-ence signal and the power envelope of the output of the delay andsum beamformer are used to estimate the residual reverberation inthe output of the delay and sum beamformer. The signal is then en-hanced using a spectral enhancement technique. An advantage ofthe proposed method is that it requires a minimum amount of a pri-ori knowledge, since we only require an estimate of the Direction ofArrival (DOA) of the desired speech source.

The outline of this paper is as follows, in Section 2 the problemis described. In Section 3 we describe the proposed dereverberationalgorithm. Evaluation using simulated RIRs are presented in Section4. Discussion and conclusions can be found in Section 5.

2. PROBLEM STATEMENT

The mth microphone signal (m C {1, 2}) is denoted by zm(n) andresults from the convolution of the anechoic speech signal s(n) andthe RIR between the source and the corresponding microphone. TheRIR between the source and the mth microphone, at time n, is mod-elled as a finite impulse response of length L, and is denoted byam(n) = [am,o(n),.. ., am,L- 1(n)]T. The RIR is divided intotwo parts such that

am,j (n) = ad j (n) + a (n) (1)

where j is the coefficient index, ad (n) consists of the direct path,and a' (n) consists of all echoes. In the sequel we assume that the

1-4244-0728-1/07/$20.00 ©2007 IEEE IV - 901 ICASSP 2007

Authorized licensed use limited to: Eindhoven University of Technology. Downloaded on March 29,2010 at 03:24:00 EDT from IEEE Xplore. Restrictions apply.

Zi(k l) Q(k,l1) Post Filter D(k,l1)2 ~~~~~~~~~~~G(k, 1)

Ar(k, I)Z2(k, 1) U(k,)REE1)

Fig. 1. Dual Microphone Speech Dereverberation System (REE: Re-verberant Energy Estimator).

microphone array is steered towards the desired source using an es-timate of the DOA of the direct signal, i.e., the direct speech signalsin zi (n) and Z2 (n) are time-aligned. The mth microphone signal isgiven by

zm (n) = am (n)) s(n) + (a' (n)) s(n) ' (2)

d,m(n) rm (n)

where s(n) = [s(n),... s(n -L + 1)]T, dm(n) is the desired(direct) speech component, and rm (n) denotes the reverberant com-ponent which contains all reflections. Using the Short-Time FourierTransform (STFT), we have in the time-frequency domain

Zm(k, I) = Dm(k, 1) + Rm(k, 1) Vm E {1, 2}, (3)

where k represents the frequency bin index, and 1 the frame index.Figure 1 shows the proposed dual-microphone speech derever-

beration system. The time-frequency signal Q(k, 1) is the output ofa delay and sum beamformer (in this case with zero delay), i.e.,

Q(k, 1) = 1 (Z1 (k, 1) + Z2 (k, 1))= D(k, 1) + Rq(k, 1),

where D(k, 1) denotes the direct speech, and Rq(k, 1) = (RI (k, 1)+R2 (k, 1)) denotes the residual reverberation ofthe speech in Q(k, 1).The reference signal U(k, 1) is constructed using the differencebetween the two microphone signals, i.e.,

(4)

In case there are no steering errors the direct signal is perfectlyblocked, i.e., DI (k, 1) -D2(k, 1) = 0, such that

U(k, 1) = 2 (RI (k, 1) -R2 (k, 1)) .(5)We can now see that U(k, 1) contains the (spatially filtered) rever-beration.

Note that the exact relation between U (k, 1) and Rq (k, 1) is verycomplex due to the spatial filtering, e.g., for low frequencies the de-lay and sum beamformer is omnidirectional, while the null beam-former, which is used to create the reference signal, will not onlysuppress the direct signal but also some reflections. However, us-ing the statistical reverberation model used in [7] it can be shownthat for frequencies above the Schroeder frequency S{ U(k, 1) 12 }S{ Rq(k, 1) 2}, where S{ } denotes the mathematical expectation.

The spectral speech component D(k, 1) is obtained by applyinga frame and frequency dependent spectral gain function G(k, 1) (seeSection 3) to the spectral component Q(k, 1), i.e.,

D(k,l) = G(k,l) Q(k, 1). (6)

The dereverberated speech signal d(n) can be obtained using theinverse STFT and the weighted overlap-add method.

3. PROPOSED METHOD

In this section we show how the residual reverberant energy can beestimated using the reference signal. Additionally, we design a postfilter which uses this estimate to enhance the speech signal.

3.1. Reverberant Energy Estimator

First the power envelopes of the output of the delay and sum beam-former Q(k, 1) and the reference signal U(k, 1) are recursively esti-mated, using

Aq(k, 1) = 3Aq(k, 1 -1) + (1 -) Q(k, 1) 2,and

A,(k, 1) = 3Azu(k, 1 -1) + (1 -3) U(k, 1) 2,

(7)

(8)

respectively, where 3 (0 < 3 < 1) is the forgetting factor.Let us assume that the estimated residual reverberant energy in

frequency bin k at frame 1 can be estimated using

(9)

where W(k) is a frequency dependent constant. The parameter Acan be used to control the end point ofthe uncompensated part oftheresidual reverberation, e.g., by increasing A one can reduce only latereflections while leaving the early reflections intact. The end point ismeasured with respect to the arrival time of the direct speech signal.Note that A is a positive integer value. The time related to A is givenby 'F, where fs denotes the sampling frequency and F denotes theframe rate in samples of the STFT. The frame rate F depends on thewindow length and the overlap of the STFT.

We now define an error signal Ae(k, 1) as,

(10)

An adaptive algorithm is used to minimize the following quadraticcost function

such that

J = (Ae (k, l))2, (1 1)

W(k, 1 + 1) = W(k, 1) -VJw,2(12)

where ,u denotes the step-size parameter, and VJw denotes the gra-dient with respect to W(k, 1), which is given by

VJw =-2Ae(k,l)A,u(k, - A). (13)

Note that Ae(k, 1) and A,(k, 1) are real and positive values for all kand 1.

3.2. Post Filter

Many spectral enhancement techniques are described in the litera-ture. Spectral subtraction methods are the most widely used due tothe simplicity of implementation and the low computational load,which makes them the primary choice for real-time applications. Acommon feature of this technique is that the interference reductionprocess can be related to the estimation of a short-time spectral at-tenuation factor [9]. Since the spectral components are assumed tobe statistically independent, this factor is adjusted individually as afunction ofthe relative local aposteriori Signal to Interference Ratio(SIR) on each frequency. The a posteriori SIR is defined as

(k, l1) <A Q(k, 1) 2A, (k, 1)

(14)

IV - 902

I

A, (k, 1) = W (k) A,,, (k, 1 A) I

A, (k, 1) = A q (k, 1) k (k, 1).

U (k, 1) = 1 (Zi (k, 1) Z2 (k, 1))2

Authorized licensed use limited to: Eindhoven University of Technology. Downloaded on March 29,2010 at 03:24:00 EDT from IEEE Xplore. Restrictions apply.

Using informal listening tests we concluded that magnitude sub-traction gives very good performance. The gain function related tothe magnitude subtraction is given by [9]

G(k, 1) = max { 1'y(k- iGmin 1 (15) s

W: 200(

g.owhere Gmin is a lower-bound constraint for the spectral gain functionwhich allows us to control the maximum amount of reverberationthat is reduced. In the following experiments Gmin was set to 0.1,which corresponds to maximum attenuation of 20 dB.

4. EVALUATION

In this section we present evaluation results that were obtained usingsynthetically reverberated signals. One speech fragment which con-

sists of a female voice of 20 seconds and a male voice of 20 seconds,sampled at 8 kHz, was used in all experiments. The synthetic RIRswere generated using the image method [10], and the reflection co-

efficients were set such that the reverberation time, denoted by T60was equal to approximately 200, 400 and 600 ms. Experiments were

conducted using different distances between the source and the cen-

ter of the array, denoted by d, ranging from 1 to 2 m. The distancebetween the two microphones was set to 10 cm.

The analysis window of the STFT was a 256 point Hammingwindow, and the overlap between two successive frames is set to75%. Each frame is zero padded with 256 points to avoid wrap

around errors. The forgetting factor 3 in (7) and (8) was set to 0.9,and the step-size /, in (12) was set to 0.2.

We used the segmental Signal to Interference Ratio (SIRseg),Bark Spectral Distortion (BSD), and a recently proposed evaluationmeasure developed by Wen and Naylor called the Reverberation De-cay Tail (RDT) [11] to evaluate the proposed algorithm. The RDTjointly characterizes the relative energy in the tail of the RIR and therate of decay. In [12] the RDT measure was tested using three dere-verberation methods, the results were compared to the subjectiveamount of reverberation indicated by 26 normal hearing subjects.The results showed a strong correlation between the RDT values andthe amount of reverberation perceived by the subjects. Note thathigher RDT values correspond to a higher amount of relative energy

in the tail and/or a slower decay rate. The (properly delayed) ane-

choic speech signal was used as a reference signal for these speechquality measures. As a reference dereverberation method we showthe quality measures calculated from the output ofthe delay and sumbeamformer (DSB). In Table 1 the results are shown for d = 1 m andd = 2, and A = 0 1. The quality measures are calculated using 40seconds of speech data after the filter coefficients have converged.We can see that the SIRseg is increased in almost all scenarios. TheBSD measure indicates that the average Bark spectral distance isslightly increased. The RDT values are very consistent and indicate a

clear improvement in all cases.

In Figure 2 the spectrogram of the anechoic signal, the micro-phone signal zi (n) and the output of the proposed algorithm forA = 0 and A = 16 are depicted (d = 2 m and T60 = 400 ms).Note that the effect of overlap-masking is reduced and that the firstreflections can be preserved by increasing A. In Figure 3 the micro-phone signal zi (n) and the output of the proposed algorithm, usingd = 2 m, T60 = 400 ms and A = 8, are depicted. In both figures iscan be seen that the smearing caused by reverberation, is reduced.

'The results are available for listening on the following web page:http://www.sps. ele. tue.nlmembersle. a.p.habetslicasspO7.

400(Proposed A = 0

4000Proposed A = 16

3000 3000

0 000.5 1 0 005

Time (sec) Time (sec)

Fig. 2. Spectrograms of the anechoic, reverberant and proposed sig-nalsusingA = OandA = 16(d = 2mandT6o = 400 ms).

Anechoic

0.5-

05TT10.

0.5 1.5 2.5

Reverberant

, 05

$ -05

-1 _0 0.5 1.5 2.5

Proposed A =

, 05

0.5 1.5

Fig. 3. Anechoic, reverberant and proposed (AT60 = 400 ms) signals.

2.5

8, d = 2 m, and

In case the DOA estimation is not perfect the direct speech sig-nal will leak into the reference signal. To study the effects of steeringerrors due to errors in the DOA estimate we introduced a steering er-

ror of 5 degrees. The spectrogram of the processed signals, with andwithout steering error, are depicted in Figure 4. We can see that theproposed algorithm is still able to suppress a significant amount ofreverberation. However, it can also be seen that some additional dis-tortion was introduced by the proposed dereverberation algorithm.

IV - 903

N

>

C: 200u

Time (sec) Time (sec)

Irl

-10

Authorized licensed use limited to: Eindhoven University of Technology. Downloaded on March 29,2010 at 03:24:00 EDT from IEEE Xplore. Restrictions apply.

Table 1. Experimental results in terms of segmental Signal to Interference Ratio (SIRseg), Bark Spectral Distortion (BSD) and ReverberationDecay Tail (RDT) for A = 0.

T60 = 200 msSIRseg BSD RDT

T60 = 400 msSIRseg BSD RDT

T60 = 600 msSIRseg BSD

Unprocessed 8.40 dB 0.05 dB 53 -0.13 dB 0.13 dB 250 -4.31 dB 0.20 dB 5681 m DSB 9.03 dB 0.04 dB 42 0.37 dB 0.10 dB 175 -3.95 dB 0.17 dB 463_ Proposed 6.83 dB 0.06 dB 23 [ 2.30 dB 0.13 dB 126 j -0.26 dB 0.18 dB 180

3.52 dB4.41 dB4.05 dB

0.15 dB0.12 dB0.18 dB

897466

-4.17 dB-3.35 dB-0.12 dB

0.31 dB0.23 dB0.34 dB

454337200

-8.15 dB-7.43 dB-2.83 dB

0.41 dB0.33 dB0.45 dB

7. REFERENCES

Proposed4000

'N 3000

G 2000

4;)" 1000

0

4000

N' 3000

2000

lon

2 3 4Time (sec)

Prono.sedi with 9 stecriniq err(

Time (sec)

Fig. 4. Spectrograms of the processed signal with and without a

steering error of 5 degrees (A = 0, d = 2 m, and T6o = 400 ms).

5. DISCUSSION AND CONCLUSIONS

In this paper we proposed a dual-microphone speech dereverberationalgorithm. A GSC type of structure was used to enhance the desiredspeech signal. We proposed to use a reference signal to enhancethe output of the delay and sum beamfomer. The advantage of theproposed solution is that we only require an estimate of the DOA.Although no additional interferences have been taken into account,i.e., coherent or non-coherent noise sources, we would like to pointout that the power envelope of the reverberant component could alsobe estimated in a noisy environment (see for example [7]). Experi-mental results have shown that the proposed solution can be used toreduce the reverberation while keeping speech distortion low. Futureresearch will focus on the extension to multi-microphones, which al-lows better estimation ofthe residual reverberant energy, and to morerealistic situations where additional interferences are present.

6. ACKNOWLEDGEMENT

The authors express there thanks to Jimi Wen from the Imperial Col-lege London, United Kingdom, for making the code for the RDT mea-

sure available.

[1] J. B. Allen, D. A. Berkley, and J. Blauert, "Multimicro-phone signal-processing technique to remove room reverber-ation from speech signals," Journal of the Acoustical SocietyofAmerica, vol. 62, no. 4, pp. 912-915, 1977.

[2] Y. Huang, J. Benesty, and J. Chen, "Identification of acousticMIMO systems: Challenges and opportunities," Signal Pro-cessing, vol. 6, no. 86, pp. 1278-1295, 2006.

[3] B. Yegnanarayana and P. S. Murthy, "Enhancement of rever-

berant speech using LP residual signal," IEEE Transactions on

Speech andAudio Processing, vol. 8, no. 3, pp. 267-281, 2000.[4] N. D. Gaubitch, P. A. Naylor, and D. Ward, "On the use of

linear prediction for dereverberation of speech," in Proc. oftheInternational Workshop on Acoutsic Echo and Noise Control(IWAENC'03), Kyoto, Japan, 2003, pp. 99-102.

[5] K. Lebart and J.M. Boucher, "A new method based on spectralsubtraction for speech dereverberation," Acta Acoustica, vol.87, pp. 359-366, 2001.

[6] E.A.P. Habets, "Multi-Channel Speech Dereverberation basedon a Statistical Model of Late Reverberation," in Proc. of the30th IEEE International Conference on Acoustics, Speech, andSignal Processing (ICASSP 2005), Philadelphia, USA, March2005, pp. 173-176.

[7] E.A.P. Habets, S. Gannot, and I. Cohen, "Dual-MicrophoneSpeech Dereverberation in a Noisy Environment," in Proc. of

the IEEE International Symposium on Signal Processing andInformation Technology (ISSPIT 2006), Vancouver, Canada,August 2006, pp. 651-655.

[8] L.J. Griffiths and C.W. Jim, "An Alternate Approach to Lin-early Constrained Adaptive Beamforming," IEEE Transactionon Antennas and Propagation, vol. 1, no. 30, pp. 27-34, 1982.

[9] S. F. Boll, "Suppression of acoustic noise in speech using spec-tral subtraction," IEEE Trans. Acoust., Speech, Signal Process-ing, vol. 27, no. 2, pp. 112-120, April 1979.

[10] J. B. Allen and D. A. Berkley, "Image method for efficientlysimulating small room acoustics," Journal of the AcousticalSociety ofAmerica, vol. 65, no. 4, pp. 943-950, 1979.

[11] J.Y.C. Wen and P. Naylor, "An evaluation measure for rever-

berant speech using tail decay modelling," in Proc. ofthe Eu-ropean Signal Processing Conference (EUSIPCO 2006), Flo-rence, Italy, 2006, pp. 1-4.

[12] J.Y.C. Wen, N.D. Gaubitch, E.A.P. Habets, T. Myatt, and P.A.Naylor, "Evaluation of Speech Dereverberation Algorithmsusing the MARDY Database," in Proc. of the 10th Interna-tional Workshop ofAcoutsic Echo andNoise Control (IWAENC2006), Paris, France, September 2006, pp. 1-4.

IV - 904

d Method

2 mUnprocessedDSBProposed

RDT

939766296

u

Authorized licensed use limited to: Eindhoven University of Technology. Downloaded on March 29,2010 at 03:24:00 EDT from IEEE Xplore. Restrictions apply.