p. 1DSP-II
DSP Everywhere…Applications of DSP in Audio and
Digital Communications
Simon Doclo
Dept. Elec. Engineering, K.U.Leuven
www.esat.kuleuven.ac.be/~doclo/
14/12/01 Simon Doclo DSP Everywhere… p. 2
• DSP in Digital Communications
• 3G-systems: UMTS, CDMA
• Wireless systems: GSM, WLAN
• Modems: Cable, ADSL, VDSL
• Line echo cancellation
Introduction
• Satellite communications
• Optical communication
14/12/01 Simon Doclo DSP Everywhere… p. 3
• DSP in Audio Applications
Introduction
• Hearing aids / cochlear implants
• Hands-free telephony
• Tele-conferencing
• Voice-controlled systems
• Audio effects
• Audio and speech coding
14/12/01 Simon Doclo DSP Everywhere… p. 4
• Other applications
• Medical applications
• Cryptography
• Process control in chemical, pharmaceutical, energy plants
• Image and video processing
• …
Introduction
Anywhere (digital) signals are present, DSP-techniques are required!
14/12/01 Simon Doclo DSP Everywhere… p. 5
Overview
• Introduction
• DSP in digital communications systems:– xDSL-modems: modulation, equalisation
• DSP in audio applications:– Hands-free communication: echo, noise and reverberation– Basic techniques:
• Acoustic echo cancellation (AEC)
• Multi-microphone beamforming
– Application: hearing aids
• Conclusion
14/12/01 Simon Doclo DSP Everywhere… p. 6
Telephone Line modems
• High-speed data communication:– optical, cable, wireless, telephone line
• Telephone Line Modems– voice-band modems : up to 56kbits/sec in 0…4kHz band– ADSL modems : up to 6Mbits/sec in 30kHz…1MHz band– VDSL modems : up to 52Mbits/sec in …10MHz band
• Time to download 10 Mbyte-file:
Modem Time
56 Kbps voice-band modem 24 minutes
128 Kbps ISDN 10 minutes
6 Mbps ADSL 13 seconds
52 Mbps VDSL 1.5 seconds
14/12/01 Simon Doclo DSP Everywhere… p. 7
xDSL Modems
• ADSL : ‘Asymmetric Digital Subscriber Line’• HDSL : ‘High Speed Digital Subscriber Line’• VDSL : ‘Very High Speed Digital Subscriber Line’ …-1993: ADSL spurred by interest in video-on-demand (VOD)1995 : ADSL/VOD interest decline1996 : ADSL technology trials prove viability.1997-... : ADSL deployment, reoriented to data applications, as telco’s reaction to cable operators offering high- speed internet access with cable modems2000-… : VDSL
14/12/01 Simon Doclo DSP Everywhere… p. 8
xDSL Modems
• Analog/digital telephone network: BW 3 kHz, SNR 35 dB Shannon capacity
• ADSL/VDSL: higher bandwidth, lower SNR + impairments• Bitrate depends on length of copper line
Upstream
Downstream
SubscriberCentral office
Copper wire
vb.
300 m6.4 Mbps52 MbpsVDSL
3 km640 Kbps6 MbpsADSL
LengthUpDown
12 MHz
1.1 MHz
Bandwidth
kbits/sec35)1(log. 2 SNRBW
14/12/01 Simon Doclo DSP Everywhere… p. 9
• Modulation-technique : DMT (Discrete Multitone)• Basic idea:
– Decompose frequency into ‘tones’ (FFT/IFFT)– Assign bits according to SNR per tone
– ADSL spec (=ANSI standard): • 256 tones, 512-point (I)FFTs• carrier spacing fo= 4.3215 kHz, basic sampling rate 2.21 MHz
(=512*4.3215 kHz)
– VDSL (=proposal): up to 4096 tones, same carrier spacing
DMT Principles: IFFT/FFT-based modulation
frequentie
SNR
frequentie
bits/toon
14/12/01 Simon Doclo DSP Everywhere… p. 10
ADSL Spectrum
• ADSL spectrum :
14/12/01 Simon Doclo DSP Everywhere… p. 11
• Frequency-dependent channel attenuation introduces inter-symbol interference (ISI) equalization
• Coupling between wires in same or adjacent binders introduces crosstalk (XT)– Near-end Xtalk (NEXT)– Far-end Xtalk (FEXT)– Other systems (e.g. HPNA)
• Radio Frequency Interference (RFI): e.g. AM broadcast, amateur radio
• Noise: e.g. impulsive noise (=high bursts of short duration)• Echo: due to hybrid impedance mismatch echo cancellation
Conclusion: Need advanced modulation, DSP,etc. !
Communication impairments (1)
useful signal
FEXTNEXT
14/12/01 Simon Doclo DSP Everywhere… p. 12
Communication impairments (2)
• ADSL channel attenuation, crosstalk, noise
14/12/01 Simon Doclo DSP Everywhere… p. 13
Modulation - Demodulation (1)
• DMT-transmission block scheme:
S/P
FFT
IFFT P/S
Discreteequivalent
channel
4-QAM
8-QAM
Bitstream
(freq domain)
Modulation
(IFFT)
Time domain
signal
Demodulation
(FFT)
Equalisation
FEQ
14/12/01 Simon Doclo DSP Everywhere… p. 14
Modulation - Demodulation (2)
• Transmission: modulation is realized by means of 2N-point Inverse Discrete Fourier Transform (IFFT) (example N=4 )
• Receiver: demodulation with inverse operation, i.e. FFT
domainfrequency in symbolth -m
*1
*2
*3
4
3
2
1
0
matrixDFT inverse
49423528211470
42363024181260
35302520151050
2824201612840
211815129630
14121086420
76543210
00000000
domainin time symbolth m
7
6
5
4
3
2
1
0
.
m
m
m
m
m
m
m
m
m
m
m
m
m
m
m
m
a
a
a
a
a
a
a
a
WWWWWWWW
WWWWWWWW
WWWWWWWW
WWWWWWWW
WWWWWWWW
WWWWWWWW
WWWWWWWW
WWWWWWWW
s
s
s
s
s
s
s
s
Nj
eW
.
realreal
14/12/01 Simon Doclo DSP Everywhere… p. 15
The Magic Prefix Trick (1)
Additional feature : before transmission, a ‘prefix’
is added to each time-domain
symbol, i.e. the last
samples are copied and
put up front :
domain-in time symbolth -m
7
6
5
4
3
2
1
0
in -addprefix
88
22
sequence dtransmitte
7
6
5
4
3
2
1
0
8
7
.0
m
m
m
m
m
m
m
m
x
x
m
m
m
m
m
m
m
m
m
m
s
s
s
s
s
s
s
s
I
I
s
s
s
s
s
s
s
s
s
s
)2 (example
14/12/01 Simon Doclo DSP Everywhere… p. 16
The Magic Prefix Trick (2)
Prefix insertion : • in the receiver, the samples corresponding to the
prefix are removed (=unused) :
S/P
FFT
FEQ
IFFT P/S
Discreteequivalent
channel
14/12/01 Simon Doclo DSP Everywhere… p. 17
The Magic Prefix Trick (3)
• if channel impulse response has length L (= L non-zero taps) and ( is prefix length), then all ‘transient effects’ between symbols are confined to the prefix period :
1L
Tx-side Rx-side
Tone 3
Tone 2
Tone 1
Tone 0
Tone 3
Tone 2
Tone 1
Tone 0
Prefix From IFFT Guardband To FFT
*
ch(t)
r(t)s(t)
Channel
14/12/01 Simon Doclo DSP Everywhere… p. 18
The Magic Prefix Trick (4)
• Magic trick fails if , resulting in
– inter-symbol-interference (ISI) = interference from previous symbol(s) (same carrier)
– inter-carrier interference (ICI) = interference from other carriers
• In the receiver, after removing the samples corresponding to the prefix, the i-th tone is observed, multiplied by a factor H(i.fo), i.e. the channel response for frequency f=i.fo
• ‘Prefix trick’ is based on a linear convolution (filtering by channel impulse response) being turned into a circular convolution, which corresponds to component-wise multiplication in frequency domain easy equalization !
1L
14/12/01 Simon Doclo DSP Everywhere… p. 19
Overview
• Introduction
• DSP in digital communications systems:– xDSL-modems: modulation, equalisation
• DSP in audio applications:– Hands-free communication: echo, noise and reverberation– Basic techniques:
• Acoustic echo cancellation (AEC)
• Multi-microphone beamforming
– Application: hearing aids
• Conclusion
14/12/01 Simon Doclo DSP Everywhere… p. 20
Hands-free communication
• Recorded microphone signals are corrupted by:• Far-end echoes acoustic echo cancellation• Acoustic background noise noise suppression• Room reverberation dereverberation
• Application: hands-free telephony, hearing aids, voice control
14/12/01 Simon Doclo DSP Everywhere… p. 21
Signal model: some maths…
• Multi-microphone signal enhancement algorithms:– Extract clean speech/audio signal from microphone recordings– Exploit spatial and frequency diversity between speech and noise
• Microphone signals (m=1…M):
• Output signal:
compute filters g[k] :– echo cancellation:– noise reduction/dereverberation:
• gm[k] cancels noise components• gm[k] focuses on speech s[k]
echo endfarnoiseionreverberat
][][][][][][
kxkhknkskhky fmmmm
][][][][][1
kxkgkykgkz fM
mmm
][kyM
][2 ky
][1 ky ][1 kg
][2 kg
][kg M
][kz
][kx
][kg f
+
+
+
_
unknown
][][ khkg fm
f
known (=loud-speaker signal)
14/12/01 Simon Doclo DSP Everywhere… p. 22
Overview
• Introduction
• DSP in digital communications systems:– xDSL-modems: modulation, equalisation
• DSP in audio applications:– Hands-free communication: echo, noise and reverberation– Basic techniques:
• Acoustic echo cancellation (AEC)
• Multi-microphone beamforming
– Application: hearing aids
• Conclusion
14/12/01 Simon Doclo DSP Everywhere… p. 23
Acoustic echo cancellation (AEC)
Suppress acoustic and line echo:– to guarantee normal conversation conditions : users do not like to
hear a delayed and filtered version of their own voice– to prevent the closed-loop system from becoming unstable if
amplification is too high
14/12/01 Simon Doclo DSP Everywhere… p. 24
• Propagation of sound waves in an acoustic environment results in– signal attenuation– spectral distortion
• The attenuation and distortion can be modeled quite well as a linear filtering operation
• Non-linear distortion mainly stems from the loudspeakers. Its effect is typically of second order, therefore (often) not taken into account
• The linear filter h[k] modeling the acoustic path between loudspeaker and microphone is represented by the acoustic impulse response
Room Acoustics
14/12/01 Simon Doclo DSP Everywhere… p. 25
– direct path impulse and early reflections, which depend on the geometry of the room
Acoustic Impulse Response (1)
– dead time
Different parts:
– an exponentially decaying tail called reverberation, coming from multiple reflections
– For typical applications the impulse reponse is between 100 and 400 ms long several 100 to 1000 taps @ 8-16 kHz memory requirement for circular buffers in DSP
– Because people move around in the recording room, the acoustic impulse response is highly time-varying
14/12/01 Simon Doclo DSP Everywhere… p. 26
Acoustic Impulse Response (2)
ESAT speech laboratory :
T60 120 ms
Paleis voor Schone Kunsten :
T60 1500 ms
Original speech signal :
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
-0.4
-0.2
0
0.2
0.4
0.6
0.8
1
Time (sec)
Am
plitu
de
Impulse response PSK row 9
0 0.02 0.04 0.06 0.08 0.1 0.12
-0.2
-0.1
0
0.1
0.2
0.3
Time (sec)
Am
plitu
de
Impulse response SpeechLab
14/12/01 Simon Doclo DSP Everywhere… p. 27
Acoustic Impulse Response : FIR or IIR ?
• If the acoustic impulse response is modeled as– an FIR filter many hundreds to several thousands of
filter taps are required– an IIR filter filter order can be reduced, but still several
hundreds of filter coefficients are required (=‘bad’ model for acoustic impulse response)
– Remark: IIR-filters good model for classical filters (LP,HP,BP,BS)
• hence FIR models are typically used in practice – as they are guaranteed to be stable– as adaptive filtering techniques are called for:
• FIR adaptive filters are easier than IIR adaptive filters
14/12/01 Simon Doclo DSP Everywhere… p. 28
AEC based on Adaptive Filtering
• Goal: Identify acoustic impulse response h[k] and subtract filtered loudspeaker signal from microphone signal
• Thanks to the adaptivity – time-varying acoustics can be tracked– AEC is ‘self-learning’– performance superior to performance of conventional techniques
][kyM
][2 ky
][1 ky ][1 kg
][2 kg
][kg M
][kz
][kx
][kg f
+
+
+
_
14/12/01 Simon Doclo DSP Everywhere… p. 29
Adaptive Filtering Algorithms
• Algorithm: 2 steps– Filter loudspeaker signal error signal indicates how close
this signal is to recorded microphone signal– Update filter: update depends on error signal
error signaldesired signal
filtered signal
14/12/01 Simon Doclo DSP Everywhere… p. 30
Normalized Least Mean Square (NLMS)
with
L is the adaptive filter length, is the adaptation stepsize, is a regularization parameter and k is the discrete-time index
]1[
][
Lkx
kx
k x
][
][][][
][
1 ke
kykdke
ky
kk
Tk
kk
kTk
xxx
ww
wx
Data filtering
Filter update
]1[
]0[
Lw
w
k wCircular data buffer
Filter coefficients
14/12/01 Simon Doclo DSP Everywhere… p. 31
Control Algorithm
• ‘AEC is more than just an adaptive filter’ :
– adaptive filter is supplemented with control software, which mainly controls the adaptation speed (e.g. no adaptation during double-talk)
– In practice echo suppression is limited to 30 dB due to time-variance, non-linearities, finite filterlength postprocessing (e.g. center-clipping)
14/12/01 Simon Doclo DSP Everywhere… p. 32
Real-time DSP Implementation (1)
• AEC-implementation on DSP (lab equipment):– TMS320C44 @ 50 MHz : data acquisition (ADC/DAC) – TMS320C40 @ 50 MHz : acoustic echo cancellation
(AEC)
AEC ADC/DAC
14/12/01 Simon Doclo DSP Everywhere… p. 33
Real-time DSP Implementation (2)
• Adaptive filtering part : several algorithms can be selected– NLMS : time-domain algorithm– PB-FDAF : frequency-domain algorithm (better performance)
• Control software– double-talk detection– non-linear postprocessing algorithm
• Variable sampling rate – Common sampling rates for speech applications: 8 kHz, 16 kHz– for audio applications: 22.05 kHz, 44.1 kHz, 48 kHz
• Echo paths up to 325 ms can be modeled and tracked with the FDAF based on LMS at 8 kHz sampling frequency and 16 ms delay
14/12/01 Simon Doclo DSP Everywhere… p. 34
Real-time DSP Implementation (3)
Execution times for the most important blocks of the DSP code were measured :
N=768
FFT-size=128
fs=8000 Hz
block 64 samples = 8 ms
14/12/01 Simon Doclo DSP Everywhere… p. 35
Demo
Output AEC
Near-end signal
Output AEC
Double-talk without
Detection
Local
speaker
Far-end signal
14/12/01 Simon Doclo DSP Everywhere… p. 36
Overview
• Introduction
• DSP in digital communications systems:– xDSL-modems: modulation, equalisation
• DSP in audio applications:– Hands-free communication: echo, noise and reverberation– Basic techniques:
• Acoustic echo cancellation (AEC)
• Multi-microphone beamforming
– Application: hearing aids
• Conclusion
14/12/01 Simon Doclo DSP Everywhere… p. 37
Beamforming basics
• Background/history: antenna array design for RADAR• Array elements are combined electronically such that:
– array can be steered towards specific direction higher directivity– beam shaping is possible
• Beamforming for hands-free communication : – focus beam on speech source(s) speech enhancement and
dereverberation– put spatial nulls in direction of noise sources noise reduction
• Classification:– fixed beamforming: data-independent fixed filters gm[k]
e.g. delay-and-sum, weighted-sum, filter-and-sum
– adaptive beamforming: data-dependent adaptive filters gm[k]e.g. LCMV-beamformer, Generalized Sidelobe Canceller
14/12/01 Simon Doclo DSP Everywhere… p. 38
• Microphone signals are delayed and summed together array can be virtually steered to angle
• Angular selectivity is obtained, based on constructive ( =) and destructive ( ) interference
• Uniform delay-and-sum beamforming implies– Uniform array equal inter-microphone distance – Uniformly distributed delays
Delay-and-sum beamforming (1)
d
cos)1( dm
d
2
m
1
M
1
M
eG
mj
m
)(
M
mmm ky
Mkz
1
][1
][
sm
m fc
d cos
dmdm )1(
)1(mm
14/12/01 Simon Doclo DSP Everywhere… p. 39
• Spatial directivity pattern H(,) for uniform DS-beamformer
• H(,) has sinc-like shape and is frequency-dependent
Delay-and-sum beamforming (2)
)2/sin(
)2/sin(
),(
2/
2/1
)cos(cos)1(
j
jM
M
m
fc
dmj
e
Me
eHs
02000
40006000
8000 045
90135
180
0.2
0.4
0.6
0.8
1
Angle (deg)Frequency (Hz)
-20
-10
0
90
270
180 0
Spatial directivity pattern for f=5000 Hz
M=5 microphonesd=3 cm inter-microphone distance=60 steering anglefs=16 kHz sampling frequency
14/12/01 Simon Doclo DSP Everywhere… p. 40
• For an ambiguity, called spatial aliasing, occurs.
This is analogous to time-domain aliasing where now the spatial sampling (=d) is too large.
Delay-and-sum beamforming (3)
cos1
d
cf
sf
cd max
0
2000
4000
6000
8000 050
100150
0.20.40.60.8
1
Angle (deg)
Frequency (Hz)
M=5, =60, fs=16 kHz, d=8 cm
Spatial aliasing
14/12/01 Simon Doclo DSP Everywhere… p. 41
• Better directivity patterns than DS-beamformer are obtained with weighted-sum and filter-and-sum beamformers– e.g. Frequency-independent directivity pattern
Filter-and-sum beamformer
][kyM
][2 ky
][1 ky ][1 kg
][2 kg
][kg M
][kz
][kx
][kg f
+
+
+
_
M=8Logarithmic arrayL=50=90 fs=8 kHz
01000
20003000
045
90135
180
0
0.5
1
Angle (deg)
Frequency (Hz)
14/12/01 Simon Doclo DSP Everywhere… p. 42
• Adaptive filter-and-sum structure:– Minimize noise output power, while maintaining a chosen frequency
response in look direction (and/or other linear constraints)– LCMV = Linearly Constrained Minimum Variance
• minimize variance of output z[k]
• in order to avoid desired signal to be distorted or cancelled out,J linear constraints are added
Adaptive beamforming
][kyM
][2ky
][1ky
][1ky ][1kg ][1kf ][2kg
][1kf ][kgM
][1kf
][kz
][1ky
Speaker
Noise
gRggg
][min][min 2 kkzE yyT
JJMLT with bCbgC ,,
14/12/01 Simon Doclo DSP Everywhere… p. 43
Generalized Sidelobe Canceller (1)
• GSC consists of three parts:– Fixed (delay-and-sum) beamformer, in order to achieve spatial
alignment of speech source speech reference– Blocking matrix, placing spatial nulls in the direction of the speech
source noise references– Multi-channel adaptive filter with delay
Postproc
14/12/01 Simon Doclo DSP Everywhere… p. 44
• Blocking matrix Ca :– creating maximum M-1 independent noise references by placing
spatial nulls in look-direction– different possibilities: e.g. Griffiths-Jim, Walsh
broadside
Generalized Sidelobe Canceller (2)
1111TC
1001
0101
0011TaC
1111
1111
1111TaC 0
20004000
60008000 0 45 90 135 180
0
0.5
1
1.5
Angle (deg)
Frequency (Hz)
• Problems of GSC:– impossible to reduce noise from look-direction– reverberation effects cause signal leakage in noise reference adaptive filter is only updated when no speech is present !
14/12/01 Simon Doclo DSP Everywhere… p. 45
Overview
• Introduction
• DSP in digital communications systems:– xDSL-modems: modulation, equalisation
• DSP in audio applications:– Hands-free communication: echo, noise and reverberation– Basic techniques:
• Acoustic echo cancellation (AEC)
• Multi-microphone beamforming
– Application: hearing aids
• Conclusion
14/12/01 Simon Doclo DSP Everywhere… p. 46
Application: Hearing Aids (1)
• Hearing problems are very common nowadays• Most of the users are dissatisfied with the performance of
their hearing aid in noisy environments (cocktail party effect)
increase speech intelligibility by reducing background noise• Traditional hearing aids:
– one microphone, analog, limited signal processing– amplification of all incoming sound without distinction between
different sound sources
• Enabling technologies:– microphone miniaturisation integrate multiple microphones into
one hearing aid– micro-electronics: size ASIC < 10 mm2, low power consumption– advanced DSP techniques (noise reduction, feedback suppression)
14/12/01 Simon Doclo DSP Everywhere… p. 47
• Improvement of speech intelligibility by reduction of background noise• BTE hearing aid with 2 (or more) closely-spaced microphones• GSC in switched mode:
• Beamfomer : weights can be adapted during speech• Noise suppression (ANC) : only adaptation during noise• Speech detection : determine when speech is present
Application: Hearing Aids (2)
Noise Source
Result
Noise reference
Delay 1
Adaptive filter 1
Speech reference
Adaptive filter 2
Delay 2
Speech detection
Speech source Beamformer Noise suppression
14/12/01 Simon Doclo DSP Everywhere… p. 48
Conclusion
• DSP-techniques can be found in many every-day products:– audio applications: CD, MiniDisc, hands-free telephony– communications: GSM, modems, WLAN– medical applications: hearing aids, cochlear implants
• Implementation differences:– sampling rate, memory requirements, complexity
• Basic techniques:– filters, filterbanks, FFT/IFFT frequency filtering– adaptive filters track changing systems– multi-sensor systems spatial filtering