1 maxent 2007 r. f. astudillo, d. kolossa and r. orglmeister
TRANSCRIPT
1
MAXENT 2007
R. F. Astudillo, D. Kolossa and R. Orglmeister
2
MAXENT 2007
R. F. Astudillo, D. Kolossa and R. Orglmeister
PROPAGATION OF STATISTICAL INFORMATION THROUGH NON-LINEAR FEATURE EXTRACTIONS FOR ROBUST
SPEECH RECOGNITION
Overview:
1. Introduction: Automatic speech recognition.2. Problem: Imperfect noise suppression.3. Proposed solution: Uncertainty propagation.4. Tests & results.5. Conclusions.
R. F. Astudillo, D. Kolossa and R. Orglmeister - TU-Berlin
3
MAXENT 2007
R. F. Astudillo, D. Kolossa and R. Orglmeister
Automatic Speech Recognizer (ASR)
• Feature extraction transforms signal into a domain more suitable for recognition.
• Speech recognizer models abstract speech components like phonemes or triphones, generates transcription.
• Most of speech recognition applications need noise suppression preprocessing.
4
MAXENT 2007
R. F. Astudillo, D. Kolossa and R. Orglmeister
• Non-linear transformations that imitate the way humans process speech.
• Robust against inter-speaker and intra-speaker variability.
• Mel-cepstral and RASTA-PLP transformations.
Feature Extraction
5
MAXENT 2007
R. F. Astudillo, D. Kolossa and R. Orglmeister
Speech Recognition
• Statistical models are used to model speech.
• Hidden Markov models with mixture of Gaussians (multivariable) for the emitting states.
Example:Mel-cepstral features
6
MAXENT 2007
R. F. Astudillo, D. Kolossa and R. Orglmeister
Noise Suppression
• MMSE-LSA bayesian estimation [Ephraim1985] is one of the most used.
• Leaves residual noise.
• Introduces artifacts in speech.
• Most methods obtain an estimation of the short-time spectrum (STFT) of the clean signal .
Problem: Imperfect estimation.
7
MAXENT 2007
R. F. Astudillo, D. Kolossa and R. Orglmeister
Solution: Modeling Uncertainty of Estimation
We model each element of the STFT as a complex Gaussian random distribution .
• Mean set equal to estimated clean value .
• Parameter controls the uncertainty.
8
MAXENT 2007
R. F. Astudillo, D. Kolossa and R. Orglmeister
Propagation of Uncertainty
• We propagate first and second order moments of the distributions.
• Correlation between feature appears (covariance).
• Resulting uncertainty is combined with statistical model parameters for robust speech recognition
9
MAXENT 2007
R. F. Astudillo, D. Kolossa and R. Orglmeister
Propagation of Uncertainty
• We propagate first and second order moments of the distributions.
• Correlation between feature appears (covariance).
• Resulting uncertainty is combined with statistical model parameters for robust speech recognition
10
MAXENT 2007
R. F. Astudillo, D. Kolossa and R. Orglmeister
Approaches to Uncertainty Propagation
Analytic solutions Imply complex calculations. Specific for each transformation.
Pseudo-Montecarlo Unscented Transform [Julier1996]. Inefficient for high number of dimensions
(i.e. STFT 256 dim./frame).
►Piecewise Propagation Efficient combination of both methods. Valid for many feature extractions (i.e. MELSPEC, MFCC, RASTA-PLP).
11
MAXENT 2007
R. F. Astudillo, D. Kolossa and R. Orglmeister
Piecewise Uncertainty Propagation
Exemplified with Mel-Ceptral transformation:
1. Modulus extraction (non-linear).2. Filterbank (linear).3. Logarithm (non-linear).4. Discrete-cosine-transform (linear).5. Delta and acceleration coefficients (linear).
12
MAXENT 2007
R. F. Astudillo, D. Kolossa and R. Orglmeister
Propagation through Modulus
• By integrating the phase of a complex Gaussian distribution we obtain the Rice distribution.
• Mean and variance can be calculated as:
were L is the Legendre polynom.
13
MAXENT 2007
R. F. Astudillo, D. Kolossa and R. Orglmeister
Propagation through filterbank
• Each filter output m is a weighted sum of frequency moduli.
• It can be expressed as a matrix multiplication.
• Mean and variance can be calculated as:
14
MAXENT 2007
R. F. Astudillo, D. Kolossa and R. Orglmeister
Full Covariance and other linear transformations
• DCT, delta and acceleration can be computed similarly.
• Covariance after filterbank is no longer diagonal.
• Additional computation costs.
15
MAXENT 2007
R. F. Astudillo, D. Kolossa and R. Orglmeister
Propagation through Logarithm
• Non-linear transformation
• Distribution after filterbank difficult to model
• not diagonal
• Dimesionality of the Mel features much smaller than the STFT features
► Unscented transform can be applied efficiently
16
MAXENT 2007
R. F. Astudillo, D. Kolossa and R. Orglmeister
Unscented Transform
• Only points must be propagated.
• Points on the th covariace contour and the mean.
• = feature dimension
• Example for =2
17
MAXENT 2007
R. F. Astudillo, D. Kolossa and R. Orglmeister
Unscented Transform II
• Mean and covariances are calculated by using weighted averages:
• Parameter allows higher moments of the distribution to be considered.
18
MAXENT 2007
R. F. Astudillo, D. Kolossa and R. Orglmeister
Use of Uncertainty
• After propagation of uncertainty, missing feature techniques or uncertainty decoding may be applied.
• These techniques combine uncertainty and model information to ignore or reestimate noisy features.
Parametersof state f1
19
MAXENT 2007
R. F. Astudillo, D. Kolossa and R. Orglmeister
Use of Uncertainty II
• Modified imputation [Kolossa2005] showed the best performance.
• It reestimates features for state q by maximizing the probability:
• Assuming multivariate Gaussian distribution for uncertaintyand model:
20
MAXENT 2007
R. F. Astudillo, D. Kolossa and R. Orglmeister
Recognition Tests TI-DIGITS database
% correct identified words
Windnoise Streetnoise
Test Type Uncertainty -15dB 5dB -15dB 5dB
Clean Speech ( ) 98.76
Noisy ( ) 28.44 87.94 22.87 92.43
MMSE-LSA ( ) 34.78 75.27 36.63 92.43
+Aprox. uncertainty 46.68 88.72 22.72 94.90
+Ideal uncertainty 51.93 94.28 48.53 96.45
0
0
0
• 200 files (20 different speakers).
• Best, second best results.
21
MAXENT 2007
R. F. Astudillo, D. Kolossa and R. Orglmeister
Conclusions
• The use of uncertainty in Mel-cepstral domain is useful to compensate imperfect estimation during noise suppression.
• Piecewise uncertainty propagation is valid for multiple feature extractions.
• Better estimation of uncertainty should improve the results.
22
MAXENT 2007
R. F. Astudillo, D. Kolossa and R. Orglmeister
Thank You!
Some literature:
[Ephraim1985] Y. Ephraim, and D. Malah, Acoustics, Speech, and Signal Processing, IEEE Transactions on 33, 443–445 (1985).
[Julier1996] S. Julier, and J. Uhlmann, A general method for approximating nonlineartransformations of probability distributions, Tech. rep., University of Oxford, UK (1996).
[Kolossa2005] D. Kolossa, A. Klimas, and R. Orglmeister, “Separation and robust recognition of noisy, convolutive speech mixtures using time-frequency masking and missing data techniques,” Applications of Signal Processing to Audio and Acoustics, 2005. IEEE Workshop on, 2005, pp. 82-85.