1 multidimensional model order selection. 2 motivation stock markets: one example of [1] [1]: m....
TRANSCRIPT
1
Multidimensional Model Order SelectionMultidimensional Model Order Selection
2
MotivationMotivation
Stock Markets: One example of [1]
[1]: M. Loteran, “Generating market risk scenarios using principal components analysis: methodological and practical considerations”, in the Federal Reserve Board, March, 1997.
Information: Long Term Government Bond interest rates.
Canada, USA, 6 European countries and Japan. Result: by visual inspection of the Eigenvalues (EVD).
Three main components: Europe, Asia and North America.
3
MotivationMotivation
Ultraviolet-visible (UV-vis) Spectrometry [2]
[2]: K. S. Von Age, R. Bro, and P. Geladi, “Multi-way analysis with applications in the chemical sciences,” Wiley, Aug. 2004.
Non-identified substanceRadiation
Wa
vele
ng
th
Oxidation statepH
samples
Result: successful application of tensor calculus.
In [2], the model order is estimated via the core consistency
analysis (CORCONDIA) by visual inspection.
4
MotivationMotivation
Sound source localization
[3]: A. Quinlan and F. Asano, “Detection of overlapping speech in meeting recordings using the modified exponential fitting test,” in Proc. 15th European Signal Processing Conference (EUSIPCO 2007), Poznan, Poland.
Microphone array
Sound source 1
Sound source 2
Applications: interfaces between humans and robots and data
processing. MOS: Corrected Frequency Exponential Fitting Test [3]
5
MotivationMotivation
Wind tunnel evaluation
[4]: C. El Kassis, “High-resolution parameter estimation schemes for non-uniform antenna arrays,” PhD Thesis, SUPELEC, Universite Paris-Sud XI, 2009. (Wind tunnel photo provided by Renault)
MOS: No technique is applied. [4]
Wind
Array
Source: Carine El Kassis [4].
6
Receive array: 1-D or 2-D
Frequency
Time
Transmit array: 1-D or 2-D
Direction of Arrival (DOA)
Delay
Doppler shift
Direction of Departure (DOD)
MotivationMotivation Channel model
7
MotivationMotivation
An unlimited list of applications Radar; Sonar; Communications; Medical imaging; Chemistry; Food industry; Pharmacy; Psychometrics; Reflection seismology; EEG; …
8
OutlineOutline
Motivation Introduction Tensor calculus One dimensional Model Order Selection
Exponential Fitting Test Multidimensional Model Order Selection (R-D MOS)
Novel contributions• R-D Exponential Fitting Test (R-D EFT)• Closed-form PARAFAC based model order selection (CFP-MOS)
Comparisons Conclusions
9
OutlineOutline
Motivation Introduction Tensor calculus One dimensional Model Order Selection
Exponential Fitting Test Multidimensional Model Order Selection (R-D MOS)
Novel contributions• R-D Exponential Fitting Test (R-D EFT)• Closed-form PARAFAC based model order selection (CFP-MOS)
Comparisons Conclusions
10
IntroductionIntroduction
The model order selection (MOS) is required for the principal component analysis (PCA). is the amount of principal components of the data. has several schemes based on the Eigenvalue Decomposition (EVD). can be estimated via other properties of the data, e.g., removing
components until reaching the noise level or shift invariance property of the data.
The multidimensional model order selection (R-D MOS) requires a multidimensional structure of the data, which is taken into
account (this additional information is ignored by one dimensional MOS). gives an improved performance compared to the MOS. based on tensor calculus, e.g., instead of EVD and SVD, the Higher Order
Singular Value Decomposition (HOSVD) [5] is computed.
[5]: L. de Lathauwer, B. de Moor, and J. Vanderwalle, “A multilinear singular value decomposition”, SIAM J. Matrix Anal. Appl., vol. 21(4), 2000.
11
IntroductionIntroduction A large number of model order selection (MOS) schemes have been proposed in
the literature. However, most of the proposed MOS schemes are compared only to Akaike’s
Information Criterion (AIC) [6] and Minimum Description Length (MDL) [6]; the Probability of correct Detection (PoD) of these schemes is a function of
the array size (number of snapshots and number of sensors). In [7], we have proposed expressions for the 1-D AIC and 1-D MDL. Moreover, for
matrix based data in the presence of white Gaussian noise, the Modified Exponential Fitting Test (M-EFT) outperforms 12 state-of-the-art matrix based model order selection
techniques for different array sizes. For colored noise, the M-EFT is not suitable, as well as several other MOS
schemes, and the RADOI [8] reaches the best PoD according to our comparisons.[6]: M. Wax and T. Kailath “Detection of signals by information theoretic criteria”, in IEEE Trans. on
Acoustics, Speech, and Signal Processing, vol. ASSP-33, pp. 387-392, 1974.
[7]: J. P. C. L. da Costa, A. Thakre, F. Roemer, and M. Haardt, “Comparison of model order selection techniques for high-resolution parameter estimation algorithms,” in Proc. 54th International Scientific Colloquium (IWK), (Ilmenau, Germany), Sept. 2009.
[8]: E. Radoi and A. Quinquis, “A new method for estimating the number of harmonic components in noise with application in high resolution radar,” EURASIP Journal on Applied Signal Processing, 2004.
12
IntroductionIntroduction
One of the most well-known multidimensional model order selection schemes in the literature is the Core Consistency Analysis (CORCONDIA) [9] a subjective MOS scheme, i.e., depends on the visual interpretation.
In [10], we have proposed the Threshold-CORCONDIA (T-CORCONDIA) which is non-subjective, and its PoD is close, but still inferior to the 1-D AIC and
1-D MDL. By taking into account the multidimensional structure of the data, we extend the
M-EFT to the R-D EFT [10] for applications with white Gaussian noise. For applications with colored noise, we proposed the Closed-Form PARAFAC
based Model Order Selection (CFP-MOS) scheme, which outperforms the state-of-the-art colored noise scheme RADOI [11].
[9]: R. Bro and H.A.L. Kiers. A new efficient method for determining the number of components in
PARAFAC models. Journal of Chemometrics, 17:274–286,2003.
[10]: J. P. C. L. da Costa, M. Haardt, and F. Roemer, “Robust methods based on the HOSVD for estimating
the model order in PARAFAC models,” in Proc. 5-th IEEE Sensor Array and Multichannel Signal
Processing Workshop (SAM 2008), (Darmstadt, Germany), pp. 510 - 514, July 2008.
[11]: J. P. C. L. da Costa, F. Roemer, and M. Haardt, “Multidimensional model order via closed-form
PARAFAC for arbitrary noise correlations,” submitted to ITG Workshop on Smart Antennas 2010.
13
OutlineOutline
Motivation Introduction Tensor calculus One dimensional Model Order Selection
Exponential Fitting Test Multidimensional Model Order Selection (R-D MOS)
Novel contributions• R-D Exponential Fitting Test (R-D EFT)• Closed-form PARAFAC based model order selection (CFP-MOS)
Comparisons Conclusions
14
Tensor algebraTensor algebra
3-D tensor = 3-way array
n-mode products between and
Unfoldings
M1
M2
M3
“1-mode vectors”
“2-mode vectors”
“3-mode vectors”
i.e., all the n-mode vectors multiplied from the left-hand-side by
11 22
15
The Higher-Order SVD (HOSVD)The Higher-Order SVD (HOSVD) Singular Value Decomposition Higher-Order SVD (Tucker3)
“Full HOSVD”
Low-rank approximation (truncated HOSVD)
“Economy size HOSVD”
“Full SVD”
Matrix data model
signal partsignal part noise partnoise part
rank rank dd
Tensor data model
signal partsignal part noise partnoise part
rank rank dd
“Economy size SVD”
Low-rank approximation
16
OutlineOutline
Motivation Introduction Tensor calculus One dimensional Model Order Selection
Exponential Fitting Test Multidimensional Model Order Selection (R-D MOS)
Novel contributions• R-D Exponential Fitting Test (R-D EFT)• Closed-form PARAFAC based model order selection (CFP-MOS)
Comparisons Conclusions
17
Exponential Fitting Test (EFT)Exponential Fitting Test (EFT) Observation is a superposition of noise and signal
The noise eigenvalues still exhibit the exponential profile [12,13]
We can predict the profileof the noise eigenvaluesto find the “breaking point”
Let P denote the number of candidate noise eigenvalues.
• choose the largest P such that the P noise eigenvalues can be fitted with a decaying exponential
d = 3, M = 8, SNR = 20 dB, N = 10[12]: J. Grouffaud, P. Larzabal, and H. Clergeot, “Some properties of ordered eigenvalues of a wishart
matrix: application in detection test and model order selection,” in Proceedings of the IEEE
International Conference on Acoustics, Speech and Signal Processing (ICASSP’96).
[13]: A. Quinlan, J.-P. Barbot, P. Larzabal, and M. Haardt, “Model order selection for short data: An
exponential fitting test (EFT),” EURASIP Journal on Applied Signal Processing, 2007
18
Exponential Fitting Test (EFT)Exponential Fitting Test (EFT)
Start with P = 1
Predict M-1 based on M
Compare this predictionwith actual eigenvalue
relative distance:
In our case it agrees, we continue
d = 3, M = 8, SNR = 20 dB, N = 10
19
Exponential Fitting Test (EFT)Exponential Fitting Test (EFT)
Now, P = 2
Predict M-2 based on M-1 and M
relative distance
d = 3, M = 8, SNR = 20 dB, N = 10
20
Exponential Fitting Test (EFT)Exponential Fitting Test (EFT)
Now, P = 3
Predict M-3 based on M-2, M-1, and M
relative distance
d = 3, M = 8, SNR = 20 dB, N = 10
21
Exponential Fitting Test (EFT)Exponential Fitting Test (EFT)
Now, P = 4
Predict M-4 based on M-3, M-2, M-1, and M
relative distance
d = 3, M = 8, SNR = 20 dB, N = 10
22
Exponential Fitting Test (EFT)Exponential Fitting Test (EFT)
Now, P = 5
Predict M-5 based on M-4 , M-3, M-2, M-1, and M
relative distance
The relative distance becomes very big, we have found the break point.
d = 3, M = 8, SNR = 20 dB, N = 10
23
OutlineOutline
Motivation Introduction Tensor calculus One dimensional Model Order Selection
Exponential Fitting Test Multidimensional Model Order Selection (R-D MOS)
Novel contributions• R-D Exponential Fitting Test (R-D EFT)• Closed-form PARAFAC based model order selection (CFP-MOS)
Comparisons Conclusions
24
RR-D Exponential Fitting Test-D Exponential Fitting Test
In the R-D case, we have a measurement tensor
This allows to define the r-mode sample covariance matrices
The eigenvalues of are denoted by for
They are related to the higher-order singular values of the HOSVD of through
r-mode eigenvalues
25
RR-D Exponential Fitting Test-D Exponential Fitting Test
The R-mode eigenvalues exhibit an exponential profile for every R
Assume . Then we can define global eigenvalues
The global eigenvalues also follow an exponential profile, since
The product across modes enhances the signal-to-noise ratio and improves the fit to an exponential profile
R-D exponential profile
26
RR-D Exponential Fitting Test-D Exponential Fitting Test
Comparison between the global eigenvalues profile and the profile of the last unfolding
R-D exponential profile
27
RR-D Exponential Fitting Test-D Exponential Fitting Test
Is an extended version of the M-EFT operating on the
Exploits the fact that the global eigenvalues still exhibit an exponential profile
The enhanced SNR and the improved fit lead to significant improvements in the performance
Is able to adapt to arrays of arbitrary size and dimension through the adaptive definition of global eigenvalues
R-D EFT
28
OutlineOutline
Motivation Introduction Tensor calculus One dimensional Model Order Selection
Exponential Fitting Test Multidimensional Model Order Selection (R-D MOS)
Novel contributions• R-D Exponential Fitting Test (R-D EFT)• Closed-form PARAFAC based model order selection (CFP-MOS)
Comparisons Conclusions
29
Another way to look at the SVD
decomposition into a sum of rank one matrices also referred to as principal components (PCA)
Tensor case:
SVD and PARAFACSVD and PARAFAC
+ +=
+ +=
30
HOSVD and PARAFACHOSVD and PARAFAC
HOSVD PARAFAC
Identity tensor Core tensor
• Core tensor usually is full. R-D STE [14] • Identity tensor is always diagonal. CFP-PE [15]
[14]: M. Haardt, F. Roemer, and G. Del Galdo, ``Higher-order SVD based subspace estimation to improve
the parameter estimation accuracy in multi-dimensional harmonic retrieval problems,'' IEEE
Trans. Signal Processing, vol. 56, pp. 3198 - 3213, July 2008.
[15]: J. P. C. L. da Costa, F. Roemer, and M. Haardt, “Robust R-D parameter estimation via closed-form
PARAFAC,” submitted to ITG Workshop on Smart Antennas 2010.
31
Closed-form solution to PARAFACClosed-form solution to PARAFAC
The task of PARAFAC analysis: Given (noisy) measurements
and the model order d, findsuch that
Here is the higher-order Frobenius norm (sum of squared magnitude of all elements).
[16]:F. Roemer and M. Haardt, “A closed-form solution for multilinear PARAFAC decompositions,” in Proc. 5-th IEEE Sensor Array and Multichannel Signal Processing Workshop (SAM 2008), (Darmstadt, Germany), pp. 487 - 491, July 2008.
Our approach: based on simultaneous matrix diagonalizations (“closed-form”). By applying the closed-form PARAFAC (CFP) [16]
R*(R-1) simultaneous matrix diagonalizations (SMD) are possible; R*(R-1) estimates for each factor are possible; selection of the best solution by different heuristics (residuals of the SMD) is
done
32
For P = 2, i.e., P < d
Closed-form PARAFAC basedClosed-form PARAFAC basedModel Order SelectionModel Order Selection
+=
+= Assuming that d = 3, and solutions with the two smallest residuals of the SMD. Using the same principle as in [17], the error is minimized when P = d. Due to the permutation ambiguities, the components of different tensors are ordered using the amplitude based approach proposed in [18].
[17]:J.-M. Papy, L. De Lathauwer, and S. Van Huffel, “A shift invariance-based order-selection technique for exponential data modelling,” in IEEE Signal Processing Letters, vol. 14, No. 7, pp. 473 - 476, July 2007.
[18]:M. Weis, F. Roemer, M. Haardt, D. Jannek, and P. Husar, “Multi-dimensional Space-Time-Frequency component analysis of event-related EEG data using closed-form PARAFAC,” in Proc. IEEE Int. Conf. Acoust., Speech, and Signal Processing (ICASSP), (Taipei, Taiwan), pp. 349-352, Apr. 2009.
For P = 4, i.e., P > d
+ += +
+ += +
33
OutlineOutline
Motivation Introduction Tensor calculus One dimensional Model Order Selection
Exponential Fitting Test Multidimensional Model Order Selection (R-D MOS)
Novel contributions• R-D Exponential Fitting Test (R-D EFT)• Closed-form PARAFAC based model order selection (CFP-MOS)
Comparisons Conclusions
34
ComparisonsComparisons
35
ComparisonsComparisons
36
ComparisonsComparisons
37
ComparisonsComparisons
38
ComparisonsComparisons
39
ComparisonsComparisons
40
ComparisonsComparisons
41
ComparisonsComparisons
42
ComparisonsComparisons
43
ComparisonsComparisons
44
ComparisonsComparisons
45
ComparisonsComparisons
46
ComparisonsComparisons
47
ComparisonsComparisons
48
ComparisonsComparisons
49
ComparisonsComparisons
50
ComparisonsComparisons
51
ComparisonsComparisons
52
OutlineOutline
Motivation Introduction Tensor calculus One dimensional Model Order Selection
Exponential Fitting Test Multidimensional Model Order Selection (R-D MOS)
Novel contributions• R-D Exponential Fitting Test (R-D EFT)• Closed-form PARAFAC based model order selection (CFP-MOS)
Comparisons Conclusions
53
ConclusionsConclusions State-of-the-art one dimensional and multidimensional model order selection
techniques were presented; For one dimensional scenarios:
in the presence of white Gaussian noise
• Modified Exponential Fitting Test (M-EFT) in the presence of severe colored Gaussian noise
• RADOI For multidimensional scenarios:
in the presence of white Gaussian noise
• R-dimensional Exponential Fitting Test (R-D EFT) in the presence of colored noise
• Closed-form PARAFAC based Model Order Selection (CFP-MOS) scheme
The mentioned schemes are applicable to problems with a PARAFAC data model, which are found in several scientific fields.
54
Thank you for your attention!Thank you for your attention!Vielen Dank für Ihre Aufmerksamkeit!Vielen Dank für Ihre Aufmerksamkeit!