the meaning of distances in spectral analysispeople.ece.umn.edu/~georgiou/papers/plenary1.pdf ·...
TRANSCRIPT
![Page 1: The meaning of Distances in Spectral Analysispeople.ece.umn.edu/~georgiou/papers/plenary1.pdf · 2007-12-16 · geometric mean of f 1 f ... in spectral analysis Operational significance](https://reader033.vdocuments.net/reader033/viewer/2022050105/5f439c1e3e36fe60e101f355/html5/thumbnails/1.jpg)
The meaning ofDistances in Spectral Analysis
46th IEEE Conference on Decision & Control
Plenary presentation
Tryphon Georgiou
Electrical & Computer EngineeringUniversity of Minnesota
![Page 2: The meaning of Distances in Spectral Analysispeople.ece.umn.edu/~georgiou/papers/plenary1.pdf · 2007-12-16 · geometric mean of f 1 f ... in spectral analysis Operational significance](https://reader033.vdocuments.net/reader033/viewer/2022050105/5f439c1e3e36fe60e101f355/html5/thumbnails/2.jpg)
Meaning of distances
• maximal separation (L∞)• energy-like content (L2)• integral of flow-rate (L1)
New Orleans, December 2007 2
![Page 3: The meaning of Distances in Spectral Analysispeople.ece.umn.edu/~georgiou/papers/plenary1.pdf · 2007-12-16 · geometric mean of f 1 f ... in spectral analysis Operational significance](https://reader033.vdocuments.net/reader033/viewer/2022050105/5f439c1e3e36fe60e101f355/html5/thumbnails/3.jpg)
Power spectra
⇒
0 0.5 1 1.5 2 2.5 310!6
10!5
10!4
10!3
10!2
10!1
Periodogram, Blackman-Tukey, Levinson, Durbin, Burg, . . .
New Orleans, December 2007 3
![Page 4: The meaning of Distances in Spectral Analysispeople.ece.umn.edu/~georgiou/papers/plenary1.pdf · 2007-12-16 · geometric mean of f 1 f ... in spectral analysis Operational significance](https://reader033.vdocuments.net/reader033/viewer/2022050105/5f439c1e3e36fe60e101f355/html5/thumbnails/4.jpg)
Spectral analysis
0 0.5 1 1.5 2 2.5 310!6
10!5
10!4
10!3
10!2
10!1
+
=
. . . u(0), u(1), . . .+
...
u(k) =∫ejkθdX(θ) E{u(k)u(k + `)} =
∫ej`θf (θ)dθ
New Orleans, December 2007 4
![Page 5: The meaning of Distances in Spectral Analysispeople.ece.umn.edu/~georgiou/papers/plenary1.pdf · 2007-12-16 · geometric mean of f 1 f ... in spectral analysis Operational significance](https://reader033.vdocuments.net/reader033/viewer/2022050105/5f439c1e3e36fe60e101f355/html5/thumbnails/5.jpg)
Signals vs. power densities
time-signals
(u1 − u2) “error signal”
power distributions
0 0.5 1 1.5 2 2.5 310!6
10!5
10!4
10!3
10!2
10!1
0 0.5 1 1.5 2 2.5 310!6
10!5
10!4
10!3
10!2
(f1 − f2) is not a “signal”
New Orleans, December 2007 5
![Page 6: The meaning of Distances in Spectral Analysispeople.ece.umn.edu/~georgiou/papers/plenary1.pdf · 2007-12-16 · geometric mean of f 1 f ... in spectral analysis Operational significance](https://reader033.vdocuments.net/reader033/viewer/2022050105/5f439c1e3e36fe60e101f355/html5/thumbnails/6.jpg)
Communications
Speech analysis/coding
New Orleans, December 2007 6
![Page 7: The meaning of Distances in Spectral Analysispeople.ece.umn.edu/~georgiou/papers/plenary1.pdf · 2007-12-16 · geometric mean of f 1 f ... in spectral analysis Operational significance](https://reader033.vdocuments.net/reader033/viewer/2022050105/5f439c1e3e36fe60e101f355/html5/thumbnails/7.jpg)
Medical diagnostics
Noninvasive temperature sensing
Temperature field
with E. Ebbini & A.N. AminiIn IEEE Trans. on Biomedical Engineering, 2005
New Orleans, December 2007 7
![Page 8: The meaning of Distances in Spectral Analysispeople.ece.umn.edu/~georgiou/papers/plenary1.pdf · 2007-12-16 · geometric mean of f 1 f ... in spectral analysis Operational significance](https://reader033.vdocuments.net/reader033/viewer/2022050105/5f439c1e3e36fe60e101f355/html5/thumbnails/8.jpg)
Medical diagnostics Radar (SAR)
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 110
−1
100
101
102
http://www.sandia.gov/radar/images/3dsar.gif
New Orleans, December 2007 8
![Page 9: The meaning of Distances in Spectral Analysispeople.ece.umn.edu/~georgiou/papers/plenary1.pdf · 2007-12-16 · geometric mean of f 1 f ... in spectral analysis Operational significance](https://reader033.vdocuments.net/reader033/viewer/2022050105/5f439c1e3e36fe60e101f355/html5/thumbnails/9.jpg)
Quantitative analysis
↗
different
methods
↘
0 0.5 1 1.5 2 2.5 310!6
10!5
10!4
10!3
10!2
10!1
0 0.5 1 1.5 2 2.5 310!6
10!5
10!4
10!3
10!2
How can we compare power spectra?
New Orleans, December 2007 9
![Page 10: The meaning of Distances in Spectral Analysispeople.ece.umn.edu/~georgiou/papers/plenary1.pdf · 2007-12-16 · geometric mean of f 1 f ... in spectral analysis Operational significance](https://reader033.vdocuments.net/reader033/viewer/2022050105/5f439c1e3e36fe60e101f355/html5/thumbnails/10.jpg)
Quantitative analysis
→
same
method
→
0 0.5 1 1.5 2 2.5 310!6
10!5
10!4
10!3
10!2
0 0.5 1 1.5 2 2.5 310!6
10!5
10!4
10!3
10!2
10!1
How can we compare power spectra?
New Orleans, December 2007 10
![Page 11: The meaning of Distances in Spectral Analysispeople.ece.umn.edu/~georgiou/papers/plenary1.pdf · 2007-12-16 · geometric mean of f 1 f ... in spectral analysis Operational significance](https://reader033.vdocuments.net/reader033/viewer/2022050105/5f439c1e3e36fe60e101f355/html5/thumbnails/11.jpg)
How can we compare power spectra?
Question:what is a natural notion of distancebetween power spectral densities?
quantify uncertainty
signal classification, detect structural changes
system identification, tune algorithms, sensor technology
New Orleans, December 2007 11
![Page 12: The meaning of Distances in Spectral Analysispeople.ece.umn.edu/~georgiou/papers/plenary1.pdf · 2007-12-16 · geometric mean of f 1 f ... in spectral analysis Operational significance](https://reader033.vdocuments.net/reader033/viewer/2022050105/5f439c1e3e36fe60e101f355/html5/thumbnails/12.jpg)
Plan of the talkMetrics based on
prediction theory (Szego, Kolmogorov)
parallels with information geometry (Fisher, Rao)
transport geometry (Monge-Kantorovich)
Case studies & applications Prediction-basedgeometry
Informationgeometry
Transportgeometry
applications
New Orleans, December 2007 12
![Page 13: The meaning of Distances in Spectral Analysispeople.ece.umn.edu/~georgiou/papers/plenary1.pdf · 2007-12-16 · geometric mean of f 1 f ... in spectral analysis Operational significance](https://reader033.vdocuments.net/reader033/viewer/2022050105/5f439c1e3e36fe60e101f355/html5/thumbnails/13.jpg)
Setting. . . u−1, u0, u1, u2, . . . . . . u−1, u0, u1, u2, . . .
0 0.5 1 1.5 2 2.5 310
−6
10−4
10−2
100
0 0.5 1 1.5 2 2.5 310
−6
10−5
10−4
10−3
10−2
f1(θ) f2(θ)
New Orleans, December 2007 13
![Page 14: The meaning of Distances in Spectral Analysispeople.ece.umn.edu/~georgiou/papers/plenary1.pdf · 2007-12-16 · geometric mean of f 1 f ... in spectral analysis Operational significance](https://reader033.vdocuments.net/reader033/viewer/2022050105/5f439c1e3e36fe60e101f355/html5/thumbnails/14.jpg)
What is it we would like to have?
distance(
0 0.5 1 1.5 2 2.5 310
−6
10−5
10−4
10−3
10−2
10−1
, 0 0.5 1 1.5 2 2.5 310
−6
10−5
10−4
10−3
10−2 )
f1(θ), f2(θ)
• metric• meaningful & natural
candidates?Kullback-Leibler, Bregman, Itakura-Saito, Makhoul,..
convex functionalsperceptual qualities
New Orleans, December 2007 14
![Page 15: The meaning of Distances in Spectral Analysispeople.ece.umn.edu/~georgiou/papers/plenary1.pdf · 2007-12-16 · geometric mean of f 1 f ... in spectral analysis Operational significance](https://reader033.vdocuments.net/reader033/viewer/2022050105/5f439c1e3e36fe60e101f355/html5/thumbnails/15.jpg)
Linear prediction
One-step-ahead prediction: upresent − upresent|past
with upresent|past :=∑
pastαkuk
E{|upresent − upresent|past|2} = variance of prediction error
New Orleans, December 2007 15
![Page 16: The meaning of Distances in Spectral Analysispeople.ece.umn.edu/~georgiou/papers/plenary1.pdf · 2007-12-16 · geometric mean of f 1 f ... in spectral analysis Operational significance](https://reader033.vdocuments.net/reader033/viewer/2022050105/5f439c1e3e36fe60e101f355/html5/thumbnails/16.jpg)
Szego’s theorem
One-step-ahead prediction:
least error variance = exp
{1
2π
∫log f (θ)dθ
}G. Szego
it is a geometric mean. . .
exp{13
(log f1 + log f2 + log f3)} = 3√f1f2f3
New Orleans, December 2007 16
![Page 17: The meaning of Distances in Spectral Analysispeople.ece.umn.edu/~georgiou/papers/plenary1.pdf · 2007-12-16 · geometric mean of f 1 f ... in spectral analysis Operational significance](https://reader033.vdocuments.net/reader033/viewer/2022050105/5f439c1e3e36fe60e101f355/html5/thumbnails/17.jpg)
Degradation of prediction error variance
Use f2 to design a predictor (assuming uf2,time).
Then compare how this performs on uf1,time against the optimal based on f1.
degraded variance︷ ︸︸ ︷E{|uf1,present −
∑past
af2,past uf1,past|2}− optimal variance
optimal variance≥ 0
New Orleans, December 2007 17
![Page 18: The meaning of Distances in Spectral Analysispeople.ece.umn.edu/~georgiou/papers/plenary1.pdf · 2007-12-16 · geometric mean of f 1 f ... in spectral analysis Operational significance](https://reader033.vdocuments.net/reader033/viewer/2022050105/5f439c1e3e36fe60e101f355/html5/thumbnails/18.jpg)
Degradation of prediction variance
degraded variance︷ ︸︸ ︷E{|uf1,present −
∑past
af2,past uf1,past|2}
optimal variance=
arithmetic mean of(f1
f2
)geometric mean of
(f1
f2
)
=
(1
2π
∫(f1
f2)dθ)
exp(
12π
∫log(f1
f2)dθ)
arithmetic over geometric mean (≥ 1)
New Orleans, December 2007 18
![Page 19: The meaning of Distances in Spectral Analysispeople.ece.umn.edu/~georgiou/papers/plenary1.pdf · 2007-12-16 · geometric mean of f 1 f ... in spectral analysis Operational significance](https://reader033.vdocuments.net/reader033/viewer/2022050105/5f439c1e3e36fe60e101f355/html5/thumbnails/19.jpg)
Riemannian metric
f1 = f,f2 = f + ∆
degraded variance︷ ︸︸ ︷E{|uf1,present −
∑past
af2,past uf1,past|2}− optimal variance
optimal variance'
δ(f, f + ∆) =1
2π
∫ (∆
f
)2
dθ −(
1
2π
∫ (∆
f
)dθ
)2
variance-like: (mean square) - (arithmetic-mean)2
New Orleans, December 2007 19
![Page 20: The meaning of Distances in Spectral Analysispeople.ece.umn.edu/~georgiou/papers/plenary1.pdf · 2007-12-16 · geometric mean of f 1 f ... in spectral analysis Operational significance](https://reader033.vdocuments.net/reader033/viewer/2022050105/5f439c1e3e36fe60e101f355/html5/thumbnails/20.jpg)
Geodesics
Paths fr (r ∈ [0, 1]) between f0, f1 of minimal length∫ 1
0
√δ (fr, fr+dr)
each point represents a different power spectral density
New Orleans, December 2007 20
![Page 21: The meaning of Distances in Spectral Analysispeople.ece.umn.edu/~georgiou/papers/plenary1.pdf · 2007-12-16 · geometric mean of f 1 f ... in spectral analysis Operational significance](https://reader033.vdocuments.net/reader033/viewer/2022050105/5f439c1e3e36fe60e101f355/html5/thumbnails/21.jpg)
Geodesics
The geodesics are exponential families:
fr = f0
(f1
f0
)r
, r∈[0, 1]
= exp{ (1− r) log (f0) + r log (f1) }
0 0.5 1 1.5 2 2.5 310
−6
10−5
10−4
10−3
10−2
10−1
0 0.5 1 1.5 2 2.5 310
−6
10−5
10−4
10−3
10−2
morphing
New Orleans, December 2007 21
![Page 22: The meaning of Distances in Spectral Analysispeople.ece.umn.edu/~georgiou/papers/plenary1.pdf · 2007-12-16 · geometric mean of f 1 f ... in spectral analysis Operational significance](https://reader033.vdocuments.net/reader033/viewer/2022050105/5f439c1e3e36fe60e101f355/html5/thumbnails/22.jpg)
Geodesic distance: metric
The path-length is
d(f0, f1) :=
√1
2π
∫ π
−π
(log
f1
f0
)2
dθ −(
1
2π
∫ π
−πlog
(f1
f0
)dθ
)2
variance-like distance on logarithms: (mean square) - (arithmetic-mean)2
scale-insensitive, “shape” recognizer
0 0.5 1 1.5 2 2.5 3 3.510
−5
10−4
10−3
10−2
10−1
100
0 0.5 1 1.5 2 2.5 3−3
−2.5
−2
−1.5
−1
−0.5
0
0.5
1
1.5
2
2.5
In IEEE Trans. on Signal Processing, Aug. 2007 log f1
f0= log(f1)− log(f0)
New Orleans, December 2007 22
![Page 23: The meaning of Distances in Spectral Analysispeople.ece.umn.edu/~georgiou/papers/plenary1.pdf · 2007-12-16 · geometric mean of f 1 f ... in spectral analysis Operational significance](https://reader033.vdocuments.net/reader033/viewer/2022050105/5f439c1e3e36fe60e101f355/html5/thumbnails/23.jpg)
Prediction-basedgeometry
Informationgeometry
New Orleans, December 2007 23
![Page 24: The meaning of Distances in Spectral Analysispeople.ece.umn.edu/~georgiou/papers/plenary1.pdf · 2007-12-16 · geometric mean of f 1 f ... in spectral analysis Operational significance](https://reader033.vdocuments.net/reader033/viewer/2022050105/5f439c1e3e36fe60e101f355/html5/thumbnails/24.jpg)
Information geometry – parallels
f ; p : probability density
I = Ep{(∂λ log pλ)2} δλ2
Fisher information metric
I =∑∆2
p
R. Fisher
C.R. Rao
New Orleans, December 2007 24
![Page 25: The meaning of Distances in Spectral Analysispeople.ece.umn.edu/~georgiou/papers/plenary1.pdf · 2007-12-16 · geometric mean of f 1 f ... in spectral analysis Operational significance](https://reader033.vdocuments.net/reader033/viewer/2022050105/5f439c1e3e36fe60e101f355/html5/thumbnails/25.jpg)
Information geometry – parallels
Expected “message-length increase”:
H(p1|p0) =(−∑
p1 log(p0))−(−∑
p1 log(p1))
Fisher information metricp0 = pp1 = p + ∆
I =∑∆2
p
R. Leibler
S. Kullback
New Orleans, December 2007 25
![Page 26: The meaning of Distances in Spectral Analysispeople.ece.umn.edu/~georgiou/papers/plenary1.pdf · 2007-12-16 · geometric mean of f 1 f ... in spectral analysis Operational significance](https://reader033.vdocuments.net/reader033/viewer/2022050105/5f439c1e3e36fe60e101f355/html5/thumbnails/26.jpg)
Information geometry – parallels
Geodesics: great circles
p 7→ √p ∈ Sphere p(1)p(2)p(3)
7→√p(1)√p(2)√p(3)
Geodesic distance: ArclengthBattacharyya distance
New Orleans, December 2007 26
![Page 27: The meaning of Distances in Spectral Analysispeople.ece.umn.edu/~georgiou/papers/plenary1.pdf · 2007-12-16 · geometric mean of f 1 f ... in spectral analysis Operational significance](https://reader033.vdocuments.net/reader033/viewer/2022050105/5f439c1e3e36fe60e101f355/html5/thumbnails/27.jpg)
Information vs. prediction-based
∑∆2
pvs.
∫ (∆
f
)2
−(∫
∆
f
)2
p 7→ √p vs. f 7→ log f
great circles vs. logarithmic families
New Orleans, December 2007 27
![Page 28: The meaning of Distances in Spectral Analysispeople.ece.umn.edu/~georgiou/papers/plenary1.pdf · 2007-12-16 · geometric mean of f 1 f ... in spectral analysis Operational significance](https://reader033.vdocuments.net/reader033/viewer/2022050105/5f439c1e3e36fe60e101f355/html5/thumbnails/28.jpg)
Information geometry – parallels
Ability to differentiate decreases p(1)p(2)p(3)
7→ M
p(1)p(2)p(3)
Chentsov’s theorem:
Stochastic maps are contractive
under Fisher metric
and
Fisher metric is the uniqueRiemannian metric with this property
New Orleans, December 2007 28
![Page 29: The meaning of Distances in Spectral Analysispeople.ece.umn.edu/~georgiou/papers/plenary1.pdf · 2007-12-16 · geometric mean of f 1 f ... in spectral analysis Operational significance](https://reader033.vdocuments.net/reader033/viewer/2022050105/5f439c1e3e36fe60e101f355/html5/thumbnails/29.jpg)
What is the analog for power spectra?
addititive noise
f 7→ f + fnoise
multiplicative noise
f 7→ f ? fnoise
continuity of moments (second-order statistics)
f 7→ integrals of f
New Orleans, December 2007 29
![Page 30: The meaning of Distances in Spectral Analysispeople.ece.umn.edu/~georgiou/papers/plenary1.pdf · 2007-12-16 · geometric mean of f 1 f ... in spectral analysis Operational significance](https://reader033.vdocuments.net/reader033/viewer/2022050105/5f439c1e3e36fe60e101f355/html5/thumbnails/30.jpg)
Prediction-basedgeometry
Informationgeometry
Transportgeometry
New Orleans, December 2007 30
![Page 31: The meaning of Distances in Spectral Analysispeople.ece.umn.edu/~georgiou/papers/plenary1.pdf · 2007-12-16 · geometric mean of f 1 f ... in spectral analysis Operational significance](https://reader033.vdocuments.net/reader033/viewer/2022050105/5f439c1e3e36fe60e101f355/html5/thumbnails/31.jpg)
Transport geometry
Monge-Kantorovich problemminimize cost of transferring mass∫
cost(x→ y) × mass(dx, dy)
G. Monge
L. Kantorovich
New Orleans, December 2007 31
![Page 32: The meaning of Distances in Spectral Analysispeople.ece.umn.edu/~georgiou/papers/plenary1.pdf · 2007-12-16 · geometric mean of f 1 f ... in spectral analysis Operational significance](https://reader033.vdocuments.net/reader033/viewer/2022050105/5f439c1e3e36fe60e101f355/html5/thumbnails/32.jpg)
Transport for power spectra
Transport-based metric
distances do not increase
under additive noiseand multiplicative noise
with power ≤ 1
+ continuity of statistics
metric = min (cost of transport(f0, f1) + normalization)
with Johan Karlsson (KTH) & Mir Shahrouz Takyar
New Orleans, December 2007 32
![Page 33: The meaning of Distances in Spectral Analysispeople.ece.umn.edu/~georgiou/papers/plenary1.pdf · 2007-12-16 · geometric mean of f 1 f ... in spectral analysis Operational significance](https://reader033.vdocuments.net/reader033/viewer/2022050105/5f439c1e3e36fe60e101f355/html5/thumbnails/33.jpg)
Prediction-basedgeometry
Transportgeometry
applications
New Orleans, December 2007 33
![Page 34: The meaning of Distances in Spectral Analysispeople.ece.umn.edu/~georgiou/papers/plenary1.pdf · 2007-12-16 · geometric mean of f 1 f ... in spectral analysis Operational significance](https://reader033.vdocuments.net/reader033/viewer/2022050105/5f439c1e3e36fe60e101f355/html5/thumbnails/34.jpg)
Fitting geodesics
K.F. Gauss
Least squares: The theory of motion of heavenly bodies, Gauss, K.F.
New Orleans, December 2007 34
![Page 35: The meaning of Distances in Spectral Analysispeople.ece.umn.edu/~georgiou/papers/plenary1.pdf · 2007-12-16 · geometric mean of f 1 f ... in spectral analysis Operational significance](https://reader033.vdocuments.net/reader033/viewer/2022050105/5f439c1e3e36fe60e101f355/html5/thumbnails/35.jpg)
Tracking with geodesics
with Xianhua Jiang
New Orleans, December 2007 35
![Page 36: The meaning of Distances in Spectral Analysispeople.ece.umn.edu/~georgiou/papers/plenary1.pdf · 2007-12-16 · geometric mean of f 1 f ... in spectral analysis Operational significance](https://reader033.vdocuments.net/reader033/viewer/2022050105/5f439c1e3e36fe60e101f355/html5/thumbnails/36.jpg)
Voice & sounds
John Weissmuller’s MGM Tarzan Yell
http://www.complxmind.com
New Orleans, December 2007 36
![Page 37: The meaning of Distances in Spectral Analysispeople.ece.umn.edu/~georgiou/papers/plenary1.pdf · 2007-12-16 · geometric mean of f 1 f ... in spectral analysis Operational significance](https://reader033.vdocuments.net/reader033/viewer/2022050105/5f439c1e3e36fe60e101f355/html5/thumbnails/37.jpg)
Images & more
Geometric active contours
∂∂tCurve = ∇Curve metric(finside, foutside)
with Romeil Sandhu and Allen Tannenbaum
New Orleans, December 2007 37
![Page 38: The meaning of Distances in Spectral Analysispeople.ece.umn.edu/~georgiou/papers/plenary1.pdf · 2007-12-16 · geometric mean of f 1 f ... in spectral analysis Operational significance](https://reader033.vdocuments.net/reader033/viewer/2022050105/5f439c1e3e36fe60e101f355/html5/thumbnails/38.jpg)
Images & more
with Romeil Sandhu and Allen Tannenbaum
New Orleans, December 2007 38
![Page 39: The meaning of Distances in Spectral Analysispeople.ece.umn.edu/~georgiou/papers/plenary1.pdf · 2007-12-16 · geometric mean of f 1 f ... in spectral analysis Operational significance](https://reader033.vdocuments.net/reader033/viewer/2022050105/5f439c1e3e36fe60e101f355/html5/thumbnails/39.jpg)
Concluding thoughtsMetricsin spectral analysis
• Operational significance
• Effect of natural transformations
New Orleans, December 2007 39
![Page 40: The meaning of Distances in Spectral Analysispeople.ece.umn.edu/~georgiou/papers/plenary1.pdf · 2007-12-16 · geometric mean of f 1 f ... in spectral analysis Operational significance](https://reader033.vdocuments.net/reader033/viewer/2022050105/5f439c1e3e36fe60e101f355/html5/thumbnails/40.jpg)
Thank you for your attention
thanks toXianhua Jiang Johan Karlsson Romeil Sandhu Mir Shahrouz Takyar
Allen Tannenbaum & Anders Lindquist
National Science Foundation, AFOSR, and Hermes-Luh endowment
New Orleans, December 2007 40