signal processing and representation theory lecture 3
TRANSCRIPT
![Page 1: Signal Processing and Representation Theory Lecture 3](https://reader034.vdocuments.net/reader034/viewer/2022051316/56649f1d5503460f94c34077/html5/thumbnails/1.jpg)
Signal Processingand
Representation Theory
Lecture 3
![Page 2: Signal Processing and Representation Theory Lecture 3](https://reader034.vdocuments.net/reader034/viewer/2022051316/56649f1d5503460f94c34077/html5/thumbnails/2.jpg)
Outline:• Review• Spherical Harmonics• Rotation Invariance• Correlation and Wigner-D Functions
![Page 3: Signal Processing and Representation Theory Lecture 3](https://reader034.vdocuments.net/reader034/viewer/2022051316/56649f1d5503460f94c34077/html5/thumbnails/3.jpg)
Representation Theory
ReviewGiven a representation of a group G onto an inner product space V, decomposing V into the direct sum of irreducible sub-representations:
V=V1…Vnmakes it easier to:
– Compute the correlation between two vectors: fewer multiplications are needed
– Obtain G-invariant information: more transformation invariant norms can be obtained
![Page 4: Signal Processing and Representation Theory Lecture 3](https://reader034.vdocuments.net/reader034/viewer/2022051316/56649f1d5503460f94c34077/html5/thumbnails/4.jpg)
Representation Theory
ReviewIn the case that the group G is commutative, the irreducible sub-representations Vi are all one-complex-dimensional, (Schur’s Lemma).
Example:
If V is the space of functions on a circle, represented by n-dimensional arrays, and G is the group of 2D rotations:
– Correlation can be done in O(n log n) time (using the FFT)
– We can obtain n/2-dimensional, rotation invariant descriptors
![Page 5: Signal Processing and Representation Theory Lecture 3](https://reader034.vdocuments.net/reader034/viewer/2022051316/56649f1d5503460f94c34077/html5/thumbnails/5.jpg)
Representation Theory
What happens when the group G is not commutative?
Example:
If V is the space of functions on a sphere and G is the group of 3D rotations:
– How quickly can we correlate?
– How much rotation invariant information can we get?
![Page 6: Signal Processing and Representation Theory Lecture 3](https://reader034.vdocuments.net/reader034/viewer/2022051316/56649f1d5503460f94c34077/html5/thumbnails/6.jpg)
Outline:• Review• Spherical Harmonics• Rotation Invariance• Correlation and Wigner-D Functions
![Page 7: Signal Processing and Representation Theory Lecture 3](https://reader034.vdocuments.net/reader034/viewer/2022051316/56649f1d5503460f94c34077/html5/thumbnails/7.jpg)
Representation Theory
Spherical Harmonic DecompositionGoal:
Find the irreducible sub-representations of the group of 3D rotation acting on the space of spherical functions.
![Page 8: Signal Processing and Representation Theory Lecture 3](https://reader034.vdocuments.net/reader034/viewer/2022051316/56649f1d5503460f94c34077/html5/thumbnails/8.jpg)
Representation Theory
Spherical Harmonic DecompositionPreliminaries:
If f is a function defined in 3D, we can get a function on the unit sphere by looking at the restriction of f to points with norm 1.
![Page 9: Signal Processing and Representation Theory Lecture 3](https://reader034.vdocuments.net/reader034/viewer/2022051316/56649f1d5503460f94c34077/html5/thumbnails/9.jpg)
Representation Theory
Spherical Harmonic DecompositionPreliminaries:
A polynomial p(x,y,z) is homogenous of degree d if it is the linear sum of monomials of degree d:
d
d
ddd
dddd
dd
dddd
dd
za
zyaxa
zyaxyayxaxa
yaxyayxaxazyxp
0,
10,11,1
10,1
21,1
22,1
11,1
0,01
1,01
1,0,0),,(
![Page 10: Signal Processing and Representation Theory Lecture 3](https://reader034.vdocuments.net/reader034/viewer/2022051316/56649f1d5503460f94c34077/html5/thumbnails/10.jpg)
Representation Theory
Spherical Harmonic DecompositionPreliminaries:
We can think of the space of homogenous polynomials of degree d in x, y, and z as:
where Pd(x,y) is the space of homogenous polynomials of degreed d in x and y.
ddddd zyxPzyxPzyxPyxPzyxP ),(),(),(),(),,( 0
111
![Page 11: Signal Processing and Representation Theory Lecture 3](https://reader034.vdocuments.net/reader034/viewer/2022051316/56649f1d5503460f94c34077/html5/thumbnails/11.jpg)
Representation Theory
Spherical Harmonic DecompositionPreliminaries:
If we let Pd(x,y,z) be the set of homogenous polynomials of degree d, then Pd(x,y,z) is a vector-space of dimension:
2
)1()1(1
2
0
ddi
d
i
![Page 12: Signal Processing and Representation Theory Lecture 3](https://reader034.vdocuments.net/reader034/viewer/2022051316/56649f1d5503460f94c34077/html5/thumbnails/12.jpg)
Representation Theory
Spherical Harmonic DecompositionObservation:
If M is any 3x3 matrix, and p(x,y,z) is a homogenous polynomial of degree d:
then p(M(x,y,z)) is also a homogenous polynomial of degree d:
333231
232221
131211
mmm
mmm
mmm
M
d
j
jd
k
jkjdkkj zyxazyxp
0 0,),,(
jd
j
jd
k
kjdkkj zmymxmzmymxmzmymxmazyxMp )()()(),,( 333231
0 0232221131211,
![Page 13: Signal Processing and Representation Theory Lecture 3](https://reader034.vdocuments.net/reader034/viewer/2022051316/56649f1d5503460f94c34077/html5/thumbnails/13.jpg)
Representation Theory
Spherical Harmonic DecompositionIf V is the space of functions on the sphere, we can consider the sub-space of functions on the sphere that are restrictions of homogenous polynomials of degree d.
Since a rotation will map a homogenous polynomial of degree d back to a homogenous polynomial of degree d, these sub-spaces are sub-representations.
![Page 14: Signal Processing and Representation Theory Lecture 3](https://reader034.vdocuments.net/reader034/viewer/2022051316/56649f1d5503460f94c34077/html5/thumbnails/14.jpg)
Representation Theory
Spherical Harmonic DecompositionIn general, the space of homogenous polynomials of degree d has dimension (d+1)+(d)+(d-1)+…+1:
d
j
jd
k
jkjdkkj zyxazyxp
0 0,),,(
![Page 15: Signal Processing and Representation Theory Lecture 3](https://reader034.vdocuments.net/reader034/viewer/2022051316/56649f1d5503460f94c34077/html5/thumbnails/15.jpg)
Representation TheorySpherical Harmonic DecompositionIf (x,y,z) is a point on the sphere, we know that this point satisfies:
Thus, if q(x,y,z)Pd(x,y,z), then even though in general, the polynomial:
is a homogenous polynomial of degree d+2, its restriction to the sphere is actually a homogenous polynomial of degree d.
1222 zyx
))(,,( 222 zyxzyxq
![Page 16: Signal Processing and Representation Theory Lecture 3](https://reader034.vdocuments.net/reader034/viewer/2022051316/56649f1d5503460f94c34077/html5/thumbnails/16.jpg)
Representation Theory
Spherical Harmonic Decomposition
So, while the sub-spaces Pd(x,y,z) are sub-representations, they are not irreducible as Pd-2(x,y,z)Pd(x,y,z).
To get the irreducible sub-representations, we look at the spaces:
),,(),,( 2 zyxPzyxPV ddd
![Page 17: Signal Processing and Representation Theory Lecture 3](https://reader034.vdocuments.net/reader034/viewer/2022051316/56649f1d5503460f94c34077/html5/thumbnails/17.jpg)
Representation Theory
Spherical Harmonic DecompositionAnd the dimension of these sub-representations is:
12
)1()1(1
)1()1(
),,(dim),,(dimdim
2
0
2
0
2
00
2
d
iidd
ii
zyxPzyxPV
d
i
d
i
d
i
d
i
ddd
![Page 18: Signal Processing and Representation Theory Lecture 3](https://reader034.vdocuments.net/reader034/viewer/2022051316/56649f1d5503460f94c34077/html5/thumbnails/18.jpg)
Representation Theory
Spherical Harmonic DecompositionThe spherical harmonics of frequency d are an orthonormal basis for the space of functions Vd.
If we represent a point on a sphere in terms of its angle of elevation and azimuth:
with 0π and 0 <2π …
sinsin,cos,sincos,
![Page 19: Signal Processing and Representation Theory Lecture 3](https://reader034.vdocuments.net/reader034/viewer/2022051316/56649f1d5503460f94c34077/html5/thumbnails/19.jpg)
Representation Theory
Spherical Harmonic DecompositionThe spherical harmonics are functions Ylm, with l0 and -lml spanning the sub-representations Vl:
Span
Span
Span
),(),,(,),,(),,(
),(),,(),,(
),(
11
11
01
111
000
kk
kk
kk
kkk YYYYV
YYYV
YV
![Page 20: Signal Processing and Representation Theory Lecture 3](https://reader034.vdocuments.net/reader034/viewer/2022051316/56649f1d5503460f94c34077/html5/thumbnails/20.jpg)
Representation Theory
Spherical Harmonic DecompositionFact:
If we have a function defined on the sphere, sampled on a regular nxn grid of angles of elevation and azimuth, the forward and inverse spherical harmonic transforms can be computed in O(n2 log2n).
Like the FFT, the fast spherical harmonic transform can be thought of as a change of basis, and a brute force method would take O(n4) time.
![Page 21: Signal Processing and Representation Theory Lecture 3](https://reader034.vdocuments.net/reader034/viewer/2022051316/56649f1d5503460f94c34077/html5/thumbnails/21.jpg)
Representation Theory
What are the spherical harmonics Ylm(,)?
![Page 22: Signal Processing and Representation Theory Lecture 3](https://reader034.vdocuments.net/reader034/viewer/2022051316/56649f1d5503460f94c34077/html5/thumbnails/22.jpg)
Representation Theory
What are the spherical harmonics Ylm?Conceptually:
The Ylm are the different homogenous polynomials of degree l:
4
1,0
0 Y
)(8
3,
8
3,
)(8
3,
11
01
11
izxY
yY
izxY
222
12
22202
12
222
)(32
15,
)(8
15,
)2(16
5,
)(8
15,
)(32
15,
izxY
iyzxyY
zyxY
iyzxyY
izxY
![Page 23: Signal Processing and Representation Theory Lecture 3](https://reader034.vdocuments.net/reader034/viewer/2022051316/56649f1d5503460f94c34077/html5/thumbnails/23.jpg)
Representation Theory
What are the spherical harmonics Ylm?Technically:
Where the Plm are the associated Legendre polynomials:
Where the Pl are the Legendre polynomials:
imml
ml eP
mlmll
Y cos)!(
)!(
4
12,
)(1)1(2/2 zPdzd
zzP lm
mmmm
l
dttttzi
zP nl
12/12212
1
![Page 24: Signal Processing and Representation Theory Lecture 3](https://reader034.vdocuments.net/reader034/viewer/2022051316/56649f1d5503460f94c34077/html5/thumbnails/24.jpg)
Representation Theory
What are the spherical harmonics Ylm?Functionally:
The Ylm are the eigen-values of the Laplacian operator:
),(),(2 ff
![Page 25: Signal Processing and Representation Theory Lecture 3](https://reader034.vdocuments.net/reader034/viewer/2022051316/56649f1d5503460f94c34077/html5/thumbnails/25.jpg)
Representation Theory
What are the spherical harmonics Ylm?Visually:
The Ylm are spherical functions whose number of lobes get larger as the frequency, l, gets bigger:
l=1
l=2
l=3
l=0
![Page 26: Signal Processing and Representation Theory Lecture 3](https://reader034.vdocuments.net/reader034/viewer/2022051316/56649f1d5503460f94c34077/html5/thumbnails/26.jpg)
Representation Theory
What are the spherical harmonics Ylm?What is important about the spherical harmonics is that they are an orthonormal basis for the (2d+1)-dimensional sub-representations, Vd, of the group of 3D rotations acting on the space of spherical functions.
![Page 27: Signal Processing and Representation Theory Lecture 3](https://reader034.vdocuments.net/reader034/viewer/2022051316/56649f1d5503460f94c34077/html5/thumbnails/27.jpg)
Representation Theory
Sub-Representations
![Page 28: Signal Processing and Representation Theory Lecture 3](https://reader034.vdocuments.net/reader034/viewer/2022051316/56649f1d5503460f94c34077/html5/thumbnails/28.jpg)
Representation Theory
Sub-Representations
![Page 29: Signal Processing and Representation Theory Lecture 3](https://reader034.vdocuments.net/reader034/viewer/2022051316/56649f1d5503460f94c34077/html5/thumbnails/29.jpg)
Representation Theory
Sub-Representations
![Page 30: Signal Processing and Representation Theory Lecture 3](https://reader034.vdocuments.net/reader034/viewer/2022051316/56649f1d5503460f94c34077/html5/thumbnails/30.jpg)
Representation Theory
Sub-Representations
![Page 31: Signal Processing and Representation Theory Lecture 3](https://reader034.vdocuments.net/reader034/viewer/2022051316/56649f1d5503460f94c34077/html5/thumbnails/31.jpg)
Outline:• Review• Spherical Harmonics• Rotation Invariance• Correlation and Wigner-D Functions
![Page 32: Signal Processing and Representation Theory Lecture 3](https://reader034.vdocuments.net/reader034/viewer/2022051316/56649f1d5503460f94c34077/html5/thumbnails/32.jpg)
Representation Theory
InvarianceGiven a spherical function f, we can obtain a rotation invariant representation by expressing f in terms of its spherical harmonic decomposition:
where each flVl:
l
lm
ml
mll Yaf ),(),(
0
),(),(l
lff
![Page 33: Signal Processing and Representation Theory Lecture 3](https://reader034.vdocuments.net/reader034/viewer/2022051316/56649f1d5503460f94c34077/html5/thumbnails/33.jpg)
Representation Theory
InvarianceWe can then obtain a rotation invariant representation by storing the size of each fl independently:
where:221212 l
lll
ll
lll aaaaf
,,,,)( 10 lffff
![Page 34: Signal Processing and Representation Theory Lecture 3](https://reader034.vdocuments.net/reader034/viewer/2022051316/56649f1d5503460f94c34077/html5/thumbnails/34.jpg)
Representation Theory
Invariance
Spherical Harmonic Decomposition
+ += +
![Page 35: Signal Processing and Representation Theory Lecture 3](https://reader034.vdocuments.net/reader034/viewer/2022051316/56649f1d5503460f94c34077/html5/thumbnails/35.jpg)
Representation Theory
Invariance
+ += +
+ + +
Constant 1st Order 2nd Order 3rd Order
![Page 36: Signal Processing and Representation Theory Lecture 3](https://reader034.vdocuments.net/reader034/viewer/2022051316/56649f1d5503460f94c34077/html5/thumbnails/36.jpg)
Representation Theory
Invariance
+ + +
Constant 1st Order 2nd Order 3rd Order
Ψ
![Page 37: Signal Processing and Representation Theory Lecture 3](https://reader034.vdocuments.net/reader034/viewer/2022051316/56649f1d5503460f94c34077/html5/thumbnails/37.jpg)
Representation Theory
InvarianceLimitations:
By storing only the energy in the different frequencies, we discard information that does not depend on the pose of the model:
– Inter-frequency information
– Intra-frequency information
![Page 38: Signal Processing and Representation Theory Lecture 3](https://reader034.vdocuments.net/reader034/viewer/2022051316/56649f1d5503460f94c34077/html5/thumbnails/38.jpg)
+
Representation Theory
InvarianceInter-Frequency information:
22.5o90o
=
= +
![Page 39: Signal Processing and Representation Theory Lecture 3](https://reader034.vdocuments.net/reader034/viewer/2022051316/56649f1d5503460f94c34077/html5/thumbnails/39.jpg)
Representation Theory
InvarianceIntra-Frequency information:
![Page 40: Signal Processing and Representation Theory Lecture 3](https://reader034.vdocuments.net/reader034/viewer/2022051316/56649f1d5503460f94c34077/html5/thumbnails/40.jpg)
Representation Theory
Invariance
…
……O(n2)
O(n)
![Page 41: Signal Processing and Representation Theory Lecture 3](https://reader034.vdocuments.net/reader034/viewer/2022051316/56649f1d5503460f94c34077/html5/thumbnails/41.jpg)
Representation Theory
Invariance
…
……O(n2)
O(n)
![Page 42: Signal Processing and Representation Theory Lecture 3](https://reader034.vdocuments.net/reader034/viewer/2022051316/56649f1d5503460f94c34077/html5/thumbnails/42.jpg)
Representation Theory
Invariance
…
……O(n2)
O(n)
![Page 43: Signal Processing and Representation Theory Lecture 3](https://reader034.vdocuments.net/reader034/viewer/2022051316/56649f1d5503460f94c34077/html5/thumbnails/43.jpg)
Representation Theory
Invariance
…
……O(n2)
O(n)
![Page 44: Signal Processing and Representation Theory Lecture 3](https://reader034.vdocuments.net/reader034/viewer/2022051316/56649f1d5503460f94c34077/html5/thumbnails/44.jpg)
Representation Theory
Invariance
…
……O(n2)
O(n)
![Page 45: Signal Processing and Representation Theory Lecture 3](https://reader034.vdocuments.net/reader034/viewer/2022051316/56649f1d5503460f94c34077/html5/thumbnails/45.jpg)
Outline:• Review• Spherical Harmonics• Rotation Invariance• Correlation and Wigner-D Functions
![Page 46: Signal Processing and Representation Theory Lecture 3](https://reader034.vdocuments.net/reader034/viewer/2022051316/56649f1d5503460f94c34077/html5/thumbnails/46.jpg)
Representation Theory
Wigner-D FunctionsThe Wigner-D functions are an orthogonal basis of complex-valued functions defined on the space of rotations:
with l0 and -lm,m’l.
'', ,)( ml
ml
lmm YRYRD
![Page 47: Signal Processing and Representation Theory Lecture 3](https://reader034.vdocuments.net/reader034/viewer/2022051316/56649f1d5503460f94c34077/html5/thumbnails/47.jpg)
Representation TheoryWigner-D FunctionsFact:If we are given a function defined on the group of 3D rotations, sampled on a regular nxnxn grid of Euler angles, the forward and inverse spherical harmonic transforms can be computed in O(n4) time.
Like the FFT and the FST, the fast Wigner-D transform can be thought of as a change of basis, and a brute force method would take O(n6) time.
![Page 48: Signal Processing and Representation Theory Lecture 3](https://reader034.vdocuments.net/reader034/viewer/2022051316/56649f1d5503460f94c34077/html5/thumbnails/48.jpg)
Representation Theory
MotivationGiven two spherical functions f and g we would like to compute the distance between f and g at every rotation. To do this, we need to be able to compute the correlation:
Corr(f,g,R)=f,R(g)at every rotation R.
![Page 49: Signal Processing and Representation Theory Lecture 3](https://reader034.vdocuments.net/reader034/viewer/2022051316/56649f1d5503460f94c34077/html5/thumbnails/49.jpg)
Representation Theory
CorrelationIf we express f and g in terms of their spherical harmonic decompositions:
00
),(),(),(),(l
l
lm
ml
ml
l
l
lm
ml
ml YbgYaf
![Page 50: Signal Processing and Representation Theory Lecture 3](https://reader034.vdocuments.net/reader034/viewer/2022051316/56649f1d5503460f94c34077/html5/thumbnails/50.jpg)
0 ',',
'
0 ',
''
0',
'
''
''
''
0'
'
''
''
''
0
0'
'
''
''
''
0
)(
,
,
,
,)(,
l
l
lmm
lmm
ml
ml
l
l
lmm
ml
ml
ml
ml
ll
l
lm
l
lm
ml
ml
ml
ml
l
m
lm
ml
ml
l
m
lm
ml
ml
l
m
lm
ml
ml
l
m
lm
ml
ml
RDba
YRYba
YRYba
YRbYa
YbRYagRf
Representation Theory
CorrelationThen the correlation of f with g at a rotation R is given by:
![Page 51: Signal Processing and Representation Theory Lecture 3](https://reader034.vdocuments.net/reader034/viewer/2022051316/56649f1d5503460f94c34077/html5/thumbnails/51.jpg)
0 ',
',' )()(,
l
l
lmm
lmm
ml
ml RDbagRf
Representation Theory
CorrelationSo that we get an expression for the correlation of f with g as some linear combination of the Wigner-D functions:
![Page 52: Signal Processing and Representation Theory Lecture 3](https://reader034.vdocuments.net/reader034/viewer/2022051316/56649f1d5503460f94c34077/html5/thumbnails/52.jpg)
Representation Theory
CorrelationThe complexity of correlating two spherical functions sampled on a regular nxn grid is:
– Forward spherical harmonic transform: O(n2 log2n)
00
),(),(),(),(l
l
lm
ml
ml
l
l
lm
ml
ml YbgYaf
![Page 53: Signal Processing and Representation Theory Lecture 3](https://reader034.vdocuments.net/reader034/viewer/2022051316/56649f1d5503460f94c34077/html5/thumbnails/53.jpg)
Representation Theory
CorrelationThe complexity of correlating two spherical functions sampled on a regular nxn grid is:
– Forward spherical harmonic transform: O(n2 log2n)
– Multiplying frequency terms: O(n3)
0 ',
'' ,)(,l
l
lmm
ml
ml
ml
ml YRYbagRf
![Page 54: Signal Processing and Representation Theory Lecture 3](https://reader034.vdocuments.net/reader034/viewer/2022051316/56649f1d5503460f94c34077/html5/thumbnails/54.jpg)
Representation Theory
CorrelationThe complexity of correlating two spherical functions sampled on a regular nxn grid is:
– Forward spherical harmonic transform: O(n2 log2n)
– Multiplying frequency terms: O(n3)
– Inverse Wigner-D transform: O(n4))(,)(
0, ',',
' gRfRDbal
l
lmm
lmm
ml
ml
![Page 55: Signal Processing and Representation Theory Lecture 3](https://reader034.vdocuments.net/reader034/viewer/2022051316/56649f1d5503460f94c34077/html5/thumbnails/55.jpg)
Representation Theory
CorrelationThe complexity of correlating two spherical functions sampled on a regular nxn grid is:
– Forward spherical harmonic transform: O(n2 log2n)
– Multiplying frequency terms: O(n3)
– Inverse Wigner-D transform: O(n4)
Total complexity of correlation is: O(n4)
(Note that a brute force approach would take O(n5): For each of O(n3) rotations we would have to perform an O(n2) dot-product computation.)