geometric fourier analysis in computational vision
TRANSCRIPT
Geometric Fourier Analysis in Geometric Fourier Analysis in Computational VisionComputational Vision
Biological and Mathematical BackgroundBiological and Mathematical Background
Jacek TurskiJacek TurskiUniversity of HoustonUniversity of Houston--DTNDTN
99thth International Conference on Cognitive and Neural Systems International Conference on Cognitive and Neural Systems
Boston University, May 18Boston University, May 18--21, 200521, 2005
Outline
I. The Brain Visual pathway1. Biological Facts2. Local V1 Topography3. On Global Topography4. Cyclopean Symmetry
II. GFA for Computational Vision5. Mathematical Background6. The Conformal Camera7. Projective Fourier Analysis
III. On Binocular Image Processing 8. Cortical Image Processing9. Binocular Vision
- Cortical Images- Horopter
References
[1] M. Balasubraminian, J. Polimeni and E.L. Schwartz, The V1-V2-V3 complex: Quasiconformal dipole maps in primate striate and extra-striate cortex, Neural Networks, 15, 1157-1163, 2002.
[2] J. Turski, Geometric Fourier Analysis for Computational Vision, JFAA, 11, 1-23, 2005.
Acknowledgement: Some pictures have been borrowed from Eric Schwartz web page http://www.cns.bu.edu/~eric
1. Biological Facts
��������������� ��������������������� ��������������������� ��������������������� ��������������������� ��������������������� ��������������������� ��������������������� ������
50% of the primary visual cortex (V1) is used toprocess input from the foveal region of 4% of theretinal area
��������������������������������������������������������������������������������
Image is transmitted from the retina to the visual cortex along the visual pathway in a precise,although changing, retinotopic arrangement, resulting in many maps of features (topography, orientation, eye dominance, motion direction, etc) superimposed in visual cortex.
-20 º
-40 º
-60 º
-80 º
20 º 40 º
60 º
80º
0 º
2. Local V1 Topography
With the ganglion density (away from the central part of the fovea) and a uniform V1 packing call density ,
gives the coordinate mapping
which defines the local V1 topography
c��r�rdrd� � cdud�
�k lnr, �� � k lnz
z � k lnz
��r� � �r2
After the work of Eric Schwartz and his group:
(1) The mapping is an accepted approximation of the V1
topographic structure.
(2) A better model represents a full topography in terms of the mapping
w � k ln z�az�b
w � k ln�z � a�
3. On Global Topography
5. Mathematical Background
Geometric Fourier Analysis (GFA)Geometric Fourier Analysis (GFA) – Harmonic Analysis associated with the Unitary Group Representations
Note: The Classical Fourier Transform is associated with the unitary representations of the group of translations.
We constructed Computational Harmonic Analysis of the group SL(2,C) -- the group that provides image projective transformations in the Conformal Camera. Remarkably, the resulting image representation is also well adapted to retinotopic mapping of the brain’s visual pathway.
The Conformal Camera
x 2 � 1
Image plane:
z � g � z � dz�cbz�a
; g �a b
c d� SL�2,��
The embedding: x2�iyx 3�ix 1
�z1z2
� � � � 2
g z1z2
� � induces g � z on slopes of lines z2 � z z1
Complex Projective GeometryComplex Projective Geometryz�� � k � z
z� � h � z
z x 2
p
x
x 1
S�0,1,0�2
p � �
p �
�x1,1,x3� � x3 � ix1Complex Projective transformationsComplex Projective transformations
are finite iterations of
Image conformal – projective transformations:
f�z� � f�g�1 � z�
h � z ���1z��
� and k � z ��z��
�� z��
The Camera Geometry of the Image Plane
6. The Conformal Camera
The conformal camera reduces the projective degrees of freedom to a minimal set of image projective transformations.
We will show later that this set of projective transformations is relevant to the way the human vision system acquires the understanding of 3D scenes.
6. The Conformal Camera
Image plane
S�0,1,0�2
7. Projective Fourier Analysis
�f �s, k� � i
2� f�z�|z|�is�1 z
|z|
�kdzd z
G ive n the action of SL�2,� �: � � z � g � z � � ,de compos e a patte rn f � L 2 �� � in te rms of theirre ducible unitary re pre s e ntations of SL�2,� �pre s e nt in L 2 �� �.
The Projective Fourier Transform (PFT)
f�z� � 1�2�� 2
� k���k�� �
�f �s, k�|z| is�1 z
|z|
kds
The Inverse PFT
Geometric Fourier Analysis of the Conformal Camera
The Borel Characters �k,s(B) = |z|-is(|z/|z|)-k
7. Projective Fourier Analysis
Gauss Decomposition� SL�2,�� .� NB, where N �
1 0
� 1and
B �z �
0 z�1, implies that the Borel subgroup B exhausts the
projective part of SL�2,��.
8. Cortical Image Processing
• PFT is the standard FT in log-polar• We discretize PFT and compute it by FFT in log-polar
�u, �� where u � ln r
Nonuniform Log-polar Sampling
– Retinal Image
Inverse DPFT in Log-polar Coordinates
– Cortical Image
The Discrete PFT and Its Inverse (Log-polar)
Projective “covariance” of PFT
fm ,n� � 1
MNk�0
M�1
�l�0
N�1
��f k,le
�u m,n�
ei2�u m,n� k/Tei�m,n
�lL
�f m,n � �k�0
M�1 �l�0N�1 fk,le
uk e �i2�umk/T e�i2��nlL
fk,l �1
MN �m�0M�1�n�0
N�1 �f m,ne�uk e i2�umk/Tei2��nlL
um ,n� � i�m ,n
� � lnzm ,n� � ln�g�1 � zm ,n�
8. Cortical Image Processing
9. Binocular Vision
Binocular Disparity
Cortical Projection of the Right Visual Field
Right Eye
Left Eye
Right Hemisphere
Left Hemisphere
V2 & V3
V1
V1
V2 & V3
Fixationpoint
LGN
LGN
Fovea
Fovea
Cortical Projection of the Left Visual Field
Right Visual Field
Left Visual Field
- Cortical Images
Cortical Image from The Left Eye
Cortical Image from The Right EyeThe Projection onto the Right Eye Retina
The Projection onto the Left Eye Retina
- Horopter
Theorem. For binocular system with the conformal cameras, the horopters in the visual plane are conics, closely matching the empirical horopters.
Cortical Image of the Right Visual Field
V2 &V3
V1
V1
V2 & V3
Fixation point
M
Horopter = Zero-Disparity Curve
Cortical Image of the Left Visual Field
With retinas acting as if they were spheres, the geometric horopters are circles, known as the Vieth-Muller circles.
Conclusions
Because: (1) the camera with silicon retina sensors
produces the image similar to the topographic image in V1,
(2) the line singularity most likely exists in the fovea,the head–eye–visual cortex integrated system makes possible an efficient, biologically realistic computational approach to binocular vision for robotic systems