invariants to convolution. motivation – template matching
TRANSCRIPT
![Page 1: Invariants to convolution. Motivation – template matching](https://reader035.vdocuments.net/reader035/viewer/2022062423/56649f305503460f94c4b1d9/html5/thumbnails/1.jpg)
Invariants to convolution
![Page 2: Invariants to convolution. Motivation – template matching](https://reader035.vdocuments.net/reader035/viewer/2022062423/56649f305503460f94c4b1d9/html5/thumbnails/2.jpg)
Motivation – template matching
![Page 3: Invariants to convolution. Motivation – template matching](https://reader035.vdocuments.net/reader035/viewer/2022062423/56649f305503460f94c4b1d9/html5/thumbnails/3.jpg)
Image degradation models
Space-variant
Space-invariant convolution
![Page 4: Invariants to convolution. Motivation – template matching](https://reader035.vdocuments.net/reader035/viewer/2022062423/56649f305503460f94c4b1d9/html5/thumbnails/4.jpg)
Two approaches
Traditional approach: Image restoration (blind deconvolution) and traditional features
Proposed approach: Invariants to convolution
![Page 5: Invariants to convolution. Motivation – template matching](https://reader035.vdocuments.net/reader035/viewer/2022062423/56649f305503460f94c4b1d9/html5/thumbnails/5.jpg)
Invariants to convolution
for any admissible h
![Page 6: Invariants to convolution. Motivation – template matching](https://reader035.vdocuments.net/reader035/viewer/2022062423/56649f305503460f94c4b1d9/html5/thumbnails/6.jpg)
The moments under convolution
Geometric/central
Complex
![Page 7: Invariants to convolution. Motivation – template matching](https://reader035.vdocuments.net/reader035/viewer/2022062423/56649f305503460f94c4b1d9/html5/thumbnails/7.jpg)
PSF is centrosymmetric (N = 2) and has a unit integral
Assumptions on the PSF
supp(h)
![Page 8: Invariants to convolution. Motivation – template matching](https://reader035.vdocuments.net/reader035/viewer/2022062423/56649f305503460f94c4b1d9/html5/thumbnails/8.jpg)
Common PSF’s are centrosymmetric
![Page 9: Invariants to convolution. Motivation – template matching](https://reader035.vdocuments.net/reader035/viewer/2022062423/56649f305503460f94c4b1d9/html5/thumbnails/9.jpg)
Invariants to centrosymmetric convolution
where (p + q) is odd
What is the intuitive meaning of the invariants?
“Measure of anti-symmetry”
![Page 10: Invariants to convolution. Motivation – template matching](https://reader035.vdocuments.net/reader035/viewer/2022062423/56649f305503460f94c4b1d9/html5/thumbnails/10.jpg)
Invariants to centrosymmetric convolution
![Page 11: Invariants to convolution. Motivation – template matching](https://reader035.vdocuments.net/reader035/viewer/2022062423/56649f305503460f94c4b1d9/html5/thumbnails/11.jpg)
Invariants to centrosymmetric convolution
![Page 12: Invariants to convolution. Motivation – template matching](https://reader035.vdocuments.net/reader035/viewer/2022062423/56649f305503460f94c4b1d9/html5/thumbnails/12.jpg)
Convolution invariants in FT domain
![Page 13: Invariants to convolution. Motivation – template matching](https://reader035.vdocuments.net/reader035/viewer/2022062423/56649f305503460f94c4b1d9/html5/thumbnails/13.jpg)
Convolution invariants in FT domain
![Page 14: Invariants to convolution. Motivation – template matching](https://reader035.vdocuments.net/reader035/viewer/2022062423/56649f305503460f94c4b1d9/html5/thumbnails/14.jpg)
Relationship between FT and moment invariants
where M(k,j) is the same as C(k,j) but with geometric moments
![Page 15: Invariants to convolution. Motivation – template matching](https://reader035.vdocuments.net/reader035/viewer/2022062423/56649f305503460f94c4b1d9/html5/thumbnails/15.jpg)
Template matching
sharp image
with the templates
the templates (close-up)
![Page 16: Invariants to convolution. Motivation – template matching](https://reader035.vdocuments.net/reader035/viewer/2022062423/56649f305503460f94c4b1d9/html5/thumbnails/16.jpg)
Template matching
a frame where
the templates were
located successfully
the templates (close-up)
![Page 17: Invariants to convolution. Motivation – template matching](https://reader035.vdocuments.net/reader035/viewer/2022062423/56649f305503460f94c4b1d9/html5/thumbnails/17.jpg)
Template matching performance
![Page 18: Invariants to convolution. Motivation – template matching](https://reader035.vdocuments.net/reader035/viewer/2022062423/56649f305503460f94c4b1d9/html5/thumbnails/18.jpg)
Valid region
Boundary effect in template matching
![Page 19: Invariants to convolution. Motivation – template matching](https://reader035.vdocuments.net/reader035/viewer/2022062423/56649f305503460f94c4b1d9/html5/thumbnails/19.jpg)
Invalid region
- invariance is
violated
Boundary effect in template matching
![Page 20: Invariants to convolution. Motivation – template matching](https://reader035.vdocuments.net/reader035/viewer/2022062423/56649f305503460f94c4b1d9/html5/thumbnails/20.jpg)
Boundary effect in template matching
zero-padding
![Page 21: Invariants to convolution. Motivation – template matching](https://reader035.vdocuments.net/reader035/viewer/2022062423/56649f305503460f94c4b1d9/html5/thumbnails/21.jpg)
• N-fold rotation symmetry, N > 2
• Axial symmetry
• Circular symmetry
• Gaussian PSF
• Motion blur PSF
Other types of the blurring PSF
The more we know about the PSF, the more invariants and the higher discriminability we get
![Page 22: Invariants to convolution. Motivation – template matching](https://reader035.vdocuments.net/reader035/viewer/2022062423/56649f305503460f94c4b1d9/html5/thumbnails/22.jpg)
Discrimination power
The null-space of the blur invariants = a set of all images having the same symmetry as the PSF.
One cannot distinguish among symmetric objects.
![Page 23: Invariants to convolution. Motivation – template matching](https://reader035.vdocuments.net/reader035/viewer/2022062423/56649f305503460f94c4b1d9/html5/thumbnails/23.jpg)
Recommendation for practice
It is important to learn as much as possible about the PSF and to use proper invariants for object recognition
![Page 24: Invariants to convolution. Motivation – template matching](https://reader035.vdocuments.net/reader035/viewer/2022062423/56649f305503460f94c4b1d9/html5/thumbnails/24.jpg)
Combined invariants
Convolution and rotation
For any N
![Page 25: Invariants to convolution. Motivation – template matching](https://reader035.vdocuments.net/reader035/viewer/2022062423/56649f305503460f94c4b1d9/html5/thumbnails/25.jpg)
Robustness of the invariants
![Page 26: Invariants to convolution. Motivation – template matching](https://reader035.vdocuments.net/reader035/viewer/2022062423/56649f305503460f94c4b1d9/html5/thumbnails/26.jpg)
Satellite image registration by combined invariants
![Page 27: Invariants to convolution. Motivation – template matching](https://reader035.vdocuments.net/reader035/viewer/2022062423/56649f305503460f94c4b1d9/html5/thumbnails/27.jpg)
Registration algorithm
• Independent corner detection in both images• Extraction of salient corner points• Calculating blur-rotation invariants of a circular neighborhood of each extracted corner• Matching corners by means of the invariants• Refining the positions of the matched control
points• Estimating the affine transform parameters by
a least-square fit• Resampling and transformation of the sensed
image
![Page 28: Invariants to convolution. Motivation – template matching](https://reader035.vdocuments.net/reader035/viewer/2022062423/56649f305503460f94c4b1d9/html5/thumbnails/28.jpg)
( v11, v21, v31, …
)
( v12, v22, v32, …
)
min distance(( v1k, v2k, v3k, … ) , ( v1m, v2m, v3m, … ))k,m
Control point detection
![Page 29: Invariants to convolution. Motivation – template matching](https://reader035.vdocuments.net/reader035/viewer/2022062423/56649f305503460f94c4b1d9/html5/thumbnails/29.jpg)
Control point matching
![Page 30: Invariants to convolution. Motivation – template matching](https://reader035.vdocuments.net/reader035/viewer/2022062423/56649f305503460f94c4b1d9/html5/thumbnails/30.jpg)
Registration result
![Page 31: Invariants to convolution. Motivation – template matching](https://reader035.vdocuments.net/reader035/viewer/2022062423/56649f305503460f94c4b1d9/html5/thumbnails/31.jpg)
Application in MBD
![Page 32: Invariants to convolution. Motivation – template matching](https://reader035.vdocuments.net/reader035/viewer/2022062423/56649f305503460f94c4b1d9/html5/thumbnails/32.jpg)
Combined blur-affine invariants
• Let I(μ00,…, μPQ) be an affine moment invariant. Then I(C(0,0),…,C(P,Q)), where C(p,q) are blur invariants, is a combined blur-affine invariant (CBAI).
![Page 33: Invariants to convolution. Motivation – template matching](https://reader035.vdocuments.net/reader035/viewer/2022062423/56649f305503460f94c4b1d9/html5/thumbnails/33.jpg)
Digit recognition by CBAI
CBAI AMI
![Page 34: Invariants to convolution. Motivation – template matching](https://reader035.vdocuments.net/reader035/viewer/2022062423/56649f305503460f94c4b1d9/html5/thumbnails/34.jpg)
Documented applications of convolution and combined invariants
• Character/digit/symbol recognition in the presence of vibration, linear motion or
out-of-focus blur
• Robust image registration (medical, satellite, ...)• Detection of image forgeries
![Page 35: Invariants to convolution. Motivation – template matching](https://reader035.vdocuments.net/reader035/viewer/2022062423/56649f305503460f94c4b1d9/html5/thumbnails/35.jpg)
Moment-based focus measure
• Odd-order moments blur invariants
• Even-order moments blur/focus measure
If M(g1) > M(g2) g2 is less blurred
(more focused)
![Page 36: Invariants to convolution. Motivation – template matching](https://reader035.vdocuments.net/reader035/viewer/2022062423/56649f305503460f94c4b1d9/html5/thumbnails/36.jpg)
Other focus measures
• Gray-level variance
• Energy of gradient
• Energy of Laplacian
• Energy of high-pass bands of WT
![Page 37: Invariants to convolution. Motivation – template matching](https://reader035.vdocuments.net/reader035/viewer/2022062423/56649f305503460f94c4b1d9/html5/thumbnails/37.jpg)
Desirable properties of a focus measure
• Agreement with a visual assessment
• Unimodality
• Robustness to noise
• Robustness to small image changes
![Page 38: Invariants to convolution. Motivation – template matching](https://reader035.vdocuments.net/reader035/viewer/2022062423/56649f305503460f94c4b1d9/html5/thumbnails/38.jpg)
Agreement with a visual assessment
![Page 39: Invariants to convolution. Motivation – template matching](https://reader035.vdocuments.net/reader035/viewer/2022062423/56649f305503460f94c4b1d9/html5/thumbnails/39.jpg)
Robustness to noise
![Page 40: Invariants to convolution. Motivation – template matching](https://reader035.vdocuments.net/reader035/viewer/2022062423/56649f305503460f94c4b1d9/html5/thumbnails/40.jpg)
Sunspots – atmospheric turbulence
![Page 41: Invariants to convolution. Motivation – template matching](https://reader035.vdocuments.net/reader035/viewer/2022062423/56649f305503460f94c4b1d9/html5/thumbnails/41.jpg)
Moments are generally worse than wavelets and other differential focus measures because they are too sensitive to local changes but they are very robust to noise.
![Page 42: Invariants to convolution. Motivation – template matching](https://reader035.vdocuments.net/reader035/viewer/2022062423/56649f305503460f94c4b1d9/html5/thumbnails/42.jpg)
![Page 43: Invariants to convolution. Motivation – template matching](https://reader035.vdocuments.net/reader035/viewer/2022062423/56649f305503460f94c4b1d9/html5/thumbnails/43.jpg)
Invariants to elastic deformations
![Page 44: Invariants to convolution. Motivation – template matching](https://reader035.vdocuments.net/reader035/viewer/2022062423/56649f305503460f94c4b1d9/html5/thumbnails/44.jpg)
How can we recognize objects on curved surfaces ...
![Page 45: Invariants to convolution. Motivation – template matching](https://reader035.vdocuments.net/reader035/viewer/2022062423/56649f305503460f94c4b1d9/html5/thumbnails/45.jpg)
Moment matching
• Find the best possible fit by minimizing the error and set
![Page 46: Invariants to convolution. Motivation – template matching](https://reader035.vdocuments.net/reader035/viewer/2022062423/56649f305503460f94c4b1d9/html5/thumbnails/46.jpg)
The bottle
![Page 47: Invariants to convolution. Motivation – template matching](https://reader035.vdocuments.net/reader035/viewer/2022062423/56649f305503460f94c4b1d9/html5/thumbnails/47.jpg)
![Page 48: Invariants to convolution. Motivation – template matching](https://reader035.vdocuments.net/reader035/viewer/2022062423/56649f305503460f94c4b1d9/html5/thumbnails/48.jpg)
Orthogonal moments
Motivation for using OG moments
• Stable calculation by recurrent relations
• Easier and stable image reconstruction
- set of orthogonal polynomials
![Page 49: Invariants to convolution. Motivation – template matching](https://reader035.vdocuments.net/reader035/viewer/2022062423/56649f305503460f94c4b1d9/html5/thumbnails/49.jpg)
Numerical stability
How to avoid numerical problems with high
dynamic range of geometric moments?
![Page 50: Invariants to convolution. Motivation – template matching](https://reader035.vdocuments.net/reader035/viewer/2022062423/56649f305503460f94c4b1d9/html5/thumbnails/50.jpg)
Standard powers
![Page 51: Invariants to convolution. Motivation – template matching](https://reader035.vdocuments.net/reader035/viewer/2022062423/56649f305503460f94c4b1d9/html5/thumbnails/51.jpg)
Orthogonal polynomials
Calculation using recurrent relations
![Page 52: Invariants to convolution. Motivation – template matching](https://reader035.vdocuments.net/reader035/viewer/2022062423/56649f305503460f94c4b1d9/html5/thumbnails/52.jpg)
Two kinds of orthogonality
• Moments (polynomials) orthogonal on a unit square
• Moments (polynomials) orthogonal on a unit disk
![Page 53: Invariants to convolution. Motivation – template matching](https://reader035.vdocuments.net/reader035/viewer/2022062423/56649f305503460f94c4b1d9/html5/thumbnails/53.jpg)
Moments orthogonal on a square
is a system of 1D orthogonal polynomials
![Page 54: Invariants to convolution. Motivation – template matching](https://reader035.vdocuments.net/reader035/viewer/2022062423/56649f305503460f94c4b1d9/html5/thumbnails/54.jpg)
Common 1D orthogonal polynomials
• Legendre <-1,1>
• Chebyshev <-1,1>
• Gegenbauer <-1,1>
• Jacobi <-1,1> or <0,1>
• (generalized) Laguerre <0,∞)
• Hermite (-∞,∞)
![Page 55: Invariants to convolution. Motivation – template matching](https://reader035.vdocuments.net/reader035/viewer/2022062423/56649f305503460f94c4b1d9/html5/thumbnails/55.jpg)
Legendre polynomials
Definition
Orthogonality
![Page 56: Invariants to convolution. Motivation – template matching](https://reader035.vdocuments.net/reader035/viewer/2022062423/56649f305503460f94c4b1d9/html5/thumbnails/56.jpg)
Legendre polynomials explicitly
![Page 57: Invariants to convolution. Motivation – template matching](https://reader035.vdocuments.net/reader035/viewer/2022062423/56649f305503460f94c4b1d9/html5/thumbnails/57.jpg)
Legendre polynomials in 1D
![Page 58: Invariants to convolution. Motivation – template matching](https://reader035.vdocuments.net/reader035/viewer/2022062423/56649f305503460f94c4b1d9/html5/thumbnails/58.jpg)
Legendre polynomials in 2D
![Page 59: Invariants to convolution. Motivation – template matching](https://reader035.vdocuments.net/reader035/viewer/2022062423/56649f305503460f94c4b1d9/html5/thumbnails/59.jpg)
Legendre polynomials
Recurrent relation
![Page 60: Invariants to convolution. Motivation – template matching](https://reader035.vdocuments.net/reader035/viewer/2022062423/56649f305503460f94c4b1d9/html5/thumbnails/60.jpg)
Legendre moments
![Page 61: Invariants to convolution. Motivation – template matching](https://reader035.vdocuments.net/reader035/viewer/2022062423/56649f305503460f94c4b1d9/html5/thumbnails/61.jpg)
Moments orthogonal on a disk
Radial part
Angular part
![Page 62: Invariants to convolution. Motivation – template matching](https://reader035.vdocuments.net/reader035/viewer/2022062423/56649f305503460f94c4b1d9/html5/thumbnails/62.jpg)
Moments orthogonal on a disk
• Zernike
• Pseudo-Zernike
• Orthogonal Fourier-Mellin
• Jacobi-Fourier
• Chebyshev-Fourier
• Radial harmonic Fourier
![Page 63: Invariants to convolution. Motivation – template matching](https://reader035.vdocuments.net/reader035/viewer/2022062423/56649f305503460f94c4b1d9/html5/thumbnails/63.jpg)
Zernike polynomialsDefinition
Orthogonality
![Page 64: Invariants to convolution. Motivation – template matching](https://reader035.vdocuments.net/reader035/viewer/2022062423/56649f305503460f94c4b1d9/html5/thumbnails/64.jpg)
Zernike polynomials – radial part in 1D
![Page 65: Invariants to convolution. Motivation – template matching](https://reader035.vdocuments.net/reader035/viewer/2022062423/56649f305503460f94c4b1d9/html5/thumbnails/65.jpg)
Zernike polynomials – radial part in 2D
![Page 66: Invariants to convolution. Motivation – template matching](https://reader035.vdocuments.net/reader035/viewer/2022062423/56649f305503460f94c4b1d9/html5/thumbnails/66.jpg)
Zernike polynomials
![Page 67: Invariants to convolution. Motivation – template matching](https://reader035.vdocuments.net/reader035/viewer/2022062423/56649f305503460f94c4b1d9/html5/thumbnails/67.jpg)
Zernike moments
Mapping of Cartesian coordinates x,y to polar coordinates r,φ:
• Whole image is mapped inside the unit disk
• Translation and scaling invariance
![Page 68: Invariants to convolution. Motivation – template matching](https://reader035.vdocuments.net/reader035/viewer/2022062423/56649f305503460f94c4b1d9/html5/thumbnails/68.jpg)
Zernike moments
![Page 69: Invariants to convolution. Motivation – template matching](https://reader035.vdocuments.net/reader035/viewer/2022062423/56649f305503460f94c4b1d9/html5/thumbnails/69.jpg)
Rotation property of Zernike moments
The magnitude is preserved, the phase is shifted by ℓθ.
Invariants are constructed by phase cancellation
![Page 70: Invariants to convolution. Motivation – template matching](https://reader035.vdocuments.net/reader035/viewer/2022062423/56649f305503460f94c4b1d9/html5/thumbnails/70.jpg)
Zernike rotation invariants
Phase cancellation by multiplication
Normalization to rotation
![Page 71: Invariants to convolution. Motivation – template matching](https://reader035.vdocuments.net/reader035/viewer/2022062423/56649f305503460f94c4b1d9/html5/thumbnails/71.jpg)
Image reconstruction
• Direct reconstruction from geometric moments
• Solution of a system of equations• Works for very small images only• For larger images the system is ill-conditioned
![Page 72: Invariants to convolution. Motivation – template matching](https://reader035.vdocuments.net/reader035/viewer/2022062423/56649f305503460f94c4b1d9/html5/thumbnails/72.jpg)
Image reconstruction by direct computation
12 x 12 13 x 13
![Page 73: Invariants to convolution. Motivation – template matching](https://reader035.vdocuments.net/reader035/viewer/2022062423/56649f305503460f94c4b1d9/html5/thumbnails/73.jpg)
Image reconstruction
• Reconstruction from geometric moments via FT
![Page 74: Invariants to convolution. Motivation – template matching](https://reader035.vdocuments.net/reader035/viewer/2022062423/56649f305503460f94c4b1d9/html5/thumbnails/74.jpg)
Image reconstruction via Fourier transform
![Page 75: Invariants to convolution. Motivation – template matching](https://reader035.vdocuments.net/reader035/viewer/2022062423/56649f305503460f94c4b1d9/html5/thumbnails/75.jpg)
Image reconstruction
• Image reconstruction from OG moments on
a square
• Image reconstruction from OG moments on
a disk (Zernike)
![Page 76: Invariants to convolution. Motivation – template matching](https://reader035.vdocuments.net/reader035/viewer/2022062423/56649f305503460f94c4b1d9/html5/thumbnails/76.jpg)
Image reconstruction from Legendre moments
![Page 77: Invariants to convolution. Motivation – template matching](https://reader035.vdocuments.net/reader035/viewer/2022062423/56649f305503460f94c4b1d9/html5/thumbnails/77.jpg)
Image reconstruction from Zernike moments
Better for polar raster
![Page 78: Invariants to convolution. Motivation – template matching](https://reader035.vdocuments.net/reader035/viewer/2022062423/56649f305503460f94c4b1d9/html5/thumbnails/78.jpg)
Image reconstruction from Zernike moments
![Page 79: Invariants to convolution. Motivation – template matching](https://reader035.vdocuments.net/reader035/viewer/2022062423/56649f305503460f94c4b1d9/html5/thumbnails/79.jpg)
Reconstruction of a noise-free image
![Page 80: Invariants to convolution. Motivation – template matching](https://reader035.vdocuments.net/reader035/viewer/2022062423/56649f305503460f94c4b1d9/html5/thumbnails/80.jpg)
Reconstruction of a noisy image
![Page 81: Invariants to convolution. Motivation – template matching](https://reader035.vdocuments.net/reader035/viewer/2022062423/56649f305503460f94c4b1d9/html5/thumbnails/81.jpg)
Reconstruction of a noisy image
![Page 82: Invariants to convolution. Motivation – template matching](https://reader035.vdocuments.net/reader035/viewer/2022062423/56649f305503460f94c4b1d9/html5/thumbnails/82.jpg)
Summary of OG moments• OG moments are used because of their favorable numerical properties, not because of theoretical contribution • OG moments should be never used outside the area
of orthogonality• OG moments should be always calculated by recurrent relations, not by expanding into powers• Moments OG on a square do not provide easy rotation invariance• Even if the reconstruction from OG moments is seemingly simple, moments are not a good tool for image compression