1 1 contour enhancement and completion via left-invariant second order stochastic evolution...
Post on 20-Dec-2015
215 views
TRANSCRIPT
![Page 1: 1 1 Contour Enhancement and Completion via Left-Invariant Second Order Stochastic Evolution Equations on the 2D-Euclidean Motion Group Erik Franken, Remco](https://reader037.vdocuments.net/reader037/viewer/2022110322/56649d475503460f94a229e7/html5/thumbnails/1.jpg)
11
Contour Enhancement and Completion via Left-Invariant Second Order
Stochastic Evolution Equations on the 2D-Euclidean Motion Group
Erik Franken, Remco Duits,Markus van Almsick
Eindhoven University of Technology
Department of Biomedical Engineering
EURANDOM workshop
“Image Analysis and Inverse Problems”
December 13th 2006, Eindhoven, NL
![Page 2: 1 1 Contour Enhancement and Completion via Left-Invariant Second Order Stochastic Evolution Equations on the 2D-Euclidean Motion Group Erik Franken, Remco](https://reader037.vdocuments.net/reader037/viewer/2022110322/56649d475503460f94a229e7/html5/thumbnails/2.jpg)
22
Outline
• Introduction to Orientation Scores• Invertible Orientation Scores• Operations in Orientation Scores• The Direction Process for Contour completion• Nonlinear diffusion for Contour enhancement• Conclusions
![Page 3: 1 1 Contour Enhancement and Completion via Left-Invariant Second Order Stochastic Evolution Equations on the 2D-Euclidean Motion Group Erik Franken, Remco](https://reader037.vdocuments.net/reader037/viewer/2022110322/56649d475503460f94a229e7/html5/thumbnails/3.jpg)
33
1. The retina contains receptive fields of varying sizes multi-scale sampling device
2. Primary visual cortex is multi-orientation
Biological Inspiration
• Cells in the primary visual cortex are orientation-specific
• Strong connectivity between cells that respond to (nearly) the same orientation
Measurement in Primary Visual Cortex
Bosking et al., J. Neuroscience 17:2112-2127, 1997
![Page 4: 1 1 Contour Enhancement and Completion via Left-Invariant Second Order Stochastic Evolution Equations on the 2D-Euclidean Motion Group Erik Franken, Remco](https://reader037.vdocuments.net/reader037/viewer/2022110322/56649d475503460f94a229e7/html5/thumbnails/4.jpg)
44
image
Orientation Scores
From 2D image f(x,y) to orientation score Uf(x,y,θ) with position (x,y) and orientation θ
x
y
xorientation score
y
![Page 5: 1 1 Contour Enhancement and Completion via Left-Invariant Second Order Stochastic Evolution Equations on the 2D-Euclidean Motion Group Erik Franken, Remco](https://reader037.vdocuments.net/reader037/viewer/2022110322/56649d475503460f94a229e7/html5/thumbnails/5.jpg)
55
Orientation Score is a Function on SE(2)
Properties of SE(2)• Group element
translation rotation• Group product
• Group inverseg¡ 1 = (¡ R ¡ 1
µ b; ¡ µ)
SE (2) = R2o SO(2) = Euclidean Motion GroupWe consider orientation score U 2 L2(SE (2))
![Page 6: 1 1 Contour Enhancement and Completion via Left-Invariant Second Order Stochastic Evolution Equations on the 2D-Euclidean Motion Group Erik Franken, Remco](https://reader037.vdocuments.net/reader037/viewer/2022110322/56649d475503460f94a229e7/html5/thumbnails/6.jpg)
66
Approach: Image Processing /analysis via Orientation Scores
“Enhancement” operation
Initial image
“Enhanced” image
Orientation score transformation
Inverse orientation score transformation
Segmented structures
Segment structures of interest
![Page 7: 1 1 Contour Enhancement and Completion via Left-Invariant Second Order Stochastic Evolution Equations on the 2D-Euclidean Motion Group Erik Franken, Remco](https://reader037.vdocuments.net/reader037/viewer/2022110322/56649d475503460f94a229e7/html5/thumbnails/7.jpg)
77
Outline
• Introduction to Orientation Scores• Invertible Orientation Scores• Operations in Orientation Scores• The Direction Process for Contour completion• Nonlinear diffusion for Contour enhancement• Conclusions
![Page 8: 1 1 Contour Enhancement and Completion via Left-Invariant Second Order Stochastic Evolution Equations on the 2D-Euclidean Motion Group Erik Franken, Remco](https://reader037.vdocuments.net/reader037/viewer/2022110322/56649d475503460f94a229e7/html5/thumbnails/8.jpg)
88
Invertible Orientation Score Transformation
i.e. “fill up the entire Fourier spectrum”.
Image to orientation score
Orientation score to image:
Stable reconstruction requires
Oriented wavelet
![Page 9: 1 1 Contour Enhancement and Completion via Left-Invariant Second Order Stochastic Evolution Equations on the 2D-Euclidean Motion Group Erik Franken, Remco](https://reader037.vdocuments.net/reader037/viewer/2022110322/56649d475503460f94a229e7/html5/thumbnails/9.jpg)
99
Invertible Orientation Score TransformationDesign considerations: reconstruction,
directional, spatial localization, quadrature, discrete number of orientations
re(Ã) im(Ã) F [Ã]P Nµ
j =0 F [R j sµ Ã]dµ
re(WÃf )(¢;µ)example image f j(WÃf )(¢;µ)j
![Page 10: 1 1 Contour Enhancement and Completion via Left-Invariant Second Order Stochastic Evolution Equations on the 2D-Euclidean Motion Group Erik Franken, Remco](https://reader037.vdocuments.net/reader037/viewer/2022110322/56649d475503460f94a229e7/html5/thumbnails/10.jpg)
1010
Outline
• Introduction to Orientation Scores• Invertible Orientation Scores• Operations in Orientation Scores• The Direction Process for Contour completion• Nonlinear diffusion for Contour enhancement• Conclusions
![Page 11: 1 1 Contour Enhancement and Completion via Left-Invariant Second Order Stochastic Evolution Equations on the 2D-Euclidean Motion Group Erik Franken, Remco](https://reader037.vdocuments.net/reader037/viewer/2022110322/56649d475503460f94a229e7/html5/thumbnails/11.jpg)
1111
Represents the “net” operator. It is Euclidean-invariant iff is left-invariant, i.e.
Left Invariant Operators
L g©U = ©L gU; i 2 f »;´;µg
(L gU)(h) = U(g¡ 1h)Where is the left-regular
representation L g
![Page 12: 1 1 Contour Enhancement and Completion via Left-Invariant Second Order Stochastic Evolution Equations on the 2D-Euclidean Motion Group Erik Franken, Remco](https://reader037.vdocuments.net/reader037/viewer/2022110322/56649d475503460f94a229e7/html5/thumbnails/12.jpg)
1212
Left-invariant Derivative Operators
areleft-invariant derivatives onEuclidean motion group, i.e.
@»
@́
@µ
x
µ
y
@»@́
@µ@»; @́; and @µ
L g@iU = @iL gU; i 2 f »;´;µg
Not all left-invariant derivatives on SE(2)do commute![@µ;@»] = @µ@» ¡ @»@µ = @́; [@µ; @́] = ¡ @»
=Tangent to line structures
= orthogonal to line structures
![Page 13: 1 1 Contour Enhancement and Completion via Left-Invariant Second Order Stochastic Evolution Equations on the 2D-Euclidean Motion Group Erik Franken, Remco](https://reader037.vdocuments.net/reader037/viewer/2022110322/56649d475503460f94a229e7/html5/thumbnails/13.jpg)
1313
Convection-diffusion PDEs on SE(2)
convection diffusion
Time process
Resolvent process
![Page 14: 1 1 Contour Enhancement and Completion via Left-Invariant Second Order Stochastic Evolution Equations on the 2D-Euclidean Motion Group Erik Franken, Remco](https://reader037.vdocuments.net/reader037/viewer/2022110322/56649d475503460f94a229e7/html5/thumbnails/14.jpg)
1414
Linear & Left-invariant Operators are G-convolutionsNormal 3D convolution – versus G-convolution on SE(2)
“G-Kernel”
![Page 15: 1 1 Contour Enhancement and Completion via Left-Invariant Second Order Stochastic Evolution Equations on the 2D-Euclidean Motion Group Erik Franken, Remco](https://reader037.vdocuments.net/reader037/viewer/2022110322/56649d475503460f94a229e7/html5/thumbnails/15.jpg)
1515
Outline
• Introduction to Orientation Scores• Invertible Orientation Scores• Operations in Orientation Scores• The Direction Process for Contour
completion• Nonlinear diffusion for Contour enhancement• Conclusions
![Page 16: 1 1 Contour Enhancement and Completion via Left-Invariant Second Order Stochastic Evolution Equations on the 2D-Euclidean Motion Group Erik Franken, Remco](https://reader037.vdocuments.net/reader037/viewer/2022110322/56649d475503460f94a229e7/html5/thumbnails/16.jpg)
1616
Direction Process on SE(2)Resolvent of linear PDE
Random walker interpretation
![Page 17: 1 1 Contour Enhancement and Completion via Left-Invariant Second Order Stochastic Evolution Equations on the 2D-Euclidean Motion Group Erik Franken, Remco](https://reader037.vdocuments.net/reader037/viewer/2022110322/56649d475503460f94a229e7/html5/thumbnails/17.jpg)
1717
Stochastic Completion Fields
Collision probability of 2 random walkers on SE(2):
Forward Backward
The mode line (in red) is the most likely connection curvebetween the two points
![Page 18: 1 1 Contour Enhancement and Completion via Left-Invariant Second Order Stochastic Evolution Equations on the 2D-Euclidean Motion Group Erik Franken, Remco](https://reader037.vdocuments.net/reader037/viewer/2022110322/56649d475503460f94a229e7/html5/thumbnails/18.jpg)
1818
An example
Noisy input image
Greens functions:
“Simple” enhancement viaOrientation score
Stochastic completion field
![Page 19: 1 1 Contour Enhancement and Completion via Left-Invariant Second Order Stochastic Evolution Equations on the 2D-Euclidean Motion Group Erik Franken, Remco](https://reader037.vdocuments.net/reader037/viewer/2022110322/56649d475503460f94a229e7/html5/thumbnails/19.jpg)
1919
Exact Solution by Duits and Van Almsick
Explicit PDE problem, case (Mumford) :Analytic Solution of Greens function?
![Page 20: 1 1 Contour Enhancement and Completion via Left-Invariant Second Order Stochastic Evolution Equations on the 2D-Euclidean Motion Group Erik Franken, Remco](https://reader037.vdocuments.net/reader037/viewer/2022110322/56649d475503460f94a229e7/html5/thumbnails/20.jpg)
2020
Practical approximations
![Page 21: 1 1 Contour Enhancement and Completion via Left-Invariant Second Order Stochastic Evolution Equations on the 2D-Euclidean Motion Group Erik Franken, Remco](https://reader037.vdocuments.net/reader037/viewer/2022110322/56649d475503460f94a229e7/html5/thumbnails/21.jpg)
2121
Exact Green’s Function versus Approximation
The smaller the better approximation
![Page 22: 1 1 Contour Enhancement and Completion via Left-Invariant Second Order Stochastic Evolution Equations on the 2D-Euclidean Motion Group Erik Franken, Remco](https://reader037.vdocuments.net/reader037/viewer/2022110322/56649d475503460f94a229e7/html5/thumbnails/22.jpg)
2222
How? •G-convolution with exact/approximate Green’s function•Finite element implementation in Fourier domain (August / Duits)•Explicit numerical schemes (Zweck and Williams)
Application of the Direction process
Non-linear enhancement step
Initial image
Image with completed contours
Orientation score transformation
Inverse orientation score transformation
Compute Stochastic Completion Field
What?
•Orientation-score gray-scale transformation (i.e. taking a power)•Angular/spatial thinning
![Page 23: 1 1 Contour Enhancement and Completion via Left-Invariant Second Order Stochastic Evolution Equations on the 2D-Euclidean Motion Group Erik Franken, Remco](https://reader037.vdocuments.net/reader037/viewer/2022110322/56649d475503460f94a229e7/html5/thumbnails/23.jpg)
2323
Automatic Contour completion by SE(2)-convolution
![Page 24: 1 1 Contour Enhancement and Completion via Left-Invariant Second Order Stochastic Evolution Equations on the 2D-Euclidean Motion Group Erik Franken, Remco](https://reader037.vdocuments.net/reader037/viewer/2022110322/56649d475503460f94a229e7/html5/thumbnails/24.jpg)
2424
Outline
• Introduction to Orientation Scores• Invertible Orientation Scores• Operations in Orientation Scores• The Direction Process for Contour completion• Nonlinear diffusion for Contour
enhancement• Conclusions
![Page 25: 1 1 Contour Enhancement and Completion via Left-Invariant Second Order Stochastic Evolution Equations on the 2D-Euclidean Motion Group Erik Franken, Remco](https://reader037.vdocuments.net/reader037/viewer/2022110322/56649d475503460f94a229e7/html5/thumbnails/25.jpg)
2525
The Diffusion Equation on Imagesf = image
u = scale space of image
D = diffusion tensor
Linear diffusion Perona&Malik Coherence-enhancing diff.
![Page 26: 1 1 Contour Enhancement and Completion via Left-Invariant Second Order Stochastic Evolution Equations on the 2D-Euclidean Motion Group Erik Franken, Remco](https://reader037.vdocuments.net/reader037/viewer/2022110322/56649d475503460f94a229e7/html5/thumbnails/26.jpg)
2626
Diffusion equation in orientation scorescurvature
Diffusion orthogonal tooriented structures
Diffusion tangent tooriented structures
Diffusion inorientation
Evolvingorientationscore
Rotating tangent spacecoordinate basis
Left-invariantderivatives
are left-invariant derivatives onEuclidean motion group, i.e.
@»
@́
@µ
x
µ
y
@»@́
@µ
@»; @́; and @µ
L g@iU = @iL gU; i 2 f »;´;µg
![Page 27: 1 1 Contour Enhancement and Completion via Left-Invariant Second Order Stochastic Evolution Equations on the 2D-Euclidean Motion Group Erik Franken, Remco](https://reader037.vdocuments.net/reader037/viewer/2022110322/56649d475503460f94a229e7/html5/thumbnails/27.jpg)
2727
Example diffusion kernels
![Page 28: 1 1 Contour Enhancement and Completion via Left-Invariant Second Order Stochastic Evolution Equations on the 2D-Euclidean Motion Group Erik Franken, Remco](https://reader037.vdocuments.net/reader037/viewer/2022110322/56649d475503460f94a229e7/html5/thumbnails/28.jpg)
2828
• Oriented regions: D’11 and D33 small, D22 large and κ according to estimate
• Non-oriented regions: D’11 large, D22=D33 large, κ = 0
@»
@́
@µ
x
µ
y
@»@́
@µ
How to Choose Conductivity Coefficients
![Page 29: 1 1 Contour Enhancement and Completion via Left-Invariant Second Order Stochastic Evolution Equations on the 2D-Euclidean Motion Group Erik Franken, Remco](https://reader037.vdocuments.net/reader037/viewer/2022110322/56649d475503460f94a229e7/html5/thumbnails/29.jpg)
2929
Measure for Orientation Strength
• Hessian in Orientation Score
Note: non-symmetric due to non-commuting operators!
• Gaussian Derivatives can be used, if one ensures to first take orientational derivatives and then spatial derivatives.
• Measure for orientation strength:
![Page 30: 1 1 Contour Enhancement and Completion via Left-Invariant Second Order Stochastic Evolution Equations on the 2D-Euclidean Motion Group Erik Franken, Remco](https://reader037.vdocuments.net/reader037/viewer/2022110322/56649d475503460f94a229e7/html5/thumbnails/30.jpg)
3030
Curvature estimation
• If a vector points tangent to a structure in the orientation score, the curvature in that point is:
Ideally zero
• Estimation of v:1. Determine eigenvectors and eigenvalues of
2. Select the 2 eigenvectors closest to the ξ,θ-plane
3. Take eigenvector corresponding to the largest eigenvalue
![Page 31: 1 1 Contour Enhancement and Completion via Left-Invariant Second Order Stochastic Evolution Equations on the 2D-Euclidean Motion Group Erik Franken, Remco](https://reader037.vdocuments.net/reader037/viewer/2022110322/56649d475503460f94a229e7/html5/thumbnails/31.jpg)
3131
@»
@́
@µ
x
µ
y
@»@́
@µ
Chosen Conductivity Coefficients
![Page 32: 1 1 Contour Enhancement and Completion via Left-Invariant Second Order Stochastic Evolution Equations on the 2D-Euclidean Motion Group Erik Franken, Remco](https://reader037.vdocuments.net/reader037/viewer/2022110322/56649d475503460f94a229e7/html5/thumbnails/32.jpg)
3232
• Numerical scheme: explicit, left-invariant finite differences.
• Using B-spline interpolation cf. Unser et al.
Implementation
![Page 33: 1 1 Contour Enhancement and Completion via Left-Invariant Second Order Stochastic Evolution Equations on the 2D-Euclidean Motion Group Erik Franken, Remco](https://reader037.vdocuments.net/reader037/viewer/2022110322/56649d475503460f94a229e7/html5/thumbnails/33.jpg)
3333
Diffusion in orientation score Coherence enhancing diffusion Results
Size: 128 x 128 x 64
![Page 34: 1 1 Contour Enhancement and Completion via Left-Invariant Second Order Stochastic Evolution Equations on the 2D-Euclidean Motion Group Erik Franken, Remco](https://reader037.vdocuments.net/reader037/viewer/2022110322/56649d475503460f94a229e7/html5/thumbnails/34.jpg)
3434
Collagen image
Diffusion in orientation score Coherence enhancing diffusion
Size: 200 x 200 x 64
![Page 35: 1 1 Contour Enhancement and Completion via Left-Invariant Second Order Stochastic Evolution Equations on the 2D-Euclidean Motion Group Erik Franken, Remco](https://reader037.vdocuments.net/reader037/viewer/2022110322/56649d475503460f94a229e7/html5/thumbnails/35.jpg)
3535
Results – with/without curvature estimation
![Page 36: 1 1 Contour Enhancement and Completion via Left-Invariant Second Order Stochastic Evolution Equations on the 2D-Euclidean Motion Group Erik Franken, Remco](https://reader037.vdocuments.net/reader037/viewer/2022110322/56649d475503460f94a229e7/html5/thumbnails/36.jpg)
3636
Outline
• Introduction to Orientation Scores• Invertible Orientation Scores• Operations in Orientation Scores• The Direction Process for Contour completion• Nonlinear diffusion for Contour enhancement• Conclusions
![Page 37: 1 1 Contour Enhancement and Completion via Left-Invariant Second Order Stochastic Evolution Equations on the 2D-Euclidean Motion Group Erik Franken, Remco](https://reader037.vdocuments.net/reader037/viewer/2022110322/56649d475503460f94a229e7/html5/thumbnails/37.jpg)
3737
Conclusions
• We developed a framework for image processing via Orientation scores. Important notion: An orientation score is a function on SE(2) use group theory.
• Useful for noisy medical images with (crossing) elongated structures
• Found Analytic Solution of Greens functions• Stochastic Completion Fields of images using
G-convolutions• Non-linear diffusion on orientation scores to
enhance crossing line structures
![Page 38: 1 1 Contour Enhancement and Completion via Left-Invariant Second Order Stochastic Evolution Equations on the 2D-Euclidean Motion Group Erik Franken, Remco](https://reader037.vdocuments.net/reader037/viewer/2022110322/56649d475503460f94a229e7/html5/thumbnails/38.jpg)
3838
Current/Future work
• Improving adaptive/nonlinear evolutions on SE(2)– Numerical methods– Nonlinearities
• Applying in medical applications– 2-photon microscopy images of Collagen fibers– High Angular Resolution Diffusion Imaging
• Apply same mathematics in other groups, e.g. SE(3) and similitude groups.
![Page 39: 1 1 Contour Enhancement and Completion via Left-Invariant Second Order Stochastic Evolution Equations on the 2D-Euclidean Motion Group Erik Franken, Remco](https://reader037.vdocuments.net/reader037/viewer/2022110322/56649d475503460f94a229e7/html5/thumbnails/39.jpg)
3939
![Page 40: 1 1 Contour Enhancement and Completion via Left-Invariant Second Order Stochastic Evolution Equations on the 2D-Euclidean Motion Group Erik Franken, Remco](https://reader037.vdocuments.net/reader037/viewer/2022110322/56649d475503460f94a229e7/html5/thumbnails/40.jpg)
4040
Line enhancement in 3D via invertible orientation scores
Application: Enhancement Adam-Kiewitz vessel
![Page 41: 1 1 Contour Enhancement and Completion via Left-Invariant Second Order Stochastic Evolution Equations on the 2D-Euclidean Motion Group Erik Franken, Remco](https://reader037.vdocuments.net/reader037/viewer/2022110322/56649d475503460f94a229e7/html5/thumbnails/41.jpg)
4141
Enhancement Kidney-Boundaries in Ultra-sound images
Via Orientation Scores:
![Page 42: 1 1 Contour Enhancement and Completion via Left-Invariant Second Order Stochastic Evolution Equations on the 2D-Euclidean Motion Group Erik Franken, Remco](https://reader037.vdocuments.net/reader037/viewer/2022110322/56649d475503460f94a229e7/html5/thumbnails/42.jpg)
4242
Medical Application: Cardiac ArrhythmiasHeart rythm disorder by extra conductive path/spot
Catheters in hart provide intracardiogram and can burn focal spots/lines
Navigation by X-ray
Detection cathethers in X-ray navigation
automatic 3D-cardiac mapping from bi-plane.
![Page 43: 1 1 Contour Enhancement and Completion via Left-Invariant Second Order Stochastic Evolution Equations on the 2D-Euclidean Motion Group Erik Franken, Remco](https://reader037.vdocuments.net/reader037/viewer/2022110322/56649d475503460f94a229e7/html5/thumbnails/43.jpg)
4343
Efficient calculation of G-convolutions
• Using steerable filters in the orientation score + inspired by Fourier transform on SE(2)
Algorithmic complexity can be reduced from
to