deformable contours 1 dr. e. ribeiro. classical methods 2 thresholding edge detection an image of...
TRANSCRIPT
![Page 1: Deformable Contours 1 Dr. E. Ribeiro. Classical Methods 2 Thresholding Edge detection An image of blood vessel Slide by: Chunming Li, Vanderbilt University](https://reader037.vdocuments.net/reader037/viewer/2022110322/56649d1c5503460f949f19e6/html5/thumbnails/1.jpg)
Deformable Contours
1
Dr. E. Ribeiro
![Page 2: Deformable Contours 1 Dr. E. Ribeiro. Classical Methods 2 Thresholding Edge detection An image of blood vessel Slide by: Chunming Li, Vanderbilt University](https://reader037.vdocuments.net/reader037/viewer/2022110322/56649d1c5503460f949f19e6/html5/thumbnails/2.jpg)
Classical Methods
2
Thresholding Edge detectionAn image of blood vessel
Slide by: Chunming Li, Vanderbilt University
![Page 3: Deformable Contours 1 Dr. E. Ribeiro. Classical Methods 2 Thresholding Edge detection An image of blood vessel Slide by: Chunming Li, Vanderbilt University](https://reader037.vdocuments.net/reader037/viewer/2022110322/56649d1c5503460f949f19e6/html5/thumbnails/3.jpg)
An Advanced Method: Active Contour Model
3Slide by: Chunming Li, Vanderbilt University
![Page 4: Deformable Contours 1 Dr. E. Ribeiro. Classical Methods 2 Thresholding Edge detection An image of blood vessel Slide by: Chunming Li, Vanderbilt University](https://reader037.vdocuments.net/reader037/viewer/2022110322/56649d1c5503460f949f19e6/html5/thumbnails/4.jpg)
Edges vs. boundaries
Edges useful signal to indicate occluding boundaries, shape.
Here the raw edge output is not so bad…
…but quite often boundaries of interest are fragmented, and we have extra “clutter” edge points.
Images from D. Jacobs
![Page 5: Deformable Contours 1 Dr. E. Ribeiro. Classical Methods 2 Thresholding Edge detection An image of blood vessel Slide by: Chunming Li, Vanderbilt University](https://reader037.vdocuments.net/reader037/viewer/2022110322/56649d1c5503460f949f19e6/html5/thumbnails/5.jpg)
Deformable contours
Given: initial contour (model) near desired object
a.k.a. active contours, snakes
(Single frame)
Fig: Y. Boykov[Snakes: Active contour models, Kass, Witkin, & Terzopoulos, ICCV1987]
![Page 6: Deformable Contours 1 Dr. E. Ribeiro. Classical Methods 2 Thresholding Edge detection An image of blood vessel Slide by: Chunming Li, Vanderbilt University](https://reader037.vdocuments.net/reader037/viewer/2022110322/56649d1c5503460f949f19e6/html5/thumbnails/6.jpg)
Deformable contours
Given: initial contour (model) near desired object
a.k.a. active contours, snakes
(Single frame)
Fig: Y. Boykov
Goal: evolve the contour to fit exact object boundary
[Snakes: Active contour models, Kass, Witkin, & Terzopoulos, ICCV1987]
![Page 7: Deformable Contours 1 Dr. E. Ribeiro. Classical Methods 2 Thresholding Edge detection An image of blood vessel Slide by: Chunming Li, Vanderbilt University](https://reader037.vdocuments.net/reader037/viewer/2022110322/56649d1c5503460f949f19e6/html5/thumbnails/7.jpg)
![Page 8: Deformable Contours 1 Dr. E. Ribeiro. Classical Methods 2 Thresholding Edge detection An image of blood vessel Slide by: Chunming Li, Vanderbilt University](https://reader037.vdocuments.net/reader037/viewer/2022110322/56649d1c5503460f949f19e6/html5/thumbnails/8.jpg)
Why do we want to fit deformable shapes?
• Non-rigid, deformable objects can change their shape over time, e.g. lips, hands.
Figure from Kass et al. 1987
![Page 9: Deformable Contours 1 Dr. E. Ribeiro. Classical Methods 2 Thresholding Edge detection An image of blood vessel Slide by: Chunming Li, Vanderbilt University](https://reader037.vdocuments.net/reader037/viewer/2022110322/56649d1c5503460f949f19e6/html5/thumbnails/9.jpg)
Why do we want to fit deformable shapes?
• Some objects have similar basic form but some variety in the contour shape.
![Page 10: Deformable Contours 1 Dr. E. Ribeiro. Classical Methods 2 Thresholding Edge detection An image of blood vessel Slide by: Chunming Li, Vanderbilt University](https://reader037.vdocuments.net/reader037/viewer/2022110322/56649d1c5503460f949f19e6/html5/thumbnails/10.jpg)
Deformable contours: intuition
Image from http://www.healthline.com/blogs/exercise_fitness/uploaded_images/HandBand2-795868.JPG
Figure from Shapiro & Stockman
![Page 11: Deformable Contours 1 Dr. E. Ribeiro. Classical Methods 2 Thresholding Edge detection An image of blood vessel Slide by: Chunming Li, Vanderbilt University](https://reader037.vdocuments.net/reader037/viewer/2022110322/56649d1c5503460f949f19e6/html5/thumbnails/11.jpg)
Deformable contours
initial intermediate final
a.k.a. active contours, snakesHow is the current contour adjusted to find the new contour at each iteration?
• Define a cost function (“energy” function) that says how good a possible configuration is.
• Seek next configuration that minimizes that cost function.
![Page 12: Deformable Contours 1 Dr. E. Ribeiro. Classical Methods 2 Thresholding Edge detection An image of blood vessel Slide by: Chunming Li, Vanderbilt University](https://reader037.vdocuments.net/reader037/viewer/2022110322/56649d1c5503460f949f19e6/html5/thumbnails/12.jpg)
Snakes energy functionThe total energy (cost) of the current snake is defined as:
externalinternaltotal EEE +=
A good fit between the current deformable contour and the target shape in the image will yield a low value for this cost function.
Internal energy: encourage prior shape preferences: e.g., smoothness, elasticity, particular known shape.
External energy (“image” energy): encourage contour to fit on places where image structures exist, e.g., edges.
![Page 13: Deformable Contours 1 Dr. E. Ribeiro. Classical Methods 2 Thresholding Edge detection An image of blood vessel Slide by: Chunming Li, Vanderbilt University](https://reader037.vdocuments.net/reader037/viewer/2022110322/56649d1c5503460f949f19e6/html5/thumbnails/13.jpg)
10))(),(()( ≤≤= ssysxsν
Parametric curve representation(continuous case)
Fig from Y. Boykov
![Page 14: Deformable Contours 1 Dr. E. Ribeiro. Classical Methods 2 Thresholding Edge detection An image of blood vessel Slide by: Chunming Li, Vanderbilt University](https://reader037.vdocuments.net/reader037/viewer/2022110322/56649d1c5503460f949f19e6/html5/thumbnails/14.jpg)
External energy: intuition• Measure how well the curve matches the image data• “Attract” the curve toward different image features– Edges, lines, etc.
![Page 15: Deformable Contours 1 Dr. E. Ribeiro. Classical Methods 2 Thresholding Edge detection An image of blood vessel Slide by: Chunming Li, Vanderbilt University](https://reader037.vdocuments.net/reader037/viewer/2022110322/56649d1c5503460f949f19e6/html5/thumbnails/15.jpg)
How do edges affect “snap” of rubber band?
Think of external energy from image as gravitational pull towards areas of high contrast
External image energy
Magnitude of gradient- (Magnitude of gradient)
( )22 )()( IGIG yx +−22 )()( IGIG yx +
![Page 16: Deformable Contours 1 Dr. E. Ribeiro. Classical Methods 2 Thresholding Edge detection An image of blood vessel Slide by: Chunming Li, Vanderbilt University](https://reader037.vdocuments.net/reader037/viewer/2022110322/56649d1c5503460f949f19e6/html5/thumbnails/16.jpg)
External image energy
• Image I(x,y)• Gradient images and
• External energy at a point v(s) on the curve is
• External energy for the whole curve:
),( yxGx ),( yxGy
)|))((||))((|())(( 22 sGsGsE yxexternal ννν +−=
∫=1
0
))(( dssEE externalexternal ν
10))(),(()( ≤≤= ssysxsν
![Page 17: Deformable Contours 1 Dr. E. Ribeiro. Classical Methods 2 Thresholding Edge detection An image of blood vessel Slide by: Chunming Li, Vanderbilt University](https://reader037.vdocuments.net/reader037/viewer/2022110322/56649d1c5503460f949f19e6/html5/thumbnails/17.jpg)
Internal energy: intuition
http://www3.imperial.ac.uk/pls/portallive/docs/1/52679.JPG
A priori, we want to favor smooth shapes, contours with low curvature, contours similar to a known shape, etc. to balance what is actually observed (i.e., in the gradient image).
![Page 18: Deformable Contours 1 Dr. E. Ribeiro. Classical Methods 2 Thresholding Edge detection An image of blood vessel Slide by: Chunming Li, Vanderbilt University](https://reader037.vdocuments.net/reader037/viewer/2022110322/56649d1c5503460f949f19e6/html5/thumbnails/18.jpg)
Internal energyFor a continuous curve, a common internal energy term is the “bending energy”. At some point v(s) on the curve, this is:
The more the curve bends the larger this energy value is.
The weights α and β dictate how much influence each component has.
Elasticity,
Tension
Stiffness,
Curvature
sd
ddsd
sEinternal 2
2
))((
22
ννβαν +=
∫=1
0
))(( dssEE internalinternal νInternal energy for whole curve:
![Page 19: Deformable Contours 1 Dr. E. Ribeiro. Classical Methods 2 Thresholding Edge detection An image of blood vessel Slide by: Chunming Li, Vanderbilt University](https://reader037.vdocuments.net/reader037/viewer/2022110322/56649d1c5503460f949f19e6/html5/thumbnails/19.jpg)
Dealing with missing data
• The smoothness constraint can deal with missing data:
[Figure from Kass et al. 1987]
![Page 20: Deformable Contours 1 Dr. E. Ribeiro. Classical Methods 2 Thresholding Edge detection An image of blood vessel Slide by: Chunming Li, Vanderbilt University](https://reader037.vdocuments.net/reader037/viewer/2022110322/56649d1c5503460f949f19e6/html5/thumbnails/20.jpg)
Total energy(continuous form)
// bending energy
// total edge strength
under curve
externalinternaltotal EEE γ+=
∫=1
0
internal ))(( dssEEinternal ν
∫=1
0
))(( dssEE externalexternal ν
![Page 21: Deformable Contours 1 Dr. E. Ribeiro. Classical Methods 2 Thresholding Edge detection An image of blood vessel Slide by: Chunming Li, Vanderbilt University](https://reader037.vdocuments.net/reader037/viewer/2022110322/56649d1c5503460f949f19e6/html5/thumbnails/21.jpg)
Parametric curve representation(discrete form)
• Represent the curve with a set of n points
10),( −== niyx iii Kν …
![Page 22: Deformable Contours 1 Dr. E. Ribeiro. Classical Methods 2 Thresholding Edge detection An image of blood vessel Slide by: Chunming Li, Vanderbilt University](https://reader037.vdocuments.net/reader037/viewer/2022110322/56649d1c5503460f949f19e6/html5/thumbnails/22.jpg)
Discrete energy function:external term
21
0
2 |),(||),(| iiy
n
iiixexternal yxGyxGE ∑
−
=
+−=
Discrete image gradients
),( yxGx ),( yxGy
• If the curve is represented by n points
![Page 23: Deformable Contours 1 Dr. E. Ribeiro. Classical Methods 2 Thresholding Edge detection An image of blood vessel Slide by: Chunming Li, Vanderbilt University](https://reader037.vdocuments.net/reader037/viewer/2022110322/56649d1c5503460f949f19e6/html5/thumbnails/23.jpg)
Summary: elastic snake• A simple elastic snake is defined by– A set of n points,– An internal elastic energy term– An external edge based energy term
• To use this to locate the outline of an object– Initialize in the vicinity of the object– Modify the points to minimize the total
energy
![Page 24: Deformable Contours 1 Dr. E. Ribeiro. Classical Methods 2 Thresholding Edge detection An image of blood vessel Slide by: Chunming Li, Vanderbilt University](https://reader037.vdocuments.net/reader037/viewer/2022110322/56649d1c5503460f949f19e6/html5/thumbnails/24.jpg)
Level Set Representation of Curves
24
zero level
zero level
Slide by: Chunming Li, Vanderbilt University
![Page 25: Deformable Contours 1 Dr. E. Ribeiro. Classical Methods 2 Thresholding Edge detection An image of blood vessel Slide by: Chunming Li, Vanderbilt University](https://reader037.vdocuments.net/reader037/viewer/2022110322/56649d1c5503460f949f19e6/html5/thumbnails/25.jpg)
Level Set Method (Osher and Sethian, 1988)
25
• Curve evolution:
where F is the speed function, N is normal vector to the curve C
• Level set formulation:
N
Slide by: Chunming Li, Vanderbilt University
![Page 26: Deformable Contours 1 Dr. E. Ribeiro. Classical Methods 2 Thresholding Edge detection An image of blood vessel Slide by: Chunming Li, Vanderbilt University](https://reader037.vdocuments.net/reader037/viewer/2022110322/56649d1c5503460f949f19e6/html5/thumbnails/26.jpg)
26
Geodesic active contour
![Page 27: Deformable Contours 1 Dr. E. Ribeiro. Classical Methods 2 Thresholding Edge detection An image of blood vessel Slide by: Chunming Li, Vanderbilt University](https://reader037.vdocuments.net/reader037/viewer/2022110322/56649d1c5503460f949f19e6/html5/thumbnails/27.jpg)
27
Geodesic active contour
![Page 28: Deformable Contours 1 Dr. E. Ribeiro. Classical Methods 2 Thresholding Edge detection An image of blood vessel Slide by: Chunming Li, Vanderbilt University](https://reader037.vdocuments.net/reader037/viewer/2022110322/56649d1c5503460f949f19e6/html5/thumbnails/28.jpg)
Geodesic active contour
28
![Page 29: Deformable Contours 1 Dr. E. Ribeiro. Classical Methods 2 Thresholding Edge detection An image of blood vessel Slide by: Chunming Li, Vanderbilt University](https://reader037.vdocuments.net/reader037/viewer/2022110322/56649d1c5503460f949f19e6/html5/thumbnails/29.jpg)
29
Segmentation using statistical models (Rousson and Deriche, 2002)
Energy functional
Probability describing the pixel values inside region i
![Page 30: Deformable Contours 1 Dr. E. Ribeiro. Classical Methods 2 Thresholding Edge detection An image of blood vessel Slide by: Chunming Li, Vanderbilt University](https://reader037.vdocuments.net/reader037/viewer/2022110322/56649d1c5503460f949f19e6/html5/thumbnails/30.jpg)
30
Segmentation using statistical models (Rousson and Deriche, 2002)
Energy functional
Probability describing the pixel values inside region i
The energy functional can be rewritten as:
![Page 31: Deformable Contours 1 Dr. E. Ribeiro. Classical Methods 2 Thresholding Edge detection An image of blood vessel Slide by: Chunming Li, Vanderbilt University](https://reader037.vdocuments.net/reader037/viewer/2022110322/56649d1c5503460f949f19e6/html5/thumbnails/31.jpg)
Two-phase case
31
![Page 32: Deformable Contours 1 Dr. E. Ribeiro. Classical Methods 2 Thresholding Edge detection An image of blood vessel Slide by: Chunming Li, Vanderbilt University](https://reader037.vdocuments.net/reader037/viewer/2022110322/56649d1c5503460f949f19e6/html5/thumbnails/32.jpg)
Two-phase case
32
Energy functional
Level set function
![Page 33: Deformable Contours 1 Dr. E. Ribeiro. Classical Methods 2 Thresholding Edge detection An image of blood vessel Slide by: Chunming Li, Vanderbilt University](https://reader037.vdocuments.net/reader037/viewer/2022110322/56649d1c5503460f949f19e6/html5/thumbnails/33.jpg)
Two-phase case
33
Energy functional
Heaviside step function
Smooth approximation
Length term
![Page 34: Deformable Contours 1 Dr. E. Ribeiro. Classical Methods 2 Thresholding Edge detection An image of blood vessel Slide by: Chunming Li, Vanderbilt University](https://reader037.vdocuments.net/reader037/viewer/2022110322/56649d1c5503460f949f19e6/html5/thumbnails/34.jpg)
Indicator functions (partitions)
34
![Page 35: Deformable Contours 1 Dr. E. Ribeiro. Classical Methods 2 Thresholding Edge detection An image of blood vessel Slide by: Chunming Li, Vanderbilt University](https://reader037.vdocuments.net/reader037/viewer/2022110322/56649d1c5503460f949f19e6/html5/thumbnails/35.jpg)
Indicator functions (partitions)
35
![Page 36: Deformable Contours 1 Dr. E. Ribeiro. Classical Methods 2 Thresholding Edge detection An image of blood vessel Slide by: Chunming Li, Vanderbilt University](https://reader037.vdocuments.net/reader037/viewer/2022110322/56649d1c5503460f949f19e6/html5/thumbnails/36.jpg)
Distance function (level set function)
36
![Page 37: Deformable Contours 1 Dr. E. Ribeiro. Classical Methods 2 Thresholding Edge detection An image of blood vessel Slide by: Chunming Li, Vanderbilt University](https://reader037.vdocuments.net/reader037/viewer/2022110322/56649d1c5503460f949f19e6/html5/thumbnails/37.jpg)
Two-phase case
37
Estimating the Parameters of the Gaussian densities
![Page 38: Deformable Contours 1 Dr. E. Ribeiro. Classical Methods 2 Thresholding Edge detection An image of blood vessel Slide by: Chunming Li, Vanderbilt University](https://reader037.vdocuments.net/reader037/viewer/2022110322/56649d1c5503460f949f19e6/html5/thumbnails/38.jpg)
Two-phase case
![Page 39: Deformable Contours 1 Dr. E. Ribeiro. Classical Methods 2 Thresholding Edge detection An image of blood vessel Slide by: Chunming Li, Vanderbilt University](https://reader037.vdocuments.net/reader037/viewer/2022110322/56649d1c5503460f949f19e6/html5/thumbnails/39.jpg)
Experiments
39
![Page 40: Deformable Contours 1 Dr. E. Ribeiro. Classical Methods 2 Thresholding Edge detection An image of blood vessel Slide by: Chunming Li, Vanderbilt University](https://reader037.vdocuments.net/reader037/viewer/2022110322/56649d1c5503460f949f19e6/html5/thumbnails/40.jpg)
Gray level
40
![Page 41: Deformable Contours 1 Dr. E. Ribeiro. Classical Methods 2 Thresholding Edge detection An image of blood vessel Slide by: Chunming Li, Vanderbilt University](https://reader037.vdocuments.net/reader037/viewer/2022110322/56649d1c5503460f949f19e6/html5/thumbnails/41.jpg)
Color and Texture
41
![Page 42: Deformable Contours 1 Dr. E. Ribeiro. Classical Methods 2 Thresholding Edge detection An image of blood vessel Slide by: Chunming Li, Vanderbilt University](https://reader037.vdocuments.net/reader037/viewer/2022110322/56649d1c5503460f949f19e6/html5/thumbnails/42.jpg)
Results
42Slide by: Chunming Li, Vanderbilt University
![Page 43: Deformable Contours 1 Dr. E. Ribeiro. Classical Methods 2 Thresholding Edge detection An image of blood vessel Slide by: Chunming Li, Vanderbilt University](https://reader037.vdocuments.net/reader037/viewer/2022110322/56649d1c5503460f949f19e6/html5/thumbnails/43.jpg)
3D Segmentation of Corpus Callosum
43Slide by: Chunming Li, Vanderbilt University
![Page 44: Deformable Contours 1 Dr. E. Ribeiro. Classical Methods 2 Thresholding Edge detection An image of blood vessel Slide by: Chunming Li, Vanderbilt University](https://reader037.vdocuments.net/reader037/viewer/2022110322/56649d1c5503460f949f19e6/html5/thumbnails/44.jpg)
Result
44
Synthetic noisy image
Slide by: Chunming Li, Vanderbilt University
![Page 45: Deformable Contours 1 Dr. E. Ribeiro. Classical Methods 2 Thresholding Edge detection An image of blood vessel Slide by: Chunming Li, Vanderbilt University](https://reader037.vdocuments.net/reader037/viewer/2022110322/56649d1c5503460f949f19e6/html5/thumbnails/45.jpg)
2D Segmentation of Real Color Images
45
A real image of potatoes
Slide by: Chunming Li, Vanderbilt University
![Page 46: Deformable Contours 1 Dr. E. Ribeiro. Classical Methods 2 Thresholding Edge detection An image of blood vessel Slide by: Chunming Li, Vanderbilt University](https://reader037.vdocuments.net/reader037/viewer/2022110322/56649d1c5503460f949f19e6/html5/thumbnails/46.jpg)
2D Vessel Segmentation
46Slide by: Chunming Li, Vanderbilt University
![Page 47: Deformable Contours 1 Dr. E. Ribeiro. Classical Methods 2 Thresholding Edge detection An image of blood vessel Slide by: Chunming Li, Vanderbilt University](https://reader037.vdocuments.net/reader037/viewer/2022110322/56649d1c5503460f949f19e6/html5/thumbnails/47.jpg)
Segmentation of White Matter in MR images
47Slide by: Chunming Li, Vanderbilt University
![Page 48: Deformable Contours 1 Dr. E. Ribeiro. Classical Methods 2 Thresholding Edge detection An image of blood vessel Slide by: Chunming Li, Vanderbilt University](https://reader037.vdocuments.net/reader037/viewer/2022110322/56649d1c5503460f949f19e6/html5/thumbnails/48.jpg)
Effect of the Level Set Regularization
48
Final zero level contour Final level set function
Without level set regularization
Slide by: Chunming Li, Vanderbilt University
![Page 49: Deformable Contours 1 Dr. E. Ribeiro. Classical Methods 2 Thresholding Edge detection An image of blood vessel Slide by: Chunming Li, Vanderbilt University](https://reader037.vdocuments.net/reader037/viewer/2022110322/56649d1c5503460f949f19e6/html5/thumbnails/49.jpg)
3D Vessel Segmentation
49
MRA Vessel Segmentation
Slide by: Chunming Li, Vanderbilt University
![Page 50: Deformable Contours 1 Dr. E. Ribeiro. Classical Methods 2 Thresholding Edge detection An image of blood vessel Slide by: Chunming Li, Vanderbilt University](https://reader037.vdocuments.net/reader037/viewer/2022110322/56649d1c5503460f949f19e6/html5/thumbnails/50.jpg)
3-D Ultrasound
50http://mrcas.mpe.ntu.edu.sg/research/imgpro/
3d_level_set.htm
![Page 51: Deformable Contours 1 Dr. E. Ribeiro. Classical Methods 2 Thresholding Edge detection An image of blood vessel Slide by: Chunming Li, Vanderbilt University](https://reader037.vdocuments.net/reader037/viewer/2022110322/56649d1c5503460f949f19e6/html5/thumbnails/51.jpg)
3-D Ultrasound
51