jan kamenický mariánská 2008. 2 we deal with medical images ◦ different viewpoints - multiview...
TRANSCRIPT
Jan KamenickýMariánská 2008
2
We deal with medical images◦ Different viewpoints - multiview◦ Different times - multitemporal◦ Different sensors – multimodal
Area-based methods (no features)
Transformation model Cost function minimization
3
Transformation model◦ Displacement field u(x)
4
)()( xuxxT ( ) ( )R SI x I T x
Transformation model◦ Displacement field u(x)
Cost function◦ Similarity measure (external forces)◦ Smoothing (penalization) term (internal forces)◦ Additional constraints (landmarks, volume
preservation)
5
)()( xuxxT ( ) ( )R SI x I T x
( ; , ) ( ; , ) ( ) ( )softR S R SC T I I S T I I P T C T
Transformation model◦ Displacement field u(x)
Cost function◦ Similarity measure (external forces)◦ Smoothing (penalization) term (internal forces)◦ Additional constraints (landmarks, volume
preservation)
Minimization
6
ˆ ˆarg min ( ; , ) arg min ( ; , )R S R ST
T C T I I C I I
)()( xuxxT ( ) ( )R SI x I T x
( ; , ) ( ; , ) ( ) ( )softR S R SC T I I S T I I P T C T
Translation Rigid (Euler)
◦ Translation, rotation Similarity
◦ Translation, rotation, scaling Affine B-splines
◦ Control points - regular grid on reference image
8
3( ) ( )k x
k kx N
T x x p x x
9
Sum of Squared Differences Normalized Correlation Coefficients Mutual Information Normalized Gradient Field
10
Sum of Squared Differences (SSD)◦ Equal intensity distribution (same modality)
Normalized Correlation Coefficients Mutual Information Normalized Gradient Field
11
21( ; , ) ( ) ( )
i R
R S R i S ixR
SSD I I I x I T x
Sum of Squared Differences Normalized Correlation Coefficients (NCC)
◦ Linear relation between intensity values (but still same modality)
Mutual Information Normalized Gradient Field
12
2 2
( ) ( )
( ; , )( ) ( )
i R
i R i R
R i R S i Sx
R S
R i R S i Sx x
I x I I T x I
NCC I II x I I T x I
Sum of Squared Differences Normalized Correlation Coefficients Mutual Information
◦ Any statistical dependence
Normalized Gradient Field
13
2
( , ; )( ; , ) ( , ; ) log
( ) ( ; )S R
R Ss L r L R S
p r sMI I I p r s
p r p s
Mutual Information (MI)◦ From entropy
14
( , ) ( ) ( | )
( ) ( | )
( ) ( ) ( , )
MI X Y H X H X Y
H Y H Y X
H X H Y H X Y
2( ) ( ) log ( ), ( ) 1 x X x X
H X p x p x p x
( , )2 ( ) ( )( , ) ( , ) log
X Y
p x yp x p y
y Y x X
MI X Y p x y
Mutual Information (MI)◦ From Kullback-Leibler distance
15
( )( )( , ) ( ) log p iq i
i
KL p q p i
( , )2 ( ) ( )( , ) ( , ) log
X Y
p x yp x p y
y Y x X
MI X Y p x y
Mutual Information (MI)◦ For images
p(x) … normalized image histogram
◦ Normalized Mutual Information (NMI)
16
( , ; )2 ( ) ( ; )( ; , ) ( , ; ) log
R S
S R
p r sR S p r p s
s L r L
MI I I p r s
2 2
2
( ) log ( ) ( ; ) log ( ; )
( ; , )( , ; ) log ( , ; )
R S
S R
R R S Sr L s L
R S
s L r L
p r p r p s p m
NMI I Ip r s p r s
( ) ( )
( , )
H X H Y
H X Y
Mutual Information (MI)◦ Joint probability estimation
Using B-spline Parzen windows
and are defined by the histogram bins widths
17
( )( )1( , ; )
i R
S iR iR S
xR R S
s I T xr I xp r s w w
R S
Sum of Squared Differences Normalized Correlation Coefficients Mutual Information Normalized Gradient Field (NGF)
◦ Based on edges
18
2 2
2( , ) ( )en I x I I x e
2( ; , ) ( , ) ( ( ), )
i R
TR S e R i e S i
x
NGF I I n I x n I T x
Elastic◦ Elastic potential (motivated by material
properties)
Fluid◦ Viscous fluid model (based on Navier-Stokes)
Diffusion◦ Much faster
19
2 2
4 2,
[ ] divj k
elasx k x j
j k
P u u u u dx
212[ ]diff
ll
P u u dx
Curvature◦ Doesn’t penalize affine transformation
Bending energy (Thin plate splines)
20
2 2
, ,
[ ] ( )p
q r
u
x xp q r
P u x dx
212
1
[ ]d
curvl
l
P u u dx
21
curvature
diffusion
elastic
fluid
Landmarks (fiducial markers)◦ “Hard” constraint
◦ “Soft” constraint
Volume preservation
22
( ) , 1, 2,...,j j j jC r u r t j m
2
1
msoftjj
C C
log det ( )R
softC u x dx
Full Grid
◦ Used with multi-resolution Random
◦ Random subset of voxels is selected◦ Improved speed
23
25
1 ,
0,1,2,...k k k ka d
k
Gradient Descent (GD)◦ Linear rate of convergence
Quasi-Newton Nonlinear Conjugate Gradient Stochastic Gradient Descent Evolution Strategy
26
1 ( )k k k ka g
Gradient Descent Quasi-Newton (QN)
◦ Can be superlinearly convergent
Nonlinear Conjugate Gradient Stochastic Gradient Descent Evolution Strategy
27
11 ( ) ( )k k k kH g
Gradient Descent Quasi-Newton Nonlinear Conjugate Gradient (NCG)
◦ Superlinear rate of convergence can be achieved
Stochastic Gradient Descent Evolution Strategy
28
1( )k k k kd g d
Gradient Descent Quasi-Newton Nonlinear Conjugate Gradient Stochastic Gradient Descent (SGD)
◦ Similar to GD, but uses approximation of the gradient (Kiefer-Wolfowitz, Simultaneous Perturbation, Robbins-Monro)
Evolution Strategy
29
Gradient Descent Quasi-Newton Nonlinear Conjugate Gradient Stochastic Gradient Descent Evolution Strategy (ES)
◦ Covariance matrix adaptation◦ Tries several possible directions (randomly
according to the covariance matrix of the cost function), the best are chosen and their weighted average is used
30
Data complexity◦ Gaussian pyramid◦ Laplacian pyramid◦ Wavelet pyramid
Transformation complexity◦ Transformation superposition◦ Different B-spline grid density
31
Registration toolkit based on ITK Handles many methods
◦ Similarity measures (SSD, NCC, MI, NMI)◦ Transformations (rigid, affine, B-splines)◦ Optimizers (GD, SGD-RM)◦ Samplers, Interpolators, Multi-resolution, …
http://elastix.isi.uu.nl
32