registration by continuous optimisationcampar.in.tum.de/twiki/pub/defregtutorial/webhome/... ·...
TRANSCRIPT
![Page 1: Registration by continuous optimisationcampar.in.tum.de/twiki/pub/DefRegTutorial/WebHome/... · Registration by continuous optimisation Stefan Klein Erasmus MC, the Netherlands](https://reader034.vdocuments.net/reader034/viewer/2022042121/5e9ab4b4a3f0f80d994e4c7e/html5/thumbnails/1.jpg)
Registration by continuous optimisation
Stefan Klein
Erasmus MC, the Netherlands
Biomedical Imaging Group Rotterdam (BIGR)
![Page 2: Registration by continuous optimisationcampar.in.tum.de/twiki/pub/DefRegTutorial/WebHome/... · Registration by continuous optimisation Stefan Klein Erasmus MC, the Netherlands](https://reader034.vdocuments.net/reader034/viewer/2022042121/5e9ab4b4a3f0f80d994e4c7e/html5/thumbnails/2.jpg)
Registration = optimisation
1
C
txty
![Page 3: Registration by continuous optimisationcampar.in.tum.de/twiki/pub/DefRegTutorial/WebHome/... · Registration by continuous optimisation Stefan Klein Erasmus MC, the Netherlands](https://reader034.vdocuments.net/reader034/viewer/2022042121/5e9ab4b4a3f0f80d994e4c7e/html5/thumbnails/3.jpg)
Registration = optimisation
1
C
txty
![Page 4: Registration by continuous optimisationcampar.in.tum.de/twiki/pub/DefRegTutorial/WebHome/... · Registration by continuous optimisation Stefan Klein Erasmus MC, the Netherlands](https://reader034.vdocuments.net/reader034/viewer/2022042121/5e9ab4b4a3f0f80d994e4c7e/html5/thumbnails/4.jpg)
Registration = optimisation
1
C
txty
![Page 5: Registration by continuous optimisationcampar.in.tum.de/twiki/pub/DefRegTutorial/WebHome/... · Registration by continuous optimisation Stefan Klein Erasmus MC, the Netherlands](https://reader034.vdocuments.net/reader034/viewer/2022042121/5e9ab4b4a3f0f80d994e4c7e/html5/thumbnails/5.jpg)
Registration = optimisation
1
C
txty
![Page 6: Registration by continuous optimisationcampar.in.tum.de/twiki/pub/DefRegTutorial/WebHome/... · Registration by continuous optimisation Stefan Klein Erasmus MC, the Netherlands](https://reader034.vdocuments.net/reader034/viewer/2022042121/5e9ab4b4a3f0f80d994e4c7e/html5/thumbnails/6.jpg)
Example
2
![Page 7: Registration by continuous optimisationcampar.in.tum.de/twiki/pub/DefRegTutorial/WebHome/... · Registration by continuous optimisation Stefan Klein Erasmus MC, the Netherlands](https://reader034.vdocuments.net/reader034/viewer/2022042121/5e9ab4b4a3f0f80d994e4c7e/html5/thumbnails/7.jpg)
Example
2fixed image moving image
![Page 8: Registration by continuous optimisationcampar.in.tum.de/twiki/pub/DefRegTutorial/WebHome/... · Registration by continuous optimisation Stefan Klein Erasmus MC, the Netherlands](https://reader034.vdocuments.net/reader034/viewer/2022042121/5e9ab4b4a3f0f80d994e4c7e/html5/thumbnails/8.jpg)
Example
2fixed image moving image
![Page 9: Registration by continuous optimisationcampar.in.tum.de/twiki/pub/DefRegTutorial/WebHome/... · Registration by continuous optimisation Stefan Klein Erasmus MC, the Netherlands](https://reader034.vdocuments.net/reader034/viewer/2022042121/5e9ab4b4a3f0f80d994e4c7e/html5/thumbnails/9.jpg)
Example
2fixed image moving image
![Page 10: Registration by continuous optimisationcampar.in.tum.de/twiki/pub/DefRegTutorial/WebHome/... · Registration by continuous optimisation Stefan Klein Erasmus MC, the Netherlands](https://reader034.vdocuments.net/reader034/viewer/2022042121/5e9ab4b4a3f0f80d994e4c7e/html5/thumbnails/10.jpg)
3
• F(x) = fixed image, M(x) = moving image
x = voxel coordinate
• Transformation function: T(x ; p)
p = vector of transformation parameters
• Cost function: C( p )
measures similarity of fixed image F(x) and
deformed moving image M( T(x; p) )
• Find p that minimises C
Math
![Page 11: Registration by continuous optimisationcampar.in.tum.de/twiki/pub/DefRegTutorial/WebHome/... · Registration by continuous optimisation Stefan Klein Erasmus MC, the Netherlands](https://reader034.vdocuments.net/reader034/viewer/2022042121/5e9ab4b4a3f0f80d994e4c7e/html5/thumbnails/11.jpg)
4
Iterative optimisation
pk+1 = pk + ak . dk
dk = search direction
ak = step size
gradient descent:kkk )(
Cgp
pd
![Page 12: Registration by continuous optimisationcampar.in.tum.de/twiki/pub/DefRegTutorial/WebHome/... · Registration by continuous optimisation Stefan Klein Erasmus MC, the Netherlands](https://reader034.vdocuments.net/reader034/viewer/2022042121/5e9ab4b4a3f0f80d994e4c7e/html5/thumbnails/12.jpg)
pk+1 = pk - ak . gk
P1 p1 g1
p2 p2 g2
p3 = p3 - ak . g3
::
::
::
k+1 k k
Gradient descent
5
![Page 13: Registration by continuous optimisationcampar.in.tum.de/twiki/pub/DefRegTutorial/WebHome/... · Registration by continuous optimisation Stefan Klein Erasmus MC, the Netherlands](https://reader034.vdocuments.net/reader034/viewer/2022042121/5e9ab4b4a3f0f80d994e4c7e/html5/thumbnails/13.jpg)
pk+1 = pk - ak . gk
p1 p1 g1
p2 p2 g2
p3 = p3 - ak . g3
::
::
::
k+1 k k
=Cp1 k
Gradient descent
5
![Page 14: Registration by continuous optimisationcampar.in.tum.de/twiki/pub/DefRegTutorial/WebHome/... · Registration by continuous optimisation Stefan Klein Erasmus MC, the Netherlands](https://reader034.vdocuments.net/reader034/viewer/2022042121/5e9ab4b4a3f0f80d994e4c7e/html5/thumbnails/14.jpg)
6
Cost function derivative
Example for mean of squared differences:
x
x
x
xp
TpxTx
ppxTx
p
pxTxp
M));((M)(F
N
2
M));((M)(F
N
2C
));((M)(FN
1)(C
t
2
![Page 15: Registration by continuous optimisationcampar.in.tum.de/twiki/pub/DefRegTutorial/WebHome/... · Registration by continuous optimisation Stefan Klein Erasmus MC, the Netherlands](https://reader034.vdocuments.net/reader034/viewer/2022042121/5e9ab4b4a3f0f80d994e4c7e/html5/thumbnails/15.jpg)
7
Choice of dk
pk+1 = pk + ak . dk
![Page 16: Registration by continuous optimisationcampar.in.tum.de/twiki/pub/DefRegTutorial/WebHome/... · Registration by continuous optimisation Stefan Klein Erasmus MC, the Netherlands](https://reader034.vdocuments.net/reader034/viewer/2022042121/5e9ab4b4a3f0f80d994e4c7e/html5/thumbnails/16.jpg)
Choice of dk
8
C
p1p2
gradient descent
![Page 17: Registration by continuous optimisationcampar.in.tum.de/twiki/pub/DefRegTutorial/WebHome/... · Registration by continuous optimisation Stefan Klein Erasmus MC, the Netherlands](https://reader034.vdocuments.net/reader034/viewer/2022042121/5e9ab4b4a3f0f80d994e4c7e/html5/thumbnails/17.jpg)
Choice of dk
8
C
p1p2
smarter steps
![Page 18: Registration by continuous optimisationcampar.in.tum.de/twiki/pub/DefRegTutorial/WebHome/... · Registration by continuous optimisation Stefan Klein Erasmus MC, the Netherlands](https://reader034.vdocuments.net/reader034/viewer/2022042121/5e9ab4b4a3f0f80d994e4c7e/html5/thumbnails/18.jpg)
Choice of dk
8
C
p1p2
cheaper steps
![Page 19: Registration by continuous optimisationcampar.in.tum.de/twiki/pub/DefRegTutorial/WebHome/... · Registration by continuous optimisation Stefan Klein Erasmus MC, the Netherlands](https://reader034.vdocuments.net/reader034/viewer/2022042121/5e9ab4b4a3f0f80d994e4c7e/html5/thumbnails/19.jpg)
pk+1 = pk + ak . dk
gradient descent: dk = - gk
Newton: dk = - [Hk]-1 gk
quasi-Newton: dk = - Bk gk
conjugate gradient: dk = - gk + βk dk-1
stochastic gradient: dk - gk
Choice of dk
smarter steps
cheaper steps
9
![Page 20: Registration by continuous optimisationcampar.in.tum.de/twiki/pub/DefRegTutorial/WebHome/... · Registration by continuous optimisation Stefan Klein Erasmus MC, the Netherlands](https://reader034.vdocuments.net/reader034/viewer/2022042121/5e9ab4b4a3f0f80d994e4c7e/html5/thumbnails/20.jpg)
10
Experimental comparison
Cardiac CT, 97x97x97 voxels, artifically deformed
![Page 21: Registration by continuous optimisationcampar.in.tum.de/twiki/pub/DefRegTutorial/WebHome/... · Registration by continuous optimisation Stefan Klein Erasmus MC, the Netherlands](https://reader034.vdocuments.net/reader034/viewer/2022042121/5e9ab4b4a3f0f80d994e4c7e/html5/thumbnails/21.jpg)
11
Experimental comparison
Cardiac CT, 97x97x97 voxels, artifically deformed
![Page 22: Registration by continuous optimisationcampar.in.tum.de/twiki/pub/DefRegTutorial/WebHome/... · Registration by continuous optimisation Stefan Klein Erasmus MC, the Netherlands](https://reader034.vdocuments.net/reader034/viewer/2022042121/5e9ab4b4a3f0f80d994e4c7e/html5/thumbnails/22.jpg)
12
Experimental comparison
Error measure:
x
xTxT )(ˆ)(N
1e
![Page 23: Registration by continuous optimisationcampar.in.tum.de/twiki/pub/DefRegTutorial/WebHome/... · Registration by continuous optimisation Stefan Klein Erasmus MC, the Netherlands](https://reader034.vdocuments.net/reader034/viewer/2022042121/5e9ab4b4a3f0f80d994e4c7e/html5/thumbnails/23.jpg)
Experimental comparison
computation time
e [mm]
gradient descent
quasi-Newton
conjugate gradient
stochastic gradient
0
0.5
1
1.5
2
2.5
3
0.001 0.01 0.1 1 10 100 1000
![Page 24: Registration by continuous optimisationcampar.in.tum.de/twiki/pub/DefRegTutorial/WebHome/... · Registration by continuous optimisation Stefan Klein Erasmus MC, the Netherlands](https://reader034.vdocuments.net/reader034/viewer/2022042121/5e9ab4b4a3f0f80d994e4c7e/html5/thumbnails/24.jpg)
14
Choice of ak
pk+1 = pk + ak . dk
![Page 25: Registration by continuous optimisationcampar.in.tum.de/twiki/pub/DefRegTutorial/WebHome/... · Registration by continuous optimisation Stefan Klein Erasmus MC, the Netherlands](https://reader034.vdocuments.net/reader034/viewer/2022042121/5e9ab4b4a3f0f80d994e4c7e/html5/thumbnails/25.jpg)
Choice of ak
C
p1p2
15
Too small steps
![Page 26: Registration by continuous optimisationcampar.in.tum.de/twiki/pub/DefRegTutorial/WebHome/... · Registration by continuous optimisation Stefan Klein Erasmus MC, the Netherlands](https://reader034.vdocuments.net/reader034/viewer/2022042121/5e9ab4b4a3f0f80d994e4c7e/html5/thumbnails/26.jpg)
Choice of ak
C
p1p2
15
Too large steps
![Page 27: Registration by continuous optimisationcampar.in.tum.de/twiki/pub/DefRegTutorial/WebHome/... · Registration by continuous optimisation Stefan Klein Erasmus MC, the Netherlands](https://reader034.vdocuments.net/reader034/viewer/2022042121/5e9ab4b4a3f0f80d994e4c7e/html5/thumbnails/27.jpg)
16
Choice of ak
constant: ak = a
slowly decaying: ak = f ( k ) = a / ( A + k )a
exact line search: ak = argmina C ( pk + a dk )
inexact line search: ak argmina C ( pk + a dk ) [Wolfe conditions]
adaptive: ak = F ( progress in previous iterations )
pk+1 = pk + ak . dk
![Page 28: Registration by continuous optimisationcampar.in.tum.de/twiki/pub/DefRegTutorial/WebHome/... · Registration by continuous optimisation Stefan Klein Erasmus MC, the Netherlands](https://reader034.vdocuments.net/reader034/viewer/2022042121/5e9ab4b4a3f0f80d994e4c7e/html5/thumbnails/28.jpg)
Stochastic gradient descent with
adaptive strategy for ak
)(sigmoidtt
)tA/(a)t(f
)t(f
1kTkk1k
kk
kkk1k
a
gg
gpp
-5 0 5
-1
0
1
17
0 250 5000
10
20
![Page 29: Registration by continuous optimisationcampar.in.tum.de/twiki/pub/DefRegTutorial/WebHome/... · Registration by continuous optimisation Stefan Klein Erasmus MC, the Netherlands](https://reader034.vdocuments.net/reader034/viewer/2022042121/5e9ab4b4a3f0f80d994e4c7e/html5/thumbnails/29.jpg)
Stochastic gradient descent with
adaptive strategy for ak
)(sigmoidtt
)tA/(a)t(f
)t(f
1kTkk1k
kk
kkk1k
a
gg
gpp
-5 0 5
-1
0
1
17
0 250 5000
10
20
Choose a such that:
max. voxel displacement per iteration < [mm]
(with 95% probability)
![Page 30: Registration by continuous optimisationcampar.in.tum.de/twiki/pub/DefRegTutorial/WebHome/... · Registration by continuous optimisation Stefan Klein Erasmus MC, the Netherlands](https://reader034.vdocuments.net/reader034/viewer/2022042121/5e9ab4b4a3f0f80d994e4c7e/html5/thumbnails/30.jpg)
Experimental comparison
• 6 prostate MR image pairs: nonrigid registration
• evaluation measure:
overlap of manual segmentations after registration
18
![Page 31: Registration by continuous optimisationcampar.in.tum.de/twiki/pub/DefRegTutorial/WebHome/... · Registration by continuous optimisation Stefan Klein Erasmus MC, the Netherlands](https://reader034.vdocuments.net/reader034/viewer/2022042121/5e9ab4b4a3f0f80d994e4c7e/html5/thumbnails/31.jpg)
Experimental comparison
[mm]
A
non-adaptive2000
0.0
31
25
0.0
62
5
0.1
25
0.2
5
0.5
1.0
2.0
4.0
8.0
1.25
2.5
5
10
20
40
80
160
320
0.8
0.85
0.9
0.95
[mm]
A
adaptive2000
0.0
31
25
0.0
62
5
0.1
25
0.2
5
0.5
1.0
2.0
4.0
8.0
1.25
2.5
5
10
20
40
80
160
320
0.8
0.85
0.9
0.95
non-adaptive adaptive
19
![Page 32: Registration by continuous optimisationcampar.in.tum.de/twiki/pub/DefRegTutorial/WebHome/... · Registration by continuous optimisation Stefan Klein Erasmus MC, the Netherlands](https://reader034.vdocuments.net/reader034/viewer/2022042121/5e9ab4b4a3f0f80d994e4c7e/html5/thumbnails/32.jpg)
Experimental comparison
20
Experiments with:
• brain, lung, prostate
• CT, MRI
• sum of squared differences, mutual information,
normalized mutual information
• rigid, nonrigid
size voxel ,20 A
good results in all experiments!
![Page 33: Registration by continuous optimisationcampar.in.tum.de/twiki/pub/DefRegTutorial/WebHome/... · Registration by continuous optimisation Stefan Klein Erasmus MC, the Netherlands](https://reader034.vdocuments.net/reader034/viewer/2022042121/5e9ab4b4a3f0f80d994e4c7e/html5/thumbnails/33.jpg)
Local similarity measures
21
• MI = mutual information
• LMI = localised mutual information
=
assumes grey-value distribution does
not vary over image domain
)( x
xMIN
1 (aka: regional MI, conditional MI,
spatial information encoded MI)
![Page 34: Registration by continuous optimisationcampar.in.tum.de/twiki/pub/DefRegTutorial/WebHome/... · Registration by continuous optimisation Stefan Klein Erasmus MC, the Netherlands](https://reader034.vdocuments.net/reader034/viewer/2022042121/5e9ab4b4a3f0f80d994e4c7e/html5/thumbnails/34.jpg)
Local similarity measures
21
• MI = mutual information
• LMI = localised mutual information
=
assumes grey-value distribution does
not vary over image domain
)( x
xMIN
1 (aka: regional MI, conditional MI,
spatial information encoded MI)
can be efficiently implemented with
stochastic gradient descent!
![Page 35: Registration by continuous optimisationcampar.in.tum.de/twiki/pub/DefRegTutorial/WebHome/... · Registration by continuous optimisation Stefan Klein Erasmus MC, the Netherlands](https://reader034.vdocuments.net/reader034/viewer/2022042121/5e9ab4b4a3f0f80d994e4c7e/html5/thumbnails/35.jpg)
Summary
• Parametric formulation can be solved by continuous optimisation
• Derivative-based methods: require
• Extensive literature
• Basic method: gradient descent
• Popular choice:
quasi-Newton or conjugate gradient icm inexact line search
• “Recommended”:
stochastic gradient descent with adaptive step sizes
pC
22
![Page 36: Registration by continuous optimisationcampar.in.tum.de/twiki/pub/DefRegTutorial/WebHome/... · Registration by continuous optimisation Stefan Klein Erasmus MC, the Netherlands](https://reader034.vdocuments.net/reader034/viewer/2022042121/5e9ab4b4a3f0f80d994e4c7e/html5/thumbnails/36.jpg)
Literature
• Nocedal & Wright: Numerical Optimization
• IEEE Trans. Image Processing 2007 - Klein, Staring, Pluim
Evaluation of optimization methods for nonrigid medical image registration using mutual
information and B-splines
• Int. J. Computer Vision 2009 - Klein, Pluim, Staring, Viergever
Adaptive stochastic gradient descent optimisation for image registration
• IEEE Trans. Image Processing 2000 - Thevenaz, Unser
Optimization of mutual information for multiresolution image registration
23
![Page 37: Registration by continuous optimisationcampar.in.tum.de/twiki/pub/DefRegTutorial/WebHome/... · Registration by continuous optimisation Stefan Klein Erasmus MC, the Netherlands](https://reader034.vdocuments.net/reader034/viewer/2022042121/5e9ab4b4a3f0f80d994e4c7e/html5/thumbnails/37.jpg)
• Rigid and nonrigid registration
• Various cost functions, transformation models, multiresolution
strategies etc.
• Many optimisation algorithms implemented
• Free: http://elastix.isi.uu.nl
• Based on Insight ToolKit (ITK): http://www.itk.org
• IEEE Trans. Medical Imaging 2010 - Klein, Staring, Murphy,
Viergever, Pluim – elastix: a toolbox for intensity based medical image registration24