July 2010

Image RegistrationTechniques, Benchmarking, Strategy

Surgical Planning Laboratory

Center for Neurological Imaging

July 2010

Lidwien Veugen

Supervision by Dominik S. Meier, PhD

July 2010

Contents

- IntroductionImage Registration, 3D Slicer

- TheoryTransformations, Similarity Metrics

- BenchmarkingTime/Memory vs Iterations/Samples

- Registration Strategies

- Registration CasesBrains, PET-CT, EMPIRE10

July 2010

Introduction

Image Registration:

- Process of matching multiple image by optimal transformation

3D Slicer:

- Free Open Source Software program

- Huge amount of Registration Modules/Methods

July 2010

3D Slicer

July 2010

Theory Transformations

Mapping points from original spatial coordinates to new spatial coordinates: (u,v,w) = T{(x,y,z)}

Rigid TransformRotation + Translation

(u,v,w) = R*(x,y,z) + t

6 DOF

Affine TransformRotation + Translation + Scaling + Shear

(u,v,w) = A*(x,y,z) + t

12 DOF

July 2010

Theory Transformations

BSplineSpline: function defined piecewice by polynomialsCubic grid of moving control points describes deformation

3 DOF per control point

BrainsDemonWarpThirion + Maxwell: Image registration based on optical flow

Boundaries are semi-permeable membranes with effectors/demons

High DOF

July 2010

Theory Transformation

BRAINSFit- Rigid, Affine, BSpline

- Mutual Information

- 6/12/higher DOF

Expert Automated Registration- Pipelines: Rigid, Affine, BSpline

- MutualInfo + MeanSqE + NormCorr

- 6/12/higher DOF

Plastimatch- Pipeline: Rigid/Affine, BSpline(s)

- MutualInfo + MeanSqE

- 6/12/higher DOF

July 2010

Theory Similarity Metrics

Tells to what degree two images are aligned

Based on: intensity, landmarks

Mutual Information- Measure of the statistical dependence between two random variables:

Information about image A that is shared by B and vice versa

- Maximized if the two images are spatially aligned

- Based on Shannon entropy H: measure of intensity prediction

- Fast measure

MI A,B H B H B | A

H(A) pAa

a log pA a

July 2010

Theory Similarity Metrics

Normalized Cross Correlation- Based on cross correlation

- Maximized if the two images are spatially aligned

- Intra-patient + Intra-modality

- Time consuming

Mean Squared Difference- Summation of the squared differences between two images

- Minimized if the two images are spatially aligned

- Intra-patient + Intra-modality

- Time consuming

MSD1

mnA x,y B x,y 2

y1

n

x1

m

NCC A x,y A

y1

n

x1

m

B x,y B

A x,y A 2y1

n

x1

m

B x,y B 2y1

n

x1

m

July 2010

Theory Optimization

Optimization algorithm:

Tries to find a global solution to an energy function

- Gradient descent

- Statistical optimization

- Line search algorithm

- One-plus-one evolutionary

- Multiresolution

July 2010

Registration Accuracy

Subtraction

Fixed - MovingRegistered

Checkerboard

Alternating squares from

fixed and moving image

July 2010

BenchmarkingEffect of the amount of iterations and samples on CPU

time and memory for different modules/methods

Rigid: 4 methods

Affine: 7 methods

BSpline: 2 methods

Default: Samples = 10000, Iterations = 200

Iterations: 11 values, ranging from 25 to 20 000

Samples: 20 values, ranging from 25 to 10 000 000

July 2010

Benchmarking

Fast results with:

SPL Dell Linux Cluster of 50 computers

Creates log-file of every job Matlab

July 2010

Benchmarking Results

July 2010

Benchmarking Results

Time vs Iterations- Not much effect

- Increase: Brainsfit, Exp.Autom.

- Decrease: Multiresolution

- Constant: BSpline modules

Time vs Samples- Increase: All modules, except:

- Decrease: Exp.Autom. NormCr- 10 800 seconds (0.003% 13%)

- Rigid < Affine < BSpline

Memory vs IterationsNot much effect

- Increase: All modules, except:

- Constant: Brainsfit, Multires

- Lowest: 2MB; Highest: 155MB

Memory vs Samples- Increase: All modules, except:

- Decrease: Exp.Autom. NormCr

- Lowest: Rigid (10-100MB)

- Highest: BSpline (400-1300MB)

July 2010

Registration Cases

Slicer Registration Case Library

July 2010

Registration Strategies

Choice of Transformation

Modality, Subject, Inter/Intra, Part of body

Choice of Similarity Metric

Inter/Intra, Time/Accuracy

Focus

Time/Accuracy/Memory Sim.metric/iterations/samples

Fixed Image

Resolution/Contrast

July 2010

Registration Cases

EMPIRE10Evaluation of Methods for Pulmonary Image Registration 2010

= Challenge of International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI)

20 Pairs of chest CT scans: variety scanners, voxel size, breathing phase

Evaluation: Lung boundaries, Fissures, Landmarks, Singularities

July 2010

Registration Cases

July 2010

Registration Cases

EMPIRE10 - Registration Pipeline:1. Fast Affine Registration

2. Fast nonrigid Bspline Registration (grid = 7)

3. Fast nonrigid Bspline Registration (grid = 12)

4. Fast nonrigid Bspline Registration (grid = 17)

5. BrainsDemonWarp

July 2010

Registration CasesEMPIRE10 - Quality RegistrationSubtraction + MATLAB help in evaluation registration:

Median pixelvalue of absolute subtracted image: the lower the better

July 2010

Registration Cases

fMRI alignment to structural scan (T1)- Fixed: T1 scan (anatomical reference)- Moving: fMRI scan - Problem: Low tissue contrast, acquisition related distortions

T1fMRI

Registration based on ventricles only

July 2010

Registration Cases

Aging Mobility Study 2 year follow-up

- 2 Exams at different times: nonrigid (BSpline)

- Incorrect axis-info

- Fixed: MPRAGE- Moving: T2, FLAIR

MPRAGE

T1

FLAI RMPRAGE

T1

FLAI R

Exam 1 Exam 2

July 2010

Registration Cases

Inter-subject Normal brain MIDASexample

- Fixed: T1- Moving: T2, MRA - Interpatient: non-rigid (BSpline)

MRA

Patient 2

T1T2

MRA

Patient 1

T1T2

July 2010

Registration Cases

PET-CT Fusion 2- Intersubject: nonrigid

BSpline, BrainsDemonWarp- Fixed: CT-scan patient 1- Moving: CT-scan patient 2- Problem: Different posture

Patient 1 Patient 2

CT

PET

CT

PET

July 2010

Registration Cases

Brain Intersubject PNL-XNAT- Intersubject: nonrigid (BSpline)- Problems with (too much) BSpline

Patient 1

MRI

Patient 2

MRI

July 2010

Registration Cases

Brain Intersubject OrientationFlx- Intersubject: nonrigid (BSpline)- Fixed: T1- Moving: T2- Problems with nested transformations

Patient 1 Patient 2

MRIT1

T2

T1

T2

July 2010

- Fixed: Colin27- Moving: Patient- Orientation!

Registration Cases

Brain Intersubject Dartmouth

Montreal Neurological Institue:

Colin27 for group analysis in MRI studies

Colin27

MRI

Patient

MRI

July 2010

Acknowledgements

Finally, I would like to thank everybody from CNI for the possibility to do an internship here!

Thanks to my supervisor Dominik S. Meier, PhD

July 2010

Questions?

?