dramms: deformable registration via attribute matching and...

DRAMMS: Deformable Registration via Attribute Matching and Mutual-Saliency weighting

Yangming Ou, Christos Davatzikos

Section of Biomedical Image Analysis (SBIA)University of Pennsylvania

2

Outline

1. Background 2. Motivations3. Framework4. Methods5. Results6. Discussions

3

Image Registration is the process of finding the optimal transformation that aligns different imaging data into spatial correspondence.[Maintz & Viergever’98, Lester & Arridge’99, Hill’01, Zitova’03, Pluim’03, Crum’04, Holden’08]

Source (Subject) Target (Template)S2T

(overlaid on T)Transformation

(Deformation Field)

1. Background – Definition of Registration

4

1. Background – Registration Literature

Division of Most Registration Methods:

Category 1

Landmark/feature-based

Category 2

voxel-wise (intensity-based)

DRAMMS

5

1. Background – Registration Literature (1)

Category 1: Landmark/feature-based methods[Davatzikos’96, Thompson’98, Rohr’01, Johnson’02, Shen’02, Joshi’00, Chui’03, ...]

[Figure from Rohl’03].

Pros: 1) Intuitive; 2) Fast;

Cons:1) Errors in landmark detection & matching;2) Task-specific: different registration tasks

need different landmark detection methods.

Expected: “General-purpose” registration methods!

brain heart breast prostate

brain Abrain B

heart Aheart B

breast Abreast B

prostate Aprostate B

Not suitable for general-purpose

6

1. Background – Registration Literature (2)

Category 2: (Intensity-based) Voxel-wise methods[Christensen'94, Collins'94, Thirion'98, Rueckert'99, Vercauteren'07, Glocker'08, ...]

Joint Histogram After Reg.Images Under Registration

Pro: General-purpose registration methods (only rely on intensities).Con: 1) 2) => motivations for DRAMMS

Assumption:

Consistentrelationship between intensity distributions

[figure from Rueckert’99]

A B

A B

[figures from Papademetris]

A2B

Bblack

blackwhite

white

A2B

Bblack whiteblack

white

7

Outline

1. Background2. Motivations3. Framework4. Methods5. Results6. Discussions

8

2. Motivations (1): Why?

Histological Image (Prostate)

MR Image (Same Prostate)

Challenge 1:No consistent relationship in intensity distributions

intensity-based voxel-wise methods (e.g. MI) fail.

blackblack blackwhite

BlackWhite

BlackMatching Ambiguity

Reason:Matching ambiguity <= characterizing voxels only by intensities.

Not Distinctive!

9

2. Motivations (1): How?

Proposed Solution to Challenge 1:

- To reduce matching ambiguity, 1-dim image intensity => high-dim attribute vector

Attribute Matching

DRAMMS

Similarity map (by attributes)

High similarity

Low similarity

10

2. Motivations (2)

Challenge 2: Partial loss of correspondence

Histological Image (Prostate)

MR Image (Prostate)

Inspiration 2:

A continuous weighting mechanism for all voxels:

- Weight high for voxels able to establish reliable correspondence;=> let them drive the registration

- Weight low for voxels not able to establish reliable correspondence. => reduce their negative impact to the registration

Normal Brain Brain w/ lesion

Mutual-Saliency weighting

DRAMMS

11

Outline


12

Framework

1. Attribute MatchingTo reduce matching ambiguities

2. Mutual-Saliency weightingTo account for loss of correspondence

Deformable Registration via

DRAMMS

u T(u)

A BT?

1. Attribute-Matching2. Mutual-Saliency

13

Outline

1. Background2. Motivations3. Framework4. Methods

4.1. Attribute Extraction and Selection4.2. Mutual-Saliency Weighting4.3. Implementation

5. Results6. Discussions

14

4.1.1. Attribute Extraction

Ideal Attributes1) Generally Applicable: to diverse registration tasks;2) Discriminative: voxels similar iff true correspondence.

Recent work[Shen and Davatzikos’01, Liu’02, Xue’04, Verma’04, Wu’07, etc]- Intensity attributes- Edge attributes- Tissue membership attributes (based on segmentation)- Geometric moment invariant attributes- Wavelet attributes- Local histogram attributes

15

4.1.1 Attribute Extraction – Gabor Attributes

x

A(0)(x)

A(1)(x)

A(2)(x)

A(3)(x)

A(x)

Gabor filter bank (multi-scale, multi-orientation)

16

Why Gabor Attributes?

Reason 0: Sometimes, maybe one reason is enough ☺

Dennis Gabor (1900-1979)

Nobel Prize in Physics (1971)

17


Reason 1/3: General Applicability

Success in Texture segmentation [e.g., Jain’91];Cancer detection [e.g., Zhang’04];Prostate tissue differentiation [e.g., Zhan’06]; Brain registration tasks [e.g., Liu’02, Verma’04, Elbacary’06];…

brain heart breast prostate

18


Reason 2/3: Multi-scale and Multi-orientation.characterize voxels distinctively

Original Image

orientation

scale

Gabor Attributes

scale

orientation

19


Reason 3/3: Suitable for Registration

High Freq.Gabor Attributes

Edge maps => relatively independent of intensity distributions

Low Freq.Gabor Attributes

Smoothed (coarse) version => reduce local minimum in reg..

Original Image

20

Gabor Attributes characterize voxels distinctively

Special Points Ordinary Points

21

4.1.2. Select Optimal Gabor Attributes

Why? 1) Non-orthogonality among Gabor filters

redundancy;2) Attribute vector A( ) being too long

computational expensive.

How?Step 1: Select training voxel pairs:

Step 2: Select attribute on training voxel pairs:Training voxel pairs

by iterative backward elimination and forward inclusion.

Mutual-Saliency

Mutual-Saliency

22

Role of Gabor Attributes and Optimal Gabor Attributes

distinctiveness

23

Role of Gabor Attributes and Optimal Gabor Attributes

distinctiveness

24

Outline


4.1. Attribute Extraction and Selection4.2. Mutual-Saliency Weighting4.3. Implementation


25

Framework

1. Attribute MatchingTo reduce matching ambiguities



DRAMMS

u T(u)

A BT?

1. Attribute-Matching2. Mutual-Saliency

26

4.2. Mutual-Saliency weighting

Recent work [Bond’05, Wu’07, Mahapha’08]

Their approach: Higher weights for more salient regions

Their assumption: Salient regions more likely to establish reliable correspondence.

Saliency in one image=> Matching reliability between two images?

A counter-example

Our work: saliency in one image => mutual-saliency b/w two images

Directly measure matching reliability[Anandan’89, McEache’97]

27

4.2. Mutual Saliency (MS) weighting

Idea:True correspondence should

Calculation of MS:

similarity

u T(u)

similar to each other;not similar to anything else.

where

Delta fun.

Reliable matching

High MS value

28

Role of Mutual-Saliency Map

Account for partial loss of correspondence

Source image Target image

Registrationwithout MS map

Registrationwith MS map

MS map

29

Outline


4.1. Attribute Extraction and Selection4.2. Mutual-Saliency Weighting 4.3. Implementation


30

4.3. Implementation

Optimized and regularized by Free Form Deformation (FFD) model [Rueckert’99]

Diffeomorphism FFD [Rueckert’06]

Multi-resolution to reduce local minimaGradient descent optimizationImplemented in CRun on 2.8G CPU, Unix OS

31

Outline

1. Background2. Motivations3. Framework4. Methods5. Results

5.1. Cross-subject registration;5.2. Multi-modality registration;5.3. Longitudinal registration;5.4. Atlas construction.

6. Discussions

32

5.1. Cross-subject Registrations

A (Subject) B (Template)

Brain

Cardiac

Evaluate registration accuracy by mean sq. diff. (MSD) and corr. coef. (CC)

between registered image and target image.

A2B

A2B deformation

deformation

33

5.1. Cross-subject Registrations

Observations:

1) In images that intensity-based method can register, attribute matching increased registration accuracy considerably;

2) Each of DRAMMS’ components provides additive improvement for registration accuracy.

34

5.2. Multi-modality Registrations

Human ProstateHistology MR Histology2MR Mutual-saliency

MR

Histological

Joint histogram after registration

35

5.2. Multi-modality Registrations

Mouse Brain

Histology MR Histology2MR Mutual-saliency

Joint histogram after registration

36

5.3. Longitudinal Registration

37

5.4. Atlas Construction

Images from 30 training subjects

template

38

5.4 Atlas Construction (cont.)

By intensity-based FFD (mutual-information)

By DRAMMS

39

5.4. Atlas Construction (cont.)

Lesion

Low MS weight

40

5.4. Atlas Construction (cont.)

Mean Mutual-Saliency Map in 3D

41

Outline


42

Discussions

DRAMMS: a general-purpose registration method;DiffeomorphismImproves MI-based methods, especially when 1) no consistent relationship between intensity distributions;2) loss of correspondence

43

DRAMMS

A bridge between two categories of methods

Category 1

Landmark/feature-based

Category 2

voxel-wise (intensity-based)

DRAMMS

44

DRAMMS – bridge 1

Category 1

Landmark-based

Category 2

voxel-wise

DRAMMSAttribute Matching

Still using all voxelsEvery voxel will become a landmark to some extent.

45

DRAMMS – bridge 2

Category 1

Landmark-based

Category 2

voxel-wise

DRAMMSMutual-Saliency weighting

Weight = 1 for all voxelsWeight =

1 for landmarks

0 otherwise

46

Take-home message

1. (optimal) Attribute MatchingTo reduce matching ambiguities



DRAMMS

47

Thank you!

Code to be available:

(Lab) https://www.rad.upenn.edu/sbia/

(Personal)https://www.rad.upenn.edu/sbia/Yangming.Ou/

dramms: deformable registration via attribute matching and...

Documents