Label Inference Encoded with Local and Global Patch PriorsSiqi Bao and Albert C. S. Chung

{sbao, achung}@cse.ust.hk, Lo Kwee-Seong Medical Image Analysis Laboratory,Department of Computer Science and Technology, Hong Kong University of Science and Technology

The Hong Kong University of Science and Technology

1. Introduction and Motivation§ Reliable and accurate subcortical structure segmentation in brain

magnetic resonance (MR) images has been playing a significant role

in clinical diagnosis, treatment planning and therapeutic assessment.

§ Hippocampus segmentation in brain MR images is extremely crucial

in predicting the progression of Alzheimer's disease (AD). There are

approximately 5.3 million Americans living with AD in 2015.

§ Atlas based segmentation first performs non-rigid image registration

between atlas and the target unlabeled image, and then propagates

the label map from the corresponding atlas to the target image.

§ Due to the serious overlap of intensity profiles among different

tissues, the conventional patch prior can be adversely impacted and

become misleading during the final label inference procedure.

2. Methodologyv Background

3. Experiments

ZoomIn Fig. 1. Comparison of segmentation results for left Hippocampus by ANTs (Blue curve) & manually labelled ground truth (Red curve).

The label inference problem is formulated on an undirected graph, 𝑮 = (𝑽, 𝑬). Nodes set 𝑽 consists of 3 subsets (foreground seeds 𝑽𝑭, background seeds 𝑽𝑩 and candidate nodes 𝑽𝑪) and is chosen from pixels in the target image by evaluating their signed distances 𝒅𝒊.

𝝆 ≤ 𝒅 ≤ 𝝆 + 𝜺

𝒅 = 𝟎

−(𝝆 + 𝜺) ≤ 𝒅 ≤ −𝝆

𝒘𝒊𝑭 𝒘𝒊𝑩

𝒘𝒊𝒋

𝒗𝒊𝑭 𝑩

𝒗𝒋

Fig. 2. Left: Node selection. Right: Graph construction of label inference. Blue nodes: candidate nodes; Red and Black nodes: foreground and background seeds.

For each candidate node 𝒗𝒊, its connecting edges fall into 2 categories: connection with its neighbors through image lattice and the direct linkage with graph seeds.

Lattice connection:

Non-rigid registration prior:

v Local Patch Prior with Sparse LearningFor the target patch, we seek for an optimal sparse combination of atlas patches by solving the following sparse learning problem:

where 𝒇 is a vector converted from the 3-D target patch 𝑷𝟏 𝒗𝒊 and 𝑨is a matrix which collects small patches inside larger atlas patches 𝑷𝟐 𝒗𝒊 , acting as dictionary bases in sparse learning. 𝜷 includes sparse coefficients for each atlas patch and 𝝀 is one tuning parameter.

Local Patch Prior:

For 𝜷𝒌 ≠ 𝟎, 𝒄𝒌 ∈ {𝟎, 𝟏}, which measures the consistency between the sign of 𝜷𝒌 and label of node 𝒗𝒊𝒌.

v Global Patch Prior with CNN

v Label Inference

Global Patch Prior:

𝟐𝟎×𝟐𝟎

Input Output

𝐿EFGFH Nodes Selection 𝑣J

Registration Priors 𝑤JLM ,𝑤JNM

Lattice Connection 𝑤JO

Sparse Learning

Random Walker

𝐿E𝑥J𝐼E

𝐼R, 𝐿R

Signed Distance

Gaussian Function

CNN

Local Patch Priors 𝑤JLS ,𝑤JNS

Global Patch Priors 𝑤JLT ,𝑤JNT

Fig. 3. CNN architecture to capture global prior. Gray square: input patch with a size of 𝟐𝟎×𝟐𝟎; Blue cube: convolution layers; Pink cube: polling layers; Green strip: fully connected layers; Red and Black neurons: probability as foreground and background, respectively.

Besides local patch prior, the high-level structural properties of each subcortical structure are also taken into consideration and global patch prior is extracted with Convolutional Neural Networks.

With the unique solution obtained for 𝒙𝒊, the labels of candidate nodes can be updated: 𝑳 𝒗𝒊 = 𝟏 if 𝒙𝒊 ≥

𝟏𝟐

and 𝑳 𝒗𝒊 = 𝟎 otherwise.

0.65

0.7

0.75

0.8

0.85

0.9

0.95

Tha Cau Puta Pal Hip Amy Ave

Segmentation results on IBSR database

MV

WV

SP

Our Method

0.78

0.8

0.82

0.84

0.86

0.88

Puta Cau Hip Ave

Segmentation results on LPBA40 database

MV

WV

SP

Our Method

* We would like to acknowledge the financial support by the Hong Kong Research Grants Council under Grant 16203115.

Evaluate our method on two publicly available MR brain databases and compare with Majority Voting (MV), Weighted Voting (WV) and intensity similarity as patch prior (SP) methods.