kapitel 13 “interactive segmentation” – p. 1 interactive segmentation live-wire approach ...

Kapitel 13 “Interactive Segmentation” – p. 1

Interactive Segmentation

Live-wire approach

Random walker segmentation

Kapitel 13


Live-wire approach

Live-wire approach (intelligent scissors):

The user interactively picks a seed point on the boundary.

Then, a live-wire is displayed in real time from the initial point to any subsequent position taken by the cursor.

The entire 2D boundary is specified by means of a set of live-wire segments in this manner.

The detection of segments is formulated as a graph search problem, which finds the globally optimal (minimum-weight) path between an initial start pixel and an end pixel.


Live-wire approach

E.N. Mortensen and W.A. Barrett: Intelligent scissors for image composition. SIGGRAPH, 191-198, 1995.


Exkurs: Dijkstra‘s algorithm

Problem: finding shortest paths from a source vertex to all other vertices in a graph.

Dijkstra's algorithm: Greedy solution to the single-source shortest path problem. It works on both directed and undirected graphs. All edges must have nonnegative weights.

Input: Weighted graph G = {E, V} and start vertex s∈V, such that all edge weights are nonnegative. Output: Lengths of shortest paths (or the shortest paths themselves) from the given start vertex s∈V to an end node t∈V (or all other vertices).

Kapitel 13 “Interactive Segmentation” – p.

Step 1: Assign the start node with distance zero and mark it as visited. Assign all other vertices with infinity and mark them unvisited.

Step 2: Consider the most recently visited node X with distance Dx. For each of its unvisited neighbors Y with distance Dy, replace Dy by min(Dx+ weight(X,Y), Dy)

Step 3: Choose the unvisited vertex with the smallest distance and mark it visited.

Step 4: Repeat Steps 2 and 3 until the end node is marked visited.

Step 5: Go backwards through the graph, retracing the minimum-weight path from the end node to the start node.

5




Example: Find the minimum-weight path from node s to node t.

1

1

2

32

35

2

4

s

t

Kapitel 13 “Interactive Segmentation” – p.7


1

3

51

1

2

32

35

2

4

2 31

3

4

3

1

1

2

32

35

2

4

2

1

3

4

7

3

1

1

2

32

35

2

4

32

1

2

4

5

6

1

1

2

32

35

2

4

33 1

2

4

5

6

1

1

2

32

35

2

4

33

∞

∞

∞1

2

4

5

3

1

1

2

32

35

2

4

33


Exkurs: Search algorithm from AI

Search is a classical AI problem and many solutions exists (e.g. A*)


Live-wire approach

Example:


Live-wire approach

Example: Composition

E.N. Mortensen and W.A. Barrett: Intelligent scissors for image composition. SIGGRAPH, 191-198, 1995.


Live-wire approach

Example: A cursor snap mechanism forces the mouse point to the pixel of maximum gradient magnitude within a user-specified neighborhood. Live-wire segment snaps to a boundary as the free point moves (via cursor movement). The path of the free point is shown in white. Live-wire segments from previous free point positions are shown in green.


Live-wire approach

Example: Continuous snap–drag of a live-wire to a coronary edge (the boundary is completed in about 2s)

E.N. Mortensen and W.A. Barrett:Interactive live-wire boundaryextraction. Medical Image Analysis,1(4): 331-341.


Live-wire approach

Reproducibility for live-wire and manual tracing tool. Eight users extracted five object boundaries five times with the live-wire tool and the same object boundaries three times with the manually tracing tool.

Manual tracing takes 2-3 times longer.

The boundaries are virtually identical regardless of which user is performing the task!


Live-wire approach

Extension to 3D: segment 3D volume data or time sequences of 2D images.

A.X. Falcao and J.K. Udupa: A 3D generalization of user-steered live-wire segmentation. Medical Image Analysis, 4(4), 389-402, 2000.

The user specifies contours via live-wiring on a few slices that are orthogonal to the natural slices of the original data. If these slices are selected strategically, then one obtains a sufficient number of seed points in each natural slice which enable a subsequent automatic optimal boundary detection therein.



Given labeled pixels, for each pixel: What is the probability that a random walker starting from this pixel first reaches each set of labels?

L.Grady: Random walks for image segmentation. IEEE-TPAMI, 28: 1768–1783, 2006.

(a) A two-region image (b) Use-defined seeds for each region

(c) A 4-connected lattice topology

seeded (labeled) pixelsunseeded (unlabeled) pixel

edge weight: similarity between two nodes, based one.g., intensity gradient, color changes

(d) An undirected weighted graph

low-weight edge (sharp color gradient)



The algorithm labels an unseeded pixel in following steps:

Step 1. Calculate the probability that a random walker starting at an unseeded pixel x first reaches a seed with label s

Probability that a random walker starting from each unseeded node first reaches red seed

Probability that a random walker starting from each unseeded node first reaches blue seed

0.97 0.90 0.85

0.97

0.97 0.90

0.85

0.85

0.15 0.10 0.03

0.15

0.15 0.10

0.03

0.03

0.03 0.10 0.15

0.03

0.03 0.10

0.15

0.15

0.85 0.90 0.97

0.85

0.85 0.90

0.97

0.97



Step 2. Label each pixel with the most probable seed destination

(0.97,0.03) (0.90,0.10) (0.85,0.15) (0.15,0.85) (0.10,0.90) (0.03,0.97)

(0.97,0.03) (0.85,0.15) (0.15,0.85) (0.03,0.97)

(0.97,0.03) (0.90,0.10) (0.85,0.15) (0.15,0.85) (0.10,0.90) (0.03,0.97)

A segmentation corresponding to region boundary is obtained by biasing the random walker to avoid crossing sharp color gradients



green red yellow blue

partially labeled image segmented image

Probabilities



Algorithm summary:

1.Generate weights based on image intensities.

2.Build Laplacian matrix.

3.Solve system of linear equations for each label.

4.Assign pixel (voxel) to label for which it has the highest probability.



Weight generation:

The random walker is governed by edge weights (e.g., probabilities of reaching a pixel pj from its neighboring pixel pi).

f(): intensity function

β: influences how quickly the probability decreases

The probability is higher the more likely the two pixels belong to the same region.



Laplacian matrix L:

di : degree of pixel pi

It is shown that finding the probabilities for a walk starting at some pixel to arrive at some seed pixel is related to minmizing:



Solution of system of linear equations:

Partition the pixels into two sets, VM (all marked/seed pixels, regardless of

their label) and VU (unseeded). The pixels in L and x are ordered such

that seed pixels are first and unseeded pixels are second. Then, the optimization problem becomes:

LM: edge weighted among the marked pixels

LU: edge weighted among the unmarked pixels

Kapitel 13 “Interactive Segmentation” – p.

To minimize it, the equation is differentiated with respect to unknown XU:

This is set to zero for finding the minumum of D(XU), resulting a system of |VU| linear equations:

XM: 1 for the label under consideration, and 0 for all other labels(remember: we need to solve XU for each label)

23




Cardiac segmentation across modalities



Segmentation of objects with varying size, shape and texture



Segmentation of natural images



Author webpage: http://cns.bu.edu/~lgrady

Random walker paper: http://cns.bu.edu/~lgrady/grady2006random.pdf

Random walker MATLAB code: http://cns.bu.edu/~lgrady/random_walker_matlab_code.zip

Random walker demo page: http://cns.bu.edu/~lgrady/Random_Walker_Image_Segmentation.html

kapitel 13 “interactive segmentation” – p. 1 interactive segmentation live-wire approach ...

Documents

livewire approach example

dijkstras algorithm

aa kapitel

start node

node s

wire segments

set of live

dijkstras algorithm