GEORGIA INSTITUTE OF TECHNOLOGY

Image Segmentation Using The Geodesic And

Chan-Vese Models ECE 6560

Bruno Pop-Stefanov

29/04/2013

In this paper we compare a classic edge-based active contour method to a region-based method.

1

Content 1. Introduction ....................................................................................................................................... 2

2. Mathematical Formulation ............................................................................................................ 2

2.1. The Geodesic Active Contours Model ................................................................................... 2

2.1.1. Energy Functional ............................................................................................................ 2

2.1.2. Partial Differential Equation ......................................................................................... 3

2.1.3. Interpretation .................................................................................................................... 5

2.2. The Chan-Vese Model .............................................................................................................. 6

2.2.1. Energy Functional ............................................................................................................ 6

2.2.2. Partial Differential Equation ......................................................................................... 6

2.2.3. Interpretation .................................................................................................................. 11

3. Implementation ............................................................................................................................... 11

3.1. Discretization ........................................................................................................................... 12

3.1.1. The Geodesic Active Contours Model ......................................................................... 12

3.1.2. The Chan-Vese Model .................................................................................................... 13

3.2. Managing Level Sets .............................................................................................................. 13

4. Experimental results ..................................................................................................................... 14

4.1. “Easy” Examples ..................................................................................................................... 15

4.2. “Hard” Examples..................................................................................................................... 16

5. Discussion and Improvements ..................................................................................................... 18

6. References ........................................................................................................................................ 18

7. Code Documentation ...................................................................................................................... 18

2

1. Introduction

In computer vision, image segmentation is one of the oldest and most studied problems (Szeliski, 2010). Segmentation is the process of partitioning an image into a set of distinct regions. It is a fundamental step before any high-level process. For example, before trying to recognize an object in a scene it must first be detected, i.e. its boundary must be found.

Traditional segmentation techniques include thresholding, clustering, and edge detection. In this paper, we focus on algorithms based on active contours and level sets. In particular we take a closer look at two different kinds of active contours.

The geodesic model is an edge-based active contour model. The energy functional upon which it relies is calculated over the length of the contour, and thus does not consider what is inside the contour. The Chan-Vese model (Chan & Vese, 2001) is a region-based active contour model. Its energy functional is calculated over the entire image domain, inside the contour and outside.

2. Mathematical Formulation

The mathematical principle behind active contours is to start with an initial contour 𝐶(𝑡 = 0) and deform it in time until we reach the boundary of the object we are trying to find. At each time 𝑡, an energy 𝐸 = 𝐸�𝐶(𝑡)� is associated to 𝐶(𝑡). This energy is a function (more precisely, a functional) of the contour and its expression is defined by the model we use. Making the contour evolve is equivalent to changing its energy. More precisely, the energy functional is defined so that it is minimal when the contour is exactly the boundary of the object. Thus, starting from an initial contour we apply the gradient descent method to reach a (local) minimum.

In the following, Ω ⊂ ℝ × ℝ is the image domain and 𝐼:Ω → ℝ is the image function.

2.1. The Geodesic Active Contours Model

A geodesic is a minimal distance curve. The minimal distance curve lies in a Riemannian space whose metric is defined by the image content (Caselles, Kimmel, & Sapiro, 1997).

2.1.1. Energy Functional

Its energy functional is computed over the length of the curve such that,

3

𝐸(𝐶) = � Φ𝑑𝑠

𝐶

(1)

where Φ:Ω → (0,1] is defined as

Φ(𝑥,𝑦) =1

1 + ‖∇𝐼(𝑥,𝑦)‖ , ∀(𝑥,𝑦) ∈ Ω (2)

Φ indicates the presence of an edge in the image. An edge is characterized by a non-zero gradient; the sharper the edge, the greater the norm of the gradient. Therefore, a strong edge will yield a value of Φ close to zero, whereas a flat surface will yield a value of Φ close to one. The energy is consequently low when the contour is on an edge and high on a flat surface.

2.1.2. Partial Differential Equation

The evolution of the contour is given by a gradient descent partial differential equation:

𝜕𝐶𝜕𝑡

= −∇𝐸(𝐶) (3)

We define the L2 geometric inner product for two vectors 𝛼1𝑁 and 𝛼2𝑁 (where 𝑁 is the unit normal to the curve) as,

⟨𝛼1𝑁|𝛼2𝑁⟩ = �(𝛼1𝑁) ∙ (𝛼2𝑁)𝑑𝑠

𝐶

(4)

Given a perturbation vector field 𝐶𝑡 on 𝐶, the gradient of the energy is defined by the following relationship,

⟨∇𝐸|𝐶𝑡⟩ =𝜕𝐸𝜕𝐶𝑡

=𝑑𝑑𝑡𝐸�𝐶(𝑡)�

(5) Let us therefore take the derivative of 𝐸 with respect to 𝑡 and factor 𝐶𝑡 inside the integral:

𝑑𝑑𝑡𝐸�𝐶(𝑡)� =

𝑑𝑑𝑡� Φ𝑑𝑠𝐶

(6)

The first remark is that we cannot move the derivative with respect to 𝑡 inside the integral because the integral depends on 𝐶, which depends on 𝑡. Although 𝐸 does not depend on the parameterization of the curve, we can choose one parameterization which will yield the

4

same results and allows us to take the derivative inside the integral. The solution is thus to switch the geometric 𝑠-parameter, which depends on 𝑡, with an arbitrary parameterization 𝑝, which is independent of 𝑡. We assume 𝑝 ∈ [0,1).

Since 𝑑𝑠 = �𝐶𝑝�𝑑𝑝, we can write,

𝑑𝑑𝑡𝐸�𝐶(𝑡)� =

𝑑𝑑𝑡�Φ�𝐶𝑝�𝑑𝑝1

0

= �𝑑𝑑𝑡�Φ�𝐶(𝑝, 𝑡)��𝐶𝑝��𝑑𝑝

1

0

= ��∇Φ ∙ 𝐶𝑡�𝐶𝑝� + Φ𝐼�𝐶𝑝�𝑡� 𝑑𝑝1

0

(7)

(8)

(9)

Note that,

�𝐶𝑝�𝑡 = ��𝐶𝑝 ∙ 𝐶𝑝�𝑡

=𝐶𝑝𝑡 ∙ 𝐶𝑝�𝐶𝑝�

(10)

Additionally, 𝑝 and 𝑡 are mutually independent and we can therefore change the order of the derivatives in 𝑝 and 𝑡 such that 𝐶𝑝𝑡 = 𝐶𝑡𝑝. We can further note that 𝐶𝑝

�𝐶𝑝�= 𝑇, the unit

tangent to the curve. The relationship becomes,

𝜕𝐸𝜕𝐶𝑡

= ��∇Φ ∙ 𝐶𝑡�𝐶𝑝� + Φ𝐶𝑡𝑝 ∙ 𝑇�𝑑𝑝1

0

= ��∇Φ ∙ 𝐶𝑡�𝐶𝑝� + Φ𝐶𝑡𝑠�𝐶𝑝� ∙ 𝑇�𝑑𝑝1

0

(11)

(12)

We can now switch back to the 𝑠-parameter:

𝜕𝐸𝜕𝐶𝑡

= � [∇Φ ∙ 𝐶𝑡 + 𝐶𝑡𝑠 ∙ (Φ𝑇)]𝑑𝑠𝐶

(13)

The next step is to peel off the 𝑠-derivative from 𝐶𝑡𝑠 in order to factor 𝐶𝑡:

𝜕𝐸𝜕𝐶𝑡

= � [∇Φ ∙ 𝐶𝑡 − 𝐶𝑡 ∙ (Φ𝑇)𝑠]𝑑𝑠𝐶

(14)

Note that,

5

(Φ𝑇)𝑠 = (∇Φ ∙ T)𝑇 + Φ𝑇𝑠 (15) and,

𝑇𝑠 = −𝜅𝑁 (16) where 𝜅 is the curvature. The relationship becomes,

𝜕𝐸𝜕𝐶𝑡

= � 𝐶𝑡 ∙ [∇Φ − (∇Φ ∙ T)𝑇 + Φ𝜅𝑁]𝑑𝑠𝐶

= � 𝐶𝑡 ∙ [(∇Φ ∙ N)𝑁 + Φ𝜅𝑁]𝑑𝑠𝐶

(17)

(18)

The gradient of the energy is therefore:

∇𝐸 = [(∇Φ ∙ N) + Φ𝜅]𝑁 (19) We deduce the gradient descent partial differential equation that rules the evolution of 𝐶 according to the geodesic active contour model:

𝐶𝑡 = −Φ𝜅𝑁 − (∇Φ ∙ N)𝑁 (20) In the actual implementation of the equation, we added a morphological dilation/erosion term 𝛼𝑁 which allows the contour to shrink if it is far from the image boundary or expand it is inside the boundary. We also modulate this morphological term with the edge function Φ. The final equation is, for 𝑁 the outward unit normal:

𝐶𝑡 = −Φ𝛼𝑁 −Φ𝜅𝑁 − (∇Φ ∙ N)𝑁 (21) The level-set equivalent is:

ψ𝑡 = Φ ∇ ∙ �∇ψ‖∇ψ‖

� ‖∇ψ‖ + Φ𝛼‖∇ψ‖ + ∇ψ ∙ ∇Φ (22)

2.1.3. Interpretation

Φ𝜅 is a rigidity constraint. It makes the contour smoother by reducing its length. If the curvature in a point is too large, then the contour will straighten up. Φ𝛼 is a propagation term, as explained above. ∇Φ indicates the direction of the edge. On either side of the boundary this term will pull the contour towards the edge.

6

2.2. The Chan-Vese Model

As opposed to the geodesic model, the Chan-Vese model exploits global information about the image. Its energy functional is computed over the entire domain and is designed to penalize the mismatch between the inside and the outside of the contour. A low energy corresponds to a curve which inside and outside are homogeneous and have distinct mean intensities.

2.2.1. Energy Functional

For a given curve 𝐶 over the image domain Ω, which inside is 𝑅 and outside is 𝑅𝑐, the energy of 𝐶 is given by,

𝐸(𝐶) = 𝜇 𝐿𝑒𝑛𝑔𝑡ℎ(𝐶) + 𝜈 𝐴𝑟𝑒𝑎(𝑅) + 𝜆1�(𝐼(𝑥, 𝑦) − 𝑢)2 𝑑𝑥 𝑑𝑦

𝑅

+ 𝜆2�(𝐼(𝑥,𝑦) − 𝑣)2 𝑑𝑥 𝑑𝑦𝑅𝑐

(23)

where 𝜇, 𝜈, 𝜆1 and 𝜆2 are fixed, non-negative parameters. 𝑢 and 𝑣 are two functions of 𝐶 defined as the mean intensity inside 𝐶 and outside 𝐶, respectively.

In this expression, the first term penalizes the total length of the curve. The second term penalizes the total area inside the curve. The two last terms, 𝐹1 and 𝐹2, respectively, measure how “uniform” the regions delimited by 𝐶 are.

Usually, the parameters are taken as 𝜇 = 1, 𝜈 = 0, and 𝜆1 = 𝜆2 = 1. We will assume it is so in the following.

2.2.2. Partial Differential Equation

For the sake of simplification we will assume in this paragraph that the energy functional is expressed as

𝐸(𝐶) = �𝑓(𝑥,𝑦) 𝑑𝑥 𝑑𝑦

𝑅

(24)

Then, we will deduce the gradient of the energy for the Chan-Vese model given that the energy functional for the Chan-Vese model can be written as

𝐸𝐶𝑉(𝐶) = �𝑓1(𝑥,𝑦)𝑑𝑥 𝑑𝑦

𝑅

+ �𝑓2(𝑥,𝑦) 𝑑𝑥 𝑑𝑦𝑅𝑐

(25)

7

Let us deduce the expression of ∇𝐸 from 𝐸(𝐶) (24):

𝜕𝐸𝜕𝐶𝑡

=𝑑𝑑𝑡�𝑓(𝑥,𝑦)𝑑𝑥 𝑑𝑦𝑅

(26)

We can’t take the derivative with respect to 𝑡 inside the integral because 𝑅 depends on 𝑡. We have to switch the 2-D integral for an integral along the curve and then substitute the 𝑠-parameter for a parameter 𝑝 that does not depend on 𝑡. There exists a vector field �⃗� such

that ∇ ∙ �⃗� = 𝑓 (for example, choose �⃗� = �𝐹1𝐹2� such that 𝐹1𝑥 + 𝐹2𝑦 = 𝑓). Using the divergence

theorem, we can rewrite (26) as,

𝜕𝐸𝜕𝐶𝑡

=𝑑𝑑𝑡� �⃗� ∙ 𝑁 𝑑𝑠𝐶

(27)

Let us now choose an arbitrary parameterization 𝑝 ∈ [0,1). Substituting 𝑠 for 𝑝 yields:

𝜕𝐸𝜕𝐶𝑡

=𝑑𝑑𝑡���⃗� ∙ 𝑁� �𝐶𝑝� 𝑑𝑝1

0

= � 𝑑𝑑𝑡� ��⃗� ∙ 𝑁� �𝐶𝑝� � 𝑑𝑝

1

0

= � 𝑑𝑑𝑡��⃗� ∙ 𝑁� �𝐶𝑝� + ��⃗� ∙ 𝑁��𝐶𝑝�𝑡 𝑑𝑝

1

0

(28)

(29)

(30)

Using (10) we obtain,

𝜕𝐸𝜕𝐶𝑡

= � 𝑑𝑑𝑡��⃗� ∙ 𝑁� �𝐶𝑝� + ��⃗� ∙ 𝑁� �𝐶𝑝𝑡 ∙ 𝑇� 𝑑𝑝

1

0

= � 𝑑𝑑𝑡��⃗� ∙ 𝑁� �𝐶𝑝� + ��⃗� ∙ 𝑁� �𝐶𝑡𝑝 ∙ 𝑇� 𝑑𝑝

1

0

= � �𝑑𝑑𝑡��⃗� ∙ 𝑁� + ��⃗� ∙ 𝑁� (𝐶𝑡𝑠 ∙ 𝑇)��𝐶𝑝� 𝑑𝑝

1

0

(31)

(32)

(33)

We may now substitute 𝑝 for 𝑠:

𝜕𝐸𝜕𝐶𝑡

= � �𝑑𝑑𝑡��⃗� ∙ 𝑁� + ��⃗� ∙ 𝑁� (𝐶𝑡𝑠 ∙ 𝑇)� 𝑑𝑠

𝐶

(34)

8

Note that:

𝑑𝑑𝑡��⃗� ∙ 𝑁� =

𝜕�⃗�𝜕𝑡

∙ 𝑁 + �⃗� ∙𝜕𝑁𝜕𝑡

𝑁𝑡 =𝜕𝜕𝑡

(𝐽𝑇) = 𝐽𝑇𝑡 = 𝐽 �𝐶𝑝�𝐶𝑝�

�𝑡

(35)

(36)

where 𝐽 = � 0 1

−1 0� is a rotation matrix.

�𝐶𝑝�𝐶𝑝�

�𝑡

= 𝐶𝑡𝑠 − (𝐶𝑡𝑠 ∙ 𝑇) 𝑇

= (𝐶𝑡𝑠 ∙ 𝑁) 𝑁

(37)

(38) Thus,

𝑁𝑡 = (𝐶𝑡𝑠 ∙ 𝑁) (𝐽𝑁) = −(𝐶𝑡𝑠 ∙ 𝑁) 𝑇

(39)

Additionally,

𝜕�⃗�𝜕𝑡

= �𝜕�⃗�𝜕�⃗�� 𝜕�⃗�𝜕𝑡

= �𝜕�⃗�𝜕�⃗�� 𝐶𝑡

(40)

Equation (35) now becomes,

𝑑𝑑𝑡��⃗� ∙ 𝑁� = ��

𝜕�⃗�𝜕�⃗�� 𝐶𝑡� ∙ 𝑁 − (𝐶𝑡𝑠 ∙ 𝑁)��⃗� ∙ 𝑇�

= 𝐶𝑡 ∙ ��𝜕�⃗�𝜕�⃗��𝑇

𝑁� − (𝐶𝑡𝑠 ∙ 𝑁)��⃗� ∙ 𝑇�

(41)

The derivative of the energy is therefore:

𝜕𝐸𝜕𝐶𝑡

= � �𝐶𝑡 ∙ ��𝜕�⃗�𝜕�⃗��𝑇

𝑁� − 𝐶𝑡𝑠 ∙ ���⃗� ∙ 𝑇�𝑁 − ��⃗� ∙ 𝑁� 𝑇�� 𝑑𝑠𝐶

(42)

We can now integrate by parts in order to peel off the 𝑠-derivative from 𝐶𝑡𝑠:

9

𝜕𝐸𝜕𝐶𝑡

= � 𝐶𝑡 ∙ ��𝜕�⃗�𝜕�⃗��𝑇

𝑁 + ���⃗� ∙ 𝑇�𝑁�𝑠− ���⃗� ∙ 𝑁� 𝑇�

𝑠� 𝑑𝑠

𝐶

(43)

Recall Frenet’s formulas:

𝑇𝑠 = 𝐶𝑠𝑠 = −𝜅𝑁 𝑁𝑠 = 𝜅𝑇

(44)

Let us examine the two derivatives in 𝑠:

���⃗� ∙ 𝑇�𝑁�𝑠

= ��⃗� ∙ 𝑇�𝑁𝑠 + ��⃗� ∙ 𝑇𝑠�𝑁 + ��⃗�𝑠 ∙ 𝑇�𝑁

= 𝜅��⃗� ∙ 𝑇�𝑇 − 𝜅��⃗� ∙ 𝑁�𝑁 + ��𝜕�⃗�𝜕�⃗�� 𝐶𝑠 ∙ 𝑇�𝑁

(45)

and,

���⃗� ∙ 𝑁�𝑇�𝑠

= ��⃗� ∙ 𝑁�𝑇𝑠 + ��⃗� ∙ 𝑁𝑠�𝑇 + ��⃗�𝑠 ∙ 𝑁�𝑇

= −𝜅��⃗� ∙ 𝑁�𝑁 + 𝜅��⃗� ∙ 𝑇�𝑇 + ��𝜕�⃗�𝜕�⃗�� 𝐶𝑠 ∙ 𝑁�𝑇

(46)

After simplification:

𝜕𝐸𝜕𝐶𝑡

= � 𝐶𝑡 ∙ ��𝜕�⃗�𝜕�⃗��𝑇

𝑁 + ��𝜕�⃗�𝜕�⃗�� 𝑇 ∙ 𝑇�𝑁 − ��

𝜕�⃗�𝜕�⃗�� 𝑇 ∙ 𝑁� 𝑇� 𝑑𝑠

𝐶

(47)

Note that:

�𝜕�⃗�𝜕�⃗��𝑇

𝑁 = ���𝜕�⃗�𝜕�⃗��𝑇

𝑁 � ∙ 𝑁�𝑁 + ���𝜕�⃗�𝜕�⃗��𝑇

𝑁 � ∙ 𝑇�𝑇

= �𝑁𝑇 �𝜕�⃗�𝜕�⃗�� 𝑁�𝑁 + �𝑁𝑇 �

𝜕�⃗�𝜕�⃗�� 𝑇�𝑇

(48)

Therefore, after simplification,

𝜕𝐸𝜕𝐶𝑡

= � 𝐶𝑡 ∙ �𝑁𝑇 �𝜕�⃗�𝜕�⃗�� 𝑁 + 𝑇𝑇 �

𝜕�⃗�𝜕�⃗�� 𝑇�𝑁 𝑑𝑠

𝐶

= � 𝐶𝑡 ∙ �𝑡𝑟𝑎𝑐𝑒 ��𝜕�⃗�𝜕�⃗����𝑁 𝑑𝑠

𝐶

(49)

(50)

Given,

10

�𝜕�⃗�𝜕�⃗�� = �

𝐹1𝑥 𝐹1𝑦𝐹2𝑥 𝐹2𝑦

�

(60) We deduce

𝑡𝑟𝑎𝑐𝑒 ��

𝜕�⃗�𝜕�⃗��� = 𝐹1𝑥 + 𝐹2𝑦 = ∇ ∙ �⃗� = 𝑓

(61)

Finally, we conclude that:

𝜕𝐸𝜕𝐶𝑡

= � 𝐶𝑡 ∙ (𝑓𝑁)𝑑𝑠𝐶

(62)

Therefore, the gradient descent flow is given by,

𝐶𝑡 = −𝑓𝑁 (63) where N is the unit normal. The level-set equivalent is:

ψ𝑡 = 𝑓‖∇ψ‖ (64) where ψ is the level set representing the curve (at level zero).

From that expression we deduce the gradient descent partial differential equation for the Chan-Vese model. Using the Heaviside function 𝐻 and the Dirac measure 𝛿0 respectively defined by

𝐻(𝑧) = �1, 𝑖𝑓 𝑧 ≥ 00, 𝑖𝑓 𝑧 < 0, 𝛿0(𝑧) =

𝑑𝑑𝑧𝐻(𝑧),

(65) the first term in the energy, 𝐿𝑒𝑛𝑔𝑡ℎ(𝐶), can be expressed as

𝐿𝑒𝑛𝑔𝑡ℎ(ψ = 0) = ��∇𝐻�ψ(𝑥,𝑦)��𝑑𝑥𝑑𝑦

Ω

= �𝛿0�ψ(𝑥,𝑦)�∇ψ(𝑥,𝑦)𝑑𝑥𝑑𝑦Ω

(66)

(67)

This term yields a mean curvature motion term in the Euler-Lagrange equation associated with 𝐸:

𝜇 𝛿0(ψ) ∇ ∙ �∇ψ‖∇ψ‖

� ‖∇ψ‖ (68)

The entire partial differential equation, expressed in terms of level sets, is:

11

ψ𝑡 = 𝛿0(ψ) �𝜇 ∇ ∙ �∇ψ‖∇ψ‖

� ‖∇ψ‖ − 𝜆 ‖∇ψ‖(𝐼 − 𝑢)2 + 𝜆 ‖∇ψ‖(𝐼 − 𝑣)2� (69)

2.2.3. Interpretation

The first term resembles the geometric heat equation. This mean curvature motion term makes the curve smoother. The second and third term can be combined as:

𝐹 = 𝜆 ‖∇ψ‖ [(𝐼 − 𝑣)2 − (𝐼 − 𝑢)2] = 2 𝜆 ‖∇ψ‖ (𝑢 − 𝑣) �𝐼 − 𝑢

2+𝐼 − 𝑣

2�

(70) F is called the “fitting” term. Assume the image is formed by two regions of approximately constant but different intensities (Figure 1). The object to be detected is represented by one of these two regions. Four situations are possible. (1) 𝐶 encircles the object: 𝐼 = 𝑣, 𝑢 > 𝑣, 𝐼 < 𝑢. Then 𝐹 tends to shrink the contour towards the object. (2) 𝐶 is completely inside the object: 𝐼 = 𝑢, 𝐼 > 𝑣, 𝑢 > 𝑣. Then 𝐹 tends to expand the contour towards the object boundary. (3) 𝐶 fits the object: 𝑢 > 𝑣 and 𝐼 ≈ 𝑢+𝑣

2. Then 𝐹 = 0 and 𝐶 does not change. (4) 𝐶 is both

inside and outside the object. Contour points inside the object will tend to expand, whereas contour points outside the object will tend to shrink.

3. Implementation

In order to solve the PDEs given in (22) and (69), we must discretize them. In paragraph 3.1., we discuss how to discretize each term so as to guarantee that the discretized PDE is

Figure 1 – All possible cases in the position of the curve

12

stable. We implemented each PDE using level set methods. We discuss the challenges of dealing with level sets in paragraph 3.2.

3.1. Discretization

In both models we use forward finite differences in time:

𝜓𝑡(𝑥, 𝑦, 𝑡) =𝜓(𝑥,𝑦, 𝑡 + Δ𝑡) − 𝜓(𝑥,𝑦, 𝑡)

Δ𝑡+ 𝑂(Δ𝑡)

(71)

3.1.1. The Geodesic Active Contours Model

The level set PDE (22) contains a diffusion term, a non-linear term, and a linear transport term. The diffusion term can be re-written:

∇ ∙ �

∇ψ‖∇ψ‖

� ‖∇ψ‖ =𝜓𝑥2𝜓𝑦𝑦 − 2𝜓𝑥𝜓𝑦𝜓𝑥𝑦 + 𝜓𝑦2𝜓𝑥𝑥

𝜓𝑥2 + 𝜓𝑦2

(72)

Each term in (72) should be discretized using central differences whenever possible (i.e., everywhere but on the image boundary):

𝜓𝑥(𝑥,𝑦, 𝑡) =𝜓(𝑥 + ∆𝑥,𝑦, 𝑡) − 𝜓(𝑥 − ∆𝑥, 𝑦, 𝑡)

2∆𝑥+ 𝑂(∆𝑥2)

𝜓𝑦𝑦(𝑥,𝑦, 𝑡) =𝜓(𝑥,𝑦 + ∆𝑦, 𝑡) − 2𝜓(𝑥,𝑦, 𝑡) + 𝜓(𝑥,𝑦 − ∆𝑦, 𝑡)

∆𝑦2+ 𝑂(∆𝑥2)

𝜓𝑥𝑦(𝑥,𝑦, 𝑡) ≈𝜓(𝑥 + ∆𝑥, 𝑦 + ∆𝑦, 𝑡) − 𝜓(𝑥 − ∆𝑥,𝑦 + ∆𝑦, 𝑡)

4∆𝑥∆𝑦

−𝜓(𝑥 + ∆𝑥, 𝑦 − ∆𝑦, 𝑡) − 𝜓(𝑥 − ∆𝑥,𝑦 − ∆𝑦, 𝑡)

4∆𝑥∆𝑦

(73)

The non-linear term,

Φ 𝛼 ‖∇ψ‖ = Φ 𝛼 �𝜓𝑥2 + 𝜓𝑦2,

(74) should be discretized using the entropy upwind scheme, ∀(𝑥,𝑦) ∈ Ω:

if 𝛼 Φ(𝑥,𝑦) > 0,

then Φ 𝛼 ‖∇ψ‖ = Φ 𝛼 �max(0,𝜓𝑥+) + min(0,𝜓𝑥−) + max�0,𝜓𝑦+� + min�0,𝜓𝑦−� if 𝛼 Φ(𝑥,𝑦) < 0,

then Φ 𝛼 ‖∇ψ‖ = Φ 𝛼 �min(0,𝜓𝑥+) + max(0,𝜓𝑥−) + min�0,𝜓𝑦+� + max�0,𝜓𝑦−�

(75)

13

The linear transport term,

Φ 𝛼 (∇ψ ∙ ∇Φ ) = Φ 𝛼 � Φ𝑥 ∙ ψ𝑥 + Φ𝑦 ∙ ψ𝑦 �, (76) should be discretized using the upwind scheme:

if 𝛼 Φ(𝑥,𝑦) > 0, then forward upwind if 𝛼 Φ(𝑥,𝑦) < 0, then backward upwind

(77)

In addition, the choice of ∆𝑡 should satisfy the Courant-Friedrichs-Lewy (CFL) condition:

0 ≤ ∆𝑡 ≤

∆𝑥2

2

(78)

In our implementation, ∆𝑥 = ∆𝑦 = 1 and ∆𝑡 = .25.

3.1.2. The Chan-Vese Model

The level set PDE (69) contains the same diffusion term as in (22) and two non-linear terms. The diffusion term should be discretized using central differences (73) and the non-linear terms should be discretized using the entropy upwind scheme (77) after checking the sign of the factor in front of each term.

In addition we approximate the Dirac function 𝛿0 by

𝛿0(𝑧) = 1𝜋

ℎℎ2 + 𝑧2

(79)

where ℎ = ∆𝑥 = ∆𝑦 = 1. We set ∆𝑡 = .1.

3.2. Managing Level Sets

Level set methods are a powerful tool for performing contour evolution. The level set function 𝜓:Ω → ℝ maps each pixel in the image domain to the shortest distance from that pixel to the contour. The contour corresponds to the set of points (𝑥,𝑦) for which 𝜓(𝑥,𝑦) = 0. Distances from pixels inside the contour are counted negatively. The zero level set segments Ω into two regions 𝑅 = {(𝑥, 𝑦) ∈ Ω, 𝜓(𝑥,𝑦) < 0}, the inside of the contour, and 𝑅𝑐 = {(𝑥, 𝑦) ∈ Ω, 𝜓(𝑥,𝑦) ≥ 0}, the outside of the contour. [Note that in Section 2.2.2., points inside the contour were considered to have positive distances whereas points outside the contour had negative distances. This inversion simplifies the expression of (66).]

14

At each iteration of the algorithm, we apply the discretized PDE to the level set function 𝜓. However, during the curve evolution, 𝜓 might change so much that it is not a distance function anymore. Re-initializing the signed distance function regularly prevents the level set function to become too flat. This can be done by solving the following evolution equation:

�𝜙𝑡 = sign�𝜓(𝑡)� (1 − ‖∇𝜙‖)

𝜙(0,∙) = 𝜓(𝑡,∙) (80)

The steady state of this evolution will be the signed distance function to the zero level contour of 𝜓; that is, 𝜙 will have the same sign as 𝜓 at each point, and the magnitude at a point will be the distance from that point to {𝜓 = 0}. Then, 𝜓 is reinitialized to the solution 𝜙.

The equation is discretized as follows:

𝜙(𝑥, 𝑦, 𝜏 + Δ𝜏) = 𝜙(𝑥, 𝑦, 𝜏) − Δ𝜏 sign�𝜓(𝑥, 𝑦, 𝑡)� 𝐺(∇𝜙) (81) where

𝐺(∇𝜙) =

⎩⎪⎨

⎪⎧�max�𝑎+2, 𝑏−

2� + max�𝑐+2,𝑑−2� − 1 𝑖𝑓 𝜓(𝑥,𝑦, 𝑡) > 0

�max�𝑎−2, 𝑏+2� + max�𝑐−2,𝑑+

2� − 1 𝑖𝑓 𝜓(𝑥,𝑦, 𝑡) > 0

0 𝑖𝑓 𝜓(𝑥,𝑦, 𝑡) = 0

(82)

with

𝑎 = �𝜓(𝑥,𝑦) − 𝜓(𝑥 − Δ𝑥,𝑦)� Δ𝑥⁄ 𝑏 = �𝜓(𝑥 + Δ𝑥,𝑦) − 𝜓(𝑥,𝑦)� Δ𝑥⁄ 𝑐 = �𝜓(𝑥, 𝑦) − 𝜓(𝑥,𝑦 − Δ𝑦)� Δy⁄ 𝑑 = �𝜓(𝑥, 𝑦 + Δ𝑦) − 𝜓(𝑥,𝑦)� Δ𝑦⁄ 𝑎+ = max(0,𝑎),𝑎− = min(0,𝑎), etc

(83)

This method was actually not implemented in the code for lack of time. We used a fast marching method written in C and available online (Peyre, 2006).

4. Experimental results

In this section we present numerical results obtained with the edge-based model and the region-based model on various real images, with different types of shapes and background.

Please be aware that for lack of time the function used to draw the contour given a level set function is not optimal and the contour is most of the time not “clean.”

15

4.1. “Easy” Examples

In the two following images, the geodesic model performs well. However, since the contour is initialized out of the object, the curve stops at the first significant edge, which might not

16

be what we are looking for. The Chan-Vese method tends to quickly penetrate the object to segments more regions. However, its contour is not precise.

4.2. “Hard” Examples

The two following images exhibit the edge-based active contour biggest weakness. The geodesic model has many local minima and is very sensible to initialization. The contour stops on noise and can’t segment the object properly, unless the image is smoothed beforehand. In order to preserve edges while smoothing, we used the Total Variations (TV) partial differential equation.

On the other hand Chan-Vese is not disturbed by noise and local variations. It works well when the mean intensity inside the object is very different from the mean intensity outside the object, like it is the case with Figure 3. However, in Figure 4, the distinction between foreground and background is not sharp enough to prevent the contour from penetrating the object. The contour at 𝑡 = 73 is the “cleanest” we could obtain before the contour started penetrating the object.

17

18

5. Discussion and Improvements

The geodesic method and the Chan-Vese method are based on two different principles and, thus, have different behaviors. It would be interesting to combine them into a single energy functional which would penalize being far from an edge and the mismatch between the inside and the outside of the contour as well. We have indeed noticed that the region-based method tends to segment regions which do not necessarily correspond to a well-defined boundary – like a shadow – and the contour lacks precision. However, this drawback is completely non-existent in simple cases such as synthetic images or images where objects are homogeneous.

It would also have been interesting to measure the power of active contours in tracking. We started implementing a framework to track a deforming object in a sequence after having manually defined an initial contour, but more time would have been necessary to complete it.

6. References

Caselles, V., Kimmel, R., & Sapiro, G. (1997). Geodesic Active Contours. International Journal of Computer Vision, 61-79.

Chan, T. F., & Vese, L. A. (2001). Active Contours Without Edges. IEEE Transactions On Image Processing, 266-277.

Peyre, G. (2006). Lecture 1 - Active Contours and Level Sets. Retrieved 2006, from Gabriel Peyre's personal website: http://www.ceremade.dauphine.fr/~peyre/teaching/manifold/tp1.html

Szeliski, R. (2010). Computer Vision: Algorithms and Applications. Springer.

7. Code Documentation

The implementation was written in C and C++ with Visual Studio 2010 and uses OpenCV 2.4.3. Make sure that the OpenCV dynamic libraries are in your path or the application won’t run. An executable has been compiled in the ‘Release’ directory and should run on Windows computers. Otherwise, the source code is in the ‘PDEs’ directory. In addition to the geodesic and Chan-Vese methods, the heat equation, the geometric heat equation, the total variations method, and morphological operations have been implemented. To try one of these run the console application with a picture for argument.

For example, type “PDEs.exe brain.png.” A menu prompts the user for the type of operation to apply to ‘brain.png’: either smoothing (heat equation, geometric heat equation, total

19

variations), morphological operations (opening and closing), or active contours (geodesic, Chan-Vese, but also dilation and erosion applied to level sets).

The user should then enter a number of iterations and choose an initial contour shape (circle, rectangle, and square) and size. The initial contour is centered and its position cannot be changed unless the source code is modified.

Functions implementing the geodesic and Chan-Vese methods are in ActiveContours.cpp. More precisely, iterateGeodesic() and iterateChanVese() implement one time-step of the evolution. The discretized PDE for each method can be found in those functions.

geodesic() and chanVese() are called by the main() function (in PDEs.cpp) when the user chooses a method. These functions iteratively call iterateGeodesic() and iterateChanVese(), respectively, and save the evolution of the contour in JPEG files.

computeAverages() calculates the mean intensities inside and outside the contour (𝑢 and 𝑣).

printContourOnImage() draws a contour on a grayscale OpenCV Mat object.

recalculateDistances() is used to re-initialize the signed distance function 𝜓. It finds starting points before calling the fast marching algorithm implemented by perform_front_propagation_2d(). The starting points more or less correspond to the level set zero (the contour) and thus they are used by printContourOnImage() to draw the current contour. Because the starting points don’t necessarily represent the zero level set accurately, what is drawn is not exactly the contour implicitly represented by the current level set function. Consequently, most output pictures don’t have a “clean” contour.