fuzzy sets and fuzzy techniques - lecture 13 -- fuzzy...

Fuzzy Setsand FuzzyTechniques

Laszlo G. Nyul

Outline

Motivation

Fuzzy sets(recap)

Fuzzyconnectednesstheory

FC variantsand details

Applications

References

Fuzzy Sets and Fuzzy TechniquesLecture 13 – Fuzzy connectedness

Laszlo G. Nyul

Department of Image Processing and Computer GraphicsUniversity of Szeged

2007-03-06


Laszlo G. Nyul

Outline

Motivation

Fuzzy sets(recap)



Applications

References

Outline

1 Motivation

2 Fuzzy sets (recap)

3 Fuzzy connectedness theoryFuzzy digital spaceAffinity and pathsFuzzy connected objectAlgorithm

4 FC variants and detailsDefining fuzzy spel affinityEfficient computationVectorial and relative fuzzy connectedness

5 Applications


Laszlo G. Nyul

Outline

Motivation

Fuzzy sets(recap)



Applications

References

Object characteristics in images

Graded composition

heterogeneity of intensity in theobject region due to heterogeneityof object material and blurringcaused by the imaging device

Hanging-togetherness

natural grouping of voxelsconstituting an object a humanviewer readily sees in a display ofthe scene as a Gestalt in spite ofintensity heterogeneity


Laszlo G. Nyul

Outline

Motivation

Fuzzy sets(recap)



Applications

References

Basic idea of fuzzy connectedness

• local hanging-togetherness(affinity) based on similarityin spatial location as well asin intensity(-derived features)

• global hanging-togetherness(connectedness)


Laszlo G. Nyul

Outline

Motivation

Fuzzy sets(recap)



Applications

References

Fuzzy set and relation

A fuzzy subset A of X is

A = (x , µA(x)) | x ∈ X

where µA is the membership function of A in X

µA : X → [0, 1]

A fuzzy relation ρ in X is

ρ = ((x , y), µρ(x , y)) | x , y ∈ X

with a membership function

µρ : X × X → [0, 1]


Laszlo G. Nyul

Outline

Motivation

Fuzzy sets(recap)



Applications

References

Operations on fuzzy sets

Intersection

A ∩ B = (x , µA∪B(x)) | x ∈ X µA∩B = min(µA, µB)

Union

A ∪ B = (x , µA∪B(x)) | x ∈ X µA∪B = max(µA, µB)

Complement

A = (x , µA(x)) | x ∈ X µA = 1− µA

∩ and ∪ are also called T-norm and T-conorm (S-norm).Several (corresponding pairs) of T- and S-norms exist.In the FC framework min and max are used.


Laszlo G. Nyul

Outline

Motivation

Fuzzy sets(recap)



Applications

References

Properties of fuzzy relations

ρ is reflexive if∀x ∈ X µρ(x , x) = 1

ρ is symmetric if

∀x , y ∈ X µρ(x , y) = µρ(y , x)

ρ is transitive if

∀x , z ∈ X µρ(x , z) =⋃y∈X

µρ(x , y) ∩ µρ(y , z)

ρ is similitude if it is reflexive, symmetric, and transitive

Note: this corresponds to the equivalence relation in hard sets.


Laszlo G. Nyul

Outline

Motivation

Fuzzy sets(recap)


Fuzzy digitalspace

Affinity andpaths

Fuzzy connectedobject

Algorithm


Applications

References

Fuzzy digital space

Fuzzy spel adjacency is a reflexive and symmetric fuzzyrelation α in Zn and assigns a value to a pair of spels (c , d)based on how close they are spatially.

Example

µα(c , d) =

1

‖c − d‖if ‖c − d‖ < a small distance

0 otherwise

Fuzzy digital space(Zn, α)

Scene (over a fuzzy digital space)

C = (C , f ) where C ⊂ Zn and f : C → [L,H]


Laszlo G. Nyul

Outline

Motivation

Fuzzy sets(recap)


Fuzzy digitalspace

Affinity andpaths


Algorithm


Applications

References

Fuzzy spel affinity

Fuzzy spel affinity is a reflexive and symmetric fuzzy relationκ in Zn and assigns a value to a pair of spels (c , d) based onhow close they are spatially and intensity-based-property-wise(local hanging-togetherness).

µκ(c , d) = h(µα(c , d), f (c), f (d), c , d)

Example

µκ(c , d) = µα(c , d) (w1G1(f (c) + f (d)) + w2G2(f (c)− f (d)))

where Gj(x) = exp

(−1

2

(x −mj)2

σ2j

)


Laszlo G. Nyul

Outline

Motivation

Fuzzy sets(recap)


Fuzzy digitalspace

Affinity andpaths


Algorithm


Applications

References

Paths between spels

A path pcd in C from spel c ∈ C to spel d ∈ C is any sequence〈c1, c2, . . . , cm〉 of m ≥ 2 spels in C , where c1 = c and cm = d .

Let Pcd denote the set of all possible paths pcd from c to d .Then the set of all possible paths in C is

PC =⋃

c,d∈C

Pcd


Laszlo G. Nyul

Outline

Motivation

Fuzzy sets(recap)


Fuzzy digitalspace

Affinity andpaths


Algorithm


Applications

References

Strength of connectedness

The fuzzy κ-net Nκ of C is a fuzzy subset of PC , where themembership (strength of connectedness) assigned to anypath pcd ∈ Pcd is the smallest spel affinity along pcd

µNκ(pcd) = minj=1,...,m−1

µκ(cj , cj+1)

The fuzzy κ-connectedness in C (K ) is a fuzzy relation in Cand assigns a value to a pair of spels (c , d) that is themaximum of the strengths of connectedness assigned to allpossible paths from c to d (global hanging-togetherness).

µK (c , d) = maxpcd∈Pcd

µNκ(pcd)


Laszlo G. Nyul

Outline

Motivation

Fuzzy sets(recap)


Fuzzy digitalspace

Affinity andpaths


Algorithm


Applications

References

Fuzzy κθ componentLet θ ∈ [0, 1] be a given threshold

Let Kθ be the following binary (equivalence) relation in C

µKθ(c , d) =

1 if µκ(c , d) ≥ θ

0 otherwise

Let Oθ(o) be the equivalence class of Kθ that contains o ∈ C

Let Ωθ(o) be defined over the fuzzy κ-connectedness K as

Ωθ(o) = c ∈ C |µK (o, c) ≥ θ

Practical computation of FC relies on the following equivalence

Oθ(o) = Ωθ(o)


Laszlo G. Nyul

Outline

Motivation

Fuzzy sets(recap)


Fuzzy digitalspace

Affinity andpaths


Algorithm


Applications

References

Fuzzy connected object

The fuzzy κθ object Oθ(o) of C containing o is

µOθ(o)(c) =

η(c) if c ∈ Oθ(o)

0 otherwise

that is

µOθ(o)(c) =

η(c) if c ∈ Ωθ(o)

0 otherwise

where η assigns an objectness value to each spel perhaps basedon f (c) and µK (o, c).

Fuzzy connected objects are robust to the selection of seeds.


Laszlo G. Nyul

Outline

Motivation

Fuzzy sets(recap)


Fuzzy digitalspace

Affinity andpaths


Algorithm


Applications

References

Fuzzy connectedness asa graph search problem

• Spels → graph nodes

• Spel faces → graph edges

• Fuzzy spel-affinity relation → edge costs

• Fuzzy connectedness → all-pairs shortest-path problem

• Fuzzy connected objects → connected components


Laszlo G. Nyul

Outline

Motivation

Fuzzy sets(recap)


Fuzzy digitalspace

Affinity andpaths


Algorithm


Applications

References

Computing fuzzy connectednessDynamic programming

AlgorithmInput: C, o ∈ C , κOutput: A K-connectivity scene Co = (Co , fo) of CAuxiliary data: a queue Q of spels

beginset all elements of Co to 0 except o which is set to 1push all spels c ∈ Co such that µκ(o, c) > 0 to Qwhile Q 6= ∅ do

remove a spel c from Qfval ← maxd∈Co [min(fo(d), µκ(c, d))]if fval > fo(c) then

fo(c)← fvalpush all spels e such that µκ(c, e) > 0 fval > fo(e) fval > fo(e) and µκ(c, e) > fo(e)

endifendwhile

end


Laszlo G. Nyul

Outline

Motivation

Fuzzy sets(recap)



Defining fuzzyspel affinity

Efficientcomputation

Vectorial andrelative fuzzyconnectedness

Applications

References

Fuzzy connectedness variants

• Multiple seeds per object

• Scale-based fuzzy affinity

• Vectorial fuzzy affinity

• Absolute fuzzy connectedness

• Relative fuzzy connectedness

• Iterative relative fuzzy connectedness


Laszlo G. Nyul

Outline

Motivation

Fuzzy sets(recap)






Applications

References

Components of fuzzy affinity

Fuzzy spel adjacency µα(c , d) indicates the degree of spatialadjacency of spels

The homogeneity-based component µψ(c , d) indicates thedegree of local hanging-togetherness of spels due to theirsimilarities of intensities

The object-feature-based component µφ(c , d) indicates thedegree of local hanging-togetherness of spels with respect tosome given object feature


Laszlo G. Nyul

Outline

Motivation

Fuzzy sets(recap)






Applications

References

Defining fuzzy spel affinity

Fuzzy spel affinity

µκ(c , d) = µα(c , d)g(µψ(c , d), µφ(c , d))

Expected properties of g

• range within [0, 1]

• monotonically non-increasing in both arguments

Examples

µκ =1

2µα (µψ + µφ)

µκ = µα√

µψµφ


Laszlo G. Nyul

Outline

Motivation

Fuzzy sets(recap)






Applications

References

Homogeneity-based componentNon-scale-based

Homogeneity-based component

µψ(c , d) = Wψ(|f (c)− f (d)|)

Expected properties of Wψ

• range within [0, 1] and Wψ(0) = 1

• monotonically non-increasing

• should also be related to overall homogeneity

Examples

the right-hand-side of an appropriately scaled box, trapezoid, orGaussian function


Laszlo G. Nyul

Outline

Motivation

Fuzzy sets(recap)






Applications

References

Object-feature-based componentNon-scale-based

Object-feature-based component

µφ(c , d) =

1 if c = d

Wo(c,d)Wb(c,d)+Wo(c,d) otherwise

Wo(c , d) = min[Wo(f (c)),Wo(f (d))]

Wb(c , d) = max[Wb(f (c)),Wb(f (d))]

Expected properties of Wo and Wb

• range within [0, 1]

• monotonically non-increasing

Examples

an appropriately scaled and shifted box, trapezoid, or Gaussianfunction


Laszlo G. Nyul

Outline

Motivation

Fuzzy sets(recap)






Applications

References

Scale-based affinity

Considers the following aspects

• spatial adjacency

• homogeneity (local and global)

• object feature (expected intensity properties)

• object scale


Laszlo G. Nyul

Outline

Motivation

Fuzzy sets(recap)






Applications

References

Object scale

Object scale in C at any spel c ∈ C is the radius r(c) of thelargest hyperball centered at c which lies entirely within thesame object region

The scale value can be simply and effectively estimated withoutexplicit object segmentation


Laszlo G. Nyul

Outline

Motivation

Fuzzy sets(recap)






Applications

References

Computing object scale

AlgorithmInput: C, c ∈ C , Wψ , τ ∈ [0, 1]Output: r(c)

begink ← 1while FOk (c) ≥ τ do

k ← k + 1endwhiler(c)← k

end

Fraction of the ball boundary homogeneous with the center spel

FOk(c) =

∑d∈Bk (c)

Wψs (|f (c)− f (d)|)

|Bk(c)− Bk−1(c)|


Laszlo G. Nyul

Outline

Motivation

Fuzzy sets(recap)






Applications

References

Neighborhood selection forscale-based computations

For properties based on a single spel c use

Br (c) = e ∈ C | ‖e − c‖ ≤ r(c)

For properties based on a pair of spels c and d use

Bcd(c) = e ∈ C | ‖e − c‖ ≤ min[r(c), r(d)]

Bcd(d) = e ∈ C | ‖e − d‖ ≤ min[r(c), r(d)]


Laszlo G. Nyul

Outline

Motivation

Fuzzy sets(recap)






Applications

References

Intensity variations inneighborhoods

Intensity variations (inhomogeneity) surrounding c and d(µα(c , d) > 0)

• intra-object variation

(expected to be random and to have a zero mean)

• inter-object variation

(expected to have a direction and to be larger than theintra-object variation)


Laszlo G. Nyul

Outline

Motivation

Fuzzy sets(recap)






Applications

References

Homogeneity-based componentScale-based

Directional inhomogeneity over the regions around c and d

D(c , d) =∑

e∈Bcd (c)e′∈Bcd (d)e−c=e′−d

(1−Wh(f (e)−f (e ′)))ωcd(‖e−c‖) f (e)− f (e ′)

|f (e)− f (e ′)|

Homogeneity-based component

µψs (c , d) = 1− |D(c , d)|∑e∈Bcd (c)

ωcd(‖e − c‖)


Laszlo G. Nyul

Outline

Motivation

Fuzzy sets(recap)






Applications

References

Object-feature-based componentScale-based

Filtered version of f (c) taking into account the neighborhood

fa(c) =

∑e∈Br (c)

f (e)ωc(‖e − c‖)

∑e∈Br (c)

ωc(‖e − c‖)

Object-feature-based component

µφs (c , d) =

1 if c = dWos(c , d)

Wbs(c , d) +Wos(c , d)otherwise

Wos(c , d) = min[Wos(fa(c)),Wos(fa(d))]

Wbs(c , d) = max[Wbs(fa(c)),Wbs(fa(d))]


Laszlo G. Nyul

Outline

Motivation

Fuzzy sets(recap)






Applications

References

Brain tissue segmentationFSE


Laszlo G. Nyul

Outline

Motivation

Fuzzy sets(recap)






Applications

References

Efficiency problems with FC (1)

Problemit takes too long to compute

Solutionuse more powerful computer

• scales with CPU speed but also depends on the algorithm


Laszlo G. Nyul

Outline

Motivation

Fuzzy sets(recap)






Applications

References


Problemmost time is spent with affinity computation

Solutioncompute affinity only for spels that are used

• can reduce 35–55% depending on the object


Laszlo G. Nyul

Outline

Motivation

Fuzzy sets(recap)






Applications

References


Problemmany spels are unnecessarily visited

Solutionuse pre-determined connectedness thresholds

• can reduce 70–90% depending on the object


Laszlo G. Nyul

Outline

Motivation

Fuzzy sets(recap)






Applications

References

Effect of affinity thresholds


Laszlo G. Nyul

Outline

Motivation

Fuzzy sets(recap)






Applications

References


Problemmany spels are “revisited”

Solutionreduce the number of “revisits” by ordering

• label-correcting vs. label-setting algorithms

• binary, d-ary, and Fibonacci heaps

• LIFO and FIFO lists

• hash tables of various sizes with various hash functions

• additional pointer arrays


Laszlo G. Nyul

Outline

Motivation

Fuzzy sets(recap)






Applications

References

Computing fuzzy connectednessDynamic programming

AlgorithmInput: C, o ∈ C , κOutput: A K-connectivity scene Co = (Co , fo) of CAuxiliary data: a queue Q of spels

beginset all elements of Co to 0 except o which is set to 1push all spels c ∈ Co such that µκ(o, c) > 0 to Qwhile Q 6= ∅ do

remove a spel c from Qfval ← maxd∈Co [min(fo(d), µκ(c, d))]if fval > fo(c) then

fo(c)← fvalpush all spels e such that µκ(c, e) > 0 fval > fo(e) fval > fo(e) and µκ(c, e) > fo(e)

endifendwhile

end


Laszlo G. Nyul

Outline

Motivation

Fuzzy sets(recap)






Applications

References

Computing fuzzy connectednessDijkstra’s-like

AlgorithmInput: C, o ∈ C , κOutput: A K-connectivity scene Co = (Co , fo) of CAuxiliary data: a priority queue Q of spels

beginset all elements of Co to 0 except o which is set to 1push o to Qwhile Q 6= ∅ do

remove a spel c from Q for which fo(c) is maximalfor each spel e such that µκ(c, e) > 0 do

fval ← min(fo(c), µκ(c, e))if fval > fo(e) then

fo(e)← fvalupdate e in Q (or push if not yet in)

endifendfor

endwhileend


Laszlo G. Nyul

Outline

Motivation

Fuzzy sets(recap)






Applications

References

FC with thresholdMRI


Laszlo G. Nyul

Outline

Motivation

Fuzzy sets(recap)






Applications

References

FC with thresholdCT and MRA


Laszlo G. Nyul

Outline

Motivation

Fuzzy sets(recap)






Applications

References

Vector-valued scenes

Scene

C = (C , f) where C ⊂ Zn and f = (f1, f2, . . . , fl)T

f : C → [L1,H1]× [L2,H2]× · · · × [Ll ,Hl ]

Object scale

FOk(c) =

∑d∈Bk (c)

Wh(f(c)− f(d))

|Bk(c)− Bk−1(c)|

Wh(x) = exp

(−1

2xTΣ−1

h x

)


Laszlo G. Nyul

Outline

Motivation

Fuzzy sets(recap)






Applications

References

Homogeneity-based componentVectorial scale-based

Total unidirectional inhomogeneity over the scale regionsaround c and d

D(c , d) =∑

e∈Bcd (c)e′∈Bcd (d)e−c=e′−d

(1−Wh(f(e)−f(e ′)))ωcd(‖e−c‖) f(e)− f(e ′)

|f(e)− f(e ′)|

µψvs (c , d) = 1− |D(c , d)|∑e∈Bcd (c)

ωcd(‖e − c‖)


Laszlo G. Nyul

Outline

Motivation

Fuzzy sets(recap)






Applications

References

Object-feature-based componentVectorial scale-based

Wo(fa(c)) = exp

(−1

2(fa(c)−Mo)TΣ−1

o (fa(c)−Mo)

)

µφvs (c , d) =

1 if c = d

min[Wo(fa(c)),Wo(fa(d))] otherwise


Laszlo G. Nyul

Outline

Motivation

Fuzzy sets(recap)






Applications

References

Relative fuzzy connectedness

• always at least two objects

• automatic/adaptive thresholdson the object boundaries

• objects (object seeds) “compete”for spels and the one withstronger connectedness wins


Laszlo G. Nyul

Outline

Motivation

Fuzzy sets(recap)






Applications

References

Relative fuzzy connectedness

Let O1,O2, . . . ,Om, a given set of objects (m ≥ 2),S = o1, o2 . . . , om a set of corresponding seeds, and letb(oj) = S \ oj denote the ‘background’ seeds w.r.t. seed oj .

1 define affinity for each object ⇒ κ1, κ2, . . . , κm

2 combine them into a single affinity ⇒ κ =⋃

j κj

3 compute fuzzy connectedness using κ ⇒ K

4 determine the fuzzy connected objects ⇒

Oob(o) = c ∈ C | ∀o ′ ∈ b(o) µK (o, c) > µK (o ′, c)

µOob(c) =

η(c) if c ∈ Oob(o)

0 otherwise


Laszlo G. Nyul

Outline

Motivation

Fuzzy sets(recap)






Applications

References

kNN vs. VSRFCFuzzy Setsand FuzzyTechniques

Laszlo G. Nyul

Outline

Motivation

Fuzzy sets(recap)



Applications

References

Image segmentation using FC

• MR• brain tissue, tumor, MS lesion segmentation

• MRA• vessel segmentation and artery-vein separation

• CT bone segmentation• kinematics studies• measuring bone density• stress-and-strain modeling

• CT soft tissue segmentation• cancer, cyst, polyp detection and quantification• stenosis and aneurism detection and quantification

• Digitized mammography• detecting microcalcifications

• Craniofacial 3D imaging• visualization and surgical planning


Laszlo G. Nyul

Outline

Motivation

Fuzzy sets(recap)



Applications

References

Protocols for brain MRIFuzzy Setsand FuzzyTechniques

Laszlo G. Nyul

Outline

Motivation

Fuzzy sets(recap)



Applications

References

FC segmentation of brain tissues

1 Correct for RF field inhomogeneity

2 Standardize MR image intensities

3 Compute fuzzy affinity for GM, WM, CSF

4 Specify seeds and VOI (interaction)

5 Compute relative FC for GM, WM, CSF

6 Create brain intracranial mask

7 Correct brain mask (interaction)

8 Create masks for FC objects

9 Detect potential lesion sites

10 Compute relative FC for GM, WM, CSF, LS

11 Verify the segmented lesions (interaction)


Laszlo G. Nyul

Outline

Motivation

Fuzzy sets(recap)



Applications

References

Segmentation in two phases

Phase 1: Training

performed once for each task (protocol, body region, organ)

• a few datasets are selected and used to extract the valuesfor the parameters

• mostly requires continuous user control

Phase 2: Segmentation

performed for each individual dataset

• most steps are automatic (parameters are fixed in Phase 1)

• interactive steps require• mouse clicks from the user to specify points• “cut” and “add” when correcting the brain mask


Laszlo G. Nyul

Outline

Motivation

Fuzzy sets(recap)



Applications

References

Brain tissue segmentationFSE


Laszlo G. Nyul

Outline

Motivation

Fuzzy sets(recap)



Applications

References

Brain tissue segmentationT1


Laszlo G. Nyul

Outline

Motivation

Fuzzy sets(recap)



Applications

References

Brain tissue segmentationSPGR


Laszlo G. Nyul

Outline

Motivation

Fuzzy sets(recap)



Applications

References

MS lesion quantificationFSE


Laszlo G. Nyul

Outline

Motivation

Fuzzy sets(recap)



Applications

References

MS lesion quantificationT1E


Laszlo G. Nyul

Outline

Motivation

Fuzzy sets(recap)



Applications

References

MTR analysisFuzzy Setsand FuzzyTechniques

Laszlo G. Nyul

Outline

Motivation

Fuzzy sets(recap)



Applications

References

Brain tumor quantification


Laszlo G. Nyul

Outline

Motivation

Fuzzy sets(recap)



Applications

References

Skull object from CTFuzzy Setsand FuzzyTechniques

Laszlo G. Nyul

Outline

Motivation

Fuzzy sets(recap)



Applications

References

MRA slice and MIP rendering


Laszlo G. Nyul

Outline

Motivation

Fuzzy sets(recap)



Applications

References

MRA vessel segmentation andartery/vein separation


Laszlo G. Nyul

Outline

Motivation

Fuzzy sets(recap)



Applications

References

ReferencesJ. K. Udupa and S. Samarasekera.Fuzzy connectedness and object definition: Theory, algorithms, and applications in imagesegmentation.Graphical Models and Image Processing, 58(3):246–261, 1996.

P. K. Saha, J. K. Udupa, and D. Odhner.Scale-based fuzzy connected image segmentation: Theory, algorithms, and validation.Computer Vision and Image Understanding, 77(2):145–174, 2000.

P. K. Saha and J. K. Udupa.Fuzzy connected object delineation: Axiomatic path strength definition and the case of multiple seeds.Computer Vision and Image Understanding, 83(3):275–295, 2001.

P. K. Saha and J. K. Udupa.Relative fuzzy connectedness among multiple objects: Theory, algorithms, and applications in imagesegmentation.Computer Vision and Image Understanding, 82(1):42–56, 2001.

J. K. Udupa, P. K. Saha, and R. A. Lotufo.Relative fuzzy connectedness and object definition: Theory, algorithms, and applications in imagesegmentation.IEEE Transactions on Pattern Analysis and Machine Intelligence, 24(11):1485–1500, 2002.

L. G. Nyul, A. X. Falcao, and J. K. Udupa.Fuzzy-connected 3D image segmentation at interactive speeds.Graphical Models, 64(5):259–281, 2003.

Y. Zhuge, J. K. Udupa, and P. K. Saha.Vectorial scale-based fuzzy-connected image segmentation.Computer Vision and Image Understanding, 101(3):177–193, 2006.


Laszlo G. Nyul

Outline

Motivation

Fuzzy sets(recap)



Applications

References

fuzzy sets and fuzzy techniques - lecture 13 -- fuzzy...

Documents