segmentation via graph cuts
DESCRIPTION
Segmentation via Graph Cuts. Balakrishna Thiagarajan Computer Science and Engineering State University of New York at Buffalo. Introduction. Gestalt approach ( From) Normalized Cuts and Image Segmentation. Image Segmentation. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Segmentation via Graph Cuts](https://reader036.vdocuments.net/reader036/viewer/2022081504/568145bb550346895db2c365/html5/thumbnails/1.jpg)
Segmentation via Graph Cuts
Balakrishna Thiagarajan
Computer Science and Engineering
State University of New York at Buffalo
![Page 2: Segmentation via Graph Cuts](https://reader036.vdocuments.net/reader036/viewer/2022081504/568145bb550346895db2c365/html5/thumbnails/2.jpg)
Introduction Gestalt approach
(From) Normalized Cuts and Image Segmentation
![Page 3: Segmentation via Graph Cuts](https://reader036.vdocuments.net/reader036/viewer/2022081504/568145bb550346895db2c365/html5/thumbnails/3.jpg)
Image Segmentation Segmentation refers to the process of
partitioning a digital image into multiple segments.
The goal of segmentation is to simplify and/or change the representation of an image into something that is more meaningful and easier to analyze.
Similarity (gray level difference), Proximity and continuity are important factors in segmentation.
![Page 4: Segmentation via Graph Cuts](https://reader036.vdocuments.net/reader036/viewer/2022081504/568145bb550346895db2c365/html5/thumbnails/4.jpg)
Graph representation of Image
Set of points of the feature space represented as a weighted, undirected graph, G = (V, E).
The points of the feature space are the nodes of the graph.
Edge between every pair of nodes.
![Page 5: Segmentation via Graph Cuts](https://reader036.vdocuments.net/reader036/viewer/2022081504/568145bb550346895db2c365/html5/thumbnails/5.jpg)
Graph representation of Image
Weight on each edge, w(i, j), is a function of the similarity between the nodes i and j.
Partition the set of vertices into disjoint sets where similarity within the sets is high and across the sets is low.
Therefore, Segmentation is equivalent to Graph partition.
![Page 6: Segmentation via Graph Cuts](https://reader036.vdocuments.net/reader036/viewer/2022081504/568145bb550346895db2c365/html5/thumbnails/6.jpg)
Graph representation of ImageCut: A cut in a graph is a set of edges whose
removal disconnects the graph.
A minimum cut is a cut with a minimum number of edges. It is denoted by S. In the weighted graph,
cut(A,B) = ∑ w(u,v) u E A,v E B
where A & B are two partitioned disjoint sets
![Page 7: Segmentation via Graph Cuts](https://reader036.vdocuments.net/reader036/viewer/2022081504/568145bb550346895db2c365/html5/thumbnails/7.jpg)
Segmentation by Min Cut method
By Min Cut method, the graph is partitioned into clusters.
Each cluster is considered as an image segment.
Min Cut method uses the HCS (Highly Connected Subgraphs) Algorithm to find the clusters.
![Page 8: Segmentation via Graph Cuts](https://reader036.vdocuments.net/reader036/viewer/2022081504/568145bb550346895db2c365/html5/thumbnails/8.jpg)
Segmentation by Min Cut method
Edge Connectivity is the minimum number of edges whose removal results in a disconnected graph. It is denoted by k(G).
For a graph with vertices n > 1 to be highly connected if its edge-connectivity k(G) > n/2.
A highly connected subgraph (HCS) is an induced subgraph H in G such that H is highly connected.
![Page 9: Segmentation via Graph Cuts](https://reader036.vdocuments.net/reader036/viewer/2022081504/568145bb550346895db2c365/html5/thumbnails/9.jpg)
Segmentation by Min Cut methodExample:
No. of nodes = 5 Edge Connectivity = 1
Not HCS!
A B
D C E
![Page 10: Segmentation via Graph Cuts](https://reader036.vdocuments.net/reader036/viewer/2022081504/568145bb550346895db2c365/html5/thumbnails/10.jpg)
Segmentation by Min Cut methodExample continued:
No. of nodes = 4 Edge Connectivity = 3
HCS!
A B
D C
![Page 11: Segmentation via Graph Cuts](https://reader036.vdocuments.net/reader036/viewer/2022081504/568145bb550346895db2c365/html5/thumbnails/11.jpg)
HCS Algorithm - Min Cut methodHCS(G(V,E))
begin
(H, H’,C) MINCUT(G)
if G is highly connected
then return (G)
else
HCS(H)
HCS(H’)
end if
end
![Page 12: Segmentation via Graph Cuts](https://reader036.vdocuments.net/reader036/viewer/2022081504/568145bb550346895db2c365/html5/thumbnails/12.jpg)
HCS Algorithm - Min Cut method The procedure MINCUT(G) returns H, H’ and
C where C is the minimum cut which separates G into the subgraphs H and H’.
Procedure HCS returns a graph in case it identifies it as a cluster.
Single vertices are not considered clusters and are grouped into singletons set S.
![Page 13: Segmentation via Graph Cuts](https://reader036.vdocuments.net/reader036/viewer/2022081504/568145bb550346895db2c365/html5/thumbnails/13.jpg)
Example
HCS Algorithm - Min Cut method
![Page 14: Segmentation via Graph Cuts](https://reader036.vdocuments.net/reader036/viewer/2022081504/568145bb550346895db2c365/html5/thumbnails/14.jpg)
Example Continued
HCS Algorithm - Min Cut method
![Page 15: Segmentation via Graph Cuts](https://reader036.vdocuments.net/reader036/viewer/2022081504/568145bb550346895db2c365/html5/thumbnails/15.jpg)
Example Continued
Cluster 2
Cluster 1
Cluster 3
HCS Algorithm - Min Cut method
![Page 16: Segmentation via Graph Cuts](https://reader036.vdocuments.net/reader036/viewer/2022081504/568145bb550346895db2c365/html5/thumbnails/16.jpg)
HCS Algorithm - Min Cut method The HCS Algorithm returns clusters which are
nothing but image segments.
The problem with HCS Algorithm – Min Cut method is that there is a chance that many singletons may be separated.
To overcome this we use normalized-cut for image segmentation.
![Page 17: Segmentation via Graph Cuts](https://reader036.vdocuments.net/reader036/viewer/2022081504/568145bb550346895db2c365/html5/thumbnails/17.jpg)
HCS Algorithm – Main ProblemExample
![Page 18: Segmentation via Graph Cuts](https://reader036.vdocuments.net/reader036/viewer/2022081504/568145bb550346895db2c365/html5/thumbnails/18.jpg)
HCS Algorithm – Main ProblemExample – Another possible cut
![Page 19: Segmentation via Graph Cuts](https://reader036.vdocuments.net/reader036/viewer/2022081504/568145bb550346895db2c365/html5/thumbnails/19.jpg)
Max Flow - Min Cut Method Theorem
In graph G, the maximum source-to-sink flow possible is equal to the capacity of the minimum cut in G.
(L. R. Foulds, Graph Theory Applications, 1992 Springer-Verlag New York Inc., 247-248)
![Page 20: Segmentation via Graph Cuts](https://reader036.vdocuments.net/reader036/viewer/2022081504/568145bb550346895db2c365/html5/thumbnails/20.jpg)
Max Flow - Min Cut Method Min Cut can be made by using Ford-Fulkerson
Algorithm to find the max-flow in the graph. In the graph temporarily add Source ‘s’ and
Sink ‘t’. Consider the weight associated with each each
node as a flow capacity. Starting from zero flow, increase the flow
gradually by finding a path from s to t along which more flow can be sent, until a max-flow is achieved.
![Page 21: Segmentation via Graph Cuts](https://reader036.vdocuments.net/reader036/viewer/2022081504/568145bb550346895db2c365/html5/thumbnails/21.jpg)
Max Flow - Min Cut Method
1
2
2
2
3
22
s t
1
2
11
3
22
s t
1
2
2
1
2
21
s t
1/1
1/2
2/2
1/2
1/3
1/22/2
s t
f = 0
f = 1
f = 2
f = 3
1
1
1
1
1
1
1
2
1
2
12
s t
f = 3
1
1
1
1
0/1
0/2
0/2
0/2
0/3
0/20/2
s t
f = 0
(a)
(b)
(c) (f)
(e)
(d)
![Page 22: Segmentation via Graph Cuts](https://reader036.vdocuments.net/reader036/viewer/2022081504/568145bb550346895db2c365/html5/thumbnails/22.jpg)
Max Flow - Min Cut Method After the max-flow is found, the minimum cut
is by
S = {All vertices reachable from s}
T = G \ S
1
1
2
1
2
12
s t1
1
1
1
![Page 23: Segmentation via Graph Cuts](https://reader036.vdocuments.net/reader036/viewer/2022081504/568145bb550346895db2c365/html5/thumbnails/23.jpg)
Max Flow - Min Cut Method - Illustration
(From) Image Segmentation Using Minimal Graph Cuts by Anders P. Eriksson, Olof Barr and Kalle Astrom�
![Page 24: Segmentation via Graph Cuts](https://reader036.vdocuments.net/reader036/viewer/2022081504/568145bb550346895db2c365/html5/thumbnails/24.jpg)
Normalized Cut Normalized Cut (Ncut) :
Ncut(A,B) = cut(A,B) + cut(A,B)
assoc(A,V) assoc(B,V)
where, assoc(A,V) = ∑ w(u,t) u E A, t E V
The cut is made such that the normalized cut value between the two sets A & B is minimum.
![Page 25: Segmentation via Graph Cuts](https://reader036.vdocuments.net/reader036/viewer/2022081504/568145bb550346895db2c365/html5/thumbnails/25.jpg)
Normalized Cut
Minimum cut criteria favors cutting small sets of isolated nodes in the graph
(From) Normalized Cuts and Image Segmentation –
Shi & Malik
![Page 26: Segmentation via Graph Cuts](https://reader036.vdocuments.net/reader036/viewer/2022081504/568145bb550346895db2c365/html5/thumbnails/26.jpg)
Normalized Cut
Exact solution to minimizing normalized cut is an NP-complete problem.
However, approximate discrete solutions can be found efficiently.
Normalized cut criterion can be computed efficiently by solving a generalized eigenvalue problem.
![Page 27: Segmentation via Graph Cuts](https://reader036.vdocuments.net/reader036/viewer/2022081504/568145bb550346895db2c365/html5/thumbnails/27.jpg)
Normalized Cut – Physical Interpretation Think of the weighted graph as a spring mass
system. Graph nodes physical masses Graph edges springs Graph edge weight spring stiffness Total incoming edge weights mass of the
node
![Page 28: Segmentation via Graph Cuts](https://reader036.vdocuments.net/reader036/viewer/2022081504/568145bb550346895db2c365/html5/thumbnails/28.jpg)
Normalized Cut – Physical Interpretation Imagine giving a hard shake to this spring-
mass system, forcing the nodes to oscillate in the direction perpendicular to the image plane.
Nodes that have stronger spring connections among them will likely oscillate together.
Eventually, the group will “pop” off from the image plane.
![Page 29: Segmentation via Graph Cuts](https://reader036.vdocuments.net/reader036/viewer/2022081504/568145bb550346895db2c365/html5/thumbnails/29.jpg)
Normalized Cut
(From) Normalized Cuts and Image Segmentation - Shi & Malik
![Page 30: Segmentation via Graph Cuts](https://reader036.vdocuments.net/reader036/viewer/2022081504/568145bb550346895db2c365/html5/thumbnails/30.jpg)
Average Cut
Average Cut: Averagecut(A,B) = cut(A,B) + cut(A,B) |A| |B|
The average cut measure is proportional to the total link weight cut, normalized by the sizes of the partitions.
The problem with Average Cut is that one cannot ensure the two partitions computed will have tight within-group similarity.
![Page 31: Segmentation via Graph Cuts](https://reader036.vdocuments.net/reader036/viewer/2022081504/568145bb550346895db2c365/html5/thumbnails/31.jpg)
Comparison Different types of graph cuts - MinCut,
Average Cut, and Normalized Cut were analysed by Padmanabhan Soundararajan and Sudeep Sarkar.
Totally 50 images were used. 5 different image classes with 10 images in each class.
![Page 32: Segmentation via Graph Cuts](https://reader036.vdocuments.net/reader036/viewer/2022081504/568145bb550346895db2c365/html5/thumbnails/32.jpg)
Comparison The 5 classes of images were
1. natural objects in indoor backgrounds
2. natural objects in outdoor backgrounds
3. man-made objects in indoor backgrounds
4. man-made objects in outdoor backgrounds
5. aerial images that contained man-made objects in natural surroundings.
![Page 33: Segmentation via Graph Cuts](https://reader036.vdocuments.net/reader036/viewer/2022081504/568145bb550346895db2c365/html5/thumbnails/33.jpg)
Comparison The best possible performances on each
image for the three different cut measures were not significantly different from each other.
The performance with the three measures is worst for man made objects in indoor surroundings and for aerial images.
The normalized cut based partition took significantly larger time to compute than the average and the mincut.
![Page 34: Segmentation via Graph Cuts](https://reader036.vdocuments.net/reader036/viewer/2022081504/568145bb550346895db2c365/html5/thumbnails/34.jpg)
References
1. J. Shi and J. Malik, “Normalized Cuts and Image Segmentation,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 22, no. 8, pp. 888-905, Aug. 2000.
2. Erez Hartuv and Ron Shamir, “A Clustering Algorithm based on Graph Connectivity”, Mar. 1999.
3. Padmanabhan Soundararajan and Sudeep Sark, “Analysis of MinCut, Average Cut, and Normalized Cut Measures”, May 2000.
4. Anders P. Eriksson, Olof Barr and Kalle Astrom,” Image Segmentation Using Minimal Graph Cuts”.
![Page 35: Segmentation via Graph Cuts](https://reader036.vdocuments.net/reader036/viewer/2022081504/568145bb550346895db2c365/html5/thumbnails/35.jpg)
Thank You!!