improving image segmentation quality through effective...

Improving image segmentation quality through effective region merging using a

hierarchical social metaheuristic

Abraham Duarte, Angel Sanchez, Felipe Fernandez, Antonio S. Montemayor

ClaSpec – Mardi 25 Septembre 2007

L. MacaireL. Busin



Introduction

Hierarchical social metaheuristic

Application to image segmentation

Conclusion

14 15876

4 5 11 12 13

109321

14 15876

4 5 11 12 13

109321 1

0

1 0

14 15876

4 5 11 12 13

109321 1 1 0 1

1

1

0

1

1

0

1

1

1 1 0 1 1

1 1 0 1 1

14 15876

4 5 11 12 13

109321 1 1 0 1

1

1

0

1

1

0

1

1

1 1 0 1 1

1 1 0 1 1

f 2i=Ncut i=∑

v∈C i ,u∈C i'

wvu

∑v∈C i ,u∈C i∪C i

'

wvu

∑v∈C i ,u∈C i

'

wvu

∑v∈C i

' ,u∈C i∪C i'

wvu=sC i ,C i

' sC i , g i

sC i ,C i

'

sC i' i , g i



Introduction


Applcation to image segmentation

Conclusion

Procedure Hierarchical_Social algorithm(S)varS=(V,E,W) : Initial_Society_graph;GS={gi} : Groups_structure;F1={f1} : Individuals_objective_function_structure;F2={f2} : Groups_objective_funtion_structure;i,k:1..Number_of_groups /*Group indices*/begin

GS=Get_Initial_random_partition_and_groups_structure();repeat /*group evolution*/

Compute_F1()_and_F2(); /*Objective functions*/for each gi in GS

f2(i)=min{f2(k)| pour tout k} or Autonomous_phase thenCooperative_Strategy(i);

elseCompetitive_Strategy(i);

end forUpdate_groups_structure(GS);

until termination_condition_met; /*end social evolution*/return(GS); /*Approximate optimal solution*/

end Hierarchical_Social_Algorithm

14 15876

4 5 11 12 13

109321 1 1 0 1

1

1

0

1

1

0

1

1

1 1 0 1 1

1 1 0 1 1

14 15876

4 5 11 12 13

109321 1 1 0 1

1

1

0

1

1

0

1

1

1 1 0 1 1

1 1 0 1 1

Group 1 Group 2

14 15876

4 5 11 12 13

109321 1 1 0 1

1

1

0

1

1

0

1

1

1 1 0 1 1

1 1 0 1 1

Core 1 Core 2

Group 1 Group 2

C1C2

C1'

C2'

g1=C1∪C1' g 2=C2∪C 2

'

14 15876

4 5 11 12 13

109321 1 1 0 1

1

1

0

1

1

0

1

1

1 1 0 1 1

1 1 0 1 1

Core 1 Core 2

Group 1 Group 2

f 2i=Ncut i=∑

v∈C i ,u∈C i'

wvu

∑v∈C i ,u∈C i∪C i

'

wvu

∑v∈C i ,u∈C i

'

wvu

∑v∈C i

' ,u∈C i∪C i'

wvu=sC i ,C i

' sC i , g i

sC i ,C i

'

sC i' i , g i

C1C2

C1'

C2'

g1=C1∪C1' g 2=C2∪C 2

'

14 15876

4 5 11 12 13

109321 1 1 0 1

1

1

0

1

1

0

1

1

1 1 0 1 1

1 1 0 1 1

f 2i=∑

v∈C i ,u∈C i'

wvu

∑v∈C i , u∈C i∪C i

'

wvu

∑v∈C i , u∈C i

'

wvu

∑v∈C i

' ,u∈C i∪C i'

wvu

∑v∈C1 ,u∈C1

'

wvu=w2,3w5,8w7,11=101=2

Core 1 Core 2Group 1 Group 2

Core 1 Core 2

14 15876

4 5 11 12 13

109321 1 1 0 1

1

1

0

1

1

0

1

1

1 1 0 1 1

1 1 0 1 1

f 2i =∑

v∈C i ,u∈C i'

wvu


'

wvu

∑v∈C i ,u∈C i

'

wvu

∑v∈C i

' , u∈C i∪C i'

wvu

∑v∈C1 ,u∈C1

'

wvu=w2,3w5,8w7,11=101=2

∑v∈C1 ,u∈C1∪C1

'

wvu=w1,2w1,4w2,3w2,5w4,5w4,6w5,11w5,7w6,7w7,8=9


Core 1 Core 2

14 15876

4 5 11 12 13

109321 1 1 0 1

1

1

0

1

1

0

1

1

1 1 0 1 1

1 1 0 1 1

f 2i =∑

v∈C i ,u∈C i'

wvu


'

wvu

∑v∈C i ,u∈C i

'

wvu

∑v∈C i

' , u∈C i∪C i'

wvu

∑v∈C1 ,u∈C1

'

wvu=w2,3w5,8w7,11=101=2

∑v∈C1 ,u∈C1∪C1

'

wvu=w1,2w1,4w2,3w2,5w4,5w4,6w5,11w5,7w6,7w7,8=9

∑v∈C1

' ,u∈C1∪C1'

wvu=w2,3w3,11w5,11w11,8w8,7=2


Core 1 Core 2

14 15876

4 5 11 12 13

109321 1 1 0 1

1

1

0

1

1

0

1

1

1 1 0 1 1

1 1 0 1 1

f 2i =∑

v∈C i ,u∈C i'

wvu


'

wvu

∑v∈C i ,u∈C i

'

wvu

∑v∈C i

' , u∈C i∪C i'

wvu

∑v∈C1 ,u∈C1

'

wvu=w2,3w5,8w7,11=101=2

∑v∈C1 ,u∈C1∪C1

'

wvu=w1,2w1,4w2,3w2,5w4,5w4,6w5,11w5,7w6,7w7,8=9

f 21=292

2=11

9

∑v∈C1

' ,u∈C1∪C1'

wvu=w2,3w3,11w5,11w11,8w8,7=2


Core 1 Core 2

14 15876

4 5 11 12 13

109321 1 1 0 1

1

1

0

1

1

0

1

1

1 1 0 1 1

1 1 0 1 1


f 2i=∑

v∈C i ,u∈C i'

wvu


'

wvu

∑v∈C i , u∈C i

'

wvu

∑v∈C i

' ,u∈C i∪C i'

wvu

Core 1 Core 2

f 21=292

2=11

9≈1,222

∑v∈C2 ,u∈C2

'

wvu=w9,10w12,13w14,15=111=3

∑v∈C2 ,u∈C2∪C2

'

wvu=w9,10w10,13w12,13w13,15w14,15=5

∑v∈C2

' ,u∈C2∪C2'

wvu=w9,10w9,12w12,13w12,14w14,15=5

f 22=353

5=6

5≈1,2

Cooperative strategy :Cooperative strategy : Goal = improve f2 of a group

14 15876

4 5 11 12 13

109321 1 1 0 1

1

1

0

1

1

0

1

1

1 1 0 1 1

1 1 0 1 1


Core 1 Core 2

Core 2Core 2

f 21=292

2=11

9≈1,222

f 22=353

5=6

5≈1,2


Min =

winner

14 15876

4 5 11 12 13

109321 1 1 0 1

1

1

0

1

1

0

1

1

1 1 0 1 1

1 1 0 1 1


Core 1 Core 2

f 21=292

2=11

9≈1,222

f 22=353

5=6

5≈1,2


Min =

winnerProcedure Cooperative_Strategy(i) /*Normal_NCut problem*/vargi=(Ci,Ci') : Nodes_partition_of_considered_group_gi;beginfor v=1 to Number_of_nodes_of_considered_group_gi;

if vЄCi and C_to_C'(v,i) < f2(v,i) thenCi=Ci\{v};Ci'=Ci'U{v};

end ifif vЄCi' and C'_to_C(v,i) < f2(v,i) then

Ci'=Ci'\{v};Ci=CiU{v};

end ifend forend Cooperative_Strategy

14 15876

4 5 11 12 13

109321 1 1 0 1

1

1

0

1

1

0

1

1

1 1 0 1 1

1 1 0 1 1


Core 1 Core 2

Core 2Core 2

f 21=292

2=11

9≈1,222

f 22=353

5=6

5≈1,2


Min =

winner

f 1v , i=s C i ,C i

' − ' v , iv , is C i , g i− ' v , i

sC i ,C i

' − ' v ,i v , i sC i

' , g i− ' v , i

Function C_to_C'(v,i) :

v , i=∑u∈C i

wvu ' v ,i =∑u∈C i

'

wvu

14 15876

4 5 11 12 13

109321 1 1 0 1

1

1

0

1

1

0

1

1

1 1 0 1 1

1 1 0 1 1


Core 1 Core 2

f 21=292

2=11

9≈1,222

f 22=353

5=6

5≈1,2


Min =

winner

f 1v , i=sC i ,C i

' − ' v , iv , isC i , g i− ' v , i

sC i ,C i

' − ' v , i v ,i sC i

' , g i− ' v , i


v , i=∑u∈C i


'

wvu

14 15876

4 5 11 12 13

109321 1 1 0 1

1

1

0

1

1

0

1

1

1 1 0 1 1

1 1 0 1 1


Core 1 Core 2

f 21=292

2=11

9≈1,222

f 22=353

5=6

5≈1,2


Min =

winner

f 1v , i=s C i ,C i

' − ' v , iv , is C i , g i− ' v , i

sC i ,C i

' − ' v ,i v , i sC i

' , g i− ' v , i


v , i=∑u∈C i


'

wvuExample for v=10:

10,2=w10,13=1

sC2, g 2=5

' 10,2=w10,9=1

sC2,C2' =3 sC2

' , g 2=5f 110,2=

3−115−1

3−115−1

=1,5

14 15876

4 5 11 12 13

109321 1 1 0 1

1

1

0

1

1

0

1

1

1 1 0 1 1

1 1 0 1 1


Core 1 Core 2

Core 2Core 2

f 21=292

2=11

9≈1,222

f 22=353

5=6

5≈1,2


Min =

winner

f 1v , i=sC i ,C i

' − ' v , iv , isC i , g i− ' v , i

sC i ,C i

' − ' v ,i v , i sC i

' , g i− ' v , i


v , i=∑u∈C i


'

wvu

14 15876

4 5 11 12 13

109321 1 1 0 1

1

1

0

1

1

0

1

1

1 1 0 1 1

1 1 0 1 1


Core 1 Core 2

Core 2Core 2

f 21=292

2=11

9≈1,222

f 22=353

5=6

5≈1,2


Min =

winner

f 1v , i=s C i ,C i

' − ' v , iv , is C i , g i− ' v , i

sC i ,C i

' − ' v ,i v , i sC i

' , g i− ' v , i


v , i=∑u∈C i


'

wvu

f 1v , i=sC i ,C i

' ' v , i−v , isC i , g iv ,i

sC i ,C i

' ' v ,i −v , i s C i

' , g i− ' v , i

Function C'_to_C(v,i) :

14 15876

4 5 11 12 13

109321 1 1 0 1

1

1

0

1

1

0

1

1

1 1 0 1 1

1 1 0 1 1


Core 1 Core 2

f 21=292

2=11

9≈1,222

f 22=353

5=6

5≈1,2


Min =

winnerProcedure Cooperative_Strategy(i) /*Normal_NCut problem*/vargi=(Ci,Ci') : Nodes_partition_of_considered_group_gi;beginfor v=1 to Number_of_nodes_of_considered_group_gi;

if vЄCi and C_to_C'(v,i) < f2(v,i) thenCi=Ci\{v};Ci'=Ci'U{v};

end ifif vЄCi' and C'_to_C(v,i) < f2(v,i) then

Ci'=Ci'\{v};Ci=CiU{v};

end ifend forend Cooperative_Strategy

14 15876

4 5 11 12 13

109321 1 1 0 1

1

1

0

1

1

0

1

1

1 1 0 1 1

1 1 0 1 1


Core 1 Core 2

Procedure Competitive_Strategy(i) /*Normal_NCut problem*/vargi=(Ci,Ci') : Nodes_partition_of_considered_group_gi;G={g1,..gk} : Groups_partition_of_the_graphi, j, k:1.. Number_of_groups /*Group indices*/beginfor v=1 to Number_of_nodes_of_considered_group_gi;

j = arg min{gx_to_C(v,k)}, gx_to_C'(v,k)| pour tout k}; /* j= host group */if vЄCi and (j≠i) then

Ci=Ci\{v};else if vЄCi' and (j≠i) then

Ci'=Ci'\{v}; /*remove v from gi*/if gx_to_C(v,j)



Introduction



Conclusion

RAG MRAG

12

3

12

3

if ∣xi−x j∣r x then wij=0 else

wi , j=exp−C ij I i− I j

2

I2 ×exp

−C ij xi−x j2

x2

C ij=∥E i∥×∥Ei∥∥E i∥∥Ei∥

Adjustedexperimentaly

||Ei|| and ||Ej|| are, respectively, the number of pixels in regions i and j.

Fig. (a) Synthetic chess board image. (b) Watershed segmentation. (c) Feasible society graph. (d) Society partition and groups partition.

Fig. 8. Segmentation results for image Lenna: (a) Initial image. (b) Watershed segmentation. (c) Structure of the segmentation tree. (d)–(k) Resulting segmented regions according to the segmentation tree

Fig. Segmentation results for image Pout: (a) Initial image. (b) Watershed segmentation (c) Structure of the segmentation tree. (d)–(g) Resulting segmented regions according to the segmentation tree.

Fig. 10. Segmentation results for image Windsurf: (a) Initial image. (b) Watershed segmentation (c) Structure of the segmentation tree. (d)–(i) Resulting segmented regions according to the segmentation tree.

Fig. Segmentation results for image Cameraman: (a) Initial image. (b) Watershed segmentation. (c) Structure of the segmentation tree. (d)–(i) Resulting segmented regions according to the segmentation tree.

Table shows the used standard test images uses in this work and the characteristics of their corresponding MRAGs and their first Ncut value. This Ncut value is considered as a quantitative segmentation measurement (Shi and Malik, 2000), in such a way, the lower the Ncut is, the higher segmentation quality is.

IMAGE MRAG MRAG parameters HSANodes Edges σI σx rx Ncut

Lenna 256 × 256 7156 1812344 200 200 40 0,0550Pout 480 × 320 1817 213964 200 200 60 0,1734Windsurf 480 × 320 11155 1817351 200 200 35 0,0396Cameraman 256 × 2564181 460178 200 200 35 0,0497



Introduction



Conclusion

The cooperative strategy can be considered as a local search procedure in which the quality of the solution contained in each group is autonomously improved. This process is maintained during a determined number of iterations (autonomous iterations). Moreover, the procedure only works with the individuals into a group (in order to reduce the search space).The competitive strategy can be considered as a constructive procedure and it is oriented to allow individuals exchange among groups. Individuals of groups with higher group objective functions (lower quality), can change their groups during a competitive strategy to improve their respective individual objective function. This way, the groups with lower quality tend to disappear because their individuals move to other ones with higher quality.

L'auteur pense que les resultat peuvent etre ameliores en divisant par 3 ou + plutot que 2 regions uniquement (objectives functions base sur les k-means).

Pas de preuve de convergence vers l'optimum global (et si il restait dans un minimum local initialisation cruciale ?)

Pas convaincu de l'interet de diviser gi en Ci et Ci'

improving image segmentation quality through effective...

Documents