http:// computer graphics laboratory, hiroshima city university all images are compressed

http://www.hiroshima-cu.ac.jp/

Computer Graphics Laboratory, Hiroshima City University

All images are compressed.



Interactive shadow removal from a single image using hierarchical graph cut

Daisuke Miyazaki

Yasuyuki Matsushita

Katsushi Ikeuchi


CG Lab, Hiroshima City University

Motivation



Prior work

multiple

singleautomatic

interactive

Weiss 01; Matsushita et al. 04

Finlayson et al. 04; Fredembach & Finlayson 06;Tappen et al. 05; Baba et al. 04; Arbel & Hel-Or 07;Nielsen & Madsen 07

Wu et al. 07



Contribution

PriorPriorSmoothnessSmoothness

Graph cut

)(nO )(lognOAlpha expansion

User satisfaction

Robust

Intuitive

Fast



Algorithm flow

Input Over-segmentation Lazy snapping

Shadow removal

Initial Interaction Output

11 22 33

44 55 66



Shadow matte

FI

Prior of shadowless image

Prior of shadow image

Smoothness of shadowless image

Smoothness of shadow image

Solve



Prior of shadowless image

Likelihood P

appropriate largesmall

)(log pFP



Prior of shadow image

small largeappropriate

7.07.0

0 1 pp

L0.7-NormL1-NormL2-Norm



Smoothness of shadowless image

small large

2

pF

appropriate



Smoothness of shadow image

small largeappropriate

qp



Smoothnessshadow

0

1

Interactive parameter optimization

Smoothnessshadowless

Priorshadowless

Priorshadow

0

0.25

0

0.25

0.4

0.7

InitInit InitialInitial1st1st2nd2nd3rd3rd4th4th5th5th6th6th



Approach based on alpha expansion

Alpha expansion Hierarchical graph cut

alpha=2

alpha=2,4,...

alpha=4

3

1 52 4

1

1

3

5

5

1

1

3

5

5

3

512

12

3

54

54

12 1

25

4



Hierarchical structure

32

16 48

8 24 40 56

8 16 24 32 40 48 56

16 32 48

8 16 24 32 40 48 56

16 32 48

20



Graph construction example

p a q

),(),(,0max),()()( qpqpqpqp VVVDD

),(),(,0max),()()( qpqpqpqp VVVDD

),()()( qpqp VDD

),()()( qpqp VDD

),(),( qpqp VV

qp



Algorithm

Alpha expansion Hierarchical graph cut

1. A = {{0}, {32}, {16}, {48},{8, 40}, {24, 56},{4, 20, 36, 52}, {12, 28, 44, 60}, ...,{3, 7, 11, ..., 63}}

2. for i = 0 to 11 doa. for all nodes&edges do

I. II. add nodes&edges using &

b. solve max-flow/min-cut problem3. Iterate 2 until convergence

piAp ][minarg

p p

1. A = {0, 1, 2, ..., 63}

2. for i = 0 to 63 doa. for all nodes&edges do

I. II. add nodes&edges using &

b. solve max-flow/min-cut problem3. Iterate 2 until convergence

][iA p



Output of Hierarchical graph cut

Stereomatching

[Art]

Stereomatching[Laundry]

Imagerestoration

Shadowremoval

Input Ground truth Ishikawa 2003 expansion Hierarchical cut



Speed of hierarchical graph cut

Stereo matching[Art]

Stereo matching[Laundry]

Image restoration

Shadow removal

Speed-up 6.8 11.4 16.6 3.4

Error difference +4.6% +3.0% -0.4% +0.0%

Hierarchical graph cut

Alpha expansion

[sec] Iteration250 7

EnergyEnergy

0

1.1107 1.7106

1.5106



Incremental improvement

Input Truth Finlayson 2004 Wu 2007 Initial Interactive

[Ours]

Error

Number of strokes for parameter optimization 50

Finlayson 2004

Wu 2007Ours



0 strokes (param. opt.) 1.4 sec 1 strokes 2.1 sec 6 strokes

Natural images

1.2 sec 4 strokes



2.0 sec 12 strokes 1.5 sec 2 strokes

Natural images

1.9 sec 15 strokes1.5 sec 8 strokes



5.8 sec 27 strokes 2.8 sec 40 strokes

Natural images

6.3 sec 32 strokes2.0 sec 26 strokes



Aerial images



Conclusions


Cost function User interaction



Discussions

Cost function

User interaction




(c) Daisuke Miyazaki 2009All rights reserved.

http://www.cg.info.hiroshima-cu.ac.jp/Daisuke Miyazaki, Yasuyuki Matsushita, Katsushi Ikeuchi, "Interactive shadow removal from a single

image using hierarchical graph cut," Lecture Notes in Computer Science (Proc. Asian Conference on

Computer Vision), vol. 5994, part 1, 2009.

http:// computer graphics laboratory, hiroshima city university all images are compressed

Documents