b-spline channels & channel smoothing

1

Computer Vision Laboratory

B-Spline Channels & Channel Smoothing

Michael FelsbergComputer Vision Laboratory

Linköping UniversitySWEDEN

2


General Idea of Channels

• Encode single value (linear or modular) in N-D coefficient vector (channel vector)

• Locality of encoding– Similar values in same coefficients– Dissimilar values in different coefficients

• Stability by smooth, monopolar basis functions– Small changes of value lead to small changes of

coefficients– Non-negative coefficients

3


Example for Single Value

4


Example for Multiple Values

5


Overview

• Encoding with quadratic B-splines

• Decoding strategies

• Relation to kernel-density estimation

• Relation to robust M-estimation

• Channel smoothing

• Applications

6


Quadratic B-Splines

€

B0 f( ) =

1 f <1

21

2f =

1

2

0 f >1

2

⎧

⎨

⎪ ⎪

⎩

⎪ ⎪

Bk f( ) =k + 2 f +1

2kBk−1 f +

1

2

⎛

⎝ ⎜

⎞

⎠ ⎟+

k − 2 f +1

2kBk−1 f −

1

2

⎛

⎝ ⎜

⎞

⎠ ⎟

€

B2 f( ) =

3

4− f 2 f ∈ 0,

1

2

⎡ ⎣ ⎢

⎞

⎠ ⎟

1

2

3

2− f

⎛

⎝ ⎜

⎞

⎠ ⎟2

f ∈1

2,3

2

⎡ ⎣ ⎢

⎞

⎠ ⎟

0 f ∈3

2,∞

⎡ ⎣ ⎢

⎞

⎠ ⎟

⎧

⎨

⎪ ⎪ ⎪

⎩

⎪ ⎪ ⎪

7


• The value of the nth channel at x is obtained by

• Encoding in practice:– m=round(f)– c[m-1]=(f-m-0.5)2 /2– c[m]=0.75-(f-m)2

– c[m+1]=(m-f-0.5)2 /2

B-Splines Encoding

€

cn f ,x( ) = B2 f x( ) − n( ) n =1... N

We assume f to be shiftedand rescaled such that

€

f ∈ 1.5,N − 0.5[ ]

8


Example

QuickTime™ and a decompressor

are needed to see this picture.

9


Linear Decoding

• Normalized convolution of the channel vector

• Choice of n by heuristics– Largest denominator (3-box filter)– Additional: local maximum

€

fn0=

ncn f( )n= n0 −1

n0 +1

∑

cn f( )n= n0 −1

n0 +1

∑=

cn0 +1( f ) − cn0 −1( f )

cn0 −1( f ) + cn0( f ) + cn0 +1( f )

+ n0

0

10


Quantization Effect

11


Quadratic Decoding I

• Idea: detect local maximum of B-spline interpolated channel vector– Step 1: recursive filtering to obtain

interpolation coefficients:

€

cn+ = cn + zcn−1

+

cn− = z(cn +1

− − cn+)

′ c n = 8cn−

€

c1+ = c1

cN− =

z

z2 −1cN

+

z = 2 2 − 3

12


Quadratic Decoding II

– Step 2: detect zeros

€

0 = λβ 2 + μβ + ν

λ = − ′ c n0 −2 + 2 ′ c n0 −1 − 2 ′ c n0 +1 + ′ c n0 +2( ) 2

μ = − ′ c n0 −2 + 2 ′ c n0− ′ c n0 +2( ) 2

ν = − ′ c n0 −2 − 6 ′ c n0 −1 + 6 ′ c n0 +1 + ′ c n0 +2( ) 8

13


Quadratic Decoding III

– Step 3: compute energy

– Step 4: sort the decoded values according to their energy(the energy represents the confidence)

€

E(n0) = λβ 3 3− μβ 2 2 + νβ +

′ c n0 −2 + 24 ′ c n0 −1 + 46 ′ c n0+ 24 ′ c n0 +1 + ′ c n0 +2( ) 48

The decoded values must be shiftedand rescaled to the original interval

14


Quantization Effect

15


Kernel Density Estimation I

• Given: several realizations of a stochastic variable (samples of the pdf)

• Goal: estimate pdf from samples

• Method: convolve samples with a kernel function

€

˜ p f( ) =1

Nk f − fn( )

n=1

N

∑

16


Kernel Density Estimation II

• Requirements for kernel function:– Non-negative– Integrates to one

• Expectation of estimate:

€

Ε ˜ p f( )[ ] = k f − ′ f ( ) p ′ f ( )∫ d ′ f = k∗p( ) f( )

17


Relation to C.R.

• Adding channel representation of several realizations corresponds to a sampled kernel density estimation

• Ideal interpolation with B-splines possible! €

1

Mcn fm( )

m=1

M

∑ = ˜ p f( )f = n

≈ B2 ∗p( ) f( )f = n

n =1...N

18


L2 vs. Robust Optimization

• Outliers are critical for L2 optimization:

• Idea of robust estimation:– error norm is saturated for outliers– Influence function becomes zero for

outliers

€

f0 = argmin f − f0( )∫2p f( ) df

f0 = f p f( ) df∫Error norm

Influence function

19


Robust Error Norm

E

f - f0

20


Robust Influence Function

E’

f - f0

21


Influence Function of C.R.

€

Ψ f( ) = B2 f −1( ) − B2 f +1( )

Obtained from lineardecoding:

22


Error Norm of C.R.

€

ρ f( ) = 2B3

1

2

⎛

⎝ ⎜

⎞

⎠ ⎟− B3 f +

1

2

⎛

⎝ ⎜

⎞

⎠ ⎟− B3 f −

1

2

⎛

⎝ ⎜

⎞

⎠ ⎟

Obtained by integrating the influence function:

23


Channel Smoothing

24


Channel Smoothing Example

• Discontinuity is preserved

• Constant and linear regions are correctly estimated

25


Stochastic Signals

• Stochastic signal: single realization of a stochastic process

• Ergodicity assumption:– averaging over several realizations at a

single point

can be replaced with– averaging over a neighborhood of a single

realization

26


Ergodicity & C.S.

• Ergodicity often not fulfilled for signals / features, but trivial for channels

• Ergodicity of channels implies that averaging of channels corresponds to (sampled) kernel density estimation

€

cn f ,x( )x

∑ ≈ B2 ∗p( ) f( )f = n

n =1...N

27


Quantization Effect and C.S.

28


Outlier Rejection in C.S.

29


Applications

• Image denoising

• Infilling of information

• Orientation estimation

• Edge detection

• Corner detection

• Disparity estimation

30


Image Denoising

31


Infilling of Information

32


Orientation Estimation

33


Corner Detection

34


Corner Detection

35


Corner Detection

36


Disparity Estimation

37


Disparity Estimation

38


Further Reading

• B-Spline Channel Smoothing for Robust EstimationFelsberg, M., Forssén, P.-E., Scharr, H. LiTH-ISY-R-2579January, 2004

b-spline channels & channel smoothing

Documents