ROBUST VIDEO STABILIZATION BASED ON PARTICLE FILTER

TRACKING OF PROJECTED CAMERA MOTION

IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, 2009

P48981023

冷玉琦 Yu-Chi Leng

2011/01/16

Keywords: Bootstrap filtering,motion analysis, Particle filtering, video stabilization

ABSTRACT

We propose a novel technique for video stabilization based on the particle filtering framework.

Extend the traditional use of particle filters in object tracking to tracking of the projected affine model of the camera motions.

The correspondence between scale-invariant feature transform points is used to obtain a crude estimate of the projected camera motion.

INTRODUCTION

Video cameras mounted on handheld devices and mobile platforms have become increasingly popular in the consumer market over the past few years due to a dramatic decrease in the cost of such devices.

Stabilization methods exploit the fact that camera motion causes the affine transform of the frames, which can be inverted to obtain stable frames.

THEORETICAL FOUNDATIONS

A. Camera Model

攝影機移動前後向量關係13

T

00033

T

111 ][][ xx TzyxRzyx

T

000

0

T

00 ][][ zyxz

vu

T

111

1

T

11 ][][ zyxz

vu

y

x

t

t

v

u

RR

RRs

v

u

0

0

2221

1211

1

1

011 2

22

2

21

2

12

2

1122122111 RRRRRRRR

6 個自由度 !

THEORETICAL FOUNDATIONS

B. Particle Filtering Estimation狀態向量：事後機率密度函數：

估算當前狀態：

重要性密度：

T

ykxkkkkkk ttRRRs ],,,,,[ 211211x

)( :1 kkp zx

N

i

i

kk

i

kkk wp1

:1 )()( xxzx

)()( :1:1 kkkk

i

k qpw zxzx

N

i

i

k

i

kk

N

i

i

kk

i

kkkkkkkk wdwdpE11

:1 )()(ˆ xxxxxxzxxxx

),(~ 1k

i

k q xx

THEORETICAL FOUNDATIONS

誤差向量

協方差矩陣kkk xx ˆ kkk xxe

),( kkCov ),( kkCov ee

kkk cN

Cov )(1

),( 21

2),(),( kkkk CovCov ee

2

2

2

2

2

1

1jkjj c

N

O

kjjjkMj NcN 2

2

2

2

2

1...1max

VIDEO STABILIZATION

A. Importance Density Using Scale-Invariant Features影像間的特徵點將使用 SIFT 獲得。使用特徵追蹤來得到中值向量： T

ykxkkkkkk ttRRRs ],,,,,[ 211211x

kk yx

kkkk

kkkk

kkkk

tt

RsRs

RsRsvu

svu

2221

1211

11

1....

1

....

k

i

k

T

k

i

kkG

i

k q xxxxxx 1

1

1

61 2

1exp

)2(

1),(~

有效的減少粒子數至 30 個並有同等或超越 300 個粒子所得的品質。

VIDEO STABILIZATION

B. Particle Filtering for Global Motion Estimation Between Successive Frames-權重值依粒子有多靠近真實的狀態來給定。- 選擇均方差 (mean square error, MSE) 及特徵距離做為

相似度的兩個量測量。

2

2

2exp

2

1

M

i

M

i

MSE

M

σP

2

2

2exp

2

1

F

i

F

i

feature

D

σP

Ni

i

feature

i

MSE

i

feature

i

MSEi

k PP

PPw

1

VIDEO STABILIZATION

利用離散權重近似真實狀態，得到仿射動態參數：

尺度因子、旋轉矩陣和位移與參考影像的關係：

T

ykxkkkkkk ttRRRs ]ˆˆˆˆˆˆ[ˆ 211211x

sss A

k

A

k ˆ1

kk

kkA

k

A

kRR

RRRR

2221

12111 ˆˆ

ˆˆ

k

k

y

xA

k

kk

kkk

A

k t

tT

RR

RRsT

ˆ

ˆ

ˆˆ

ˆˆˆ 1

2221

1211

VIDEO STABILIZATION

C. Intentional Motion Estimation and Motion Compensation沿著 x 方向的平移和平移速度 將可表示為：

補償非預期動態：

nTx

Tx

Tx

Txv

k

k

v

k

k 0

10

11

1

1

D

k

A

k

k

kA

k

D

kA

k

D

k

s

s TTv

uRR

s

sv

u

1

][ kk ,vu

][ ss ,vu

不穩定影像像素點

補償後穩定影像像素點

VIDEO STABILIZATION

Algorithm：

EXPERIMENTAL RESULTS

EXPERIMENTAL RESULTS

EXPERIMENTAL RESULTS

EXPERIMENTAL RESULTS

EXPERIMENTAL RESULTS

EXPERIMENTAL RESULTS

CONCLUSION

In this paper, we presented a novel approach for robust video stabilization based on particle filter estimation of projected camera motion.

An efficient implementation of particle filters for global motion estimation has been proposed based on carefully designed importance sampling.

We demonstrated experimentally that the proposed particle filtering scheme can be used to obtain an efficient and accurate motion estimation in video sequences.