sterio vision
TRANSCRIPT
-
8/18/2019 Sterio Vision
1/45
ISAN-DSP GROUP
IMAGE REGISTRATION
-
8/18/2019 Sterio Vision
2/45
ISAN-DSP GROUP
Course Syllabus
Date and Time : MW 11.00-12.00 EN 4510, LAB1 TU17-20, LAB2 TH17-20
Assessments:
Attendane ! H"#e$"%& 5'
La( and H"#e$"%& )5'
M*dte%# )0'
+*na )0'
Grading:5-100' A, 75-5' B, 70-75' B, /5-70' , /0-/5' ,
55-/0' , 50-55' , 0-50' +
References:
1. Raae . G"n3ae3 and R*a%d E. W""d, 6**ta I#ae 8%"e*n9,
Add*"n Wee, 1;;2.
2. An* nda#enta " **ta I#ae 8%"e*n9,
8%ent*e-Ha, In., 1;;.
). W**a#
-
8/18/2019 Sterio Vision
3/45
ISAN-DSP GROUP
Course Outline
1. Int%"d>t*"n
2. **ta I#ae +>nda#enta
). I#ae T%an"%#
4. I#ae Enane#ent
5. I#ae Se#entat*"n
/. I#ae "#?%e*"n
7. I#ae M"%?""
-
8/18/2019 Sterio Vision
4/45
ISAN-DSP GROUP
a?te% 1
Int%"d>t*"n t" I#ae 8%"e*n
-
8/18/2019 Sterio Vision
5/45
ISAN-DSP GROUP
What is Digital Image Processing ?
8%"e*n " a #>t*d*#en*"na ?*t>%e ( a d**ta "#?>te%
การประมวลผลสญญาณรปภาพโดยใชดจตอลคอมพวเตอร
Why we need Digital Image Processing ?
1. เพ อ!"#ก$ละจดเก%ภาพ2. เพ อปรปร&'ภาพใ(ด)*# +!โดยใช
กระว!การ"า'คณต,าสตร). เพ อช-วยใ!การวเคราะ(รปภาพ4. เพ อส'เคราะ(ภาพ5. เพ อสรา'ระการมอ'เ(%!ใ(ก
คอมพวเตอร
-
8/18/2019 Sterio Vision
6/45
-
8/18/2019 Sterio Vision
7/45
ISAN-DSP GROUP
Visal Perce!tion" #man $ye
8*t>%e %"# M*%""t Ena%ta 2000
-
8/18/2019 Sterio Vision
8/45
ISAN-DSP GROUP
Chapter 2: Digital Image Fundamentals
-
8/18/2019 Sterio Vision
9/45
ISAN-DSP GROUP
Visal Perce!tion" #man $ye %cont&'
1. Te lens "nta*n /0-70' $ate%, /' " at.
2. Te iris d*a?%a# "nt%" a#">nt " *t tat ente% te ee.
3. Light receptors *n te retina
- A(">t /-7 #**"n cones "% (%*t *t C**"n aed photopic - en*t " "ne * a(">t 150,000 ee#ent##2.
- "ne *nC"Ce *n ""% C**"n.
- "ne a%e "nent%ated *n fovea a(">t 1.5D1.5 ##2.
- A(">t 75-150 #**"n rods "% d*# *t C**"n aed scotopic
- R"d a%e en*t*Ce t" "$ eCe " *t and a%e n"t *nC"Ced ""% C**"n.
4. Blind spot * te %e*"n " e#e%ene " te "?t* ne%Ce %"# te ee.
-
8/18/2019 Sterio Vision
10/45
ISAN-DSP GROUP
Chapter 2: Digital Image Fundamentals
-
8/18/2019 Sterio Vision
11/45
ISAN-DSP GROUP
Image (ormation in #man $ye
-
8/18/2019 Sterio Vision
12/45
ISAN-DSP GROUP
Chapter 2: Digital Image Fundamentals
-
8/18/2019 Sterio Vision
13/45
ISAN-DSP GROUP
)rightness Ada!tation o* #man $ye
Position
I n t e n s i t y
Inten*t*e " >%%">nd*n ?"*nt eet ?e%e*Ced (%*tne at ea ?"*nt.
In t* *#ae, ede (et$een (a% a??ea% (%*te% "n te %*t *de and da%&e%
On te et *de.
-
8/18/2019 Sterio Vision
14/45
ISAN-DSP GROUP
In a%ea A, (%*tne ?e%e*Ced * da%&e% $*e *n a%ea B *
(%*te%. T* ?en"#en"n * aed Mach Band Effect .
)rightness Ada!tation o* #man $ye %cont&'
8"*t*"n
I n t e n 5 * t :
AB
-
8/18/2019 Sterio Vision
15/45
ISAN-DSP GROUP
)rightness Ada!tation o* #man $ye %cont&'
Simultaneous contrast . A #a F>a%e aCe eDat te a#e *nten*t
(>t te a??ea% ?%"%e*Ce da%&e% a (a&%">nd (e"#e *te%.
-
8/18/2019 Sterio Vision
16/45
ISAN-DSP GROUP
Imaging Geometry" Pers!ecti+e ,rans*ormation
Len ente%
x,X y,Y
x, y
z,Z
X,Y,Z @ $"%d ""%d*nate
λ
x,y,z @ a#e%a ""%d*nate te#
X,Y,Z
λ @ "a ent
I#ae ?ane
λ λ
λ λ
−−=
−−=
Z
Y y
Z
X x
EF. 1.1
-
8/18/2019 Sterio Vision
17/45
ISAN-DSP GROUP
−
−
−
=
=
Z Z
Z
Y Z
X
z
y
x
c
λ λ
λ
λ λ
λ
Imaging Geometry" Pers!ecti+e ,rans*ormation %cont&'
>et*"n: H"$ an $e ?%"et te %ea $"%d "(et at ,J,K "nt"te *#ae ?ane > a ?"t"%a?* *#
An$e%: Reat*"n (et$een a#e%a ""%d*nate x,y,z and $"%d ""%d*nate
X,Y,Z a%e *Cen (
S*ne "n te *#ae ?ane z * a$a 3e%", z=0, $e "n*de% "n D,
$*e 3 * neeted.
EF. 1.2
-
8/18/2019 Sterio Vision
18/45
ISAN-DSP GROUP
Imaging Geometry" Pers!ecti+e ,rans*ormation %cont&'
EF>at*"n 1.2 * n"t *nea% (ea>e " K *n te d*C*de% " $e
*nt%"d>e te homogeneous coordinate t" "Ce t* ?%"(e#.
=
Z
Y
X
wa%te*an
""%d*nate
=
k kZ
kY
kX
whH"#"ene">
""%d*nate
k @ n"n3e%" "ntant
T" "nCe%t %"# te "#"ene"> ""%d*nate wh t" te a%te*an
""%d*nate w, $e d*C*de te *%t ) "#?"nent " wh ( te ">%t
"#?"nent.
-
8/18/2019 Sterio Vision
19/45
ISAN-DSP GROUP
−−
=
−
==
λ
λ
λ
100
0100
0010
0001
1 Z k kZ
kY
kX
k
kZ
kY
kX
Pwc hh
Imaging Geometry" Pers!ecti+e ,rans*ormation %cont&'
−
=
100
0100
0010
0001
1λ
P
Te ?e%?et*Ce t%an"%#at*"n #at%*D "% te "#"ene"> ""%d*nate:
8e%?et*Ce t%an"%#at*"n (e"#e:
EF. 1.)
-
8/18/2019 Sterio Vision
20/45
ISAN-DSP GROUP
−−
=
λ
λ Z k kZ
kY kX
c h
Imaging Geometry" Pers!ecti+e ,rans*ormation %cont&'
=
−
−
−
=
−−⋅
−−⋅
−−⋅
=
z
y
x
Z
Z Z
Y Z
X
Z k kZ
Z k kY
Z k kX
c
λ
λ
λ
λ
λ
λ
λ
λ
λ
λ
λ
λ
+%"# "#"ene">
""%d*nate
We et a#e%a ""%d*nate *n te *#ae ?ane:
-
8/18/2019 Sterio Vision
21/45
ISAN-DSP GROUP
Imaging Geometry" In+erse Pers!ecti+e ,rans*ormation
=−
100
01000010
0001
1
1
λ
P hh c P w
1−= $e%e EF. 1.4
-
8/18/2019 Sterio Vision
22/45
ISAN-DSP GROUP
+"% an *#ae ?"*nt x0 ,y0, *ne "n
te *#ae ?ane z @0, $e aCe
=
k
kykx
ch0
0
0
In+erse Pers!ecti+e ,rans*ormation %cont&'
LLL
0
"%0 0
0
0
0
1 →
=
== − y
x
w
k
ky
kx
c P whh
We et te $"%d ""%d*nate :
Since the perspective transformation maps 3-D coorinates to !-D "oorinates,
we cannot #et the inverse transform $n%ess we have aitiona% information&
-
8/18/2019 Sterio Vision
23/45
ISAN-DSP GROUP
T" *nd te ">t*"n, et
=
k
kz
ky
kx
ch0
0
In+erse Pers!ecti+e ,rans*ormation %cont&'
We et
+=
+
=
+=
=
+
== −
z
z Z
z
yY
z
x X
w
z k kz
ky
kx
c P w hh
λ
λ λ
λ λ
λ
λ
λ
0
0
0
0
1 "%
BA
EF. 1.5
-
8/18/2019 Sterio Vision
24/45
ISAN-DSP GROUP
0
0
Z y
Y
Z x X
−=
−=
λ λ
λ λ
In+erse Pers!ecti+e ,rans*ormation %cont&'
+%"# EF. 1.5,
Z
Z z
−=
λ
λ We et EF. 1./
S>(t*t>t*n EF. 1./ *nt" EF.1.5, $e et
EF. 1.7
EF>at*"n 1.7 "$ tat *nCe%e ?e%?et*Ce t%an"%#at*"n %eF>*%e
*n"%#at*"n " at eat "ne "#?"nent " te $"%d ""%d*nate " te ?"*nt.
-
8/18/2019 Sterio Vision
25/45
ISAN-DSP GROUP
In+erse Pers!ecti+e ,rans*ormation %cont&'
BA and BA00
Z
y
Y Z
x
X −=−= λ
λ λ
λ Tee eF>at*"n:"$ tat ?"*nt "n L*ne ' *n te $"%d ""%d*nate
#a? t" a *ne ?"*nt *n te *#ae ?ane.
Len ente%
x,X y,Y
z,Z
x, y
λ
I#ae ?ane
X ! ,Y ! ,Z !
X 3 ,Y 3 ,Z 3
8"*nt X ( ,Y ( ,Z ( , X ! ,Y ! ,Z !, and X 3 ,Y 3 ,Z 3 #a? t"
8"*nt x,y *n te *#ae ?ane.
X ( ,Y ( ,Z (
Line L
-
8/18/2019 Sterio Vision
26/45
-
8/18/2019 Sterio Vision
27/45
ISAN-DSP GROUP
Stereo Imaging" #ow we get de!th in*ormation *rom
eyes %cont&'
Problem: $e &n"$ a#e%a ""%d*nate " te "(et "n et and%*t *#ae ?ane x(, y( and x!, y! and $ant t" "$ a% %"# te
a#e%a te "(et * "ated.
Note: $en -aD* * ?a%ae t" te %">nd, $e aCe y( = y!
x!, y!
x(, y(
λ
λ ) w
Z
I#ae 2
I#ae 1
8ane "
"ntant Z
X
O%**n " $"%d
""%d*natete#
-
8/18/2019 Sterio Vision
28/45
ISAN-DSP GROUP
1. +%"# te *nCe%e ?e%?et*Ce t%an"%#, $e "#?>te X ( and X !:
BA and BA 22
211
1 Z x
X Z x
X −=−= λ λ
λ λ
). S*ne et and %*t ene a%e e?a%ated ( d*tane ) , $e aCe
) X X += 12
2. Z ( and Z ! #>t (e eF>a, $e et 21 Z Z =
Stereo Imaging" #ow we get de!th in*ormation *rom
eyes %cont&'
4. +%"# 1, 2 and ), $e et
BA and BA 211
1 Z x ) X Z x X −=+−= λ λ
λ λ
S"C*n K *ed
12 x x
) Z
−−= λ
λ
-
8/18/2019 Sterio Vision
29/45
-
8/18/2019 Sterio Vision
30/45
ISAN-DSP GROUP
Imaging Geometry " A**ine ,rans*ormations
1. Translation
2. Scaling
3. otating
-
8/18/2019 Sterio Vision
31/45
ISAN-DSP GROUP
Image Geometry" ,ranslation o* O./ect
Y
Z
X
*?ae te "(et ( Cet"% X 0,Y 0, Z 0 $*t %e?et t"
*t "d ?"*t*"n.
=
11000
100
010
001
1
0
0
0
Z
Y
X
Z
Y
X
Z
Y
X
0
0
0
Z Z Z
Y Y Y
X X X
+=
+=
+=
X X 0,Y Y 0, Z Z 0
X ,Y , Z
A X 0 , Y
0 , Z 0 B
-
8/18/2019 Sterio Vision
32/45
ISAN-DSP GROUP
Image Geometry" ,ranslation o* (rame
Y
Z
X
0,0,0
T%anate te "%**n ?"*nt " te %a#e ( X 0,Y 0, Z 0
$*t %e?et t" te "d %a#e
−
−
−
=
11000
100
010
001
1
0
0
0
Z
Y
X
Z
Y
X
Z
Y
X
0
0
0
Z Z Z
Y Y Y
X X X
−=
−=
−=
Y ∗
Z ∗
X ∗ A X 0
, Y 0 , Z
0 B
Te "(et t* ta at te a#e ?"*t*"n. On te %a#e * #"Ced.
-
8/18/2019 Sterio Vision
33/45
-
8/18/2019 Sterio Vision
34/45
ISAN-DSP GROUP
Image Geometry" Rotating an o./ect a.ot 0-a1is
R"tate an "(et a(">t X -aD* ( x *n a ">nte%"&$*e
d*%et*"n.
Y
Z
X= X ∗
x
−=
11000
0"*n0
0*n"0
0001
1
Z
Y
X
Z
Y
X
x x
x x
θ θ
θ θ
N"te : In t* ae te "(et * #"Ced. On y and z
a%e aned $*e x t* te a#e.
-
8/18/2019 Sterio Vision
35/45
ISAN-DSP GROUP
Image Geometry" Rotating a *rame a.ot 0-a1is
R"tate te %a#e a(">t X -aD* ( x *n a ">nte%"&$*e
d*%et*"n.
Y
Z
x
−
=
110000"*n0
0*n"0
0001
1
Z
Y
X
Z
Y
X
x x
x x
θ θ
θ θ Y ∗
Z ∗
N"te : In t* ae te "(et * n"t #"Ced. Te %a#e
* %"tated *ntead.
X= X ∗
-
8/18/2019 Sterio Vision
36/45
ISAN-DSP GROUP
Image Geometry" Rotating an o./ect a.ot 2-a1is
R"tate an "(et a(">t Y -aD* ( y *n a ">nte%"&$*e
d*%et*"n.
Z
X
y
−=
11000
0"0*n
0010
0*n0"
1
Z
Y
X
Z
Y
X
y y
y y
θ θ
θ θ
N"te : In t* ae te "(et * #"Ced. On x and z
a%e aned $*e y t* te a#e.
Y= Y ∗
-
8/18/2019 Sterio Vision
37/45
ISAN-DSP GROUP
Image Geometry" Rotating a *rame a.ot 2-a1is
R"tate te %a#e a(">t Y -aD* ( y *n a ">nte%"&$*e
d*%et*"n.
Y= Y ∗
Z
X
y
N"te : In t* ae te "(et * n"t #"Ced. Te %a#e
* %"tated *ntead.
Z ∗
X ∗
−
=
11000
0"0*n
0010
0*n0"
1
Z
Y
X
Z
Y
X
y y
y y
θ θ
θ θ
-
8/18/2019 Sterio Vision
38/45
ISAN-DSP GROUP
Image Geometry" Rotating an o./ect a.ot 3-a1is
R"tate an "(et a(">t Z -aD* ( z *n a ">nte%"&$*e
d*%et*"n.
Y
Z
X
z
N"te : In t* ae te "(et * #"Ced. On x and y
a%e aned $*e z t* te a#e.
−
=
11000
0100
00"*n
00*n"
1
Z
Y
X
Z
Y
X
z z
z z
θ θ
θ θ
-
8/18/2019 Sterio Vision
39/45
ISAN-DSP GROUP
Image Geometry" Rotating a *rame a.ot 3-a1is
R"tate te %a#e a(">t Z -aD* ( z *n a ">nte%"&$*e
d*%et*"n.
Y
Z= Z ∗
X
z
Y ∗
N"te : In t* ae te "(et * n"t #"Ced. Te %a#e
* %"tated *ntead.
−=
11000
0100
00"*n
00*n"
1
Z
Y
X
Z
Y
X
z z
z z
θ θ
θ θ
X ∗
-
8/18/2019 Sterio Vision
40/45
ISAN-DSP GROUP
X
Y
Z
W"%d ""%d*nate Ste#
Image Geometry" #ow to com!te a !oint on an image
!lane *rom the world coordinate
Pro*%em+ we (now t)e location o" t)e ob*ect
an! want to (now w)ere it will be +ro*ecte!
on t)e "ilm ,image +lane-.
y
z
x
0 a # e % a 0 " " % d * n a t e
S : 5 t e #
nswer+
1. Trans"orm t)e worl! coor!inate tot)e camera coor!inate
2. Per"orm t)e +ers+ectie trans"ormation
-
8/18/2019 Sterio Vision
41/45
ISAN-DSP GROUP
Image Geometry" #ow to com!te a !oint on the image
!lane *rom the world coordinate %cont&'
Len ente%
x
y
x, y
z λ
x,y,z @ a#e%a ""%d*nate, X,Y,Z @ W"%d ""%d*nate
I#ae ?ane z @0
#e"ore using t)e +ers+ectie trans"ormation/
t)e worl! a0es X--! must coinci!e wit) t)e camera
a0es x-y-z / ,we nee! some trans"ormations-.
X,Y,Z. X
Z
Y
-
8/18/2019 Sterio Vision
42/45
ISAN-DSP GROUP
Image Geometry" 4om!te the camera coordinate *rom
the world coordinate
X
Y
Z
W"%d ""%d*nate Ste#
y
z
x
0 a # e % a 0 " " % d * n a t e
S : 5 t e #
2
3
'imbal center
1. Translate by "#
Note : +er"orm on t)e "rame
2. Pan t)e camera
,rotate about ! a0is-
. Translate by 4 5 r
Ste+s "rom 'on4ale46s boo(
3. Tilt t)e camera
,rotate about X a0is-
7. Com+ute t)e
+ers+ectie Tr.
1 " #
r
-
8/18/2019 Sterio Vision
43/45
ISAN-DSP GROUP
&ormula "rom 'on4ale46s boo(
hh w/ " P c ⋅⋅⋅⋅=
Pers+ectie tr.
Translate to t)e
image +lane
Center by z 5 r
Translate to t)egimbal center "#
8orl! coor!inate
Camera coor!inate
Image Geometry" 4om!te the camera coordinate *rom
the world coordinate %cont&'
otate by
Pan , z -
an! Tilt , x -
-
8/18/2019 Sterio Vision
44/45
ISAN-DSP GROUP
X
Y
Z
W"%d ""%d*nate Ste#
y
z
x
0 a # e % a 0 " " % d * n a t e
S : 5 t e #
2
3
'imbal center
1. Translate by "$
2. Pan t)e camera
3. Tilt t)e camera
Note : +er"orm on t)e "rame
'eneral case
. Twist t)e camera
,rotate about a0is-
7. Com+ute t)e
+ers+ectie Tr.
"$ 1
Image Geometry" 4om!te the camera coordinate *rom
the world coordinate %cont&'
-
8/18/2019 Sterio Vision
45/45
ISAN-DSP GROUP
Image Geometry" 4om!te the camera coordinate *rom
the world coordinate %cont&'
'eneral case
hh w1 P c ⋅⋅⋅=
Pers+ectie tr.
Translate tot)e image
+lane center "$
8orl! coor!inate
Camera coor!inate
otate by
Pan ,
z -/Tilt , x -/
Twist , y-