srikumar ramalingam department of computer science university of california, santa cruz...
Post on 20-Dec-2015
218 views
TRANSCRIPT
![Page 1: Srikumar Ramalingam Department of Computer Science University of California, Santa Cruz srikumar@cse.ucsc.edu 3D Reconstruction from a Pair of Images](https://reader030.vdocuments.net/reader030/viewer/2022032704/56649d4a5503460f94a279a9/html5/thumbnails/1.jpg)
Srikumar Ramalingam Department of Computer Science University of California, Santa Cruz [email protected]
3D Reconstruction from a Pair of Images
![Page 2: Srikumar Ramalingam Department of Computer Science University of California, Santa Cruz srikumar@cse.ucsc.edu 3D Reconstruction from a Pair of Images](https://reader030.vdocuments.net/reader030/viewer/2022032704/56649d4a5503460f94a279a9/html5/thumbnails/2.jpg)
- Problem Definition
- Previous Work
- Solution
- Experiments and Results
- Conclusion and Future work
Overview
![Page 3: Srikumar Ramalingam Department of Computer Science University of California, Santa Cruz srikumar@cse.ucsc.edu 3D Reconstruction from a Pair of Images](https://reader030.vdocuments.net/reader030/viewer/2022032704/56649d4a5503460f94a279a9/html5/thumbnails/3.jpg)
Problem Definition
Image-1
Image-2
3D Texture-Mapped Model
Perspective View of the Model
![Page 4: Srikumar Ramalingam Department of Computer Science University of California, Santa Cruz srikumar@cse.ucsc.edu 3D Reconstruction from a Pair of Images](https://reader030.vdocuments.net/reader030/viewer/2022032704/56649d4a5503460f94a279a9/html5/thumbnails/4.jpg)
Previous Work- Zhao, Aggarwal, Mandal and Vemuri, “3D Shape
Reconstruction from Multiple Views ”, Handbook of Image and Video Processing, pages 243-257, Al Bovik, 2000.
- Gang Xu and Zhengyou Zhang, “Epipolar Geometry in Stereo, Motion and Object Recognition”, Kluwer Academic Publishers, 1996.
- Zhang and Faugeras, “3D Dyanamic Scene Analysis-A Stereo Based Approach”, Springer-Verlag, 1992.
- Zhang, Deriche, Faugeras and Luong, “A Robust Technique for Matching Two Uncalibrated Images through the Recovery of the Unknown Epipolar Geometry”, INRIA Research Report, 1994.
- Zhang, “A New Multistage Approach to Motion and Structure Estimation: From Essential Parameters to Euclidean Motion Via Fundamental Matrix”, INRIA Research Report, 1996.
![Page 5: Srikumar Ramalingam Department of Computer Science University of California, Santa Cruz srikumar@cse.ucsc.edu 3D Reconstruction from a Pair of Images](https://reader030.vdocuments.net/reader030/viewer/2022032704/56649d4a5503460f94a279a9/html5/thumbnails/5.jpg)
Previous Work Zhang, “Determining the Epipolar Geometry and its
Uncertainity: A Review”, INRIA Research Report, July 1996. Zhang, “A Flexible New Technique for Camera Calibration”,
Technical Report, Microsoft Research, 1998. Deriche and Giraudon, “A Computational Approach for Corner
and Vertex Detection”, INRIA Research Report, 1992.
![Page 6: Srikumar Ramalingam Department of Computer Science University of California, Santa Cruz srikumar@cse.ucsc.edu 3D Reconstruction from a Pair of Images](https://reader030.vdocuments.net/reader030/viewer/2022032704/56649d4a5503460f94a279a9/html5/thumbnails/6.jpg)
Solution
-Feature Detection
-Getting Initial Set of Matches
-Medium Robust Correspondence
-Strong Robust Correspondence
-Camera Calibration
-3D Reconstruction
![Page 7: Srikumar Ramalingam Department of Computer Science University of California, Santa Cruz srikumar@cse.ucsc.edu 3D Reconstruction from a Pair of Images](https://reader030.vdocuments.net/reader030/viewer/2022032704/56649d4a5503460f94a279a9/html5/thumbnails/7.jpg)
Feature Detection : Harris Corner Detection
Corner Threshold y)R(x,
)C(k.trace - )C(det y)R(x,
I II
II I C
^2
^
^2y
^
yx
^
yx
^ 2x^
![Page 8: Srikumar Ramalingam Department of Computer Science University of California, Santa Cruz srikumar@cse.ucsc.edu 3D Reconstruction from a Pair of Images](https://reader030.vdocuments.net/reader030/viewer/2022032704/56649d4a5503460f94a279a9/html5/thumbnails/8.jpg)
Establishing Initial Set of Matches
![Page 9: Srikumar Ramalingam Department of Computer Science University of California, Santa Cruz srikumar@cse.ucsc.edu 3D Reconstruction from a Pair of Images](https://reader030.vdocuments.net/reader030/viewer/2022032704/56649d4a5503460f94a279a9/html5/thumbnails/9.jpg)
Ambiguities in the Matches
![Page 10: Srikumar Ramalingam Department of Computer Science University of California, Santa Cruz srikumar@cse.ucsc.edu 3D Reconstruction from a Pair of Images](https://reader030.vdocuments.net/reader030/viewer/2022032704/56649d4a5503460f94a279a9/html5/thumbnails/10.jpg)
Robust 1-1 Correspondence
-Medium Robust Matches
-Relaxation Techniques
-Strong Robust Matches
-Epipolar Geometry
![Page 11: Srikumar Ramalingam Department of Computer Science University of California, Santa Cruz srikumar@cse.ucsc.edu 3D Reconstruction from a Pair of Images](https://reader030.vdocuments.net/reader030/viewer/2022032704/56649d4a5503460f94a279a9/html5/thumbnails/11.jpg)
Relaxation Techniques
![Page 12: Srikumar Ramalingam Department of Computer Science University of California, Santa Cruz srikumar@cse.ucsc.edu 3D Reconstruction from a Pair of Images](https://reader030.vdocuments.net/reader030/viewer/2022032704/56649d4a5503460f94a279a9/html5/thumbnails/12.jpg)
- Winner-take-all
- Loser-take-nothing
- Some-winners-take-all - ( 1 – Max_Strength / Sec_Max_St)
Relaxation Strategies
End Result : No Ambiguities but False Matches
![Page 13: Srikumar Ramalingam Department of Computer Science University of California, Santa Cruz srikumar@cse.ucsc.edu 3D Reconstruction from a Pair of Images](https://reader030.vdocuments.net/reader030/viewer/2022032704/56649d4a5503460f94a279a9/html5/thumbnails/13.jpg)
-Epipolar Geometry and Constraint
-Least Median of Squares
Strong Robust Estimation using Epipolar Geometry
![Page 14: Srikumar Ramalingam Department of Computer Science University of California, Santa Cruz srikumar@cse.ucsc.edu 3D Reconstruction from a Pair of Images](https://reader030.vdocuments.net/reader030/viewer/2022032704/56649d4a5503460f94a279a9/html5/thumbnails/14.jpg)
Epipolar Geometry
0: 12Constraint mFmT
Fundamental Matrix (F) –3x3 matrix, which relates the corresponding points
Point corresponding to m lies on its epipolar line lm on the other image
![Page 15: Srikumar Ramalingam Department of Computer Science University of California, Santa Cruz srikumar@cse.ucsc.edu 3D Reconstruction from a Pair of Images](https://reader030.vdocuments.net/reader030/viewer/2022032704/56649d4a5503460f94a279a9/html5/thumbnails/15.jpg)
Least Median of Squares – Removal of Outliers
),(),(
..3,2,1 )}({min
212
122
2
iT
iiii
ij
mFmdFmmdresidualr
mjrmed
- 8 Matches required for estimating F matrix
- Different combinations (m) of 8 matches selected
- Least median of squares algorithm is applied
If ri < Threshold, the match is discarded.
![Page 16: Srikumar Ramalingam Department of Computer Science University of California, Santa Cruz srikumar@cse.ucsc.edu 3D Reconstruction from a Pair of Images](https://reader030.vdocuments.net/reader030/viewer/2022032704/56649d4a5503460f94a279a9/html5/thumbnails/16.jpg)
3D Reconstruction Problem is solved for Conventional Baseline Stereo System
X = b (xl+xr) / (2d)
Y = b (yl+yr) / (2d)
Z = bf / d
![Page 17: Srikumar Ramalingam Department of Computer Science University of California, Santa Cruz srikumar@cse.ucsc.edu 3D Reconstruction from a Pair of Images](https://reader030.vdocuments.net/reader030/viewer/2022032704/56649d4a5503460f94a279a9/html5/thumbnails/17.jpg)
Intrinsic Parameters (5) Extrinsic Parameters (6)
f – focal length 3 rotational parameters, 3 translational parameters
u0, v0 – Center Intrinsic Matrix(A)
ku - unit length along x direction
kv – unit length along y direction
Angle between x and y direction
mnew(u,v) = A mold(x,y)
Need to conduct an experiment to calibrate the camera
Intrinsic and Extrinsic Parameters
0 1 0 0
0 v/sinθfk 0
0 u cotθfkfk
0v
0uu
![Page 18: Srikumar Ramalingam Department of Computer Science University of California, Santa Cruz srikumar@cse.ucsc.edu 3D Reconstruction from a Pair of Images](https://reader030.vdocuments.net/reader030/viewer/2022032704/56649d4a5503460f94a279a9/html5/thumbnails/18.jpg)
3D Reconstruction- Triangulation
Robust Correspondence + Intrinsic Parameters Extrinsic Parameters
Robust Correspondence + Camera Parameters - 3D Points
TR
A
RTAP
IAP
tZYXPms
tZYXPms
TT
TT
,
]['
]0[
],,,[''
],,,[
2
1
Camera MatrixExtrinsic Parameters
![Page 19: Srikumar Ramalingam Department of Computer Science University of California, Santa Cruz srikumar@cse.ucsc.edu 3D Reconstruction from a Pair of Images](https://reader030.vdocuments.net/reader030/viewer/2022032704/56649d4a5503460f94a279a9/html5/thumbnails/19.jpg)
Reconstructed 3D Model
![Page 20: Srikumar Ramalingam Department of Computer Science University of California, Santa Cruz srikumar@cse.ucsc.edu 3D Reconstruction from a Pair of Images](https://reader030.vdocuments.net/reader030/viewer/2022032704/56649d4a5503460f94a279a9/html5/thumbnails/20.jpg)
Implementaton Pipeline
Matlab Implementations-Harris Corner Detection Algorithm (Deriche1992, Zhang1994)
-Initial Set of Matches Establishment (Zhang1994, Xu1996)
-Medium Set of Matches using Relaxation Techniques (Zhang1994, Xu1996)
-Strong Set of Matches using Epipolar Geometry (Zhang1994, Xu1996)
-Camera Calibration Experiment (Zhang1998)
-3D Points Reconstruction from Robust Matches and Camera Parameters (Zhang1994, Zhang1996, Xu1996)
-3D Polygonal Model Reconstruction (Delaunay Triangulation)
- Texture Mapping (OpenGL/C)
![Page 21: Srikumar Ramalingam Department of Computer Science University of California, Santa Cruz srikumar@cse.ucsc.edu 3D Reconstruction from a Pair of Images](https://reader030.vdocuments.net/reader030/viewer/2022032704/56649d4a5503460f94a279a9/html5/thumbnails/21.jpg)
Standard Data Sets- Corner marked
![Page 22: Srikumar Ramalingam Department of Computer Science University of California, Santa Cruz srikumar@cse.ucsc.edu 3D Reconstruction from a Pair of Images](https://reader030.vdocuments.net/reader030/viewer/2022032704/56649d4a5503460f94a279a9/html5/thumbnails/22.jpg)
Robust 1-1 Correspondence shown
![Page 23: Srikumar Ramalingam Department of Computer Science University of California, Santa Cruz srikumar@cse.ucsc.edu 3D Reconstruction from a Pair of Images](https://reader030.vdocuments.net/reader030/viewer/2022032704/56649d4a5503460f94a279a9/html5/thumbnails/23.jpg)
Color Coding for Z Coordinates after 3D Reconstruction
![Page 24: Srikumar Ramalingam Department of Computer Science University of California, Santa Cruz srikumar@cse.ucsc.edu 3D Reconstruction from a Pair of Images](https://reader030.vdocuments.net/reader030/viewer/2022032704/56649d4a5503460f94a279a9/html5/thumbnails/24.jpg)
3D Delaunay Triangulation
![Page 25: Srikumar Ramalingam Department of Computer Science University of California, Santa Cruz srikumar@cse.ucsc.edu 3D Reconstruction from a Pair of Images](https://reader030.vdocuments.net/reader030/viewer/2022032704/56649d4a5503460f94a279a9/html5/thumbnails/25.jpg)
3D Texture Mapped Model – On Rotation
![Page 26: Srikumar Ramalingam Department of Computer Science University of California, Santa Cruz srikumar@cse.ucsc.edu 3D Reconstruction from a Pair of Images](https://reader030.vdocuments.net/reader030/viewer/2022032704/56649d4a5503460f94a279a9/html5/thumbnails/26.jpg)
Real Data Sets and Results
Baskin Engineering Parking Scene – Two Images
![Page 27: Srikumar Ramalingam Department of Computer Science University of California, Santa Cruz srikumar@cse.ucsc.edu 3D Reconstruction from a Pair of Images](https://reader030.vdocuments.net/reader030/viewer/2022032704/56649d4a5503460f94a279a9/html5/thumbnails/27.jpg)
Feature Points using Corner Detection process
![Page 28: Srikumar Ramalingam Department of Computer Science University of California, Santa Cruz srikumar@cse.ucsc.edu 3D Reconstruction from a Pair of Images](https://reader030.vdocuments.net/reader030/viewer/2022032704/56649d4a5503460f94a279a9/html5/thumbnails/28.jpg)
Robust Set of Matches
![Page 29: Srikumar Ramalingam Department of Computer Science University of California, Santa Cruz srikumar@cse.ucsc.edu 3D Reconstruction from a Pair of Images](https://reader030.vdocuments.net/reader030/viewer/2022032704/56649d4a5503460f94a279a9/html5/thumbnails/29.jpg)
Color Coding for Z Coordinates after 3D Reconstruction
Red-Max, Green – Intermediate, Blue – Min depths
![Page 30: Srikumar Ramalingam Department of Computer Science University of California, Santa Cruz srikumar@cse.ucsc.edu 3D Reconstruction from a Pair of Images](https://reader030.vdocuments.net/reader030/viewer/2022032704/56649d4a5503460f94a279a9/html5/thumbnails/30.jpg)
3D Delaunay Triangulation
![Page 31: Srikumar Ramalingam Department of Computer Science University of California, Santa Cruz srikumar@cse.ucsc.edu 3D Reconstruction from a Pair of Images](https://reader030.vdocuments.net/reader030/viewer/2022032704/56649d4a5503460f94a279a9/html5/thumbnails/31.jpg)
Texture Mapped 3D Model of the Scene
Perspective View of the Texture Mapped 3D Model
![Page 32: Srikumar Ramalingam Department of Computer Science University of California, Santa Cruz srikumar@cse.ucsc.edu 3D Reconstruction from a Pair of Images](https://reader030.vdocuments.net/reader030/viewer/2022032704/56649d4a5503460f94a279a9/html5/thumbnails/32.jpg)
Camera Calibration Experiment
-Checker pattern
-3 images taken in different orientations
-Corners are marked
-Computation of camera parameters
![Page 33: Srikumar Ramalingam Department of Computer Science University of California, Santa Cruz srikumar@cse.ucsc.edu 3D Reconstruction from a Pair of Images](https://reader030.vdocuments.net/reader030/viewer/2022032704/56649d4a5503460f94a279a9/html5/thumbnails/33.jpg)
Conclusion and Future Work
-Increasing the number of feature points
- Multiple Images
- Alternate Algorithms
- 3D Reconstruction of Urban Scenes (Faugeras 1995)
- Registration within GIS Data
![Page 34: Srikumar Ramalingam Department of Computer Science University of California, Santa Cruz srikumar@cse.ucsc.edu 3D Reconstruction from a Pair of Images](https://reader030.vdocuments.net/reader030/viewer/2022032704/56649d4a5503460f94a279a9/html5/thumbnails/34.jpg)
Questions?