3-d depth reconstruction from a single still image 何開暘 2010.6.11
Post on 19-Dec-2015
222 views
TRANSCRIPT
![Page 1: 3-D Depth Reconstruction from a Single Still Image 何開暘 2010.6.11](https://reader035.vdocuments.net/reader035/viewer/2022062407/56649d395503460f94a131b4/html5/thumbnails/1.jpg)
3-D Depth Reconstruction from a Single Still Image
何開暘 2010.6.11
![Page 2: 3-D Depth Reconstruction from a Single Still Image 何開暘 2010.6.11](https://reader035.vdocuments.net/reader035/viewer/2022062407/56649d395503460f94a131b4/html5/thumbnails/2.jpg)
Visual Cues for Depth Perception
Monocular Cues Texture variations, texture gradients, interposition, occlusion,
known object sizes, light and shading, haze, defocus
Stereo Cues Motion Parallax and Focus Cues
![Page 3: 3-D Depth Reconstruction from a Single Still Image 何開暘 2010.6.11](https://reader035.vdocuments.net/reader035/viewer/2022062407/56649d395503460f94a131b4/html5/thumbnails/3.jpg)
image → feature → depth
Chose features that capture 3 types of cues: texture variations, texture gradients, and color
Model conditional distribution of depths given monocular image features p(d|x)
Estimate parameters by maximizing conditional log likelihood of training data
Given an image, find MAP estimate of depths
![Page 4: 3-D Depth Reconstruction from a Single Still Image 何開暘 2010.6.11](https://reader035.vdocuments.net/reader035/viewer/2022062407/56649d395503460f94a131b4/html5/thumbnails/4.jpg)
Outline
Introduction Feature Vector Probabilistic Model Experiments Reference
![Page 5: 3-D Depth Reconstruction from a Single Still Image 何開暘 2010.6.11](https://reader035.vdocuments.net/reader035/viewer/2022062407/56649d395503460f94a131b4/html5/thumbnails/5.jpg)
Feature vectors
Two types of features Absolute depth features―used to estimate absolute depth at a p
articular patch Relative features―used to estimate relative depths
Capture three types of cues Texture variation―apply Law’s masks to intensity channel Haze―apply a local averaging filter to color channels Texture gradient―apply six oriented edge filters to intensity cha
nnel
![Page 6: 3-D Depth Reconstruction from a Single Still Image 何開暘 2010.6.11](https://reader035.vdocuments.net/reader035/viewer/2022062407/56649d395503460f94a131b4/html5/thumbnails/6.jpg)
Features for Absolute Depth
Compute summary statistics of a patch i in the image I(x,y) as follows Use the output of each of the 17 (9 Law’s masks, 2 color channe
ls and 6 texture gradients) filters Fn, n=1,…,17 as:
(dimension 34)
To estimate absolute depth at a patch, local image features centered on the patch are insufficient
Use more global properties
![Page 7: 3-D Depth Reconstruction from a Single Still Image 何開暘 2010.6.11](https://reader035.vdocuments.net/reader035/viewer/2022062407/56649d395503460f94a131b4/html5/thumbnails/7.jpg)
More Global Properties
Use image features extracted at multiple spatial scales (three scale)
Features used to predict depth of a particular patch are computed from that patch as well as 4 neighboring patches (Repeated at each of the three scales)
Add to features of a patch additional summary features of the column it lies in
(5*3+4)*34=636 dimensional
![Page 8: 3-D Depth Reconstruction from a Single Still Image 何開暘 2010.6.11](https://reader035.vdocuments.net/reader035/viewer/2022062407/56649d395503460f94a131b4/html5/thumbnails/8.jpg)
![Page 9: 3-D Depth Reconstruction from a Single Still Image 何開暘 2010.6.11](https://reader035.vdocuments.net/reader035/viewer/2022062407/56649d395503460f94a131b4/html5/thumbnails/9.jpg)
Features for Relative Depth
To learn the dependencies between two neighboring patches
Compute a 10-bin histogram of each of the 17 filter outputs , giving a total of 170 features yis for each patch i at scale s
Relative depth features yijs for two neighboring patches i and j at scale s will be the differences between their histogram, i.e., yijs=yis-yjs
![Page 10: 3-D Depth Reconstruction from a Single Still Image 何開暘 2010.6.11](https://reader035.vdocuments.net/reader035/viewer/2022062407/56649d395503460f94a131b4/html5/thumbnails/10.jpg)
Outline
Introduction Feature Vector Probabilistic Model Experiments Reference
![Page 11: 3-D Depth Reconstruction from a Single Still Image 何開暘 2010.6.11](https://reader035.vdocuments.net/reader035/viewer/2022062407/56649d395503460f94a131b4/html5/thumbnails/11.jpg)
Gaussian Model
![Page 12: 3-D Depth Reconstruction from a Single Still Image 何開暘 2010.6.11](https://reader035.vdocuments.net/reader035/viewer/2022062407/56649d395503460f94a131b4/html5/thumbnails/12.jpg)
Laplacian Model
![Page 13: 3-D Depth Reconstruction from a Single Still Image 何開暘 2010.6.11](https://reader035.vdocuments.net/reader035/viewer/2022062407/56649d395503460f94a131b4/html5/thumbnails/13.jpg)
Outline
Introduction Feature Vector Probabilistic Model Experiments Reference
![Page 14: 3-D Depth Reconstruction from a Single Still Image 何開暘 2010.6.11](https://reader035.vdocuments.net/reader035/viewer/2022062407/56649d395503460f94a131b4/html5/thumbnails/14.jpg)
![Page 15: 3-D Depth Reconstruction from a Single Still Image 何開暘 2010.6.11](https://reader035.vdocuments.net/reader035/viewer/2022062407/56649d395503460f94a131b4/html5/thumbnails/15.jpg)
Result
![Page 16: 3-D Depth Reconstruction from a Single Still Image 何開暘 2010.6.11](https://reader035.vdocuments.net/reader035/viewer/2022062407/56649d395503460f94a131b4/html5/thumbnails/16.jpg)
Improving Performance of Stereovision using Monocular Cues
![Page 17: 3-D Depth Reconstruction from a Single Still Image 何開暘 2010.6.11](https://reader035.vdocuments.net/reader035/viewer/2022062407/56649d395503460f94a131b4/html5/thumbnails/17.jpg)
![Page 18: 3-D Depth Reconstruction from a Single Still Image 何開暘 2010.6.11](https://reader035.vdocuments.net/reader035/viewer/2022062407/56649d395503460f94a131b4/html5/thumbnails/18.jpg)
The average errors as a function of the distance from the camera
![Page 19: 3-D Depth Reconstruction from a Single Still Image 何開暘 2010.6.11](https://reader035.vdocuments.net/reader035/viewer/2022062407/56649d395503460f94a131b4/html5/thumbnails/19.jpg)
Reference
A.Y. Ng A. Saxena, S.H. Chung. 3-d depth reconstruction from a single still image. In International Journal of Computer Vision (IJCV), 2007.
Michels, J., Saxena, A., & Ng, A. Y. (2005). High speed obstacle avoidance using monocular vision and reinforcement learning. In 22nd international conference on machine learning (ICML).