scene reconstruction seminar presented by anton jigalin advanced topics in computer vision ( 048921...
TRANSCRIPT
![Page 1: Scene Reconstruction Seminar presented by Anton Jigalin Advanced Topics in Computer Vision ( 048921 )](https://reader035.vdocuments.net/reader035/viewer/2022062515/56649f555503460f94c79170/html5/thumbnails/1.jpg)
Scene Reconstruction
Seminar presented by Anton Jigalin
Advanced Topics in Computer Vision (048921)
![Page 2: Scene Reconstruction Seminar presented by Anton Jigalin Advanced Topics in Computer Vision ( 048921 )](https://reader035.vdocuments.net/reader035/viewer/2022062515/56649f555503460f94c79170/html5/thumbnails/2.jpg)
2
Scene Reconstruction
Scene Visualization:
Photo Tourism: 3D photo browser
Path Finder: 3D navigation interface
References: Scene Reconstruction and Visualization from Internet Photo Collections. N. Snavely. 2008.
![Page 3: Scene Reconstruction Seminar presented by Anton Jigalin Advanced Topics in Computer Vision ( 048921 )](https://reader035.vdocuments.net/reader035/viewer/2022062515/56649f555503460f94c79170/html5/thumbnails/3.jpg)
3
Scene Reconstruction
![Page 4: Scene Reconstruction Seminar presented by Anton Jigalin Advanced Topics in Computer Vision ( 048921 )](https://reader035.vdocuments.net/reader035/viewer/2022062515/56649f555503460f94c79170/html5/thumbnails/4.jpg)
4
The goal
Automatically reconstruct 3D geometry from large collections of photos from the Internet
![Page 5: Scene Reconstruction Seminar presented by Anton Jigalin Advanced Topics in Computer Vision ( 048921 )](https://reader035.vdocuments.net/reader035/viewer/2022062515/56649f555503460f94c79170/html5/thumbnails/5.jpg)
Motivation
5
New way to organize photo collections Experience of being at a place
![Page 6: Scene Reconstruction Seminar presented by Anton Jigalin Advanced Topics in Computer Vision ( 048921 )](https://reader035.vdocuments.net/reader035/viewer/2022062515/56649f555503460f94c79170/html5/thumbnails/6.jpg)
6
Related works
Image based modeling
Structure from motion (SfM)
Moviemaps
![Page 7: Scene Reconstruction Seminar presented by Anton Jigalin Advanced Topics in Computer Vision ( 048921 )](https://reader035.vdocuments.net/reader035/viewer/2022062515/56649f555503460f94c79170/html5/thumbnails/7.jpg)
7
Challenges
Unknown cameras position
Unknown internal camera parameters
Correspondence problem
Visualization
Algorithmic complexity
Running the complete pipeline on a 3.80GHz Intel Xeon machine with 4GB of core memory. The keypoint detection and matching phases were run in parallel on 10 such machines.
![Page 8: Scene Reconstruction Seminar presented by Anton Jigalin Advanced Topics in Computer Vision ( 048921 )](https://reader035.vdocuments.net/reader035/viewer/2022062515/56649f555503460f94c79170/html5/thumbnails/8.jpg)
8
Results
![Page 9: Scene Reconstruction Seminar presented by Anton Jigalin Advanced Topics in Computer Vision ( 048921 )](https://reader035.vdocuments.net/reader035/viewer/2022062515/56649f555503460f94c79170/html5/thumbnails/9.jpg)
9
How was this result achieved?
![Page 10: Scene Reconstruction Seminar presented by Anton Jigalin Advanced Topics in Computer Vision ( 048921 )](https://reader035.vdocuments.net/reader035/viewer/2022062515/56649f555503460f94c79170/html5/thumbnails/10.jpg)
10
Finding correspondence
Feature detection - SIFT
Feature matching - RANSAC
Epipolar constraint
![Page 11: Scene Reconstruction Seminar presented by Anton Jigalin Advanced Topics in Computer Vision ( 048921 )](https://reader035.vdocuments.net/reader035/viewer/2022062515/56649f555503460f94c79170/html5/thumbnails/11.jpg)
11
Structure from Motion Algorithm
![Page 12: Scene Reconstruction Seminar presented by Anton Jigalin Advanced Topics in Computer Vision ( 048921 )](https://reader035.vdocuments.net/reader035/viewer/2022062515/56649f555503460f94c79170/html5/thumbnails/12.jpg)
12
Initial two-frame reconstruction
Intermediate stage Final reconstruction
Structure from Motion Algorithm
![Page 13: Scene Reconstruction Seminar presented by Anton Jigalin Advanced Topics in Computer Vision ( 048921 )](https://reader035.vdocuments.net/reader035/viewer/2022062515/56649f555503460f94c79170/html5/thumbnails/13.jpg)
13
Failure modes
Repeating textures
Insufficient overlap or texture
Cascading errors
Bad initialization
![Page 14: Scene Reconstruction Seminar presented by Anton Jigalin Advanced Topics in Computer Vision ( 048921 )](https://reader035.vdocuments.net/reader035/viewer/2022062515/56649f555503460f94c79170/html5/thumbnails/14.jpg)
14
Connectivity graphs
![Page 15: Scene Reconstruction Seminar presented by Anton Jigalin Advanced Topics in Computer Vision ( 048921 )](https://reader035.vdocuments.net/reader035/viewer/2022062515/56649f555503460f94c79170/html5/thumbnails/15.jpg)
15
Scene Visualization:
Photo Tourism: 3D photo browser
![Page 16: Scene Reconstruction Seminar presented by Anton Jigalin Advanced Topics in Computer Vision ( 048921 )](https://reader035.vdocuments.net/reader035/viewer/2022062515/56649f555503460f94c79170/html5/thumbnails/16.jpg)
16
Photo Tourism: Main components
Rendering engine
Navigation interface
Tools for scene annotation
![Page 17: Scene Reconstruction Seminar presented by Anton Jigalin Advanced Topics in Computer Vision ( 048921 )](https://reader035.vdocuments.net/reader035/viewer/2022062515/56649f555503460f94c79170/html5/thumbnails/17.jpg)
17
Transitions between photographs
![Page 18: Scene Reconstruction Seminar presented by Anton Jigalin Advanced Topics in Computer Vision ( 048921 )](https://reader035.vdocuments.net/reader035/viewer/2022062515/56649f555503460f94c79170/html5/thumbnails/18.jpg)
18
Navigation interface
Object-based navigation
Stabilized slideshow
![Page 19: Scene Reconstruction Seminar presented by Anton Jigalin Advanced Topics in Computer Vision ( 048921 )](https://reader035.vdocuments.net/reader035/viewer/2022062515/56649f555503460f94c79170/html5/thumbnails/19.jpg)
19
Scene annotation
Annotate regions
Register new photographs
![Page 20: Scene Reconstruction Seminar presented by Anton Jigalin Advanced Topics in Computer Vision ( 048921 )](https://reader035.vdocuments.net/reader035/viewer/2022062515/56649f555503460f94c79170/html5/thumbnails/20.jpg)
20
Scene Visualization:
Path Finder: 3D navigation interface
![Page 21: Scene Reconstruction Seminar presented by Anton Jigalin Advanced Topics in Computer Vision ( 048921 )](https://reader035.vdocuments.net/reader035/viewer/2022062515/56649f555503460f94c79170/html5/thumbnails/21.jpg)
21
Best possible controls
The goals of 3D navigation:
• Exploration• Search• Object inspection
Principles for 3D scene navigation:
• Controls are scene and task dependent
• Constrained and automatic navigation
• User’s position and motion should be clear
![Page 22: Scene Reconstruction Seminar presented by Anton Jigalin Advanced Topics in Computer Vision ( 048921 )](https://reader035.vdocuments.net/reader035/viewer/2022062515/56649f555503460f94c79170/html5/thumbnails/22.jpg)
22
Pathfinder: Main components
Viewpoint scoring
Navigation controls for a scene
Rendering engine
User interface
![Page 23: Scene Reconstruction Seminar presented by Anton Jigalin Advanced Topics in Computer Vision ( 048921 )](https://reader035.vdocuments.net/reader035/viewer/2022062515/56649f555503460f94c79170/html5/thumbnails/23.jpg)
25
Scene-specific controls: Panoramas and Orbits
Panoramas
Orbits
![Page 24: Scene Reconstruction Seminar presented by Anton Jigalin Advanced Topics in Computer Vision ( 048921 )](https://reader035.vdocuments.net/reader035/viewer/2022062515/56649f555503460f94c79170/html5/thumbnails/24.jpg)
26
Path planning
High-quality rendering
Shortest path
Piecewise linear path
Path smoothing
Connectivity graph
![Page 25: Scene Reconstruction Seminar presented by Anton Jigalin Advanced Topics in Computer Vision ( 048921 )](https://reader035.vdocuments.net/reader035/viewer/2022062515/56649f555503460f94c79170/html5/thumbnails/25.jpg)
29
Limitations
Non circular orbits
User needs
Appearance compensation
Larger scenes
(a) Pisa Doumo
(b) Stonehenge
![Page 26: Scene Reconstruction Seminar presented by Anton Jigalin Advanced Topics in Computer Vision ( 048921 )](https://reader035.vdocuments.net/reader035/viewer/2022062515/56649f555503460f94c79170/html5/thumbnails/26.jpg)
30
Conclusion
Structure from motion
Photo Tourism - 3D photo browser
Path Finder - 3D navigation system
Reconstructing Rome
Capturing appearance
New photo-sharing community
Future work
![Page 27: Scene Reconstruction Seminar presented by Anton Jigalin Advanced Topics in Computer Vision ( 048921 )](https://reader035.vdocuments.net/reader035/viewer/2022062515/56649f555503460f94c79170/html5/thumbnails/27.jpg)
31
Thank you