Image Based Image Based Rendering Rendering

Image Based Image Based Rendering Rendering

Light Field• Gershun in 1936

– An illuminated objects fills the surrounding space with light reflected of its surface, establishing the object’s “light field”

• Adelson and Bergen 1991– Plenoptic Function

Plenoptic Funtions• An observer at any point in space (x,y,z) and any m

oment in time (t) can look in any direction(θ,ǿ.) and record the incoming light intensity Φ at all visible wavelength λ

• Φ= Φ(x,y,z,t, θ,ǿ., λ)

• Φ is the radiation’s spectral intensity, a scalar value denotes the incoming light’s power per unit per area per solid angle per unit wavelength interval

Image Based Rendering

• If Φis known, any view of the scene can be reconstructed

• IBR an approach to computer-generated 2D-views of natural 3D-scenes

• Approximation of the plenoptic functions from a number of conventional images

IBR Topics• View-Dependent Texture Mapping• Hybrid Geometry and Image Based Rendering• Delta Tree Representation• Multiple-Centre-of-Projection Images• Layered Depth Images• Sprites with Depth• Plenoptic Modeling and Environment Maps• Light Field Rendering and Lumigraphs

Light Field Rendering• M. Levoy• A ray is parameterized

by its intersection coordinates (u,v,s,t)

• The set of all rays that can be parameterized by two grid planes represents a light field slab

Light Field Rendering

Sampling Requirements

• S X T = • 256 x 256• U X V = • 121000 images

IBR with Controllable Illumination

• For a static scene, 3 or more images with “orthogonal” lighting can be used to render all possible lighting conditions

Set of all possible images

• Original 6 Photos with different lighting conditions

• Three basis images

Set of all possible images

• a

IBR and Video Scene with Camera Motion

• A video scene with camera motion contains a sequence of images of different viewing positions

• Applying IBR or IBM, develop algorithm to do video object segmentation

• Object elimination inside a video sequence, like the one in “Rising Sun” - Sean Connery

• Any help to video understanding ?• Evolution of Vision ?