wild dreams for cameras jack tumblin northwestern university [email protected]
Post on 16-Jan-2016
Embed Size (px)
.Wild Dreams for Cameras
Jack TumblinNorthwestern University [email protected]
From May 24 Panel Discussion on cameras atSymposium on Computational Photography & VideoMay 23-25, 2005
DefinitionsVisual Appearance: What we think we see. (Consciously-available estimates of our surroundings, made from the light reaching our eyes)
Picture: A container for visual appearance. (something we make to hold what we see, or what would like to see)
Image: A copy of light intensities. (Just one kind of picture, made by copying a scaled map of scene light intensities as a lens might)
Machine-Readable Images? scene displayScene LightIntensitiesDisplayLight IntensitiesPixel values (scene intensity? display intensity? perceived intensity? blackness/whiteness ?)display
DisplayRGB(x,y,tn)ImageI(x,y,,t)Rendering3D Scenelight sources, BRDFs,shapes,positions,movements,Eyepoint position, movement,projection,PHYSICALScenelight sources,BRDFs,shapes,positions,movements,Eyepointposition, movement,projection,PERCEIVEDVisionDigital ImagesExposure or Tone Mapping
Something Else? DisplayRGB(x,y,tn)ImageI(x,y,,t)Rendering3D Scenelight sources, BRDFs,shapes,positions,movements,Eyepoint position, movement,projection,PHYSICALScenelight sources,BRDFs,shapes,positions,movements,Eyepointposition, movement,projection,PERCEIVEDVisionDigital Pictures?
Williams 1998: Inflated Silhouettes
http://graphics.stanford.edu/workshops/ibr98/#Schedule%20of%20sessions 2D PhotoSilhouetteInflate Depth Symmetry
Williams`98: Inflated Silhouettes
Not bad! How can we do better?
Malzbender, HPlabs 2001A Mostly 2-D MethodPolynomial Texture MapsStore just 6 coefficients at each pixel, get Interactive re-lighting...
3D: Try image + other dimensions
Halle: Multiple Viewpoint Rendering (SIGG98) http://web.media.mit.edu/~halazar/sig98/halle98.pdf
Oh et. al, 2001: 2D3DManually Guided7 Hours! ? Would a more varied for camera pose help? http://graphics.lcs.mit.edu/ibedit/ibedit_s2001_cameraReady.pdf
Bixels: Picture Samples With Embedded Sharp BoundariesJack Tumblin and Prasun Choudhury Northwestern University, Evanston IL, USABixels (bilinear)Pixels (bilinear)
Results: boundary=depth discontinuity(Source data courtesy Ramesh Raskar, MERL)Source (1100x800)Boundaries (50x65)
Results: boundary=depth discontinuity(Source data courtesy Ramesh Raskar, MERL)Bixels (bilinear)50x65Pixels (bilinear)50x65
This is a daring talk for meput together quickly (by my standards), speculative and risky (work I havent done yet), and intended to encourage some collaborations.
Its a large radical goal; it is a plan for an immediate research project, with lots of follow-onsit could lead a producta very unusual kind of camera Ive dreamed about for yearsif it works, it has straightforward extensions into Rameshs work with projectors as the duals of cameras,into IBMR and my ambitions for making it a practical tool for historical preservation.What people see in a grid of pixels usually isnt machine-readable, but should be.Digital Images are 2D maps of imitated light; smooth, limited BW, deterministic; a scene outcome.Digital Pictures are containers for perceived quantities. These include 2D and/or partly 3D discontinuous, multi-valued, maybe uncertain and ambiguous: including shape hints, boundaries, estimated movement, edges, shadows, estimated reflectance, highlights, surf. texture inferred and suppressed illumination
Today I want to ask a few provocative questions, namely:Why are pixels such a good idea? and Can we improve on pixels? How?
Even if you dont really like my answers (they have some problems)I want to convince you that the way we represent digital images COULD be improved, and show you a simple attempted improvement that I call bixels:Bixels are useful because:--they describe true discontinuities in an image with subpixel accuracy; --they keep scene quantities separateSee how the green leaf, purple flower, and yellow background values are kept entirely unmixed in the bixel image at left top, but pixel image mixes them together.--these discontinuities are machine-readable, and may make images easier to--Compare, search, warp, edit, and align,--understand, using NonPhotoRealistic (NPR) arguments--link to the properties of the original scene--they do take more computing than ordinary pixels if you want to re-size them, but this computing is mostly already available in OpenGL texture mapping hardware,Its a simple and unfinished idea that I will explain mostly with pictures; the math is tedious, but straightforward and given in the paper.
There is also a C++ library for bixels is freely available on my website, and I encourage you to experiment with it.