wild dreams for cameras jack tumblin northwestern university [email protected]

of 13/13
. Wild Dreams for Cameras Wild Dreams for Cameras Jack Tumblin Jack Tumblin Northwestern University Northwestern University [email protected] [email protected] From May 24 Panel Discussion on cameras at From May 24 Panel Discussion on cameras at Symposium on Symposium on Computational Photography & Video Computational Photography & Video May 23-25, 2005 May 23-25, 2005

Post on 16-Jan-2016

214 views

Category:

Documents

0 download

Embed Size (px)

TRANSCRIPT

  • .Wild Dreams for Cameras

    Jack TumblinNorthwestern University [email protected]

    From May 24 Panel Discussion on cameras atSymposium on Computational Photography & VideoMay 23-25, 2005

  • DefinitionsVisual Appearance: What we think we see. (Consciously-available estimates of our surroundings, made from the light reaching our eyes)

    Picture: A container for visual appearance. (something we make to hold what we see, or what would like to see)

    Image: A copy of light intensities. (Just one kind of picture, made by copying a scaled map of scene light intensities as a lens might)

  • Machine-Readable Images? scene displayScene LightIntensitiesDisplayLight IntensitiesPixel values (scene intensity? display intensity? perceived intensity? blackness/whiteness ?)display

  • DisplayRGB(x,y,tn)ImageI(x,y,,t)Rendering3D Scenelight sources, BRDFs,shapes,positions,movements,Eyepoint position, movement,projection,PHYSICALScenelight sources,BRDFs,shapes,positions,movements,Eyepointposition, movement,projection,PERCEIVEDVisionDigital ImagesExposure or Tone Mapping

  • Something Else? DisplayRGB(x,y,tn)ImageI(x,y,,t)Rendering3D Scenelight sources, BRDFs,shapes,positions,movements,Eyepoint position, movement,projection,PHYSICALScenelight sources,BRDFs,shapes,positions,movements,Eyepointposition, movement,projection,PERCEIVEDVisionDigital Pictures?

  • Williams 1998: Inflated Silhouettes

    http://graphics.stanford.edu/workshops/ibr98/#Schedule%20of%20sessions 2D PhotoSilhouetteInflate Depth Symmetry

  • Williams`98: Inflated Silhouettes

    Not bad! How can we do better?

    http://graphics.stanford.edu/workshops/ibr98/#Schedule%20of%20sessions

  • Malzbender, HPlabs 2001A Mostly 2-D MethodPolynomial Texture MapsStore just 6 coefficients at each pixel, get Interactive re-lighting...

  • 3D: Try image + other dimensions

    Halle: Multiple Viewpoint Rendering (SIGG98) http://web.media.mit.edu/~halazar/sig98/halle98.pdf

  • Oh et. al, 2001: 2D3DManually Guided7 Hours! ? Would a more varied for camera pose help? http://graphics.lcs.mit.edu/ibedit/ibedit_s2001_cameraReady.pdf

  • Bixels: Picture Samples With Embedded Sharp BoundariesJack Tumblin and Prasun Choudhury Northwestern University, Evanston IL, USABixels (bilinear)Pixels (bilinear)

  • Results: boundary=depth discontinuity(Source data courtesy Ramesh Raskar, MERL)Source (1100x800)Boundaries (50x65)

  • Results: boundary=depth discontinuity(Source data courtesy Ramesh Raskar, MERL)Bixels (bilinear)50x65Pixels (bilinear)50x65

    This is a daring talk for meput together quickly (by my standards), speculative and risky (work I havent done yet), and intended to encourage some collaborations.

    Its a large radical goal; it is a plan for an immediate research project, with lots of follow-onsit could lead a producta very unusual kind of camera Ive dreamed about for yearsif it works, it has straightforward extensions into Rameshs work with projectors as the duals of cameras,into IBMR and my ambitions for making it a practical tool for historical preservation.What people see in a grid of pixels usually isnt machine-readable, but should be.Digital Images are 2D maps of imitated light; smooth, limited BW, deterministic; a scene outcome.Digital Pictures are containers for perceived quantities. These include 2D and/or partly 3D discontinuous, multi-valued, maybe uncertain and ambiguous: including shape hints, boundaries, estimated movement, edges, shadows, estimated reflectance, highlights, surf. texture inferred and suppressed illumination

    Today I want to ask a few provocative questions, namely:Why are pixels such a good idea? and Can we improve on pixels? How?

    Even if you dont really like my answers (they have some problems)I want to convince you that the way we represent digital images COULD be improved, and show you a simple attempted improvement that I call bixels:Bixels are useful because:--they describe true discontinuities in an image with subpixel accuracy; --they keep scene quantities separateSee how the green leaf, purple flower, and yellow background values are kept entirely unmixed in the bixel image at left top, but pixel image mixes them together.--these discontinuities are machine-readable, and may make images easier to--Compare, search, warp, edit, and align,--understand, using NonPhotoRealistic (NPR) arguments--link to the properties of the original scene--they do take more computing than ordinary pixels if you want to re-size them, but this computing is mostly already available in OpenGL texture mapping hardware,Its a simple and unfinished idea that I will explain mostly with pictures; the math is tedious, but straightforward and given in the paper.

    There is also a C++ library for bixels is freely available on my website, and I encourage you to experiment with it.