augmented reality with.net casey chesnut brains-n-brawn.com wisconsin.net ug november 2007
TRANSCRIPT
![Page 1: Augmented Reality with.NET casey chesnut brains-N-brawn.com Wisconsin.NET UG November 2007](https://reader035.vdocuments.net/reader035/viewer/2022062423/56649eaa5503460f94bb018d/html5/thumbnails/1.jpg)
Augmented Reality with .NET
casey chesnutbrains-N-brawn.com
Wisconsin .NET UGNovember 2007
![Page 2: Augmented Reality with.NET casey chesnut brains-N-brawn.com Wisconsin.NET UG November 2007](https://reader035.vdocuments.net/reader035/viewer/2022062423/56649eaa5503460f94bb018d/html5/thumbnails/2.jpg)
Source
• http://www.brains-N-brawn.com/wpfAugReal/• Environment : 32-bit Vista, VS 2005, .NET 3.0,
Logitech QuickCam Pro Web Cam for Notebooks
![Page 3: Augmented Reality with.NET casey chesnut brains-N-brawn.com Wisconsin.NET UG November 2007](https://reader035.vdocuments.net/reader035/viewer/2022062423/56649eaa5503460f94bb018d/html5/thumbnails/3.jpg)
brains-N-brawn.com• Pervasive Computing
– Tablet PC (MVP 03)
– Compact Framework (MVP 04)
– Advanced Web Services (MVP 05)
– Media Center (MVP 06)
– Speech / Office Communication Server (MVP 07)
– Location Based Services– Artificial Intelligence– 3D– Image Processing– Augmented Reality
![Page 4: Augmented Reality with.NET casey chesnut brains-N-brawn.com Wisconsin.NET UG November 2007](https://reader035.vdocuments.net/reader035/viewer/2022062423/56649eaa5503460f94bb018d/html5/thumbnails/4.jpg)
Audience
• DirectShow?
• 3D?
• WPF?
• Augmented Reality?
![Page 5: Augmented Reality with.NET casey chesnut brains-N-brawn.com Wisconsin.NET UG November 2007](https://reader035.vdocuments.net/reader035/viewer/2022062423/56649eaa5503460f94bb018d/html5/thumbnails/5.jpg)
Outline
• Intro to Augmented Reality
• DirectShow
• AR Libraries
• WPF 3D
![Page 6: Augmented Reality with.NET casey chesnut brains-N-brawn.com Wisconsin.NET UG November 2007](https://reader035.vdocuments.net/reader035/viewer/2022062423/56649eaa5503460f94bb018d/html5/thumbnails/6.jpg)
Introduction
• Augmented Reality (AR) - is the combination of real world data with virtual data.
![Page 7: Augmented Reality with.NET casey chesnut brains-N-brawn.com Wisconsin.NET UG November 2007](https://reader035.vdocuments.net/reader035/viewer/2022062423/56649eaa5503460f94bb018d/html5/thumbnails/7.jpg)
Audience
• What are some current examples?
![Page 8: Augmented Reality with.NET casey chesnut brains-N-brawn.com Wisconsin.NET UG November 2007](https://reader035.vdocuments.net/reader035/viewer/2022062423/56649eaa5503460f94bb018d/html5/thumbnails/8.jpg)
Current Examples
• Pilot HUDs
• QuickCam video conferencing
• PS3 Eye of Judgment
• Sports : Football, Hockey, Nascar
• Aircraft maintenance
• ARQuake
• Star Wars chess
• …
![Page 9: Augmented Reality with.NET casey chesnut brains-N-brawn.com Wisconsin.NET UG November 2007](https://reader035.vdocuments.net/reader035/viewer/2022062423/56649eaa5503460f94bb018d/html5/thumbnails/9.jpg)
Videos
• QuickCam
• Eye of Judgment
![Page 10: Augmented Reality with.NET casey chesnut brains-N-brawn.com Wisconsin.NET UG November 2007](https://reader035.vdocuments.net/reader035/viewer/2022062423/56649eaa5503460f94bb018d/html5/thumbnails/10.jpg)
Steps
• Acquire real-world data (camera, HMD, mobile devices)
• Recognize real-world data (computer vision, face detection, fudicial markers)
• Determine relative position of real-world marker to input source.
• Display composite of real-world data and virtual object (monitor, HMD, mobile device, projector).
![Page 11: Augmented Reality with.NET casey chesnut brains-N-brawn.com Wisconsin.NET UG November 2007](https://reader035.vdocuments.net/reader035/viewer/2022062423/56649eaa5503460f94bb018d/html5/thumbnails/11.jpg)
Acquire real-world data
• OpenCV / SharperCV
• DirectShow / DirectShow .NET
• Windows Media Foundation
• …
![Page 12: Augmented Reality with.NET casey chesnut brains-N-brawn.com Wisconsin.NET UG November 2007](https://reader035.vdocuments.net/reader035/viewer/2022062423/56649eaa5503460f94bb018d/html5/thumbnails/12.jpg)
DirectShow
• Used by Windows Media Player, Media Center Edition, Windows Media Encoder, Movie Maker, …
• User for audio/video playback, DVD, TV tuners, web cams, media capture, media transcoding, …
![Page 13: Augmented Reality with.NET casey chesnut brains-N-brawn.com Wisconsin.NET UG November 2007](https://reader035.vdocuments.net/reader035/viewer/2022062423/56649eaa5503460f94bb018d/html5/thumbnails/13.jpg)
DirectShow
• COM
• GraphEdt tool
• Graph, Filter, Pin, Configuration
• Filters (Source, Transform, Sink)
• DirectShow .NET (C# wrapper)– http://directshownet.sourceforge.net/
• Learning the filters is key
• Not WPF 3D friendly
![Page 14: Augmented Reality with.NET casey chesnut brains-N-brawn.com Wisconsin.NET UG November 2007](https://reader035.vdocuments.net/reader035/viewer/2022062423/56649eaa5503460f94bb018d/html5/thumbnails/14.jpg)
DirectShow .NET
• C# wrapper
• Wraps a majority of DirectShow
• Many samples available, along with GraphEdt.exe, and MSDN documentation
• Also has wrappers for Windows Media Lib and Windows Media Foundation
![Page 15: Augmented Reality with.NET casey chesnut brains-N-brawn.com Wisconsin.NET UG November 2007](https://reader035.vdocuments.net/reader035/viewer/2022062423/56649eaa5503460f94bb018d/html5/thumbnails/15.jpg)
Demo
• GraphEdt
• /dshowPlay
• /wpfArBasic - capture
![Page 16: Augmented Reality with.NET casey chesnut brains-N-brawn.com Wisconsin.NET UG November 2007](https://reader035.vdocuments.net/reader035/viewer/2022062423/56649eaa5503460f94bb018d/html5/thumbnails/16.jpg)
Recognize real-world data
• OpenCV
• Face detection
• AR Libraries (fudicial markers)
• …
![Page 17: Augmented Reality with.NET casey chesnut brains-N-brawn.com Wisconsin.NET UG November 2007](https://reader035.vdocuments.net/reader035/viewer/2022062423/56649eaa5503460f94bb018d/html5/thumbnails/17.jpg)
AR Libraries
• ArToolKit
• ArTag
• ArToolKitPlus– http://studierstube.icg.tu-graz.ac.at/handheld_
ar/artoolkitplus.php
• Studierstube Tracker
• Goblin XNA (coming)
![Page 18: Augmented Reality with.NET casey chesnut brains-N-brawn.com Wisconsin.NET UG November 2007](https://reader035.vdocuments.net/reader035/viewer/2022062423/56649eaa5503460f94bb018d/html5/thumbnails/18.jpg)
ArToolKitPlus
• Optimized for mobile devices
• C++ Lib
• GPL, no longer under development
• Not well documented, but ArToolKit is
• Single, Multi-config, Multi-distinct tracking
• Markers (ID, BCH, Patterns)
![Page 19: Augmented Reality with.NET casey chesnut brains-N-brawn.com Wisconsin.NET UG November 2007](https://reader035.vdocuments.net/reader035/viewer/2022062423/56649eaa5503460f94bb018d/html5/thumbnails/19.jpg)
Videos
• ARTag Magic Lens
• Studierstube Mobile
![Page 20: Augmented Reality with.NET casey chesnut brains-N-brawn.com Wisconsin.NET UG November 2007](https://reader035.vdocuments.net/reader035/viewer/2022062423/56649eaa5503460f94bb018d/html5/thumbnails/20.jpg)
Demo
• /wpfArBasic – AR tracking
• /wpfArTest
![Page 21: Augmented Reality with.NET casey chesnut brains-N-brawn.com Wisconsin.NET UG November 2007](https://reader035.vdocuments.net/reader035/viewer/2022062423/56649eaa5503460f94bb018d/html5/thumbnails/21.jpg)
Composite real and virtual data
• GDI
• OpenGL / Tao
• Direct3D
• XNA
• WPF 3D
• …
![Page 22: Augmented Reality with.NET casey chesnut brains-N-brawn.com Wisconsin.NET UG November 2007](https://reader035.vdocuments.net/reader035/viewer/2022062423/56649eaa5503460f94bb018d/html5/thumbnails/22.jpg)
WPF 3D
• Viewport3D
• MatrixCamera
• Light
• ModelVisual3D / Model3DGroup
• Material
• Transform3D
• HitTest
![Page 23: Augmented Reality with.NET casey chesnut brains-N-brawn.com Wisconsin.NET UG November 2007](https://reader035.vdocuments.net/reader035/viewer/2022062423/56649eaa5503460f94bb018d/html5/thumbnails/23.jpg)
WPF 3D
• 3D Programming for Windows (Petzold)
• Windows SDK Samples
![Page 24: Augmented Reality with.NET casey chesnut brains-N-brawn.com Wisconsin.NET UG November 2007](https://reader035.vdocuments.net/reader035/viewer/2022062423/56649eaa5503460f94bb018d/html5/thumbnails/24.jpg)
Demo
• /wpfArBasic – 3D model
• /wpfArVideo
![Page 25: Augmented Reality with.NET casey chesnut brains-N-brawn.com Wisconsin.NET UG November 2007](https://reader035.vdocuments.net/reader035/viewer/2022062423/56649eaa5503460f94bb018d/html5/thumbnails/25.jpg)
Future
• Augmented Reality @ Home (book)– Mostly OpenGL and C++, but at least one C#
and Tao sample. Based on ArTag library.
• Goblin XNA (AR library)– For PC use, not on 360 until XNA is extended
• …
![Page 26: Augmented Reality with.NET casey chesnut brains-N-brawn.com Wisconsin.NET UG November 2007](https://reader035.vdocuments.net/reader035/viewer/2022062423/56649eaa5503460f94bb018d/html5/thumbnails/26.jpg)
Questions