cs294-137: theory and applications of virtual reality and ... · definition: virtual reality •...
TRANSCRIPT
![Page 1: CS294-137: Theory and Applications of Virtual Reality and ... · Definition: Virtual Reality • VR is a computer technology that uses head mounted displays, sometimes in combination](https://reader034.vdocuments.net/reader034/viewer/2022050413/5f89d0b32f57df6be456ba68/html5/thumbnails/1.jpg)
CS294-137: Theory and Applications of Virtual Reality and Immersive Computing
Achin Bhowmik, James O’Brien, Allen Y. Yang
Fall, 2017
![Page 2: CS294-137: Theory and Applications of Virtual Reality and ... · Definition: Virtual Reality • VR is a computer technology that uses head mounted displays, sometimes in combination](https://reader034.vdocuments.net/reader034/viewer/2022050413/5f89d0b32f57df6be456ba68/html5/thumbnails/2.jpg)
Lecturers
AchinBhowmik
CTO&EVP,StarkeyHearingTechnologies
FormerVP,PerceptualComputing,
Intel
Email:[email protected]
JamesO’Brien
Professor,EECS
ChiefScientist,Avametric
Email:[email protected]
AllenY.Yang
ChiefScientist,FungInstitute
FormerCTO,AtheerLabs
Email:[email protected]
![Page 3: CS294-137: Theory and Applications of Virtual Reality and ... · Definition: Virtual Reality • VR is a computer technology that uses head mounted displays, sometimes in combination](https://reader034.vdocuments.net/reader034/viewer/2022050413/5f89d0b32f57df6be456ba68/html5/thumbnails/3.jpg)
Other Guest Lecturers: CAC.berkeley
![Page 4: CS294-137: Theory and Applications of Virtual Reality and ... · Definition: Virtual Reality • VR is a computer technology that uses head mounted displays, sometimes in combination](https://reader034.vdocuments.net/reader034/viewer/2022050413/5f89d0b32f57df6be456ba68/html5/thumbnails/4.jpg)
Other Resources @ Berkeley
![Page 5: CS294-137: Theory and Applications of Virtual Reality and ... · Definition: Virtual Reality • VR is a computer technology that uses head mounted displays, sometimes in combination](https://reader034.vdocuments.net/reader034/viewer/2022050413/5f89d0b32f57df6be456ba68/html5/thumbnails/5.jpg)
Other Resources @ Berkeley
l Founded in Spring 2015 l >200 members in each
semester enrollment
VR@Berkeley
![Page 6: CS294-137: Theory and Applications of Virtual Reality and ... · Definition: Virtual Reality • VR is a computer technology that uses head mounted displays, sometimes in combination](https://reader034.vdocuments.net/reader034/viewer/2022050413/5f89d0b32f57df6be456ba68/html5/thumbnails/6.jpg)
Course ScheduleWeek1(8-23):IntroductionandCapstoneOptionsWeek2(8-30):HumanPerceptionintheContextofVRWeek3(9-6):BasicUnity3D/VRProgrammingWorkshopWeek4(9-13):CourseprojectproposalpresentationWeek5(9-20):ComputerGraphicsrelatedtopicsWeek6(9-27):3DVisionrelatedtopicsWeek7(10-4):OpticsandDisplaytechnologiesWeek8(10-11):LocalizationandMapping****************Week9:(10-18)VRFilmMaking(RichardHernandez)Week10(10-25):Gaming(JackMcCauley)Week11(11-1):Telemedicine(RuzenaBajcsy/GregorijKorillo)Week12(11-8):AR/VRinArts&Design(TedSelker)Week13(11-15):ComputationalImagingforVR(RenNg)Week14(11-22):NoclassWeek15(11-29):FinalprojectpresentationWeek16(12-6):Finalprojectpresentation
![Page 7: CS294-137: Theory and Applications of Virtual Reality and ... · Definition: Virtual Reality • VR is a computer technology that uses head mounted displays, sometimes in combination](https://reader034.vdocuments.net/reader034/viewer/2022050413/5f89d0b32f57df6be456ba68/html5/thumbnails/7.jpg)
Grading PolicyYourfinalgradeswillbedeterminedbyfourfactors:1. Attendancebyyou(10%)
2. Interactionduringtheclassgivenbythelecturers(10%)
3. Course/capstoneprojectpresentation(40%)
4. Aresearchpaperdiscussingyourprojectanditsrelevantliteratureandcommercialapplications(40%)
![Page 8: CS294-137: Theory and Applications of Virtual Reality and ... · Definition: Virtual Reality • VR is a computer technology that uses head mounted displays, sometimes in combination](https://reader034.vdocuments.net/reader034/viewer/2022050413/5f89d0b32f57df6be456ba68/html5/thumbnails/8.jpg)
Recommended Reading Material• Perception:SensationandPerception
byBruceGoldstein
• VirtualReality:VirtualRealityByStevenLaValle(andcheckouthisYouTubelectures)
• ComputerGraphics:FundamentalsofCGbyPeterShirley
• ComputerVision:AnInvitationto3-DVisionbyYiMa,etal.• Display:MobileDisplays
byAchinBhowmik,etal.
• AR/VRMarketResearch:Virtual&AugmentedReality,understandingtheraceforthenextcomputingplatformbyGoldmanSachs
![Page 9: CS294-137: Theory and Applications of Virtual Reality and ... · Definition: Virtual Reality • VR is a computer technology that uses head mounted displays, sometimes in combination](https://reader034.vdocuments.net/reader034/viewer/2022050413/5f89d0b32f57df6be456ba68/html5/thumbnails/9.jpg)
Goal of the Course• UnderstandthefundamentaltheoriesthatenableVR/ARandIC
• UnderstandthemaintechnologydriversofVR/ARandICmarkets
• Becomeanexpertincriticizingthecurrentsoftwareandhardwaresolutions
• Beingabouttheinventnewsolutionsthataddressexistingneeds/painpointsofVR/ARandICapplications
• (Optional)Beingmotivatedtopursueacareerinrelevantresearchorentrepreneurialfields
![Page 10: CS294-137: Theory and Applications of Virtual Reality and ... · Definition: Virtual Reality • VR is a computer technology that uses head mounted displays, sometimes in combination](https://reader034.vdocuments.net/reader034/viewer/2022050413/5f89d0b32f57df6be456ba68/html5/thumbnails/10.jpg)
AR/VR: A Hot Market in 2016
![Page 11: CS294-137: Theory and Applications of Virtual Reality and ... · Definition: Virtual Reality • VR is a computer technology that uses head mounted displays, sometimes in combination](https://reader034.vdocuments.net/reader034/viewer/2022050413/5f89d0b32f57df6be456ba68/html5/thumbnails/11.jpg)
Holodeck: A VR Experience
![Page 12: CS294-137: Theory and Applications of Virtual Reality and ... · Definition: Virtual Reality • VR is a computer technology that uses head mounted displays, sometimes in combination](https://reader034.vdocuments.net/reader034/viewer/2022050413/5f89d0b32f57df6be456ba68/html5/thumbnails/12.jpg)
Leia’s Hologram: An AR Experience
![Page 13: CS294-137: Theory and Applications of Virtual Reality and ... · Definition: Virtual Reality • VR is a computer technology that uses head mounted displays, sometimes in combination](https://reader034.vdocuments.net/reader034/viewer/2022050413/5f89d0b32f57df6be456ba68/html5/thumbnails/13.jpg)
Brief History of VR/AR/IC
PhotographyGraphics Stereoscopy
Interaction3DAudio
![Page 14: CS294-137: Theory and Applications of Virtual Reality and ... · Definition: Virtual Reality • VR is a computer technology that uses head mounted displays, sometimes in combination](https://reader034.vdocuments.net/reader034/viewer/2022050413/5f89d0b32f57df6be456ba68/html5/thumbnails/14.jpg)
Early Forms of Paintings and Arts
CavepaintingsSince35,000–40,000BP
Writingandlanguages,Since3100BC
![Page 15: CS294-137: Theory and Applications of Virtual Reality and ... · Definition: Virtual Reality • VR is a computer technology that uses head mounted displays, sometimes in combination](https://reader034.vdocuments.net/reader034/viewer/2022050413/5f89d0b32f57df6be456ba68/html5/thumbnails/15.jpg)
Early Forms of Photography
CameraObscura,circa400BC Firstphotoonpaper,1800s Kodachrome,1935
![Page 16: CS294-137: Theory and Applications of Virtual Reality and ... · Definition: Virtual Reality • VR is a computer technology that uses head mounted displays, sometimes in combination](https://reader034.vdocuments.net/reader034/viewer/2022050413/5f89d0b32f57df6be456ba68/html5/thumbnails/16.jpg)
3D Illusion with Perspective
AmesRoom,byAdelbertAmes,Jr.,1946
![Page 17: CS294-137: Theory and Applications of Virtual Reality and ... · Definition: Virtual Reality • VR is a computer technology that uses head mounted displays, sometimes in combination](https://reader034.vdocuments.net/reader034/viewer/2022050413/5f89d0b32f57df6be456ba68/html5/thumbnails/17.jpg)
Seeing 3D from Stereo
Wheatstonemirrorstereoscope,1838 Holmesstereoscope,1861
![Page 18: CS294-137: Theory and Applications of Virtual Reality and ... · Definition: Virtual Reality • VR is a computer technology that uses head mounted displays, sometimes in combination](https://reader034.vdocuments.net/reader034/viewer/2022050413/5f89d0b32f57df6be456ba68/html5/thumbnails/18.jpg)
Audio: From Mono to Spatial 3D
Edisoncylinderphonograph,1899
DolbyStereo,1977
Inventionofheadphones,1910s
DolbyAtmosforVR
![Page 19: CS294-137: Theory and Applications of Virtual Reality and ... · Definition: Virtual Reality • VR is a computer technology that uses head mounted displays, sometimes in combination](https://reader034.vdocuments.net/reader034/viewer/2022050413/5f89d0b32f57df6be456ba68/html5/thumbnails/19.jpg)
Interaction Modalities
Keyboard,since1860s Mouse,DouglasEngelbart,1963
iPhone,2007
![Page 20: CS294-137: Theory and Applications of Virtual Reality and ... · Definition: Virtual Reality • VR is a computer technology that uses head mounted displays, sometimes in combination](https://reader034.vdocuments.net/reader034/viewer/2022050413/5f89d0b32f57df6be456ba68/html5/thumbnails/20.jpg)
Hands as 3D Input Device
![Page 21: CS294-137: Theory and Applications of Virtual Reality and ... · Definition: Virtual Reality • VR is a computer technology that uses head mounted displays, sometimes in combination](https://reader034.vdocuments.net/reader034/viewer/2022050413/5f89d0b32f57df6be456ba68/html5/thumbnails/21.jpg)
Sensorama: The First VR Prototype
MortonHeilig,1958
![Page 22: CS294-137: Theory and Applications of Virtual Reality and ... · Definition: Virtual Reality • VR is a computer technology that uses head mounted displays, sometimes in combination](https://reader034.vdocuments.net/reader034/viewer/2022050413/5f89d0b32f57df6be456ba68/html5/thumbnails/22.jpg)
Virtuality: Dawn of VR Gaming
![Page 23: CS294-137: Theory and Applications of Virtual Reality and ... · Definition: Virtual Reality • VR is a computer technology that uses head mounted displays, sometimes in combination](https://reader034.vdocuments.net/reader034/viewer/2022050413/5f89d0b32f57df6be456ba68/html5/thumbnails/23.jpg)
Tilt Brush in Virtual 3D
![Page 24: CS294-137: Theory and Applications of Virtual Reality and ... · Definition: Virtual Reality • VR is a computer technology that uses head mounted displays, sometimes in combination](https://reader034.vdocuments.net/reader034/viewer/2022050413/5f89d0b32f57df6be456ba68/html5/thumbnails/24.jpg)
CES 2017: Samsung 4D VR Experience
![Page 25: CS294-137: Theory and Applications of Virtual Reality and ... · Definition: Virtual Reality • VR is a computer technology that uses head mounted displays, sometimes in combination](https://reader034.vdocuments.net/reader034/viewer/2022050413/5f89d0b32f57df6be456ba68/html5/thumbnails/25.jpg)
IvanSutherland,1968
The First AR Prototype
![Page 26: CS294-137: Theory and Applications of Virtual Reality and ... · Definition: Virtual Reality • VR is a computer technology that uses head mounted displays, sometimes in combination](https://reader034.vdocuments.net/reader034/viewer/2022050413/5f89d0b32f57df6be456ba68/html5/thumbnails/26.jpg)
Anatomy of an AR Device: HoloLens
![Page 27: CS294-137: Theory and Applications of Virtual Reality and ... · Definition: Virtual Reality • VR is a computer technology that uses head mounted displays, sometimes in combination](https://reader034.vdocuments.net/reader034/viewer/2022050413/5f89d0b32f57df6be456ba68/html5/thumbnails/27.jpg)
Definition: Virtual Reality• VR is a computer technology that uses head mounted
displays, sometimes in combination with other sensory devices, to generate realistic images, sounds, and other sensations (touch, smell, motion, etc.) that simulate a user’s physical presence in a virtual environment.
27
![Page 28: CS294-137: Theory and Applications of Virtual Reality and ... · Definition: Virtual Reality • VR is a computer technology that uses head mounted displays, sometimes in combination](https://reader034.vdocuments.net/reader034/viewer/2022050413/5f89d0b32f57df6be456ba68/html5/thumbnails/28.jpg)
Definition: Augmented Reality• AR is a computer technology that augments a
physical, real-world environment directly or through its indirect view computer-generated sensory information, including graphics, video, and sound. AR may alter a user’s view of reality, and may also enhance one’s perception of reality.
28
![Page 29: CS294-137: Theory and Applications of Virtual Reality and ... · Definition: Virtual Reality • VR is a computer technology that uses head mounted displays, sometimes in combination](https://reader034.vdocuments.net/reader034/viewer/2022050413/5f89d0b32f57df6be456ba68/html5/thumbnails/29.jpg)
Enabling Technologies/ Open Research
• Near-EyeDisplaysandOptics
• 3DLocalization
• 3DContentCapturing
• NewHuman-ComputerInterface
![Page 30: CS294-137: Theory and Applications of Virtual Reality and ... · Definition: Virtual Reality • VR is a computer technology that uses head mounted displays, sometimes in combination](https://reader034.vdocuments.net/reader034/viewer/2022050413/5f89d0b32f57df6be456ba68/html5/thumbnails/30.jpg)
Near-Eye Optical Module
![Page 31: CS294-137: Theory and Applications of Virtual Reality and ... · Definition: Virtual Reality • VR is a computer technology that uses head mounted displays, sometimes in combination](https://reader034.vdocuments.net/reader034/viewer/2022050413/5f89d0b32f57df6be456ba68/html5/thumbnails/31.jpg)
HMD Stereo Display Challenge
AccommodationVergence
![Page 32: CS294-137: Theory and Applications of Virtual Reality and ... · Definition: Virtual Reality • VR is a computer technology that uses head mounted displays, sometimes in combination](https://reader034.vdocuments.net/reader034/viewer/2022050413/5f89d0b32f57df6be456ba68/html5/thumbnails/32.jpg)
Localization via Beacons
![Page 33: CS294-137: Theory and Applications of Virtual Reality and ... · Definition: Virtual Reality • VR is a computer technology that uses head mounted displays, sometimes in combination](https://reader034.vdocuments.net/reader034/viewer/2022050413/5f89d0b32f57df6be456ba68/html5/thumbnails/33.jpg)
Localization via Depth Perception
![Page 34: CS294-137: Theory and Applications of Virtual Reality and ... · Definition: Virtual Reality • VR is a computer technology that uses head mounted displays, sometimes in combination](https://reader034.vdocuments.net/reader034/viewer/2022050413/5f89d0b32f57df6be456ba68/html5/thumbnails/34.jpg)
Google Tango Inside-Out Localization
![Page 35: CS294-137: Theory and Applications of Virtual Reality and ... · Definition: Virtual Reality • VR is a computer technology that uses head mounted displays, sometimes in combination](https://reader034.vdocuments.net/reader034/viewer/2022050413/5f89d0b32f57df6be456ba68/html5/thumbnails/35.jpg)
360 VR Capturing: Photo Stitching
GoogleJumpVR
![Page 36: CS294-137: Theory and Applications of Virtual Reality and ... · Definition: Virtual Reality • VR is a computer technology that uses head mounted displays, sometimes in combination](https://reader034.vdocuments.net/reader034/viewer/2022050413/5f89d0b32f57df6be456ba68/html5/thumbnails/36.jpg)
From 360 VR to Real-Time 3D VR
![Page 37: CS294-137: Theory and Applications of Virtual Reality and ... · Definition: Virtual Reality • VR is a computer technology that uses head mounted displays, sometimes in combination](https://reader034.vdocuments.net/reader034/viewer/2022050413/5f89d0b32f57df6be456ba68/html5/thumbnails/37.jpg)
Lytro Immerge
![Page 38: CS294-137: Theory and Applications of Virtual Reality and ... · Definition: Virtual Reality • VR is a computer technology that uses head mounted displays, sometimes in combination](https://reader034.vdocuments.net/reader034/viewer/2022050413/5f89d0b32f57df6be456ba68/html5/thumbnails/38.jpg)
New 3D Human-Computer Interface
![Page 39: CS294-137: Theory and Applications of Virtual Reality and ... · Definition: Virtual Reality • VR is a computer technology that uses head mounted displays, sometimes in combination](https://reader034.vdocuments.net/reader034/viewer/2022050413/5f89d0b32f57df6be456ba68/html5/thumbnails/39.jpg)
New 3D Human-Computer Interface
![Page 40: CS294-137: Theory and Applications of Virtual Reality and ... · Definition: Virtual Reality • VR is a computer technology that uses head mounted displays, sometimes in combination](https://reader034.vdocuments.net/reader034/viewer/2022050413/5f89d0b32f57df6be456ba68/html5/thumbnails/40.jpg)
Connecting AR/VR and Robotics
![Page 41: CS294-137: Theory and Applications of Virtual Reality and ... · Definition: Virtual Reality • VR is a computer technology that uses head mounted displays, sometimes in combination](https://reader034.vdocuments.net/reader034/viewer/2022050413/5f89d0b32f57df6be456ba68/html5/thumbnails/41.jpg)
Course/Capstone Projects Examples
Berkeley OpenARK
• Lumus • PMD• Webcam• Motion Sensor• Microsoft Surface Pro• <$5K per unit
![Page 42: CS294-137: Theory and Applications of Virtual Reality and ... · Definition: Virtual Reality • VR is a computer technology that uses head mounted displays, sometimes in combination](https://reader034.vdocuments.net/reader034/viewer/2022050413/5f89d0b32f57df6be456ba68/html5/thumbnails/42.jpg)
Beta released early 2017
![Page 43: CS294-137: Theory and Applications of Virtual Reality and ... · Definition: Virtual Reality • VR is a computer technology that uses head mounted displays, sometimes in combination](https://reader034.vdocuments.net/reader034/viewer/2022050413/5f89d0b32f57df6be456ba68/html5/thumbnails/43.jpg)
2 Allow for breathing room
ISAACS: Immersive Semi-Autonomous Aerial Command System
![Page 44: CS294-137: Theory and Applications of Virtual Reality and ... · Definition: Virtual Reality • VR is a computer technology that uses head mounted displays, sometimes in combination](https://reader034.vdocuments.net/reader034/viewer/2022050413/5f89d0b32f57df6be456ba68/html5/thumbnails/44.jpg)
2 Allow for breathing room
Drone Fleet Control (Tomlin)
![Page 45: CS294-137: Theory and Applications of Virtual Reality and ... · Definition: Virtual Reality • VR is a computer technology that uses head mounted displays, sometimes in combination](https://reader034.vdocuments.net/reader034/viewer/2022050413/5f89d0b32f57df6be456ba68/html5/thumbnails/45.jpg)
2 Allow for breathing room
Drone Safety Control (Tomlin)
![Page 46: CS294-137: Theory and Applications of Virtual Reality and ... · Definition: Virtual Reality • VR is a computer technology that uses head mounted displays, sometimes in combination](https://reader034.vdocuments.net/reader034/viewer/2022050413/5f89d0b32f57df6be456ba68/html5/thumbnails/46.jpg)
VR in Autonomous Driving (Borrelli)