comp 4010 - lecture4 vr technology - visual and haptic displays
TRANSCRIPT
LECTURE 4: VR TECHNOLOGY – VISUAL AND HAPTIC DISPLAYS
COMP 4010 – Virtual Reality Semester 5 - 2016
Mark Billinghurst, Bruce Thomas University of South Australia
August 16th 2016
Virtual Reality Technology
Using Technology to Stimulate Senses • Simulate output
• E.g. simulate real scene
• Map output to devices • Graphics to HMD
• Use devices to stimulate the senses • HMD stimulates eyes
Visual Simulation
3D Graphics
HMD
Vision System
Brain
Example: Visual Simulation
Human-Machine Interface
Key Technologies for VR Systems • Visual Display
• Stimulate visual sense
• Audio/Tactile Display • Stimulate hearing/touch
• Tracking • Changing viewpoint • User input
• Input Devices • Supporting user interaction
Mapping Between Input and Output
Input
Output
VISUAL DISPLAY
Creating an Immersive Experience
• Head Mounted Display • Immerse the eyes
• Projection/Large Screen • Immerse the head/body
• Future Technologies • Neural implants • Contact lens displays, etc
HMD Basic Principles
• Use display with optics to create illusion of virtual screen
Key Properties of HMDs • Lens
• Focal length, Field of View • Occularity, Interpupillary distance • Eye relief, Eye box
• Display • Resolution, contrast • Power, brightness • Refresh rate
• Ergonomics • Size, weight • Wearability
Simple Magnifier HMD Design
p
q
Eyepiece (one or more lenses) Display
(Image Source)
Eye f Image
1/p + 1/q = 1/f where p = object distance (distance from image source to eyepiece) q = image distance (distance of image from the lens) f = focal length of the lens
Virtual Image
Lens Display
Virtual Image
Focal Length and Diopter
Focal Length - The distance from the surface of a lens at which rays of light converge.
Diopter - The power of a lens. Equal to 1/(focal length of the lens measured in meters)
Field of View
Monocular FOV is the angular subtense (usually expressed in degrees) of the displayed image as measured from the pupil of one eye. Total FOV is the total angular size of the displayed image visible to both eyes.
Binocular(or stereoscopic) FOV refers to the part of the displayed image visible to both eyes.
FOV may be measured horizontally, vertically or diagonally.
Ocularity • Monocular - HMD image goes to only one eye.
• Bioccular - Same HMD image to both eyes.
• Binocular (stereoscopic) - Different but matched images to each eye.
Interpupillary Distance (IPD)
! IPD is the horizontal distance between a user's eyes.
! IPD is the distance between the two optical axes in a binocular view system.
Vignetting and Eye Relief
Vignetting • The blocking or redirecting of light rays as they pass through the optical system.
Eye Relief Distance • Distance from the last optical surface in the HMD optical system to the front surface of the eye.
LEEP Optics
• Large Expanse Extra Perspective
• Very wide FOV for stereoscopic images • Higher resolution in the middle of FOV
• Lower resolution on the periphery
• Pincushion distortion
LEEP Optics
• Wide field of view optical design
Fresnel Lens
• A lens consisting of a concentric series of simple lens sections
• Result is a thin lens with a short focal length and large diameter
• More even resolution distribution
• Less distortion • from lanternroom.com
Relationship bet. angle and screen distance
0.0010.0020.0030.0040.0050.0060.0070.0080.00
R2.006.0010.00
14.00
18.00
22.00
26.00
30.00
34.00
Angle in Radians
Dist
ance
in m
m
LeepFresnel
Distortion in Lens Optics
A rectangle Maps to this
Example Distortion
Oculus Rift DK2 HTC Vive
To Correct for Distortion
• Must predistort image • This is a pixel-based distortion
• Graphics rendering uses linear interpolation!
• Too slow on most systems • Use shader programming
HMD Design Trade-offs
• Resolution vs field of view • As FOV increases, resolution decreases for fixed pixels
• Eye box vs field of view • Larger eye box limits field of view
• Size, Weight and Power vs everything else
vs.
The Perfect HMD • “Oakley look” . i.e., thin & small optics • Low cost & small image generators (OLED, LCOS, …) • Wide field of view 30oto 110o full diagonal field • Large eye box ~10 mm diameter, for eye ball movement • Large eye relief > 20 mm, for lash clearance and glasses • High resolution ~ SXGA (1280 x 1024) or higher • Low distortion < 2% • Bright hundreds of Cd/m2 • Artifact free; no “dirty windows” ; no raster; no sunlight scattering • Low weight • Other: eye tracking; battery life; connectivity….
Oculus Rift • Cost: $599 USD • FOV: 110o Horizontal • Refresh rate: 90 Hz • Resolution 1080x1200/eye • 3 DOF orientation tracking • 3 axis positional tracking
Inside an Oculus Rift
HTC Vive
29
Sony Playstation VR
30
https://www.youtube.com/watch?v=Lulzi9LmNSM
Comparison Between HMDs
Computer Based vs. Mobile VR Displays
• dsfsaf
Google Cardboard
• Released 2014 (Google 20% project) • >5 million shipped/given away • Easy to use developer tools
+ =
Version 1.0 vs Version 2.0
• Version 1.0 – Android focused, magnetic switch, small phone • Version 2.0 – Touch input, iOS/Android, fits many phones
Many Different Cardboard Viewers
Edible Google Cardboard
37
https://www.youtube.com/watch?v=beSfGrM7OXY
Gear VR
Multiple Mobile VR Viewers Available
• zxcvz
Projection/Large Display Technologies • Room Scale Projection
• CAVE, multi-wall environment • Dome projection
• Hemisphere/spherical display • Head/body inside
• Vehicle Simulator • Simulated visual display in windows
CAVE
• Developed in 1992, EVL University of Illinois Chicago • Multi-walled stereo projection environment
• Head tracked active stereo
Cruz-Neira, C., Sandin, D. J., DeFanti, T. A., Kenyon, R. V., & Hart, J. C. (1992). The CAVE: audio visual experience automatic virtual environment. Communications of the ACM, 35(6), 64-73.
Typical CAVE Setup
• 4 sides, rear projected stereo images
Demo Video – Wisconsin CAVE
https://www.youtube.com/watch?v=mBs-OGDoPDY
CAVE Variations
Stereo Projection • Active Stereo
• Active shutter glasses • Time synced signal • Brighter images • More expensive
• Passive Stereo • Polarized images • Two projectors (one/eye) • Cheap glasses (powerless) • Lower resolution/dimmer • Less expensive
Caterpillar Demo
https://www.youtube.com/watch?v=r9N1w8PmD1E
Multi-User CAVEs
• Limitation of CAVEs • Stereo projection from only one user’s viewpoint
• Solution • Higher frequency projectors and time slicing
Kulik, A., Kunert, A., Beck, S., Reichel, R., Blach, R., Zink, A., & Froehlich, B. (2011). C1x6: a stereoscopic six-user display for co-located collaboration in shared virtual environments. ACM Transactions on Graphics (TOG), 30(6), 188.
Multiuser Demo
https://www.uni-weimar.de/de/medien/professuren/vr/research/multi-user-virtual-reality/c1x6-a-stereoscopic-six-user-display/
Walt Disney Imagineering’s Digital Immersive Showroom (DISH)
Technology " Large working volume " 10.5m x 7.5m x 4.0m
" 360 Surround Projection " Front projection " Five 4K @ 120 Hz 3D projectors " One 2K @ 120 Hz 3D projectors
" Complex screen geometry " Rounded corners " Overhanging ceiling
" Tech Viz, Unreal, Panda3D
DISH Demo
https://www.youtube.com/watch?v=70hCn9PguI4
Allosphere • Univ. California Santa Barbara
• One of a kind facility
• Immersive Spherical display • 10 m diameter • Inside 3 story anechoic cube • Passive stereoscopic projection • 26 projectors • Visual tracking system for input
• See http://www.allosphere.ucsb.edu/
Kuchera-Morin, J., Wright, M., Wakefield, G., Roberts, C., Adderton, D., Sajadi, B., ... & Majumder, A. (2014). Immersive full-surround multi-user system design. Computers & Graphics, 40, 10-21.
Allosphere Demo
https://www.youtube.com/watch?v=25Ch8eE0vJg
Allosphere Research • Multi-disciplinary research
• Science, art, engineering
• Typical research projects • Brain imaging
• fMRI imaging data • Atomic bonding
• Bond simulation models • Nano medicine
• Simulate chemotherapy • Graph browser
• Mathematical visualization • Etc
Brain Imaging
Hydrogen bond
Nano Medicine
Vehicle Simulators • Combine VR displays with vehicle
• Visual displays on windows • Motion base for haptic feedback • Audio feedback
• Physical vehicle controls • Steering wheel, flight stick, etc
• Full vehicle simulation • Emergencies, normal operation, etc • Weapon operation • Training scenarios
Lexus Driving Simulator
Tank Turret Simulator
Demo: Boeing 787 Simulator
https://www.youtube.com/watch?v=3iah-blsw_U
HAPTIC/TACTILE DISPLAYS
Haptic Feedback • Greatly improves realism • When is it needed?
• Other cues occluded/obstructed • Required for task performance • High bandwidth!
• Hands and wrist are most important • High density of touch receptors
• Two kinds of feedback • Touch Feedback
• information on texture, temperature, etc. • Does not resist user contact
• Force Feedback • information on weight, and inertia. • Actively resists contact motion
Haptic Devices • Pin arrays for the finger(s) • Force-feedback "arms" • "Pager" motors • Particle brakes • Passive haptics • Many devices are application specific
• Like surgical devices
Active Haptics
• Actively resists contact motion • Dimensions?
• Force resistance • Frequency Response • Degrees of Freedom • Latency • Intrusiveness • Safety • Comfort • Portability
Force Feedback Joysticks
• WingMan Force 3D • Inexpensive ($60) • Actuators that can move the joystick given system commands
• Max 3.3 N of force • Force feedback driving wheel
Sensable Phantom
• Combined stylus input/haptic output • 6 DOF haptic feedback
Phantom Omni Demo
https://www.youtube.com/watch?v=REA97hRX0WQ
Immersion Cybergrasp
• Haptic feedback on Glove • Combined with glove input
CyberMotion (MPI Tübingen)
68
https://www.youtube.com/watch?v=GhnIK1PILig
CableRobot (MPI Tübingen)
69
https://www.youtube.com/watch?v=cJCsomGwdk0
Haptic Feedback in VR • Virtual contact
• What should we do when we know that contact has been made with a virtual object?
• The output of collision detection is the input to virtual contact • Cues for understanding the nature of contact with objects are
typically over-simplified (e.g., sound) • Training aids
• Can we convey additional information using the haptic channel?
Passive Haptics
• Not controlled by system • Pros
• Cheap • Large scale • Accurate
• Cons • Not dynamic • Limited use
UNC Being There Project
The Void- https://thevoid.com/
• Passive haptic environment • Warehouse scale VR • Wearable VR systems • Graphics overlaid on real props
The Void Demo
https://www.youtube.com/watch?v=cML814JD09g
Passive Haptic Paddle
• Using physical props to provide haptic feedback • http://www.cs.wpi.edu/~gogo/hive/
Tactile Feedback Interfaces • Goal: Stimulate skin tactile receptors • How?
• Air bellows • Jets • Actuators (commercial) • Micropin arrays • Electrical (research) • Neuromuscular stimulations (research)
Vibrotactile Cueing Devices • Vibrotactile feedback has been incorporated into many
devices • Can we use this technology to provide scalable, wearable
touch cues?
Tactile Mouse
• Logitch iFeel Mouse • Electrical Actuator
• Shakes up and down (do not disturb XY motion)
• Mouse over buttons • Haptic Bump • Rumble Pack
Vibrotactile Feedback Projects
Navy TSAS Project
TactaBoard and TactaVest
CyberTouch Glove • Immersion Corporation
• Expensive - $15000 • Six Vibrotactile actuators
• Back of finger • Palm
• Off-centered actuator motor • Rotation speed=frequency of
vibration (0-125 Hz) • When tracked virtual hand intersects with virtual object, send signal to glove to vibrate
