cosc 6326/psych6750x

of 55 /55
Cosc 6326/Psych6750X Movement, Tracking

Upload: serina-holder

Post on 03-Jan-2016




1 download

Embed Size (px)


Cosc 6326/Psych6750X. Movement, Tracking. Moving about virtual and real environments. Simulating motion through the environment visual (vection) vestibular stimulation auditory surround motion/vibration tactile surround motion/vibration - PowerPoint PPT Presentation


Page 1: Cosc 6326/Psych6750X

Cosc 6326/Psych6750X

Movement, Tracking

Page 2: Cosc 6326/Psych6750X

Moving about virtual and real environments

Page 3: Cosc 6326/Psych6750X

• Simulating motion through the environment– visual (vection)– vestibular stimulation– auditory surround motion/vibration– tactile surround motion/vibration– cues associate with active and passive limb

motion (proprioception and efference)

Page 4: Cosc 6326/Psych6750X

Motion Perception

• Human beings are highly sensitive to retinal motion– hyperacuity

– relative motion more salient than absolute

– need to account for eye movements and distance to obtain linear from angular speed

• autokinetic effect

• induced motion

– motion can define form

Page 5: Cosc 6326/Psych6750X

• apparent motion– sense of motion from sequential presentation of

discrete images– 15 frames per second normally considered

marginally acceptable– want for 30 to 60+ frames per second, ideally at

refresh rate

• patterns of motion give information about depth and 3D motion

Page 6: Cosc 6326/Psych6750X

Depth from motion

Page 7: Cosc 6326/Psych6750X

Figure ground segregation

• Motion can define form

• Boundaries of foreground image defined by accretion deletion regions

Motion in depth• Various cues: looming, changing disparity

• Time to contact/passage can be judged independent of speed

• Direction of motion in depth is also important

Page 8: Cosc 6326/Psych6750X

Motion Parallax

• translation of observer w.r.t. to scene (or of scene w.r.t. observer)– angular speed of objects across optic array

varies with distance

Page 9: Cosc 6326/Psych6750X
Page 10: Cosc 6326/Psych6750X

Motion Parallax

• if eye fixates– objects nearer than fixation point move with

observer, further objects move against observer– retinal speed varies with distance from point of


Page 11: Cosc 6326/Psych6750X

• motion parallax provides information about spatial layout of the environment and relative motion relative to the environment

• many similarities to depth from stereopsis

Page 12: Cosc 6326/Psych6750X

Self motion

Page 13: Cosc 6326/Psych6750X

Optic flow

• Flow patterns in velocity field can be analysed

• could provide direction of heading (e.g. from focus of expansion)

• perception of self-motion/motion in depth (vection)

Page 14: Cosc 6326/Psych6750X

• layout and depth of environment

• traditionally optic flow has been considered to be essential for control of locomotion

• note:– optic flow refers to velocity patterns in ‘optic arrays’

not the retina

– need to account for eye movements to extract optic flow from retinal flow

– real flow fields are complex

Page 15: Cosc 6326/Psych6750X

• Optic flow is a key component in– perception of self-motion– postural reflexes/body sway– eye movements– cybersickness

Page 16: Cosc 6326/Psych6750X

Visual direction based locomotion

• Recently the idea that locomotion relies directly on the processing of optic flow has been challenged

• Alternative is locomotion is based upon egocentric direction (see seminar)

Page 17: Cosc 6326/Psych6750X


• Normally world is stable and stationary

• A sense of self-motion can be elicited by visual motion (optic flow)

• Circular vection – illusory rotation from optic flow

• Linear vection – illusory translation from optic flow

Page 18: Cosc 6326/Psych6750X

• vection has a latency– depends on stimulus– reported as 5-30 s– believed to be due to conflict with vestibular

system, may benefit from vestibular priming

• may be partial or saturated, can drop out

Page 19: Cosc 6326/Psych6750X

• vection is– strengthened by large FOV– increased with increased optical

velocity/spatio-temporal frequency– sensitive to foreground background

relationships (driven by background)

• integrated with vestibular and motor copy (efferent) signals centrally

Page 20: Cosc 6326/Psych6750X

• Vection– can provide compelling self motion cues– enhances presence and realism

• but – is linked to cyber sickness and other negative


• main issue is mismatch between visual and other self motion cues (to be discussed later).

Page 21: Cosc 6326/Psych6750X

• as well as vection, vision can provide important static cues to orientation (as we’ll see in the tumbling room demo)

Page 22: Cosc 6326/Psych6750X

Vestibular Inputs

• organs of balance and equilibrium in inner ear

• two distinct sub-systems– semicircular canals: angular


– otoliths:linear acceleration

• self-motion perception, eye movements, postural control, orientation wrt gravity

Page 23: Cosc 6326/Psych6750X

• canal response– inertial sensors detect angular acceleration (three degrees

of freedom)– high-pass frequency response

• insensitive to low-frequency signals

• codes head velocity over relevant frequency range

– vision and other senses contribute as well but vestibular system most reliable at high frequency

– drives very low latency compensatory eye movements (vestibulo-ocular reflex VOR)

Page 24: Cosc 6326/Psych6750X

• otolith response– sensitive to three-dimensional linear

acceleration– also sensitive to gravity (head tilt or pitch)– linear acceleration due to gravity can be

confused with linear acceleration due to motion• possible to trade tilt for acceleration

Page 25: Cosc 6326/Psych6750X

Other non-visual cues to motion

• proprioceptive and somato-sensory cues can contribute to effective perception of self motion, e.g.– wind on face or body– weight on limbs, soles of feet, seat etc. can indicate

direction of gravity/accelerations– active and passive limb motions associated with

motion, efference copy– visceral receptors

Page 26: Cosc 6326/Psych6750X

Motion Displays

Page 27: Cosc 6326/Psych6750X

Actual Motion Displays walk/PW/gif/Stewart.gif

NASA space shuttle simulator

Page 28: Cosc 6326/Psych6750X

Flogiston chair Spaceball

Page 29: Cosc 6326/Psych6750X

Other issues

• active motion?

• range of possible motion limited

• requirement to stay centered in range of motion

Page 30: Cosc 6326/Psych6750X

Washout filter• Vestibular system is most sensitive to changes in

velocity (accelerometer)– provides an onset cue for self motion

– extended motion simulated with vision and other cues

• some problems with a motion base– simulating extended motion with limited travel


– need to stay near centre of range to prepare for future motion

Page 31: Cosc 6326/Psych6750X

• After onset cue, desirable to ‘home’ to a neutral position.

• Do this gradually and ideally imperceptibly – ‘washout’

• Washout filters are based on highpass filters to give washout and a fast transient response

Page 32: Cosc 6326/Psych6750X

• Wearable computers allow long range natural (physical) motion

• Many devices have been proposed to introduce somatosensory and proprioceptive cues– treadmills– bicycles– hang gliders (skis, motorcycles, …)– importance of ‘feel for vehicle controls (haptic


Page 33: Cosc 6326/Psych6750X


• Simulating locomotion may require displays for:– forward motion

– turning

– slope

– terrain roughness, solidity …

– other postures (e.g. crawling)

Page 34: Cosc 6326/Psych6750X


Page 36: Cosc 6326/Psych6750X
Page 37: Cosc 6326/Psych6750X


Page 38: Cosc 6326/Psych6750X

• critical component for immersive displays

• supports – structure from motion perception– maneuvering– locomotion– active perception– situational awareness

Page 39: Cosc 6326/Psych6750X

• Field of Regard– amount of space (in terms of visual angle) filled

by the virtual world– an effective increased effective FOV beyond

the instantaneous FOV that is provided by tracking head motion

Page 40: Cosc 6326/Psych6750X

• Tracker issues– resolution, smoothness, degrees of freedom– display lag is a major problem in head-slaved

displays (more in sensory conflict lecture):• sensor transduction, data transmission, input handling,

simulation, rendering, wait for frame refresh• when head moves or vibrates display is not updated

immediately• prediction can help

Page 41: Cosc 6326/Psych6750X

• Consequences:– fatigue, nausea (simulator sickness), poor

performance– poor fidelity and distortions of virtual space – image slip, blur or instability

• See Foxlin handout to be discussed

Page 42: Cosc 6326/Psych6750X


Page 43: Cosc 6326/Psych6750X

• Two components to navigating– way finding– travel

Page 44: Cosc 6326/Psych6750X


• Refers to determination of current location and path to desired location

• Need to maneuver through environment and avoid obstacles when wayfinding (or wandering)

• need to provide cognitive information to support location determination and wayfinding behaviour (navigational awareness)

Page 45: Cosc 6326/Psych6750X

• people believed to form cognitive maps or models of the environment

• In VR we need to be able to navigate about the VE.

• A typical goal of wearable computers is to provide navigational awareness.

Page 46: Cosc 6326/Psych6750X

• Aids to wayfinding– landmarks (natural or artificial)– path following/leaving trails– maps– memorable place names– compass/instruments– exocentric views– coordinate display, grid structure– constrained travel

Page 47: Cosc 6326/Psych6750X
Page 48: Cosc 6326/Psych6750X


• Current VR systems allow small tracked work spaces

• To move to other places in the virtual environment (or in the real world) need to support travel

• Physical travel fits real world– Locomotion on foot is natural form of travel– Vehicular travel is also familiar

• direct or indirect control and locomotion

Page 49: Cosc 6326/Psych6750X

• Travel through computer based worlds using mice and joystick is familiar to many (computer games etc)

• Travel through time is also useful

Page 50: Cosc 6326/Psych6750X

Travel Methods and Metaphors

• Real locomotion– modest distances, limited by tracking

capabilities– some tricks possible (seminar later)– typically some real locomotion supported in

addition to other techniques

• Ride along a preplanned route

Page 51: Cosc 6326/Psych6750X

• ‘Tow rope’ or river metaphor

Page 52: Cosc 6326/Psych6750X

• Fly mode– most common, lots of freedom– walk through is fly mode constrained to follow

the terrain– direction and speed control– pointer (gaze, torso, gesture, wand/joystick, …)– can drive vehicles using virtual or physical

controls, often with increased control over speed, acceleration …

Page 53: Cosc 6326/Psych6750X

• moving /scaling the world– often used in visualization

• ‘put me there’ method– jump to a defined location– ask to go to named location– define location in a exocentric, map or world-in-

miniature display– ‘stepping’ into a picture or aperture into a different


Page 54: Cosc 6326/Psych6750X

• all methods except real locomotion dissociate real from virtual travel

• Problems with this dissociation– conflict between real and physical motion– unnatural, may interfere with wayfinding– not suitable for AR or wearable applications

• Real motion is most natural but not always desirable in virtual environments (for example travelling large distances or through outer space etc)

Page 55: Cosc 6326/Psych6750X

• Interest in extending the range of real physical motion – Hi-ball and other long range trackers– vision based AR trackers & wearable systems– GPS, landmark based– York Trike project