1

Virtual Reality Technology Virtual Reality Technology and Programmingand Programming

TNM053:TNM053:Lecture 5: Tracking and I/O devicesLecture 5: Tracking and I/O devices

Tracking devicesTracking devices

Referred to ‘headReferred to ‘head--tracking’ many timestracking’ many times–– Needed to get good stereo effect with parallaxNeeded to get good stereo effect with parallax–– Essential for Essential for HMD’sHMD’s as well.as well.–– And for Augmented realityAnd for Augmented reality

Devices to locate a point in 3DDevices to locate a point in 3D–– Immediately Immediately –– sample rate (latency)sample rate (latency)–– Continuously Continuously –– refresh raterefresh rate

Also want to Orientate:Also want to Orientate:–– In 3D spaceIn 3D space–– In other degrees of freedomIn other degrees of freedom

6 Degrees of freedom6 Degrees of freedom

(X,Y,Z,θ,φ,ω)

Important featuresImportant features

AccuracyAccuracyLatencyLatency–– Immediacy of the dataImmediacy of the data

Update RateUpdate Rate–– Frequent samplesFrequent samples

RangeRange–– Affects the suitability for usesAffects the suitability for uses

TypesTypes

MechanicalMechanicalUltrasonicUltrasonicInertial Navigation SystemsInertial Navigation SystemsElectromagneticElectromagnetic‘Vision’‘Vision’--basedbasedGPS?GPS?

Mechanical devicesMechanical devices

Consists of an armatureConsists of an armatureElectroElectro--mechanical devices measure mechanical devices measure anglesangles–– Optical devicesOptical devices–– Electrical potentiometersElectrical potentiometersCompute position (and orientation) Compute position (and orientation) using known lengths and anglesusing known lengths and anglesNot very good for head trackingNot very good for head tracking

2

Ultrasonic trackersUltrasonic trackers

Lots of typesLots of typesUse mobile ultrasound emittersUse mobile ultrasound emitters–– Can be battery poweredCan be battery powered

Multiple PickMultiple Pick--ups:ups:–– Minimum of 3Minimum of 3–– More helps to avoid occlusionMore helps to avoid occlusion

Ultrasonic tracking (3dof)Ultrasonic tracking (3dof)

Ultrasonic tracking (5dof)Ultrasonic tracking (5dof) Ultrasonic tracking (6dof)Ultrasonic tracking (6dof)

Ultrasonic trackers:Ultrasonic trackers:SummarySummary

Relatively cheapRelatively cheapLightLightQuite accurate in 3 Quite accurate in 3 dofdofOften not so good in 5 (or 6) Often not so good in 5 (or 6) dofdof–– Need wide separation of emittersNeed wide separation of emitters–– Reduced sample rateReduced sample rate

Inertial NavigationInertial Navigation

Measures accelerationMeasures accelerationMeasures rate of rotationMeasures rate of rotationRequires initializationRequires initialization–– Typically based on assumptionTypically based on assumption–– Possibly fixed pointPossibly fixed point

Drifts over timeDrifts over time–– Small errors, if not random, accumulateSmall errors, if not random, accumulate

3

Gyroscopic trackingGyroscopic tracking Modern ‘Gyroscopic’ Modern ‘Gyroscopic’ trackingtracking

SmallerSmallerQuieterQuieterNo moving partsNo moving parts–– No wearNo wear

Inertial tracking:Inertial tracking:InterSenseInterSense InertiaCube2InertiaCube2

Box contains:Box contains:–– 3 orthogonal gyros3 orthogonal gyros–– 3 linear accelerometers3 linear accelerometers

Support hardware does Support hardware does the integrationthe integrationAlso magnetic sensorsAlso magnetic sensors–– avoid error buildavoid error build--upup

Inertial trackingInertial tracking

Fairly fast results (latency)Fairly fast results (latency)Fast sampling (180Hz to 90Hz)Fast sampling (180Hz to 90Hz)No interferenceNo interferenceNo occlusionNo occlusionNot accurate enoughNot accurate enough–– Accumulates errorAccumulates error–– 0.10.1° per second error° per second error–– Similar error in distance movementSimilar error in distance movement

Hence hybrid devicesHence hybrid devices

Hybrid trackingHybrid tracking

Ultrasound/InertialUltrasound/Inertial–– Ultrasound provides updates and error Ultrasound provides updates and error

correctioncorrection–– Doesn’t rely entirely on eitherDoesn’t rely entirely on either

May rely on magnetic fields as wellMay rely on magnetic fields as well–– Passive sensors pick up earth fieldPassive sensors pick up earth field

E.g. E.g. InterSenseInterSense devices in labdevices in lab

Hybrid tracking:Hybrid tracking:InterSenseInterSense IS600 mk2IS600 mk2

INS ‘SoniDiscs’

4

Hybrid tracking:Hybrid tracking:InterSenseInterSense IS900ITIS900IT

Hybrid head trackers:Hybrid head trackers:InterSenseInterSense 900 900 MiniTraxMiniTrax

Acoustic devices

Hybrid head trackersHybrid head trackers

LatencyLatencySample rate ~180HzSample rate ~180Hz–– Divided by number of ‘stations’Divided by number of ‘stations’–– Station = three emitter setStation = three emitter set

Accuracy:Accuracy:–– ±4mm, ± ±4mm, ± 0.50.5° (RMS error)° (RMS error)

Coverage ~25 square metresCoverage ~25 square metres

ElectromagneticElectromagnetic

Use a source and small detectorsUse a source and small detectorsMultiple detectors per systemMultiple detectors per system–– Typically 4Typically 4--1616

Signal produced in three planesSignal produced in three planesDetected in three planesDetected in three planesTypically medium range (<10 metres)Typically medium range (<10 metres)Suffers from interferenceSuffers from interference

Electromagnetic trackersElectromagnetic trackers

x

y

z

x

y

z

Fixed Source

Mobile sensor

PolhemusPolhemus FastTrackFastTrack

5

PolhemusPolhemus trackerstrackers

Latency ~4mSLatency ~4mSSample rate: ~120HzSample rate: ~120Hz–– Each receiver unit functions independentlyEach receiver unit functions independently–– Same sample rate regardless of number of Same sample rate regardless of number of

detectors (if the hardware supports it)detectors (if the hardware supports it)–– usually use 2usually use 2--44

Accurate!Accurate!–– ±0.07mm, ± ±0.07mm, ± 0.150.15° (RMS error)° (RMS error)

No Accumulation of errorsNo Accumulation of errors

Laser trackingLaser tracking

Fixed laser sourcesFixed laser sourcesMobile detectors receive signalMobile detectors receive signalCompute position at three pointsCompute position at three points–– Like ultrasound, fast processing requiredLike ultrasound, fast processing required

Compute 6DOF from three pointsCompute 6DOF from three points

Laser Tracking:Laser Tracking:Ascension Ascension laserBIRDlaserBIRD

Two source ‘scanner’Two source ‘scanner’Three point sensorThree point sensor–– Fitted to headsetFitted to headset

Uses stereo ‘vision’Uses stereo ‘vision’–– Gives 3D position of sensorsGives 3D position of sensors

Laser Scanner unit

Sensor(s)

Laser tracking:Laser tracking:Ascension Ascension laserBIRDlaserBIRD

6 DOF6 DOF240 Hz sampling rate240 Hz sampling rateQuite accurate:Quite accurate:–– ±1mm, ±±1mm, ±11° (RMS error)° (RMS error)

No interferenceNo interferenceSuffers from occlusionSuffers from occlusion

Laser tracking:Laser tracking:Ascension Ascension laserBIRDlaserBIRD Optical trackingOptical tracking

New in the last couple of years.New in the last couple of years.‘Vision’ systems used to track moving ‘Vision’ systems used to track moving objects automaticallyobjects automaticallyLike motionLike motion--capture for film industrycapture for film industryWorks in the infraWorks in the infra--redred

6

‘‘Optical’ tracking: Optical’ tracking: markersmarkers

Optical Tracking: Optical Tracking: CamerasCameras

Image basedImage based

Uses limited faceUses limited face--recognition to track recognition to track headhead–– Actually tracks eyesActually tracks eyesDetermines position of headDetermines position of headAdjusts display accordinglyAdjusts display accordinglyNot commonNot common–– Primarily used in display technologyPrimarily used in display technology–– E.g. i2i D4D displayE.g. i2i D4D display

i2i i2i -- D4D4D display:D display:TTakingaking the imagethe image

i2i i2i -- D4D4D display:D display:Processing the imageProcessing the image

i2i i2i -- D4D4D display:D display:Displaying the imageDisplaying the image

7

Image based tracking (2)Image based tracking (2)

Eye surgery at the University of Münster

Head Tracking: SummaryHead Tracking: Summary

Mechanical Mechanical –– not used muchnot used muchUltrasonicUltrasonicInertialInertial–– hybridhybrid

ElectromagneticElectromagneticOptical Optical –– Getting popularGetting popularLaser Laser –– not common yetnot common yetVision Vision –– very specializedvery specialized

—— No occlusionNo occlusion

What to use and why:What to use and why:

ElectromagneticElectromagnetic–– Most accurateMost accurate–– ExpensiveExpensive–– Can’t use everywhereCan’t use everywhere

Interference issues Interference issues –– Maybe with iron?Maybe with iron?HospitalsHospitals

Hybrid inertial/acousticHybrid inertial/acoustic–– Quite accurateQuite accurate–– cheapercheaper

Common Problems with Common Problems with tracking hardwaretracking hardware

TrackingTracking–– LinearityLinearity–– LatencyLatency–– Noisy signalsNoisy signals

Frequently need filteringFrequently need filtering

Cables!Cables!–– Becoming less of a problem through Becoming less of a problem through

wireless technologieswireless technologies

Virtual Reality Technology Virtual Reality Technology and Programmingand Programming

TNM053:TNM053:Lecture 5.5: I/O devicesLecture 5.5: I/O devices

I/O DevicesI/O Devices

Important aspects:Important aspects:–– Degrees of Freedom (DOF)Degrees of Freedom (DOF)–– PrecisionPrecision–– LatencyLatency–– Long Term usage effectsLong Term usage effects

Mouse Mouse –– 2 to 6 DOF2 to 6 DOFWand Wand –– 5 DOF5 DOFJoysticks Joysticks –– 6 DOF6 DOFGlove Glove –– Hand ‘gestures’Hand ‘gestures’Suit Suit –– Body ‘gestures’Body ‘gestures’

8

TrackingTracking

All the same methods applyAll the same methods apply–– Attach Attach sensor(ssensor(s) to object) to object–– it’s an it’s an interactorinteractor

Mechanical tracking is more commonMechanical tracking is more common

Mechanical trackingMechanical tracking

Consists of an armatureConsists of an armatureElectroElectro--mechanical devices measure mechanical devices measure anglesangles–– Optical devicesOptical devices–– Electrical potentiometersElectrical potentiometersCompute position (and orientation) Compute position (and orientation) using known lengths and anglesusing known lengths and anglesCommonly used in adapting toolsCommonly used in adapting tools

Optical detectorsOptical detectors Optical detectorsOptical detectors

System always knows positionSystem always knows positionCan be extremely accurateCan be extremely accurateLittle ‘wear and tear’Little ‘wear and tear’System has to be initializedSystem has to be initialized–– Bring to a fixed point and start systemBring to a fixed point and start system–– If it loses position it’s uselessIf it loses position it’s useless

Requires expensive partsRequires expensive parts

PotentiometersPotentiometers

Similar but with different propertiesSimilar but with different propertiesNo Initialization requiredNo Initialization required–– Voltage gives immediate positionVoltage gives immediate position

Can be much cheaper to buildCan be much cheaper to buildParts more prone to wear outParts more prone to wear outCan’t easily rotate through zeroCan’t easily rotate through zero–– Problem with the shoulder?Problem with the shoulder?

Mechanical devicesMechanical devices

Relatively cheapRelatively cheapCan be extremely accurateCan be extremely accurateGives better quality signalGives better quality signal–– Little or no noiseLittle or no noise–– Avoids need for filteringAvoids need for filtering

9

Stefan’s ‘Monkey’Stefan’s ‘Monkey’

HomeHome--built 3D position devicebuilt 3D position device–– Designed with motion capture in mindDesigned with motion capture in mind–– Represents a humanoid armRepresents a humanoid arm

Three stage armature (with ‘shoulder’)Three stage armature (with ‘shoulder’)Four potentiometersFour potentiometers–– Three in the ‘shoulder’Three in the ‘shoulder’–– One in the ‘elbow’One in the ‘elbow’

Stefan’s ‘monkey’ (2)Stefan’s ‘monkey’ (2)

Stefan’s ‘monkey’ (3)Stefan’s ‘monkey’ (3) Stefan’s ‘monkey’ (4)Stefan’s ‘monkey’ (4)

Presented at SIGGRAPH 2002Presented at SIGGRAPH 2002–– “Motion Capture Done Dirt Cheap”“Motion Capture Done Dirt Cheap”Total cost ~$300Total cost ~$300–– Provides for up to 16 potentiometersProvides for up to 16 potentiometers–– Add $10 for each extra potentiometerAdd $10 for each extra potentiometerhttp://www.itn.liu.se/~stegu/monkey/http://www.itn.liu.se/~stegu/monkey/–– Parts listParts list–– Circuit diagramsCircuit diagrams

Mouse interactionMouse interaction

Extension of the 2D mouseExtension of the 2D mouse–– Tries to provide 6DOFTries to provide 6DOF

Some attempts to use for interactionSome attempts to use for interaction–– Selection of objectsSelection of objects–– Selection of menu options etc.Selection of menu options etc.

Doesn’t really workDoesn’t really workPrincipally useful for navigationPrincipally useful for navigation

3D Mouse3D Mouse

LogiCADLogiCAD Magellan Magellan SpaceMouseSpaceMouse

10

More MiceMore Mice More useful mouse:More useful mouse:PolhemusPolhemus 3Ball3Ball

Provides 3(Provides 3(--6)DOF in a usable package6)DOF in a usable package

Actually nothing like a mouse at allActually nothing like a mouse at all

3D Mouse = ‘Wand’3D Mouse = ‘Wand’

Provides 5DOF (maybe 6)Provides 5DOF (maybe 6)Typically used as a pointing deviceTypically used as a pointing deviceProvides buttons for selectionProvides buttons for selectionThe 3D equivalent of the mouseThe 3D equivalent of the mouse

Wand: Wand:

Wand:Wand:InterSenseInterSense devicesdevices

Geological data analysis at Schlumberger

Wireless

Wand = JoystickWand = Joystick

6DOF makes it a joystick6DOF makes it a joystick–– Tracking systems used can provide 6Tracking systems used can provide 6

Allows complex navigationAllows complex navigationAllows wandAllows wand--like selectionlike selection

11

Joystick devicesJoystick devices

PolhemusPolhemus devicedeviceMade from game joystickMade from game joystick–– Quite literally!Quite literally!

Virtual Presence ‘SpaceStick’

GlovesGloves

Hand position trackingHand position trackingAlso track finger positionsAlso track finger positions–– Allows for ‘gestures’Allows for ‘gestures’–– Allows for picking up objectsAllows for picking up objects

Gloves: Hand trackingGloves: Hand tracking

IS900 IS900 MiniTraxMiniTrax Hand trackerHand tracker

Gloves: Finger trackingGloves: Finger tracking

Optical fibres measure bend of the finger

GlovesGloves

Of limited availability due to costOf limited availability due to costTracking devices already availableTracking devices already available–– Not the source of the costNot the source of the cost

Optical fibres and sensorsOptical fibres and sensors–– Very expensive technologyVery expensive technology–– Not much demandNot much demand–– Very expensive devicesVery expensive devices

Gloves: Done cheapGloves: Done cheap

MATTEL (NES) MATTEL (NES) PowerGlovePowerGloveProduced in 1989Produced in 1989--19911991Cost $99 originallyCost $99 originallyNow cost ~$40 (on Now cost ~$40 (on ebayebay))

12

Gloves for the userGloves for the user

Can use as a wand or joystickCan use as a wand or joystickCan use ‘hand gestures’ Can use ‘hand gestures’ –– flex fingersflex fingers–– To select objectsTo select objects–– To call up menusTo call up menus

Can interact with virtual objects in the Can interact with virtual objects in the scenescene–– Can’t feel them of courseCan’t feel them of course

SuitsSuits

Extension of the data gloveExtension of the data gloveUse trackers and flexion detectorsUse trackers and flexion detectors–– Monitor whole body positionMonitor whole body position

Mostly used for motion captureMostly used for motion capture–– Can be used interactivelyCan be used interactively–– Can use ‘body gestures’ to interactCan use ‘body gestures’ to interact

Suits Suits -- Mechanical: Mechanical: MetamotionMetamotion GypsyGypsy

Gypsy Jr – Upper body Gypsy 3 – Whole body

Suits Suits –– e/me/m tracking:tracking:Ascension Ascension MotionStarMotionStar

Real Time Real Time e/me/m trackingtracking Optical tracking: Full BodyOptical tracking: Full Body

13

SummarySummary

A dazzling array of toys to play withA dazzling array of toys to play withAny tool you want can be made virtualAny tool you want can be made virtual–– Adding tracking can be quite cheap…Adding tracking can be quite cheap…

…as long as you already have it…as long as you already have it

–– Adding optical or Adding optical or e/me/m sensors is easysensors is easy–– Building kit is not hard or expensiveBuilding kit is not hard or expensive

Working out how to interact is hardWorking out how to interact is hard