1 | S X S W 2 0 1 7

DESIGNING TRACKEDDOUG BRUEY | MICHAEL CIUFFO

OBJECTS FOR

2 | S X S W 2 0 1 7

■ Product development for the world’s leading companies

■ 150+ multi-disciplinary engineers■ Cambridge Consultants

• Global parent company• 500+ Engineers and innovators

SYNAPSE

SEATTLESAN FRANCISCO

ORLANDOBOSTON

NEW DELHI

CAMBRIDGE

HONG KONG

SINGAPORE

TOKYO

3 | S X S W 2 0 1 7

Valve is an entertainment software and technology company founded in 1996. In addition to creating several of the world's most award-winning games, Valve is also a developer of leading-edge technologies including the Source® game engine and Steam®, the premier online gaming platform.

OCT 2014 Synapse started working with Valve on their vr system

HISTORY

4 | S X S W 2 0 1 7

JUNE 2015Developer release of the base stations

APRIL 2016HTC Vive released

AUG 2016Valve announces royalty free licensing of SteamVR™ tracking

SEPT 2016SteamVR™ tracking reference design released

2017Develop firmware and assist partners integrating SteamVR™ tracking into their products

SEPT 2016 - TODAYSteamVR™ tracking training courses at synapse

• What is SteamVR™ Tracking?

• The problem it solves in VR and AR applications

• How it works to track the pose of an object

• How to make objects that track well using the technology

• How to get started using the technology

• Answer as many questions and possible

WE’LL COVER

5 | S X S W 2 0 1 7

YAW

ORIENTATION

6 | S X S W 2 0 1 7

ROLL

PITCH

POSE TRACKING

Z-Axis

z

7 | S X S W 2 0 1 7

X-Axis

Y-Axis

x

POINT(x,y,z)

y

POSITIONPOSE TRACKING

STATE OF THE ART

8 | S X S W 2 0 1 7

IMUs (Inertial Measurement Units)■ Gyroscopes and

accelerometers track orientation

External Cameras (Pose Tracking)

Internal Cameras (Inside-out Pose Tracking)■ Popular in AR■ Depth Perception &

Room Mapping

SteamVR™ Tracking(Pose Tracking)

STEAMVR™ TRACKING

9 | S X S W 2 0 1 7

■ Room scale■ Security/privacy■ Low latency & bandwidth■ Absolute position■ Eliminates challenges of camera-based tracking■ Track to submillimeter accuracy

STEAMVR™ TRACKING SYSTEM OVERVIEW

1 0 | S X S W 2 0 1 7

BASE STATION

1 1 | S X S W 2 0 1 7

TRACKED OBJECTS

1 2 | S X S W 2 0 1 7

■ Reduces shadows■ Requires synchronization

MULTIPLE BASE STATIONS

1 3 | S X S W 2 0 1 7

REFERENCE SIGNALING

1 4 | S X S W 2 0 1 7

■ Optical receivers detect the reference signals from the base station■ Photodiode converts IR light to current■ Transimpedance amplifier converts current to voltage■ Envelope detector removes the modulation frequency■ FPGA connects to all sensors to timestamp the arrival of reference signals

OPTICAL RECEIVERS

1 5 | S X S W 2 0 1 7

TRIANGULATION

1 6 | S X S W 2 0 1 7

■ The angle to all sensors creates a solution set for the position of the object■ SteamVR™ matches the measured angles to the known object geometry■ There must only be one solution to the problem!

SOLVING THE SYSTEM

1 7 | S X S W 2 0 1 7

SOLVING THE SYSTEM

1 8 | S X S W 2 0 1 7

DEMO

1 9 | S X S W 2 0 1 7

ONE TO FOUR SENSORS

■ Reference signal jitter• Motor jitter• Timestamp quantization• Laser modulation

■ Minimized by design• Base station design• Object design

■ Sources of error place requirements on object design

SOURCES OF ERROR

2 0 | S X S W 2 0 1 7

■ Baseline increases the time between laser hits■ Sources of error are angular■ The limit of the system is a minimum detectable angle■ More baseline accommodates the same angle at a greater distance■ Need baseline to overcome translation error

CONSEQUENCES OF ERROR | TRANSLATION ERROR

2 1 | S X S W 2 0 1 7

■ Baseline increases the time between laser hits■ Sources of error are angular■ The limit of the system is a minimum detectable angle■ More baseline accommodates the same angle at a greater distance■ Need baseline to overcome translation error

OVERCOMING TRANSLATION ERROR | SUFFICIENT BASELINE

2 2 | S X S W 2 0 1 7

■ Rotation orthogonal to a plane yields significant displacement■ Rotation in the plane yields much smaller displacement per degree rotation■ Error dominates the small change in distance

CONSEQUENCES OF ERROR | ROTATION ERROR

2 3 | S X S W 2 0 1 7

■ Baseline amplifies the effect of rotation■ Pose tracking detects

• Roll• Pitch• Yaw

■ Detecting all three means amplifying all three■ Need baseline in X, Y and Z axes to overcome rotation error

OVERCOMING ROTATION ERROR | BASELINE IN 3 AXES

2 4 | S X S W 2 0 1 7

■ Four visible sensors (minimum)■ Maximize the distance between sensors■ Maximize baseline in three axes

TRACKING PERFORMANCE DRIVERS

2 5 | S X S W 2 0 1 7

dx

zy2

■BPW 34 S has a ±60° field of view from normal

■At 60°, the sensitivity is 50% of 0° sensitivity

■System is specified for 5 meters at 60° off axis

■Simulation output accounts for the 60° viewing angle limitation

SENSOR FIELD OF VIEW

2 6 | S X S W 2 0 1 7

DEMO

2 7 | S X S W 2 0 1 7

VIEWING ANGLE

■ Objects that track well have geometries designed for optimal sensor placement!

■ Simulation lets us verify performance early in the process.

SENSOR PLACEMENT CRITERIA

2 8 | S X S W 2 0 1 7

X, Y, Z coordinates, facing direction

2 9 | S X S W 2 0 1 7

±60° from normal Shadows cast by the sensor object

Shadows cast by nearby objects

3 0 | S X S W 2 0 1 7

DEMO

3 1 | S X S W 2 0 1 7

HMD DESIGNER VIEWER

NUMBER OF VISIBLE SENSORS

3 2 | S X S W 2 0 1 7

INITIAL POSE POSSIBLE

3 3 | S X S W 2 0 1 7

TRANSLATION ERROR

3 4 | S X S W 2 0 1 7

ROTATION ERROR

3 5 | S X S W 2 0 1 7

REFINEMENTS | NUMBER OF VISIBLE SENSORS

3 6 | S X S W 2 0 1 7

REFINEMENTS | POSE ROTATION ERROR

3 7 | S X S W 2 0 1 7

REFINEMENTS | POSE TRANSLATION ERROR

3 8 | S X S W 2 0 1 7

DEMO

3 9 | S X S W 2 0 1 7

HMD DESIGN REFINEMENTS

■ Consumer products need to look and feel great■ Products also need to perform, especially VR products

INDUSTRIAL DESIGN CHALLENGES

MINIATURIZATION

4 0 | S X S W 2 0 1 7

FLAT SURFACES RIGHT ANGLES LOW PROFILE CURVED SURFACESTranslation error? Translation and rotation error?Sensor viewing angle?Rotation error? Sensor covering?

MECHANICAL DESIGN CHALLENGES

4 1 | S X S W 2 0 1 7

■ Facets on shapes improve performance• How many slides in the mold?

■ Sensors facing in all directions• Multiple parts to facilitate ejection

■ Sensor interconnect■ FPC design challenge

ELECTRICAL DESIGN CHALLENGES

4 2 | S X S W 2 0 1 7

RECOMMENDATIONS

4 3 | S X S W 2 0 1 7

■ Collaborate between engineering and industrial design early in the process

■ Teach industrial designers and product visionaries about sensor placement

■ Use the constraints of sensor placement as a seed for creating unique, compelling designs

■ Reduce risk early• Use the simulation tools in the HDK to validate design choices• Prototype shapes using rapid prototyping techniques and evaluation hardware• See your object track in SteamVR™ before investing in tooling

■ Laser based pose tracking system■ Solves the problem in VR and AR applications■ Tracks the pose of an object through….■ Design objects that track well using the technology

• Four visible sensors (minimum)• Maximize the distance between sensors• Maximize baseline in three axes

■ Use the software to simulate & refine design■ Collaboration between ID, ME, EE, & SW is key to successful design

SUMMARY | STEAMVR™ TRACKING

4 4 | S X S W 2 0 1 7

NEXT STEPS

partner.steamgames.com/vrtracking

STEAMVR™ TRACKING LICENSE

4 5 | S X S W 2 0 1 7

STEAMVR™ TRACKING TRAININGsynapse.com/steamvr

HARDWARE DEVELOPMENT KITtriadsemi.com/product/steamvr-tracking-hdk

■ VR Accessories■ AR Products■ Tracking Systems

WHAT ARE YOU GOING TO MAKE?

4 6 | S X S W 2 0 1 7

The information contained herein is confidential and proprietary and may not be reproduced or distributed without the consent of Synapse Product Development

QUESTIONS?

4 7 | S X S W 2 0 1 7