intel realsense hands-on lab - rome

Tips and Tricks from Real Case Studies

Matteo Valoriani - Roma – 23/09/2015 mvaloriani@gmail.com @matteovaloriani

2

Nice to Meet YouMatteo Valoriani

CEO of FifthIngenium

PhD at Politecnico of Milano

Speaker and Consultant

Microsoft MVP

Intel Software Innovator

email: mvaloriani@gmail.com

twitter: @MatteoValoriani

linkedin: https://it.linkedin.com/in/matteovaloriani

3

You have to be a magician…

or at least a good illusionist

UI evolution

4

RealSense SnapShot

Display: 8.4" OLED infinity (2560 x 1600)

Dimensions: 215.9 x 124.2 x 6.1 mm

Weight: 305 g

OS: Android OS, v4.4.2 (KitKat),

CPU: Intel Atom Z3580, Quad-core 2.3 GHz

Memory: 16 GB (+microSD), 2 GB RAM

Primary: 8 MP, 3264 x 2448 pixels,

autofocus

Secondary: 2 MP

Intel RealSense Snapshot, 2 x 720p

cameras

Dell Venue 8 7000

6

Measurement Tool

7

Refocus / PhotoEditing

8

Stereoscopic Vision

9

Depth Sensing

10

Mathematical Model

b

𝑑 = 𝑥𝑙 − 𝑥𝑟

𝑏+𝑥𝑙 − 𝑥𝑟

𝑍−𝑓=

𝑏

𝑍

Z =𝑏∗𝑓

𝑑

11

RealSense R200

R200

@60FPS, depth at 320x240,, color can be 640x480

@60FPS, depth at 480x360, color can be 320x240 or 640x480

@30FPS, depth at 320x240,, color can be 640x480, 1280x720 or 1920x1080

@30FPS, depth at 480x360, color can be 320x240, 640x480, 1280x720, or 1920x1080

The dual depth cams use a fixed focus 4:3 aspect ratio with a 70x59x46 degree field of view.

The IR is a class 1 laser in the 850nm range

RealSense F200

Understands 4 basic types of input

15

Categories of Input

Capabilities Features

Hands • Hand and Finger

Tracking • Gesture Recognition

• 22-point Hand and Finger Tracking• 9 static and dynamic mid-air gestures

Face • Face Detection and Tracking

• Multiple Face Detection and tracking

• 78-point Landmark Detection (facial features)

• Emotion Recognition (7 emotions, coming post-Beta)

• Pulse Estimation• Face Recognition (Coming post-beta)

Speech • Speech Recognition • Command and Control

• Dictation• Text to Speech

Environment • Segmentation

• 3D Scanning• Augmented Reality

• Background Removal

• 3D Object / Face / Room Scanning (Coming post-beta)

• 2D/3D Object Tracking• Scene Perception (coming post-beta)

Understands Hardware Limits

16

Competitive

technologies focus

on a living-room

experience or a

sub-set of Intel

RealSense

technology

features

Designed for close-range interactions

17

120 cm

Intel®RealSense™

3D camera

56°(v) x 72° (v)

20 cm

Leap, RealSense, Kinect

18

2,5 cm 60 cm 2 m 4 m

Coordinate Systems

19

World coordinates Image coordinates

Vertical Rages

20

60cm

58 c

m

120cm

56°

20cm

17 c

m

70cm 35cm73 c

m

Effective

Range

Gestures

Range

Effective

Range

3D Facial Traking

Range

2D Facial Traking

Range

115cm

Vertical Misalignment

21

56°

Horizontal Rages

22

60cm

87cm

120cm

72°

20cm

24cm

Effective

Range

Gestures

Range

Effective

Range

3D Facial Traking

Range

2D Facial Traking

Range

170cm

70cm 35cm108cm

50cm

Capture Volumes

23

The user is performing a hand gesture outside of the capture volume.

The camera will not see this gesture

Evaluate different settings and environment

24

RealSense Camera use IR light and Sunlight can blind the

camera!!!

• Check exposition during all day

• Verify that there isn’t direct light on the camera

Indoor/Outdoor

25

RealSense isn’t a Rugged device:

• Check temperatures (+3/33°)

• Check humidity

Indoor/Outdoor (2)

26

Comfortable positions

Your users are not GORILLAS!!!

27

User posture may affect design of a gesture

28

Input variability

29

Gesture Tracking

30

Feedback, feedback, feedback,…

31

View of user:

• User Viewport

• User Overlay

Feedback, feedback, feedback,…

32

… where actions performed for some other purpose or unconscious signs are interpreted in order to influence/improve/facilitate the actors' future interaction or day-to-day life (from Alan Dix)

• The interaction is not purposeful from the person side, but it is designed “to happen”

• It “happens” in relation to signs which are not done for that (body temperature, unconscious reactions such as blink rate, or unconscious aspects of activities such as typing rate, vocabulary shifts (e.g. modal verbs), actions done for other purposes, …

• It is designed for people acting

Manage Incidental Interaction

33

NetSenseFast and simplified way to create RealSense Applications

de

mo https://github.com/mvaloriani/NetSense/

PM> Install-Package NetSense

35

Intel RealSense

36

37