Robots for Disabilities Leapmotion Testing for

Accuracy as an Input

Method for Robotic Surgery

Anthony Brill

Matthew Moorhead

5/18/15

Robotic Surgery

• Developed to address issues with

traditional surgery techniques

o increase dexterity/precision

o reduce recovery time

o reduce physiologic tremors

• Ability to perform surgery remotely

o battlefield for wounded soldiers

o transcontinental for specialized

surgery reaching more patients

• Limited long-term studies to truly

understand benefits/disadvantages

Robotic Surgery

5/18/2015 2

Robotic Surgery

5/18/2015 3

Surgical Robots

• Current end effector manipulation

accomplished by use of a joystick-

like controller

• A more intuitive, hands-free method

has potential for ease-of-use

Robotic Surgery

5/18/2015 4

Project Goals

• Examine the accuracy of position

measurements made by the

Leapmotion sensor

• Control a robotic arm utilizing the

Leapmotion sensor as the input

Robotic Surgery

5/18/2015 5

Leapmotion

Leap Motion

• Two monochromatic IR

cameras

• Tracks infrared light (850 nm)

projected onto hands

• 8 ft3 of interaction space

• Image processing

• Tracking algorithm infers

hand position and orientation

Leapmotion

5/18/2015 7

Tool tracking

• Tracks the tip of a “tool”

• A pencil is used in this study

• Data is only used once the z-

component reaches the threshold

• Tool requirements: o longer, thinner, and straighter than

a finger

o Cylindrical

Leapmotion

5/18/2015 8

Hardware Communication

• Leap Motion data is accessed

through processing

• Robotic Arm o Tip position is sent to the arbotiX-

M Robocontroller through Serial

Com.

• Data Collection o Text-files created are exported for

analysis in excel

Leapmotion

5/18/2015 9

Robotic Arm

Forward Kinematics

• Diagram

• Transformation matrices

• End effector equation

Inverse Kinematics

Robotic Arm

5/18/2015 11

L2

q2

L1

q1

n2

n1

a2

a1

b2

b1

O

P

Q

Robotic Arm

5/18/2015 12

NRA a1 a2

n1 c1 -s1

n2 s1 c1

ARB b1 b2

a1 c2 s2

a2 -s2 c2

NRB = NRA · ARB

NRB b1 b2

n1 c1c2 + s1s2 c1s2 - c2s1

n2 c2s1 - c1s2 s1s2 + c1c2

NRB b1 b2

n1 c1-2 s2-1

n2 s1-2 c12

=

Rotation matrices:

L2

q2

L1

q1

n2

n1

a2

a1

b2

b1

O

P

Q

Robotic Arm

5/18/2015 13

NRA a1 a2

n1 c1 -s1

n2 s1 c1

x = L1c1 – L2c1-2

y = L1s1 – L2s1-2

NRB b1 b2

n1 c1-2 s2-1

n2 s1-2 c12

L2

q2

L1

q1

n2

n1

a2

a1

b2

b1

O

P

Q End effector:

rQ/O = rP/O + rQ/P = L1a1 – L2b1

= L1c1n1 + L1s1n2 – L2c1-2n1 –

L2s1-2n2

Robotic Arm

5/18/2015 14

L2

q2

L1

q1

n2

n1

r

β

α

(xc, yc)

Processing Code:

● Acquires tip position

from Leapmotion

● Maps values for

graphical interface

● Sends position

information to Arbotix

Robotic Arm

5/18/2015 15

Graphical Interface:

● Grey box - motor mount

● Red line - extension

limit of arm

● Colored line - tracked

tip position

Robotic Arm

5/18/2015 16

Drawing Results

Parameter testing:

Motor Speed: ~2.20rpm

Delay: 10ms

Observation: choppy output

Change: increase motor speed

Robotic Arm Drawing Results

5/18/2015 18

5 10 15 20 25

5 10 15 20 25 Parameter testing:

Motor Speed: ~8.79rpm

Delay: 10ms

Observation: no noticeable change

Change: reduce delay time

Robotic Arm Drawing Results

5/18/2015 19

Parameter testing:

Motor Speed: ~4.39rpm

Delay: 5ms

Observation: slightly smoother

lines at higher tip speed

Change: increase path accuracy

between points (future)

Robotic Arm Drawing Results

5/18/2015 20

5 10 15 20 25

Sensor Results

Leapmotion Sensor Results

5/18/2015 22

Experimental set up:

•Coordinate resolution • Coordinates are provided in mm’s

•Series of two lines • 25, 20, 15, 10, and 5mm apart

• Mean of x-position

• Standard deviation of x-position

•Comparison • Comparing the measured distance

to average distance from the leap

5 10 15 20 25

Leapmotion Sensor Results

5/18/2015 23

Distance (mean):

5.26mm

Standard Deviation:

Left: 0.83mm

Right:

0.43mm

Leapmotion Sensor Results

5/18/2015 24

Distance (mean):

10.80mm

Standard Deviation:

Left: 0.59mm

Right:

0.33mm

Leapmotion Sensor Results

5/18/2015 25

Distance (mean):

16.11mm

Standard Deviation:

Left: 0.52mm

Right:

0.86mm

Leapmotion Sensor Results

5/18/2015 26

Distance (mean):

21.99mm

Standard Deviation:

Left: 0.41mm

Right:

0.34mm

Leapmotion Sensor Results

5/18/2015 27

Distance (mean):

26.60mm

Standard Deviation:

Left: 0.46mm

Right:

1.11mm

Problems

● Robotic Arm

o The path of the end effector between points is random

o Does not provide smooth transition between data points

o Tip position data is not smooth

o Motor resolution provides limited accuracy

● Data Collected

o Standard deviations are fairly large

o Human error introduces significant effects on data

Problems

5/18/2015 28

Conclusion

● The Leap Motion successfully controls the robotic arm through tool tracking

o Results show a noisy output, implementing a filter would help

o Implementing a dynamic controller (PD,PID..) would improve performance

o Path planning could decrease error between data points

● Data collected from the sensor on average was within 1.5 mm of the actual distance

o Not acceptable for robotic surgery needs

Conclusion

5/18/2015 29