haptic interface

Haptic Interface

April 2006

Prof. Ed Colgate

Northwestern University

Evanston, IL USA

© J. Edward Colgate, 2006

Today’s Class

Course overview Introduction to Haptics

Haptics overview History Applications/Motivations How to design effective haptic interfaces Current challenges

Break Psychophysics



Chicago



Founded in 1851 in Evanston, IL Two campuses today: Evanston and Chicago ~8000 undergraduates and ~6000 graduates in 9

schools ~1400 undergraduates and ~700 graduates in the

McCormick School of Engineering and Applied Science

9 Departments in McCormick Applied Math; Biomedical; Chemical; Civil & Environmental;

Computer; Electrical; Industrial; Materials Science; Mechanical


Northwestern Scenes


Northwestern>Dept of Mechanical Engineering>LIMS

Prof. Michael Peshkin

Prof. Kevin Lynch

Prof. Mitra Hartmann

Not shown: Prof. Malcolm MacIver


LIMS Research

Human-Robot InteractionHaptic (touch) interfaceAssistive robots

Robot Motion PlanningUnderactuated systems

Biologically-Inspired RoboticsRobotic fishActive sensing

Prototype variable friction haptic display

Developed by John Glassmire

Robotic Ribbon Fin

Developed by Michael Epstein


Goals of this course Gain familiarity with key ideas in haptics

Haptic perception Psychophysics Design and control of haptic interfaces Passivity, Z-width Haptic Rendering

Hands-on experience with haptics Gain some familiarity with current research and

literature Identify opportunities for research in haptics


Grading

30% Class participation/contribution Ask questions! Offer opinions, insights, etc.

40% Homework 3 assignments, due Wed, Thu, Fri

30% Paper presentations Each student gives a 15 minute presentation of a

paper All papers are from 2006 Haptics Symposium


Website

http://othello.mech.northwestern.edu/~colgate/UPC/


hap·tic ('hap-tik)adj.

Of or relating to the sense of touch; tactile.[Greek haptikos, from haptesthai, to grasp, touch. (1890)]

Temperature TextureSlip Vibration Force

Location/configuration Motion ForceCompliance

Cutaneous Kinesthesia


Our focus: programmable haptic interfaces

Mainly: kinesthetic interface to virtual environments

Also: tactile interface to virtual environments

Phantom - kinesthetic

Pin Array – low frequency cutaneous


More generally: human-robot interaction

Telemanipulation Exoskeletons Physical Rehabilitation and Exercise

machines Intelligent Assist Devices (IADs) Advanced prosthetics “Near-field” telerobotics Human-robot-human


A Little History

Ray Goertz, Argonne National Lab, 1940s


Computer simulation replaces the slave manipulator

Fred Brooks, UNC Chapel Hill, 1970s

Developed to study molecular docking

User feels interaction force between molecules

Master was one of Goertz’s Didn’t work very well…


~1990 – Haptic Interface Emerges as an Engineering Discipline

Margaret Minsky’s “virtual sandpaper” system developed at the MIT Media Lab

Dov Adelstein’s force reflecting joystick developed in the MIT Biomechanics Lab

Force

Minsky, 1990


LIMS has been involved since the early days (~1991)

Paul Millman’s 4DOF Haptic Interface Originally developed with

telemanipulation in mind Never got around to

developing the slave!


LIMS continues to be active in haptics


Three-rotational virtual spring and damper


Ball in a box


Haptics has many applications

Blind Persons Programmable Braille Access to GUIs

Training Medical Procedures Astronauts

Education Computer-Aided Design

Assembly-Disassembly Human Factors

Entertainment Arcade (steering wheels) Home (game controllers)

Automotive BMW “iDrive” Haptic Touchscreens

Mobile Phones Immersion “Vibetonz”

Animation/Modeling Art Material Handling

Virtual Surfaces


TrainingVisual display alone is not sufficient for certain

types of virtual environments. To learn physical skills, such as using complicated hand tools, haptic information is a requirement

Applications


Virtual Prototyping

Applications McNeeley et al. (Boeing Corp.)


Rehabilitation

Applications


Teleoperation

Applications


Computer interface for blind users

Text-based computers can easily be enhanced to include a speech synthesizer

Graphical user interfaces are inherently visual A haptic display can help a blind computer user

interact with graphics-based operating systems


Entertainment

Applications


Underlying motivations for haptics

Looking across applications, we find common motivations:

Haptics is required to solve the problem Interfaces for the blind Phlebotomy training (task is mainly “feel”) Vibetonz – a private communication channel

Haptics improves realism and sense of immersion Entertainment Animation/modeling

Haptics provides constraint Assembly/disassembly Virtual surfaces


How to design effective haptic interfaces

A simple three-step program…

A. Understand how the human sensory and perceptual systems work

B. Use this information to develop performance metrics

C. Understand how to build/control machines that display haptic percepts and meet performance metrics


A. How haptic sensing works

Let’s see it in action…


Some terminology

“Haptic” refers to the perceptual system that draws information from the skin and kinesthesis

“Proprioception” is the unconscious perception of movement and spatial orientation arising from stimuli within the body itself

“Kinesthesia” is the sense that detects bodily position, weight, or movement of the muscles, tendons, and joints

“Tactual Stereognosis” is the perception of the form of an object by means of touch


Sensors that contribute to haptic perception

4 types of mechanoreceptors

2 types of thermoreceptors

2 types of nociceptors (free nerve endings for pain)

3 types of kinesthetic receptors


Mechanoreceptors

Mechanoreceptorsdiffer according to:

-frequency response-receptive field-location

Merkel’s Disk(SA I)

• 0.4 Hz - 100 Hz• 11 mm2 receptive field• shallow

•Curvature, shape, pressure

Meissner’s Corpuscle(FA I)

• 2 Hz - 200 Hz• 13 mm2 receptive field• shallow

• flutter vibration; tickle; texture?

Pacinian Corpuscle(FA II)

• 40 Hz - 800 Hz• 101mm2 receptive field• deep

• vibration

Ruffini Endings(SA II)

• 0.4 Hz - 100 Hz• 59 mm2 receptive field• deep

• skin stretch, force


The skin is an important organ!

Large: approximately 2 m2

Abundant sensors: ~500,000 mechanoreceptors spread across the

body ~17,000 in the glabrous (non-hairy) skin of the hand


Sensory Homunculus


Kinesthetic receptors Muscle Spindles

provide muscle length and velocity information

Golgi Tendon Organs provide tension information

Joint Afferents provide joint angle and angular velocity information Ruffini endings and

Pacinian corpuscles located in joint capsule

Note that people with artificial joints have almost normal sense of joint position


Bilateral Nature of Kinesthetic Sensing

Human Hand/Arm

Environmenteffort

flow

effort flow = Power

Unlike vision & audition, kinesthetic sensing is two-way

There is also the prospect for significant power exchange with the environment as part of a haptic interaction


Conclusions: how haptic sensing works

A vast number of sensors in both the skin and musculoskeletal system work in conjunction with the motor control system to enable sensing of mechanical stimuli Haptic sensing is bilateral

Perception clearly involves the CNS as well as the peripheral nervous system, but that is the subject of another lecture…

We’ve just barely scratched the surface!


B. Performance metrics for haptics

Performance can be assessed at various levels: Peripheral sensors From sensors to CNS Perceptual Functional


Pressure thresholds Weinstein, 1968 Pressure

measured with precisely calibrated nylon filaments pressed into skin


Point localization thresholds

Weinstein, 1968 Distance between

body point stimulated and subject’s impression of where stimulation took place

Two-point discrimination data are similar


Frequency response thresholds

Bolanowski, 1988


Just Noticeable Differences

I is the increment in intensity that, when added to stimulus intensity I, produces a just noticeable difference.

I/I = k is the “Weber fraction” Weber hypothesized that k would remain

constant across all values of I for a given modality. Not true, but often a reasonable approximation


JNDsVision (brightness, white light) 1.5%

Audition (middle pitch & moderate loudness) 10%

Smell (odor of India rubber) 25%

Taste (table salt) 33%

Kinesthesis (lifted weights) 2%

Pressure (cutaneous pressure “spot”) 14%

Length 10% or less

Velocity 10%

Acceleration 20%

Force on skin 7%

Compliance 23%

Viscosity 34%

Sources: Biggs and Srinivasan “Haptic Interfaces” Schiffman “Sensation and Perception”


Perceptual Measures

Channel capacity in bits/sec Max information flow at receptor level:

~107 bits/sec for eye ~106 bits/sec for hand ~105 bits/sec for ear

Post-processing rate for tactile information is ~2-56 bits/sec Compare to ~40 bits/sec for speech and ~30

bit/sec for reading Kaczmarek, K.A., and P. Bach-y-Rita, “TactileDisplays”, in W. Barfield, and T.A. Furness (Eds.), VirtualEnvironments and Advanced Interface Design, OxfordUniversity Press, New York, 1995, pp. 349-414.


Information Transmission Rates

Reading ~ 30 bits/sec

Kinesthetic Morse Code 2.7 bits/sec

Tadoma 12-14 bits/sec

Optacon (vibrotactile) 5.4 bits/sec (40 wpm)

Tadoma Optacon


Example of a Functional Measure

Object identification via tactual stereognosis Klatzky, Lederman and Metzger 1985 96% correct identification of 100 common objects 94% within first 5 seconds; 68% within first 3

seconds


Conclusions: haptic performance High bandwidth

Temporal to 1 kHz Spatial discrimination to 1 mm

Extraordinary sensitivity to certain stimuli 300 Hz vibration (.1 m), raised edges (<1 m)

Huge dynamic range: forces from ~0.5N to ~500N JNDs generally consistent with other senses

Better for signals (e.g. force) than impedances (e.g. compliance) Haptics does not excel as a high bandwidth channel for

structured information (characters, words, text) But Tadoma illustrates the power of a highly parallel approach

Haptics does excel at 3D object recognition


Displaying haptic percepts

Ground-based devices Body-based devices Inertial reaction devices Tactile displays


Ground-based devices

Phantom Haptic Master Cobot Hand Controller List goes on…


Body-based devices

Cybergrasp Rutgers Hand Master Again, many others


Inertial Reaction Devices

Game controllers

motors

eccentric masses

shafts


Tactile Displays

Pin arrays (Optacon, Harvard displays, many others)

Lateral stretch Electrocutaneous displays Haptic field displays


Some current challenges in haptics

Low power, embedded haptics Mobile electronics

Exploiting tactual stereognosis (e.g., for automobile instrument panels) Exoskeletal devices haven’t been the answer

Haptics over the internet (e.g., for telesurgery) Latencies are a big issue

Haptic feedback for amputees


Haptics for prosthetics

“Sensory reinnervation” provides a possible means for restoring the sense of touch to amputees

Kinesthetic and tactile sensorsHaptic

display

haptic interface

Documents

edward colgate

haptic interfaceapril

haptic haptik

tactile interface

virtual environmentsalso

kinesthetic interface

interaction force

northwestern scenes