haptic teleoperation for robot-assisted surgery -...

Haptic Teleoperation for Robot-Assisted

Surgery

Chao LIU

CR1 Research Scientist National Center for Scientific Research (CNRS)

LIRMM, UMR 5506 Montpellier, France

[email protected]

INTRODUCTION TO TELEOPERATION

| Teleoperation: “ is the extension of a person’s sensing and manipulation

capability to a remote location” - Sheridan 1989

2/45


| Brief history y Early period

fire-tongs Prod, spear, etc

Polygraph: produces a copy of a piece of writing (Thomas Jefferson ~1805)

3/45


y Teleoperation systems were developed in the mid-1940s to create capabilities for handling highly radioactive material

4/45


y First modern master-slave teleoperation system (electrically servoed instead of mechanically) - Argonne National Laboratory, US under Ray Goertz in 1954

master slave

communication 5/45


y MASCOT – (MAnipulatore Servo COntrollato Transistorizzato) Carlo Mancini et al, 1958

IX Congress of Electronics Exhibition in Roma, 1962

6/45


| Teleoperation: “ is the extension of a person’s sensing and manipulation

capability to a remote location” - Sheridan 1989

Human operator (master) acts as a supervisor, communicating to a computer information relative to a limited task (goals, constrains, plans…) and getting back information (accomplishments, difficulties, sensory data…)

Teleoperator (slave) is a machine that enables a human operator to move about, sense and mechanically manipulate objects at a distance

7/45


| Current teleoperation applications: Industry; Space, deap sea, planetary exploration; Hazard material handling; Micro/Nano manipulation etc.

8/45


| Medical and Surgical Teleoperation

The da Vinci® Surgical System Till 2013, there are 2,585 da Vinci systems installed in 2,025

hospitals around the world

9/45


| Human operator

| Teleoperator

10/45


Da Vinci® Telesurgery System | Advantages

y Dedicated for Minimally Invasive Surgery (MIS)

¾ Remove surgeon’s hand tremor ¾ Improved 3D visual monitoring ¾ Increased range of motion

¾ Enhanced dexterity ¾ Greater operation precision ¾ Wide instrument choices

VS

11/45

Da Vinci® Telesurgery System | Disadvantages

y High cost (~ 2M USD) y Steep learning curve y No force feedback to the surgeon Æ lack the tactile or haptic sensation (the cability to “feel” the tissues being operated on)


the natural feeling of operating is lost (negative effects on operation accuracy and safety)

Unilateral Control 12/45


| Challenges for teleoperation in robot-assisted surgery? y Provide force feedback to the surgeon during operation

Bilateral Control Unilateral Control

Input command (desired output) is not influenced by

any physical variable present at the output

Output state affects the input and vice-versa

13/45


| Challenges for teleoperation in robot-assisted surgery? y Deal with Time delay

Endoscopic Capsule Robots

Modular Robots

Å wireless communication

14/45

HAPTIC TELEOPERATION

How to design a haptic/bilateral teleoperation system

for robot-assisted surgery

Æ Stability

Æ Transparency

To maintain stability of the closed-loop system irrespective of the behavior of the operator or the environment -> safety!

To provide a faithful transmission of signals (positions, velocities, forces) between master and slave to couple the operator as closely as possible to the remote task -> telepresence

15/45

OUTLINE




y Time delay


| Conclusion

HAPTIC TELEOPERATION FUNDAMENTAL

Although bilateral systems can be modeled with traditional block diagram or signal-flow-graph techniques, more insight is obtained with methods drawn from the network theory which share this bilateral property

16/45


| Basic Concepts: y Lumped Parameter Elements Physical entities who’s energy (storage elements) or power (dissipative

elements) is defined by a scalar. Examples: mass resistor

y Network A system described by lumped parameter elements connected in series and

parallel. Each element consists of one constitutive relation. Example: An electric circuit consisting of R, L, C in arbitrary connections.

y Port A location where energy can move into or out of a network. An effort and

flow, are defined for each port. Examples: Electric wall socket. Contact point between human and haptic

interface.

17/45


y Effort and Flow The flow of energy between two elements is characterized by two

variables called effort and flow , of which the product is power. y Impedance and Admittance Effort and flow variables are linked together by the behavior of

systems at their ports, with relationship represented as:

Impedance Admittance

18/45


Port: location at which energy can move into and out of the system y One-port network A system which has a single port is called a one-port system or simply

a one-port

Convention: 1) Choose the sign of flow so that it is positive going into the port.

2) The effort of interest at the port is that being exerted on the port, not the effort exerted by the port on the environment.

3) Choose the sign of effort so that when energy flows into the port Æ effort has the same sign as flow

effort and flow define positive or negative power going into the network

19/45


y Two-port network A system with two ports Two-port has separate effort and flow variables defined for each

port, and a separate coordinate system (sign convention) for each port.

Teleoperation network block diagram 20/45

OUTLINE




y Time delay


| Conclusion

1/45

HAPTIC TELEOPERATION DESIGN

Passivity-Based Teleoperation

- Passivity theory is an input–output property of dynamical systems that has its origins in network theory and is concerned primarily with the exchange of energy between interconnected systems

- System passivity constitutes a sufficient condition for the stability of the system coupled to a passive environment and a bounded input energy operator, and represent a necessary condition for stability with any remote environment

21/45


y Passivity definition - system input x - system output y - “power” P entering system as A system is said to be passive if: where E is lower bounded “energy storage” function, is non-

negative “power dissipation” function. the power is either stored or dissipated total energy supplied by the system is lower bounded

May not correspond to actual physical quantities

22/45


y Important practical feature of passivity formulation: The combination of passive systems connected in either a serial or

feedback/parallel configuration is still passive

network elements could be studied separately

23/45

| Stability Robustness and Performance Evaluation Tools

operator force on the master slave force on the environment master velocity slave velocity master impedance slave impedance external force generated by the operator external force generated by environment


24/45

| Stability Robustness and Performance Evaluation Tools y Depending on the choice of the network input and output

variables y and u (effort and flow), their relationship is described in terms of an immittance matrix P such that y = P×u:

Impedance Matrix Hybrid Matrix


Admittance Matrix

Alternate Hybrid Matrix

»¼

º«¬

ª»¼

º«¬

ª »

¼

º«¬

ª

e

h

e

h

VV

zzzz

FF

2221

1211»¼

º«¬

ª»¼

º«¬

ª »

¼

º«¬

ª

e

h

e

h

FF

yyyy

VV

2221

1211

»¼

º«¬

ª»¼

º«¬

ª »

¼

º«¬

ª� e

h

e

h

FV

hhhh

VF

2221

1211»¼

º«¬

ª�»

¼

º«¬

ª »

¼

º«¬

ª

e

h

e

h

VF

gggg

FV

2221

1211

eehhT VFVFuy �

25/45


| Stability Robustness and Performance Evaluation Tools y Hybrid matrix parameter interpretation

»¼

º«¬

ª»¼

º«¬

ª »

¼

º«¬

ª� e

h

e

h

FV

hhhh

VF

2221

1211

022

021

012

011

h

e

e

e

Ve

e

Fh

e

Ve

h

Fh

h

FVh

VVh

FFh

VFh Free motion input impedance

Force scaling

Velocity scaling

Output admittance with clamped input

26/45


| Stability Robustness and Performance Evaluation Tools y Llewellyn’s absolute stability condition An LTI two-port network is absolutely stable if and only if for any P

matrix: 1. p11 and p22 are positive real (no poles in the open right-half-plane, any

imaginary poles are simple and have real and positive residues) 2. Either of the following 2 sets of inequalities is satisfied:

or

Network stability parameter

27/45


| Stability Robustness and Performance Evaluation Tools y Transparency Transparency can be described quantitatively as a match between

the environment impedance and the impedance transmitted to the operator:

»¼

º«¬

ª�

0110

idealH

A perfect transparent system is marginally absolutely stable !

28/45


| Architectures of Bilateral Teleoperation

How to realize the bilateral teleoperation system?

Teleoperation network block diagram

29/45


| Architectures of Bilateral Teleoperation

General 4-channel Architecture

30/45


| Architectures of Bilateral Teleoperation y Two channel architectures

Position-Position architecture

C2=C3=0

31/45



Position-Force architecture

C3=C4=0

32/45



33/45


| Bilateral Teleoperation Design Guidance y In practice, perfect transparency and robust stability (passivity) are

conflicting design goals for bilateral teleoperation systems

y With specific control architecture and system properties, both stability and transparency are compromised and their analysis becomes involved

Æ use the stability robustness and performance evaluation tools to numerically evaluate

34/45

OUTLINE




y Time delay


| Conclusion


Time delay

What effects? Æ Destroy system passivity! Æ Potentially unstable

35/45


Time delay

| Analysis through Scattering theory y Scattering operator : For an n-port with effort F and flow v, the scattering operator S

is defined by:

using Hybrid matrix for 2 port system

y Theorem: A system is passive if and only if the norm of its scattering

operator S is less than or equal to one:

»¼

º«¬

ª»¼

º«¬

ª »

¼

º«¬

ª� e

h

e

h

FV

hhhh

VF

2221

1211

36/45


Time delay

| Analysis through Scattering theory y With existence of time delay:

Direct transmission of force and velocity

signal with time delay is not passive!

37/45


| Wave Variable Design Method y Instead of transmitting force and velocity signals, wave variables

(input waves u and output waves v) are transmitted

38/45


| Wave Variable Design Method

Recall power and eehhT VFVFuyP �

Passive for arbitrary time delay!

39/45


| Wave Variable Design Method y Transparency

¾ since wave variable is a excessively conservative approach to system passivity, the transparency of the standard wave-variable-based system is degraded.

¾ trade-off between stability and transparency is again inherent in the teleoperation system

¾ the fundamental tuning parameter in a wave-variable based controller is the wave impedance b

40/45


| Wave Variable Design Method y Wave reflection waves are reflected at points (junctions and terminations) where

the wave carrier impedance changes (master and slave sites), which corrupt the useful information flow and cause oscillatory behavior

41/45


| Wave Variable Design Method y Impedance matching for wave reflection the impedance of the wave transmission can be matched to the

remaining system by including additional termination elements.

42/45


| Main Problem transparency and passivity are conflicting design goals

Transparency Passivity

Trade-off

Safety Maximize transparency Scaling factor Force/Position tracking etc.

Robotic surgery requirements

43/45

OUTLINE




y Time delay


| Conclusion

NEW CHALLENGES FOR HAPTIC ROBOTIC SURGERY

| New Challenges/Open Questions for Haptic Robotic Surgery ¾ Better modeling of the modules (tissue model)

¾ Active environment (beating heart, etc)

¾ Sensor (biocompatibility, sterizability, size, cost)

¾ Asymmetrices of sensor/actuator

¾ Wireless telesurgery (communication & power transfer)

¾ Cognition in telesurgery (virtual fixtures)

¾ Semi-autonomous telesurgery

¾ New slave surgical robot (concentric tube robot)

¾ … …

44/45

OUTLINE




y Time delay


| Conclusion

CONCLUSIONS

| Conclusions y Brief history of teleoperation

y Why haptic teleoperation is necessary for robot-assisted surgery

y Fundamentals of haptic teleoperation

y Passivity and transparency of haptic teleoperation

y Stability Robustness and Performance Evaluation Tools

y Time-delayed haptic teleoperation

y New challenges & open issues for haptic robot-assisted surgery

45/45

HAPTIC TELEOPERATION FOR ROBOT-ASSISTED SURGERY

Thank You!

Chao LIU

LIRMM – CNRS [email protected]

www.lirmm.fr/~liu

haptic teleoperation for robot-assisted surgery -...

Documents