computer-integrated surgery

Copyright © CISST ERC, 2002Engineering Research Center for Computer Integrated Surgical Systems and Technology

Computer-Integrated Surgery

Russell H. TaylorDept. of Computer Science/Radiology/Mechanical EngineeringDirector, Center for Computer-Integrated Surgical Systems and TechnologyThe Johns Hopkins University3400 N. Charles Street; Baltimore, Md. [email protected]

NSF Engineering Research Center for Computer-Integrated Surgical

Systems and Technology


Thanks to many people …

• J. Anderson• W. Bargar• A. Bzostek• B. Eldridge• G. Fichtinger• J. Funda• F. Frassica• R. Goldberg• S. Gomory• G. Hager• L. Joskowicz• Y. Kim• R. Kumar• D. Shen• C. Riviere• D. Yousem• A. Kalvin• C. Cutting• A. Morris

• J. Lazarus• D. Long• K. Masamune• K. Murphy• D. Yousem• B. Jaramaz• A. DiGioia• A. Lahmer• M. Borner• A. Bauer• A. Barnes• P. Jensen• J. Yao• B. Mittelstadt• M. Li• D. Sheng• D. Rothbaum• A. Gueziec• M. Talamini

• K. Chinzei• N. Hata• R. Kikinis• D. Stoianovici• A. Patriciu• L. Whitcomb• A. Okamura• C. Davatzikos• E. Atalar• E. Zerhouni• L. Kavoussi• P. Kazanzides• H. Paul• J. Wenz• E. DeJuan• J. Roy

… and many more


Prediction

The impact of computer-integrated surgical systems and technology on medical care in the next 20 years will be as great as the impact of computer-integrated manufacturing systems and technology on industrial production over the past 20 years.

Engineering Research Center for Computer Integrated Surgical Systems and Technology


Imagers and other sensors

Robotic devices

Human-machine interfaces

Interface technology

Computers and networks

Image & sensor processing, anatomical modeling, surgical

planning and control, …

Patient data bases

Anatomic atlases and

surgical task models

patient surgeon

Images & other sensor data

Command & control

Decision support &

commands


Computer-assisted planning

Patient-specific Model

Update Model

Computer-Assisted

Execution

Update Plan

Computer-Assisted

Assessment

Preoperative Intraoperative

Atlas

Postoperative

Patient




Update Model

Computer-Assisted

Execution

Update Plan

Computer-Assisted

Assessment


Atlas

PostoperativePatient

Surgical “CAD”

Surgical “TQM”

Surgical “CAM”


Process Vision: “One stop shopping” therapy

• Fully integrated and optimized planning, execution, assessment, and follow-up of minimally invasiveinterventional procedures

• … for a broad spectrum of clinical conditions

• … anywhere in the body• … with convenience

comparable to current outpatient diagnostic procedures.

Copyright © CISST ERC, 2000, 2001 NSF Engineering Research Center for Computer Integrated Surgical Systems and Technology

•Follow up•Statistics•Process learning

Pre-op Screening

Image patient and plan treatment pattern

In-scanner robotically assisted treatment

Post-op imaging

Reimagevolume

MaintainMaintainTreatmentTreatment

LogLog

One endpoint: 1-stop shopping

Copyright © CISST ERC, 2000, 2001 NSF Engineering Research Center for Computer Integrated Surgical Systems and Technology


Pre-op Screening



Post-op imaging

Reimagevolume


LogLog



System Vision: “Plug and Play” Surgery

• A modular family of subsystems, including – imaging devices and methods– modeling & analysis algorithms – robotic devices– visualization & human-machine interfaces– systems development tools …

• which can be quickly integrated with novel therapeutic end effectors to produce …

• complete and effective surgical CAD/CAM systems with predictable and certifiable performance …

• for multiple organ systems, therapeutic approaches, and imaging modalities.

Robot Unit (RU)

EngineeringCockpit

(EC)

Robot Server(RS)

CommunicationServer(CS)

SlicerBrachy Module

(BM)

MRT

Robot Module(RM)

Planning Module(PM)

Robot

DICOM

MRT Console

Tracking Server(TS)

MRC

1

2

3

4

5

6

7

8

91 0

1 1

1 2

1 3




Update Model

Computer-Assisted

Execution

Update Plan

Computer-Assisted

Assessment


Atlas

Postoperative

Patient

Surgical

Assistants


Vision: “Information-intensive” surgery

• Surgical systems interfacing to an information-rich surgical environment

• … and capable of using this information to maintain patient-specific models

• … registered in real time to the physical reality of the operating room

• … with the ability to use these models to assist surgeons in carrying out complex surgical procedures

• … through robotic assists and information display devices.


Vision: “super-human” surgical teams

• Surgical systems capable of modeling and following the progress of a surgical procedure,

• … and able to use robotic devices and sensors extending human performance

• … cooperatively with the surgeon• … in order to improve the quality of

surgical procedures and to enable interventions that would be otherwise impossible.


• Multi-institution, multi-disciplinary center– Johns Hopkins University + Medical Institutions– MIT + Brigham & Women’s Hospital– CMU + West Penn. Allegheny Health System– Others: Morgan State, Georgetown, Harvard

• University researchers, clinicians, industry• Research, Systems, Education, Outreach

NSF Engineering Research Center for Computer-Integrated Surgical

Systems and Technology


Technical barriers: Knowledge and Systems

Researchand Technology

Systems Infra-structure

Credit: K. Chinzei & N. Hata


Non-technical barriersNon-Technical Barriers:The Three Cultures

The three cultures

Academic Researchers

Clinicians

Industry


Education

Non-Technical Barriers:Education

Undergrad & postgradContinuing professional

K-12


Researc

h

and

Tech

nolog

y

System

s

Infras

tructu

re

The three

cultu

res

Education

The CISST ERC addresses these barriers

• Multidisciplinary systems –oriented research leveraging our environment

• Engineering faculty + clinicians + industry

• Education integrated into all aspects


Researc

han

d

Tech

nolog

y

System

s

Infrastr

ucture

The three

cultu

res

Education

The CISST ERC addresses these barriers

• Multidisciplinary systems –oriented research leveraging our environment

• Engineering faculty + clinicians + industry

• Education integrated into all aspects


Engineered systems & applications

Enabling core technology

Fundamental knowledge

Research-based Education

“Regular” courses

Integrate education into all aspects of research and industry programs

Project courses

Independent study & internships


Education Highlight:High School Research Project

• Intelligent hand tool for computer-assisted knee cartilage surgery

• High school student– Al Brzechko (Baltimore Polytechnic HS)

• Mentors– R. Goldberg (MS student)– P. Evans, MD (surgeon)– R. Taylor, Ph.D (faculty)

• Status– Prototype device built– Paper accepted at MICCAI 2001– Surgeon pursuing commercial partner

for next phase


Surgical Robot Lego Competition for High School Students


• Surgical CAD/CAM Systems

• Surgical Assistant Systems

• Some concluding remarks



Manual Surgery Robotic Surgery

Robotic Joint Replacement Surgery


Statistical Atlas

“Virtual CT”

Deformable 2D-3D

Planning Updated Plans

Registered Model

Visualization & Guidance

X-raysX-rays

Rigid 2D-3D

R. Taylor & J. Yao


Prior Information#

Pro

ject

ion

s

Registration

Reconstruction

?

3D models from incomplete data

100 degrees100 degrees


2D/3D Deformable Registration

Jianhua Yao


Optimized Local Therapy• Deliver optimized patterns of local

treatments (e.g., radiation seeds, injections) with an image-guided robot

• Applications

– Prostate, brain, spine, liver, bone, kidney ….

• Advantages

– Accuracy of dose– Cost and time of procedure– Lower morbidity– Avoid complications– Enable new treatments

Photo: D. Stoianovici, L. Kavoussi


Current ERC percutaneous focus applications

Spine$120 billion cost70% in lifetime

Bone Cancer600,000 metastatic

cases/year

LiverMets from colorectal130,000 new /year60,000 death /year

Prostate~210,000 new/year25% men in lifetime

~30-40% prostatectomy~20% radiation seeds

~30-40% extracapsular



Pre-op Screening



Post-op imaging

Reimagevolume


LogLog




Clinical example: X-ray guidance• Percutaneous access to kidney• Radiolucent needle driver• Robot aligns needle under x-ray

fluoroscopy guidance• Has been done both locally and

remotely

Photos: D. Stoianovici, L. Kavoussi

D. Stoianivici


X-ray guided breast biopsy


Clinical example• Kidney biopsy• Robot registered to CT

from single image using markers on end-effector

Photos: D. Stoianovici, L. Kavoussi, A. Patriciu , S. Solomon (JHU Bayview)

Other contributors: R. Susil, G. Fichtinger, K. Masamune, R. Taylor (JHU WSE)


Early ultrasound guided example: Robotic TURP

Photo: courtesy Brian Davies


• Combined Phase I/II SBIR with Burdette Medical Systems

• Goal is “one stop shopping” system for ultrasound-guided prostate brachytherapy

Ultrasound-guided prostate brachytherapy


Integration Example: Open-MR Brachytherapy System (BWH/MEL)

• MR-compatible mechanical manipulator

• Pneumatic and ultrasound motors

• 5 DOF holder


Credit: K. Chinzei & N. Hata


• Labeled images* of 20 subjects for cancer and normal tissue

• Deformably registered images to average prostate (red)

• Computed spatial distribution of cancer (green)

• Computed 5 needle positions to maximize probability of detection

• Use statistical correlation of cancer spatial distributions to optimize pattern

Statistical atlas of prostate cancerChristos Davatzikos & Dinggang Shen

* Images provided by Jianchao Zeng, Georgetown U.


In-MRI prostate biopsy

Why use MRI?• Good tissue contrast.

• Excellent sensitivity for detecting tissue abnormalities.

• May allow for ‘targeted’ biopsy and improved diagnostic sensitivity

R. Susil, G. Fichtinger, E. Atalar


In-MRI imaging

Prostate

Needle

RectalProbe



Interventional Device

Mimics design and functionality of US transrectal biopsy device, but allows for control while patient is in MRI scanner



Example: integrated MRI imaging and therapy delivery

Coaxial transmission line

LCR ScannerSurface coilRF electrodes

Combined endoscopic MRI coil & RF ablator

78 µm resolution !

Canine urethra

Beagle prostate

Photo: E. Atalar, JHU

E. Atalar, JHU


Cutting, Bookstein, Taylor, et al.


CT Overlay

Masamune, Fichtinger, Taylor, et al.




• Some concluding remarks



• Robot holds endoscope• Surgeon controls view

through various interfaces– Joystick or foot pedal– Voice recognition– Head or eye tracking

• Main current justifications– Labor cost saving– Somewhat steadier view

Example: Robotic endoscope Holders

Photo: Computer Motion, Inc


R. Taylor, K. Gruben, J. Funda


Example: Robotic retraction assistant


Our current focus: mostly Eye & ENT

Eye:– Retinal vein cannulation– Retinal translocation– …

ENT:– Stapes prosthesis– Cochlear implant– …

MicrovascularNeurosurgery….

Retina with occluded retinal vein

2 mm


Handle force

Kv

Steady Hand Guiding for Microsurgery

R. TaylorFree hand motion Steady hand motion


Successful cannulation of anapprox. 100 micron blood vessel

(Kumar, 2001)


Ultrasound Manipulation Robots

Salcludean ultrasound system Hippocrate Robot

E.Degoulange, et al


Telerobotic Surgical Augmentation

• Surgeon manipulates “master” robot

• “Slave” robot mimics motions

• Typically systems have:– Force & position

scaling– Remote visualization

via video• Can be expensive but

provide improved capability for minimally invasive procedures

Photo: Intuitive Surgical


Evolution: “Smart” telesurgery

• Situation assessment• Task strategy & decisions• Sensory-motor coordination

Augmentation System

• Sensor processing• Model interpretation• Display

atlases

• Manipulation enhancement

• Online references & decision support

• Cooperative control and “macros”

atlases

libraries

R. Taylor


Research & Technology Barriers

Augmentation System

atlases

libraries

Performance assessment• Metrics & measurements • Human baseline data• System evaluation

Robotics & cooperative control

Human-machine interfaces• Displays & visualization • Haptic input/output• Voice

Sensor processing, modeling, and anatomical atlases

Task & task element characterization

System infrastructure, integration, and end user experience

R. Taylor





• Concluding remarks


Research and technology barriers

Modeling & analysis• Segmentation• Registration• Atlases• Optimization• Visualization• Task characterization• etc.

Interface Technology• Sensing• Robotics• Human-machine

interfacesSystems• Safety & verifiability• Usability & maintainability• Performance and validation


How can we get there?

Strong and committed teams

– Surgeons– Engineers– Industry

Focus on systems that address important needs

Rapid iteration with measurable goals

Have fun!


The real bottom line: patient care

• Provide new capabilities that transcend human limitations in surgery

• Increase consistency and quality of surgical treatments

• Promote better outcomesand more cost-effectiveprocesses in surgical practice

computer-integrated surgery

Documents