jocelyne troccaz timc lab. [email protected] http

Robot registration

Jocelyne TROCCAZTIMC [email protected]://www-timc.imag.fr/gmcaohttp://www-timc.imag.fr/Jocelyne.Troccaz

Summer School on Surgical Robotics Montpellier, Sept. 07

Robot Registration – J. Troccaz

Grenoble GMCAO (CAMI) teamCreated in 1985 by Philippe CinquinHeaded from 1996 by Jocelyne TroccazStrong connection to Grenoble HospitalAbout 35 people

Grenoble HospitalWe are thereGrenoble



Research topics

Imaging

Modelling

Planning and simulation

Data fusionRobotics

Navigation

Sensors



What is it?Registration consists in determining geometric relationships between two reference framesRobot registration essentially consists in transferring the planning to the robot coordinate system

Pre-operative data

Intra-operative data

Robot

Image registration

?

?



Example



Tools

CalibrationTrackingData registration

Using:Patients’ dataExternal objects

RequiresIntrinsic robot calibration



Contents

IntroductionMethodsExamples: four main situationsConclusion



Possible reference frames of interest

Rpatient

planningRimg_sensor(i)

Intra-operativelyPre-operatively

Rpatient

Rrobot

Rlocalizer



Hardware examplesLocalizers:

Optical, US, magnetic, mechanical arm

Imaging sensors:Fluoroscopy, digital X-Ray, ultrasound



Examples A: Robodoc, ACRobot, (CAD-Implant)

Rpatient

planning


Rpatient

Rrobot



Examples B: CASPAR

Rpatient

planning


Rpatient

Rrobot

Rlocalizer



Examples C:Speedy, Cyberknife, MARS

Rpatient

planning Rimg_sensor1


Rpatient

Rrobot

Rimg-sensor2



Example D: Cyberknife+Synchrony

Rpatient

planningRimg_sensor1


RpatientRrobot

Rlocalizer

Rimg_sensor2



Registration basics

Two reference frames RA and RB and a transform TA

B to be determinedSelection of features FA in RA and FB in RB

Definition of a similarity measure (or distance) between FA and FB

Determination of TAB such that the

similarity is maximum (or distance minimum)

TAB= arg min d(FA,TA

B(FB))



Typical 3D/3D rigid registration methods

Point to point (Procrustes)

External fiducialsAnatomical landmarks

Surface registrationAnatomical surface (i.e. ICP, chamfer matching)Template [Radermacher]

Intensity-basedregistration (for images only)



Examples APre-op: planning on CT dataIntra-op: a robotDeveloped methods:

Robodoc: robot palpation of implantedfiducialsACRobot: robot palpation of anatomicalsurfaceCAD-Implant: fiducials+template (robot is pre-operative)



Robodoc (for hip surgery)

Define precisely the prosthesis position (geometrical or biomechanicalcriteria)Improve the preparation of the hip cavity



Robodoc [Taylor et al.]

1. Planning: Orthodoc

3. Intra-operative bone milling procedureusing Robodoc (based on the IBM scara robot)

2. Pre-op to intra-op registrationusing implanted titanium pins (anatomical registration in the last version)

6D force sensor



Example A.1: Robodoc


Rpatient

Rrobot

Rpatient

planning

3D/3D point-to-pointregistration

Palpation



ACRobot [Davies et al.]

« Hands on » robotKnee arthroplasty3 DOFsBone surface palpated with the robotIEEE TRA 03: registration accuracy evaluation



ACRobot registration tests

Initial estimate from 4 anatomical landmarksICP surface matchingIntra-op criterion: rms distanceDoes a small rms mean a good registration?Experiments with phantom and artificialdata:

Generated palpated points (10 to 100) with or withoutrandom noise added (max up to 1.5mm)Initial estimates in the range of +/-10mm and 2°Known translational and rotational errors



ACRobot: results

Nb pts <70 makesICP more problematicResults highlydepend on the data setsMay have a rms=0.6 witherrors of 0.8mm and 2° From [IEEE-TRA03]



Example A.2: ACRobot


Rpatient

Rrobot

Rpatient

planning

ICP 3D/3D surfaceregistration

Palpation



CAD-Implant [Champleboux et al.]

A system for dental implant assistanceA template associated to fiducials visible on CTA pre-operative robotIntra-operatively: no robot, no computersSurface registration withoutcomputers



Example A.3: CAD-Implant

Rpatient

planning


Rpatient

Rrobot

Line-to-line registration

calibration

Template matching



Example B

Close to RobodocKnee application Pre-op: planning on CT dataIntra-op: a robot, a tracking sensorDeveloped method:

implanted fiducials S for registrationfiducials S’ for motion detection



Example B: CASPAR

Rpatient

planning


Rpatient Rrobot

3D/3D point-to-pointregistration

Palpation

Rlocalizer

Motion detection



Examples C

Pre-op: planning on CT dataIntra-op: a robot, X-Ray sensorsDeveloped methods:

Speedy V1 [Lavallée89]: Direct X-ray/robot calibration and manual image registrationCyberknife V1 [Schweikard98]: Indirect X-ray/robot calibration and intensity-based registration



Speedy V1

Stereotactic neurosurgeryPre-operative MR or CTIntra-operative X-Ray (AP and lateral) – several examsDirect X-Ray/robot calibrationManual image registration (anatomical for pre-op/intra-op and markers for intra-op/intra-op)



SpeedyX-Ray extrinsiccalibration: bi-plane model

2D/2D image registration fiducials

source

(P1) (P2)

MM2M1

f1: M (u,v) M1(x1,y1)f2: M (u,v) M2(x2,y2)

image



Example C.1: Speedy V1

Rpatient



Rpatient

Rrobot

Rimg-sensor2

X-ray/robotcalibration

X-Ray calibration

imageregistration

intra-operativeimage registration



Cyberknife V1 [Schweikard et al.]

RadiotherapyapplicationComplextrajectories for improved tumordestruction (multiple radiation ports)6 DOFs requiredVery heavy tools

Traditional linearaccelerator set-up



Cyberknife V1

Planning on CT dataIntra-operatively:

Indirect X-Ray/robot calibration (via isocenter)« X-Ray/pre-computedDRRs » intensity-basedregistration (before eachbeam activation) Small motion compensation when necessary / interruption of the procedureand replanning for large motion

A Digitally ReconstructedRadiograph (DRR)



Example C.2: Cyberknife V1

Rpatient



Rpatient

Rrobot

Rimg-sensor2

Robot/Isocalibration

X-Ray calibration

iterated imageregistration

Risocenter

X-Ray/Iso calibration

Set-up



Example D: Cyberknife+synchronyPre-op: planning on CT dataIntra-op: a robot, two X-ray sensors, a localizerDeveloped methods:

X-Ray/robot calibrationX-Ray/DRR registration for head motion compensationOr fiducial-basedregistration plus real-time tracking for targetsmoving with respiration



Real-time registration

Large motion tracking[Schweikard05]

Internal fiducials (gold seeds) for initial registration External fiducials (IR diodes) for respiration trackingLearning internal/externalfiducials relationship



Example D: Cyberknife+Synchrony

Rpatient



Rpatient=Riso

Rrobot

Rlocalizer

Rimg_sensor2

Pre-learning

tracking

imagecalibration

X-Ray/robot calib.

initialregistration

indirect



Another type of solution (E)Target defined in the intra-operative imaging data*Examples

Indirect visual servoing: computing the robot position from the images

PAKY+RCM [Stoianovici et al.], LPR [Cinquin et al.], etc.

Direct visual servoing: modeling variations of the robot position to variations of the target in the images

GABIE [Morel], ZEUS [deMathelin], etc.

*if pre-operative planning: need for pre-op/intra-op registration



Example E.1

planning

Rimg_sensor=Rpatient

Intra-operatively

Rrobot

Image analysisand robot model

PAKY+RCM

LPR



Example E.2

planning

Rimg_sensor=Rpatient

Intra-operatively

Rrobot

Image analysis andImage/robot differential model

toolheart

US plane(ω1,ω2,ω3,d)=f(d(T1,P1), d(T2,P2))

T1

T2

P1

P2

GABIE



DiscussionModus operandi:many solutions

Palpation (fiducial, anatomy): easy, invasiveImaging (anatomy): more difficult, less or non invasiveTemplate: easy, limited to few applicationsNeed for updated or real-time registration?

No motionMotion detectionDiscrete motion detection and compensationContinuous motion detection and tracking



Discussion (cont’d)No universal recipies: depends on the applicationSome important issues

Intra-operative evaluation of registration accuracySafety of real-time registration

jocelyne troccaz timc lab. [email protected] http

Documents