Initial position Estimated by MSBP using DQF

Stiffness estimation

𝑐𝑖 = 𝐿 𝒑𝛽𝑅

𝑖− 𝒑𝛾

𝑅𝑖, 𝐹𝛽 𝑖

− 𝐹𝛾 𝑖

Registration estimation

𝑻 = 𝑎𝑟𝑔𝑚𝑖𝑛 𝑻 𝒑𝑖𝐶 −𝒏𝑖𝐶 𝐹𝛽 𝑖𝑐𝑖− 𝑻 𝒑𝛽

𝑅

𝑖

𝑛

𝑖=1

Rangaprasad Arun Srivatsan1, Elif Ayvali1, Preetham Chalasani2, Long Wang3, Rashid Yasin3,

Nabil Simaan3, Howie Choset1 and Russell H. Taylor2

ERC-CISST

Minimally invasive surgery (MIS) has the potential to reduce patient

trauma, post-operative complications and recovery time at the cost of

increased technical difficulty and loss of sensory feedback. This loss can add

a significant cognitive challenge to the surgeon in finding correspondence

between the intraoperative scene and the preoperative imaging data.

In robotic surgery, mechanical palpation can provide important information

about organ geometry and tissue stiffness for diagnosis, surgical guidance,

and registration of the intraoperative information to the preoperative model of

the anatomy. This research aims to introduce methods that use inference to

guide palpation, and incorporate position and force measurements for

registration, stiffness mapping and geometric model reconstruction.

Motivation

1 Biorobotics Laboratory, Carnegie Mellon University, Pittsburgh, PA 15213 – IIS1426655 2 LCSR Lab, Dept. of Computer Science, Johns Hopkins University, Baltimore MD 21218 – IIS1327657

3 ARMA Lab, Dept. of Mech. Engineering, Vanderbilt University, Nashville, TN 37235 – IIS1327566

Autonomous Search of Tumors

Simultaneous Stiffness and Registration Estimation

Gaussian Processes (GPs) provide a

probabilistic description of the

stiffness map and surface geometry.

The framework can adapt to discrete

probing motion or continuous

palpation motion.

1. R A. Srivatsan, E. Ayvali, L. Wang, R. Roy, N. Simaan and H. Choset, "Complementary Model Update: A Method for Simultaneous Registration and Stiffness Mapping in Flexible Environments", In the proceedings of the International Conference on Robotics and Automation, Stockholm, Sweden, May 2016.

2. R. A. Srivatsan, L. Wang, E. Ayvali, N. Simaan, and H. Choset, “Simultaneous Registration and Stiffness mapping of a Flexible Environment using Stiffness and Geometric Prior”, in the proceedings of the Hamlyn symposium on Medical Robotics, UK, June 2016.

3. R. A. Srivatsan, G. T. Rosen, F. M. Naina, and H. Choset, “Estimating SE(3) elements using a dual-quaternion based linear Kalman filter”, in the Proceedings of Robotics Science and Systems, Michigan, USA, June 2016.

4. R. A. Srivatsan and H. Choset, “Multiple Start Branch and Prune Filtering Algorithm for Nonconvex Optimization”, accepted for publication in proceedings of the Workshop on the Algorithmic Fundamentals of Robotics, San Francisco, USA, December 2016.

5. E. Ayvali, R. A. Srivatsan, L. Wang, R. Roy, N. Simaan, and H. Choset, "Using Bayesian Optimization to Guide Probing of a Flexible Environment for Simultaneous Registration and Stiffness Mapping", In the proceedings of the International Conference on Robotics and Automation, Stockholm, Sweden, May 2016.

6. P.Chalasani, L. Wang, R. Roy, N. Simaan, R.H.Taylor and M.Kobilarov, “ Concurrent Nonparametric Estimation of Organ Geometry and Tissue Stiffness Using Continuous Adaptive Palpation”, In the proceedings of the International Conference on Robotics and Automation, Stockholm, Sweden, May 2016.

7. E. Ayvali, A. Ansari, L. Wang, N. Simaan and H. Choset, “Utility-based Planning for Autonomous Search”, submitted to Robotics and Automation Letters, 2016. [In Review]

Force-controlled Exploration and Sensing

The registration and autonomous tumor-search algorithms are platform

independent. There have been implemented on multiple robot platforms.

The robots were operated on a hybrid force-position control to always

probe normal to the surface of the flexible environment.

Complementary Model Update (CMU), an implementation for

simultaneously estimating the registration parameters and the stiffness

map, was developed [1].

CMU estimates the registration accurately with less number of iterations

and probed points.

Filtering Implementation for Online Registration

Registration is a non-linear optimization problem, requiring linearization

when using a Kalman filter for estimation.

Dual quaternion Filtering (DQF): Registration problem is reformulated in

the space of dual-quaternions and pairs of measurements are used

simultaneously to obtain a truly linear objective function [3].

Registration is also a non-convex optimization problem.

Multiple start branch and prune (MSBP) algorithm was been developed to

estimate the global optimum especially when we have a poor the initial

guess for the registration guess [4].

A variant of CMU has been developed that uses stiffness features in

addition to geometric features [2].

This variant is particularly effective when the organ is relatively flat and

has very few geometric features.

𝒅𝒑𝟏𝑹

𝒅𝒑𝟒𝑹

𝒅𝒑𝟐𝑹

𝒅𝒑𝟑𝑹

𝒅𝒑𝟓𝑹

Robot {R} Stiff features

Probing

Model {C} Optimization

Registration guess

dp1R,f1

dp2R,f2

dp3R,f3

dp4R,f4

dp5R,f5

… Stiffness map

Geometric prior

Stiffness prior

The autonomous search framework can incorporate prior information such

as regions where the targets could be, and increase the search space

only if there is a high probability of finding a target.

Two planners have been adapted for autonomous search of tumors:

Bayesian optimization and ergodic coverage [7].

(a) Experimental setup, (b) CAD model of the organ and stiff inclusions, (c) Actual stiffness map, (d) Prior information obtained from registration and red circles showing the actual locations of the inclusions, (e) Bayesian optimization, (f) Ergodic coverage.

(a) Cartesian robot (b) 6 DoF industrial arm from Foxconn,(c) daVinci research kit (DVRK).

Publications

Autonomous Planning for Robotic Palpation to Estimate Stiffness

Distribution, Organ Geometry and Registration.

(b) (c) (a)

x

z

y Stiff feature

Motion of the probe tip

Surface

Bayesian optimization provides efficient information-guided exploration of

an organ using mechanical palpation and integration of information, such as

stiffness, to an a priori geometric model through registration [5] [6].