Bryan Willimon, Steven Hickson, Ian Walker, and Stan Birchfield

IROS 2012Vila Moura, Algarve

An Energy Minimization Approach to 3D Non-Rigid Deformable Surface Estimation Using

RGBD Data

We propose an algorithm that uses energy minimization to estimate the current configuration of a non-rigid object.

Our approach relies on a 3D nonlinear energy minimization framework to solve for the configuration using a semi-implicit scheme.

Overview

Previous Related Work on Pose Estimation

• Elbrechter et al. (IROS 2011) compare a purely mathematical representation of the paper manifold with a soft-body-physics model and demonstrate the use of their visual tracking method.

• Bersch et al. (IROS 2011) describe a method to compute valid grasp poses on the cloth which accounts for deformability.

The purpose of this approach is to minimize the energy equation of a mesh model that involves 4 terms:

Smoothness term

data Correspondence term

Depth term

Boundary term

Energy Minimization Approach

The purpose of this approach is to minimize the energy equation of a mesh model that involves 4 terms:

Energy Minimization Approach

Mesh Initialization

Energy Minimization Approach

Smoothness term

Energy Minimization Approach

data Correspondence term

Energy Minimization Approach

Depth term

Energy Minimization Approach

Front View Top View

Boundary term

Without BoundaryWith Boundary

Energy Minimization Approach

Experimental Results

• We captured RGBD video sequences of shirts and posters to test our proposed method’s ability to handle different non-rigid objects in a variety of scenarios.

Four experiments were conducted:

1) Illustrating the contribution of the depth term

2) Illustrating the contribution of the boundary term

3) Partial self-occlusion

4) Textureless shirt sequence

Experimental Results

• Illustrating the contribution of the depth term

Experimental Results

• Illustrating the contribution of the boundary term

Experimental Results

• Partial self-occlusion

Experimental Results

• Textureless shirt sequence

ConclusionWe have presented an algorithm to estimate the 3D configuration of a non-rigid object through a video sequence using feature point correspondence, depth, and boundary information.

The next step is to integrate this algorithm into a robotic system that can grasp and handle non-rigid objects in an unstructured environment.

Questions?