vision-based motion control of robots azad shademan guest lecturer cmput 412 – experimental...

Vision-Based Motion Control of Vision-Based Motion Control of RobotsRobots

Azad ShademanGuest Lecturer

CMPUT 412 – Experimental RoboticsComputing Science, University of Alberta

Edmonton, Alberta, CANADA

A. Shademan. CMPUT 412, Vision-based motion control of robots March 4, 2008

Vision-Based Control

current

desired

Left Image Right Image

A

A

A

B

BB


Vision-Based Control

Left Image Right Image

B

BB


Vision-Based Control Feedback from visual sensor (camera) to

control a robot Also called “Visual Servoing” Is it any difficult?

Images are 2D, the robot workspace is 3D 2D data 3D geometry


Where is the camera located? Eye-to-Hand

e.g.,hand/eye coordination

Eye-in-Hand


Visual Servo Control law Position-Based:

Robust and real-time pose estimation + robot’s world-space (Cartesian) controller

Image-Based:Desired image features seen from cameraControl law entirely based on image features


Position-Based

Desired pose

Estimated pose


Image-Based

Desired Image feature

Extracted image feature


Visual-motor Equationx1

x2

x3 x4

q=[q1 … q6]

Visual-Motor Equation

This Jacobian is important for motion control.


Visual-motor JacobianImage spacevelocity

Joint spacevelocity

A

A

B

B


Image-Based Control Law

1. Measure the error in image space

2. Calculate/Estimate the inverse Jacobian

3. Update new joint values


Image-Based Control Law

Desired Image feature

Extracted image feature


Jacobian calculation Analytic form available if model is known.

Known model Calibrated

Must be estimated if model is not known

Unknown model Uncalibrated


Image Jacobian (calibrated) Analytic form depends on depth estimates.

Camera/Robot transform required. No flexibility.

CameraVelocity


Image Jacobian (uncalibrated) A popular local estimator:

Recursive secant method (Broyden update):


Relaxed model assumptions

Traditionally: Local methods No global planning Difficult to show

asymptotic stability condition is ensured

The problem of traditional methods is the locality.

Calibrated vs. Uncalibrated

Model derived analytically Global asymptotic

stability Optimal planning is

possible A lot of prior knowledge

on the model

Global Model Estimation (Research result)

Optimal trajectory planning

Global stability guarantee


Synopsis of Global Visual Servoing Model Estimation (Uncalibrated) Visual-Motor Kinematics Model Global Model

Extending Linear Estimation (Visual-Motor Jacobian) to Nonlinear Estimation

Our contributions:K-NN Regression-Based EstimationLocally Least Squares Estimation


Local vs. Global

Key idea: using only the previous estimation to estimate the Jacobian

1. RLS with forgetting factor Hosoda and Asada ’94

2. 1st Rank Broyden update: Jägersand et al. ’97

3. Exploratory motion: Sutanto et al. ‘98

4. Quasi-Newton Jacobian estimation of moving object: Piepmeier et al. ‘04

Key idea: using all of the interaction history to estimate the Jacobian

Globally-Stable controller design

Optimal path planning Local methods don’t!


3 NN

K-NN Regression-based Method

q1

q2

x1

q1

q2

?


q1

q2

x1 ?

KNN(q)

(X,q)

Locally Least Squares Method


Experimental Setup

Puma 560 Eye-to-hand configuration Stereo vision Features: projection of the end-effector’s

position on image planes (4-dim) 3 DOF for control


Measuring the Estimation Error


Global Estimation Error


Noise on Estimation Quality

KNN

LLS

With increasing noise level, the error decreases


Effect of Number of Neighbors


Conclusions Presented two global methods to learn the visual-

motor function LLS (global) works better than the KNN (global) and

local updates. KNN suffers from the bias in local estimations Noise helps system identification


Eye-in-Hand Simulator


Mean-Squared-Error


Task Errors


Questions?


Position-Based Robust and real-time relative pose

estimation Extended Kalman Filter to solve the

nonlinear relative pose equations. Cons:

EKF is not the optimal estimator.Performance and the convergence of pose

estimates are highly sensitive to EKF parameters.


Overview of PBVS

2D-3D nonlinear point correspondences

IEKFWhat kind of nonlinearity?

T. Lefebvre et al. “Kalman Filters for Nonlinear Systems: A Comparison of Performance,” Intl. J. of Control, vol.

77, no. 7, pp. 639-653, May 2004.


EKF Pose Estimation

Measurement noise

State variable

Process noise

yaw pitch roll

Measurement equation is nonlinear and must be linearized.


Visual-Servoing Based on the Estimated Global Model


Control Based on Local Models


In practice: Broyden 1st-rank estimation, RLS with forgetting factor, etc.

Estimation for Local Methods

vision-based motion control of robots azad shademan guest lecturer cmput 412 – experimental...

Documents

imagebased control lawmarch

imagebased control lawmeasure

global estimation errormarch

image spacecalculateestimate

image planes

image featuresmarch

visionbased controlfeedback

estimation qualityknnllsmarch