automatic extrinsic calibration for lidar-stereo vehicle ... · • tested with hdl-64e &...

Automatic Extrinsic Calibration for Lidar-Stereo

Vehicle Sensor SetupsCarlos Guindel, Jorge Beltrán,

David Martín and Fernando García

Intelligent Systems Laboratory · Universidad Carlos III de Madrid

IEEE 20th International Conference on Intelligent Transportation Systems

Yokohama · 17 October 2017

Agenda

Motivation

Calibration algorithm

Synthetic test suite

Results

Conclusion

2

Automatic Extrinsic Calibration for Lidar-Stereo Vehicle Sensor SetupsC. Guindel, J. Beltrán et al. · ITSC 2017

Agenda

Motivation



Results

Conclusion

3


Perception systems in vehicles

• Topologies with complementary sensory modalities

4


Cameras

Stereo-visionsystems

Rangescanners

Multi-layer3D lidarscanner

• High accuracy

• 360º Field of View

• Appearance information

• Cost-effective

• Dense 3D info.

OvelappingFOVs

Correspondencebetween data

representations

Extrinsiccalibration

requiredData fusion

IVVI 2.0 Research Platform

Previous works

• Camera-to-range calibration in robotic/automotive platforms

• Complex setups / lack of generalization ability

• Strong assumptions are usually made: sensor resolution, limited pose range, environment structure,…

• Assessment of calibration methods

• Ground-truth of extrinsic parameters cannot be obtained in practice

5


Geiger et al., ICRA 2012 Velas et al., WSCG 2014

Levinson & Thrun, RSS 2013

Proposal overview6


• Stereo-vision system–multi-layer lidar calibration

• Suitable for use with different models of lidar scanners (e.g. 16-layer)

• Very different relative poses are allowed

• Performed within a reasonable time using a simple setup

Agenda

Motivation



Results

Conclusion

7



• Calibration target

• Process overview

8


• Single point of view

• Holes visible from the camera and intersected by at least 2 lidar beams

• No alignment required

Registration

Data

CAMERA

Target segmentation

CAMERA

Circlessegmentation

CAMERA

Data

LIDAR

Target segmentation

LIDAR

Circlessegmentation

LIDAR

𝝃CL =𝑡𝑥𝑡𝑦𝑡𝑧𝜙𝜃𝜓

Data representation9


Registration

Data

CAMERA

Target segmentation

CAMERA

Circlessegmentation

CAMERA

Data

LIDAR

Target segmentation

LIDAR

Circlessegmentation

LIDAR

Data representation10


Point cloud: 𝒫0𝑐Stereo matching

Left image

Right image

Point cloud: 𝒫0𝑙

• 3D point clouds, 𝒫0 = { 𝑥, 𝑦, 𝑧 }

Stereo matching

• Accuracy in the depth estimation is required (SGM)

• Border localization problem will be tackled using intensity

Data

CAMERA

Data

LIDARLIDAR: 𝒫0

𝑙

Target segmentation · Step 111


Target segmentation

CAMERA CAMERA

Target segmentation

LIDAR LIDAR




Point clouds: 𝒫0Plane model

extraction

Plane model extraction

• Random sample consensus (RANSAC)

• Tight threshold (1 cm) and requirement for the plane to be roughly parallel to the vertical axis (tol: 0.55 rad)

Remove pts. farfrom the planes

Target segmentation

CAMERA/LIDAR

LIDAR: 𝒫1𝑙CAMERA: 𝒫1

𝑐

• Extracting the points belonging to discontinuities in the target

• Successive segmentations: 𝒫𝑖0 = { 𝑥, 𝑦, 𝑧 } ⊆ 𝒫𝑖0−1

Step 1



Target segmentation

CAMERA

Left image Sobel filtering

Point cloud: 𝒫1𝑐

Keepdiscontinuities

CAMERA: 𝒫2𝑐

for every point in 𝒫1𝑙

CAMERA: Sobel edges

Target segmentation

LIDAR

Filter out

LIDAR: 𝒫2𝑙

Step 2

Step 2

(50 cm)

Circles segmentation · Step 114


Circlessegmentation

CAMERA

Circlessegmentation

LIDAR



• Getting rid of the points not belonging to the circles: target boundaries

15


Circlessegmentation

CAMERA

Step 1

Point cloud: 𝒫2𝑐 3D-line RANSAC Point cloud: 𝒫3

𝑐

Geometricalconstraints

LIDAR: 𝒫3𝑙CAMERA: 𝒫3

𝑐

• Keep only the rings where a circle is possible

• Remove the outer points

Circlessegmentation

LIDAR

Step 1



• Detecting the center of the holes

Circles segmentation

CAMERA/LIDAR

Step 2

Point cloud: 𝒫3Alignment with

XY plane2D CircleRANSAC

4 x centers + radius

Undo the alignment

Geometrical constraints

4 x centers coordinates

• 2D search: only three points are required

Circle model extraction

Camera Lidar


• Robustness against noise

17



CAMERA/LIDAR

Step 3

𝑡0

𝑡1

…

𝑡𝑁4 x center

coordinates

4 x center coordinates

4 x center coordinates

Euclidean clustering

4 x cluster centroids

CAMERA LIDAR

tol: 2 cm

Registration18


Registration


Registration19


4 x reference points, 𝒑𝑐

𝑖

Registration


CAMERA


LIDAR4 x reference

points, 𝒑𝑙𝑖

• Pure translation

• Overdetermined system of 12 equations

𝒕𝐶𝐿 = ഥ𝒑𝑙𝑖 − ഥ𝒑𝑐

𝑖

• Column-pivoting QR decomposition

Step 1

• Iterative Closest Points (ICP)

Step 2

Translation

Translation + Rotation

Composition


Agenda

Motivation



Results

Conclusion

20


Synthetic Test Suite

• Our proposal for quantitative assessment of calibration algorithms

• Exact ground-truth, but also noise and real constraints

• Simulation of sensors and their environment based on Gazebo

• Different calibration scenarios

21


Calibration target

Stereo-vision system model

Velodyne models

Gazebo models, plugins and worlds available athttp://wiki.ros.org/velo2cam_gazeboOpen source · GPLv2 License

Agenda

Motivation



Results

Conclusion

22


Experimental setup

• Using the synthetic test suite

• Nine different calibration setups

• 7 simple setups to evaluate the parameters of the transform

• 2 challenging situations

• Gaussian noise added to the sensor measurements

• Models simulated with real parameters

• 12 cm stereo baseline and 16-layer lidar

23


𝑒𝑡 = ‖𝒕 − 𝒕𝒈‖

Translation error (linear)

𝑒𝑟 = ∠(𝑹−𝟏𝑹𝒈)

Rotation error (angular)

𝑹 𝒕

Selection of the length of the window, 𝑁

Translation error (linear) Rotation error (angular)

Experiments

• Accumulation of cluster centroids over 𝑁 frames

• 𝑁 images and 𝑁 point clouds processed

• Not every window provides clusters to be accumulated

24



CAMERA/LIDAR

Step 3

Experiments

• Noise is included in the measurements from the sensors

• Gaussian noise: 𝒩 0, 𝐾𝜎02

25


𝜎0𝑐 = 0.007

CAMERA

𝜎0𝑙 = 0.008 𝑚

LIDAR

Translation error (linear) Rotation error (angular)

Robustness to noise, 𝐾

Comparison26


Geiger et al., ICRA 2012Geiger et al., ICRA 2012

• Public web toolbox

• Monocular cam., provide intrinsics

• Tested with HDL-64E & Kinect

Velas et al., WSCG 2014

• Public ROS package

• Monocular camera

• Not suitable for large pose displacements

• Tested with HDL-32E

RecreatedIn Gazebo

16 layers 32 layers 64 layers

Experiments27


Tra

ns

lati

on

err

or

Ro

tati

on

err

or

16 layers 32-layer 64-layer16-layer

Experiments28


Tra

ns

lati

on

err

or

Ro

tati

on

err

or

16 layers 32-layer 64-layer16-layer

Results

• IVVI 2.0 platform

• Bumblebee XB3 stereo system: 1280 x 960 images, 12 cm baseline

• Velodyne VLP-16

29


Results30


Results in real scenarios31


Stereo + lidar point clouds aligned

Results in real scenarios32


Lidar measurements projected on the image

Agenda

Motivation



Results

Conclusion

33


Conclusion

• Method for calibration of lidar–stereo-camera setups:

• Without user intervention

• Suitable for close-to-production devices

• Assessment of the calibration methods using advanced simulation

• Exact ground-truth in unlimited calibration scenarios

• Results validate our calibration approach

34


ROS Package available athttp://wiki.ros.org/velo2cam_calibrationOpen source · GPLv2 License

Future work

• Further testing

• Sensitivity to different stereo matching approaches (e.g. CNN-based), weather/illumination conditions,…

• Monocular camera–multi-layer lidar calibration

• Geometrical information may be extracted from the calibration target

Thank youfor your attention

Carlos Guindel · [email protected]

IEEE 20th International Conference on Intelligent Transportation Systems

Yokohama · 17 October 2017

automatic extrinsic calibration for lidar-stereo vehicle ... · • tested with hdl-64e &...

Documents