VALIDATION AND TUNNING OF DENSE STEREO-VISION SYSTEMS

USING HI-RESOLUTION 3D REFERENCE MODELS

E. REMETEAN (CNES)S. MAS (MAGELLIUM) - JB GINESTET (MAGELLIUM) - L.RASTEL (CNES)

ASTRA - 14.04.2011

ASTRA 2011 2

CONTENT

Validation methodologyHardware & software toolsPreliminary resultsStudy statusCredits

ASTRA 2011 3

Dense stereo-vision system validation

Study goals3D reconstruction accuracyRobustness wrt the scene content & lightingImpact of the stereo-vision system parametersOptimal parameter set definition for a given robotic mission

MethodologyComparison of the computed disparity maps to dense reference maps acquired with low parallax

To build a navigation map and compute a safe and effective trajectory, the Autonomous Navigation software needs a reliable knowledge of the rover surroundings

ASTRA 2011 4

Acquisition Mechanical Ground Support Equipment

Acquisition of stereo images & reference models with reduced parallax Composed of a stereo-bench, a Laser Scanner, translation linear stages

FARO Photon 20 Laser ScannerUp to 30 million 3D points / s-b f.o.v.Accuracy ≈ ±2mmMeasurement range up to 20m

Translation stages for parallax minimization

ASTRA 2011 5

Acquisition MGSE calibration

Laser Tracker (LT)

Laser Scanner(LS)

Landmark Balls

Stereo Bench(SB)

Useful area

Estimation of transfer matrix between the Laser Scanner and stereo-bench reference framesParallax minimisation

Calibration performed after any MGSE displacement on the Mars yard

Laser Tracker used to measurePosition of the landmark ballsPosition & attitude of the stereo-benchAccuracy better than 0.1mm

ASTRA 2011 6

Both indoors and outdoors (scene content & lighting conditions variation) Several exposure times for every scene (robustness studies) The content of the scene is changed rather than moving the MGSE (to avoid MGSE calibrations)

Acquisition campaigns

ASTRA 2011 7

Laser Scanner data filtering 3D-Filter software was developed for:

Interest zone selection (corresponding to stereo-bench field of view) Points clouds filtering for measurement artefacts & outliers removal

ASTRA 2011 8

Data exploitation: Perception Workshop

Comparison of real disparity (physical stereo-bench) to virtual disparity (virtual stereo-bench looking at the 3D model measured by the LS)

Real disparity computation parameters can be modified from a control panel

The filtered Laser Scanner points cloud is meshed and the disparity is computed using the virtual stereo-bench physical parameters (adjustable)

Initial virtual stereo-bench position & attitude obtained from MGSE calibration step

ASTRA 2011 9

Data exploitation: PW – Stereo-benchmarking

Similarity scores Classical window-based scores (SAD, SSD, ZNCC) 3D-distance score (a little pessimistic)

Virtual 3D cloud

Real 3D cloud

Left optical centre

3Ddist = | DepthReal – DepthVirtual |

Virtual stereo-bench position & attitude optimisation Stereo base length optimisation → indirect stereo base length measurement method

ASTRA 2011 10

Data exploitation: PW – 3D viewer 3D viewer allows to display in 3D

Real & Virtual points clouds computed from disparities Mismatches between the clouds (local similarity error)

ASTRA 2011 11

Preliminary results (1/3)

Accuracy (3Ddist)100mm stereo base CCD 4.65µm pixels Full resolution images 7x7 correlation window Virtual SB attitude & base optimisation to measure intrinsic performance

Scenes Mean Error (mm) Std dev (mm)Indoor 5.05 0.64

Outdoor 13.27 1.55

Image sub-sampling

“Pixel size” (µm)

Mean error (mm)

Mean accuracy degradation

Number of pixels to process

1/1 4.65 13.27 0% 100%

1/2 9.30 16.60 25.09% 25%

1/4 18.60 20.45 54.11% 6.25%

☺ Mean Error < Autonomous Navigation DEM cell size (40mm)

Impact of image resolution

ASTRA 2011 12

Preliminary results (2/3)

Impact of stereo-correlation window sizeCorrelation window size Mean error (mm)

Mean accuracy gain wrt 7x7

Estimated complexity wrt 7x7

9x9 12.45 6.18% +65%

7x7 13.27 0% 0%5x5 15.00 -13.04% -51%

Robustness to exposure time

Correlation ratio (%)

L \ R 5 ms 48 ms 81 ms 87 ms 135 ms 170 ms5 ms 27.24 - - - - -55 ms - 88.50 66.87 57.10 15.33 7.2790 ms - 75.03 89.68 81.58 47.92 28.70

100 ms - 62.31 82.68 90.09 77.61 55.69150 ms - 19.64 57.60 82.59 90.18 86.70200 ms - 8.10 32.99 55.30 87.42 90.19

ASTRA 2011 13

Preliminary results (3/3) Fast multi-resolution stereo-correlation algorithm

L \ R 5 ms 48 ms 81 ms 87 ms 135 ms 170 ms5 ms 81.90 - - - - -

55 ms - 92.98 91.11 92.49 86.82 82.7190 ms - 92.74 93.08 92.48 91.14 88.96

100 ms - 93.11 92.22 93.11 92.79 91.89150 ms - 91.97 92.11 92.94 93.19 92.98200 ms - 90.33 91.26 92.23 92.90 93.15

L \ R 5 ms 48 ms 81 ms 87 ms 135 ms 170 ms5 ms +24.5 - - - - -

55 ms - -7.6 -7.4 -10.1 -0.8 +2.190 ms - -9.3 -8.7 -7.2 -12.7 -14.9

100 ms - -14.7 -7.7 -9.4 -11.7 -13.6150 ms - -13.5 -13.0 -11.8 -9.1 -9.2200 ms - -17.7 -18.0 -17.6 -9.1 -9.2

Correlation ratio (%)

Mean error(% wrt mono-resolution

algorithm)

ASTRA 2011 14

Study statusToday

Validation methodology adapted for dense stereo-vision systemsIndirect stereo base estimation methodAccuracy of the studied stereo-vision system is compatible with AN requirementsGood robustness to image exposure conditionsFast multi-resolution algorithm will become the new CNES baseline

Further workFull-resolution outdoor acquisition campaignsPerformances with stereo-bench flight model demonstrator“Tough” textures campaignsStereo base length impactsSecurity margins definition for Autonomous Navigation

ASTRA 2011 15

Credits : CNES sub-contractors involved

Stereo-benchesFlight model demonstrator: CSEM / MCSEGround models: COMAT Aerospace, AR2P

Perception WorkshopMagelliumCS-SI

3D-FilterCS-SI

Validation studies & MGSE realisationMagelliumSud-Rectif

ASTRA 2011 16

Thank you for your attention!

emile.remetean@cnes.fr

Slide 1Slide 2Slide 3Slide 4Slide 5Slide 6Slide 7Slide 8Slide 9Slide 10Slide 11Slide 12Slide 13Slide 14Slide 15Slide 16