1

Real-Time Stereo-Matching for Micro Air Vehicles

Pascal Dufour13.04.2010

Master Thesis Presentation

2

Outline

• Introduction

• Stereo-Matching

• Evaluation

• Outlook

• Demo

3

PIXHAWK

• Computer vision on a MAV to enable autonomous action

• Current System: Quadrotor

• COM Express board with Intel Core 2 Duo

• Integrated GPU

4

Requirements• Efficiency:

• other mission critical processes run in parallel

• 10 Hz estimated for object avoidance

• no more than 20% load of one CPU: 20ms at 10Hz

• Robustness:

• produce consistent results, dense disparity map

5

Stereo Vision

• Two cameras allow the triangulation of 3D points

• Solve the correspondence problem

7

Rectification

• Epipolar lines align horizontally

• Search for correspondece becomes search in one dimension

8

Block-Matching

• Find correspondences by comparing blocks of pixels

• Fast implementation possible

• Difficulty:

• badly textured regions

• repeating textures

• occlusions

9

SAD

• Dissimilarity value: Sum of Absolute Differences

• Winner-take-all strategy: Disparity with lowest cost wins

10

Sliding window• Slide window across epipolar line, only compute differences

• These differences are columns and can be precomputed for each line

• Idea: reuse computed data, so matching becomes independent of window width

11

Sliding window

• When changing line: the columns from the line above can be reused

• Matching becomes independent of window height

12

Improvements: SSE4

• 128bit registers

• Single Instruction, Multiple Data

• Eight differences with one instruction

13

Improvements: SSE4

• minpos( ) returns smallest value and position in a register

• No branching required

14

Matching on the GPU

• GPU provides more computation power

• Implementation with programmable shaders:

• Compute texture differences

• Use texture sampling

• Exploit depth-test

15

Texture Differences

16

Texture Sampling

17

Sampling: Math

• Sampling can be used to compute the average of a pixel-block

18

Sampling Differences

• Sampling is applied to the texture differences

• Each pixel in the sampled texture corresponds to the average of a block of pixel intensity differences

19

Weighted Distribution

• Sum up dissimilarity values of blocks

• Weighted distribution:

• Large area has influence on the matching cost

• But center has bigger influence

20

Sum of Sampled Textures

• Sum up texels of sampled textures

• Nearest neighbor yields bad distribution

21

Sum of Sampled Textures

• Sum up texels of sampled textures with interpolation

• Weighted distribution, but not perfectly centered

23

Post-Processing

• Robustness can be increased with post-processing

• Variance in block as confidence

• Simpler: only variance of a line

• other possibility: difference between darkest and brightest pixel

24

Evaluation

• Middlebury’s Stereo-Vision Website allows evaluation of algorithms:

• percent of pixels at the correct disparity

• raw disparity map, only evaluate matching

• disparity of four different images evaluated

25

Evaluation

Tsu

kub

aC

on

es

Ted

dy

Ven

us

26

Evaluation

Tsu

kub

a

Ted

dy

Ven

us

Con

es

27

Evaluation

Algorithm Tsukuba Venus Teddy Cones

SAD: 9x9 10.9 9.82 31.6 27.0

SSD: 9x9 11.9 9.66 32.2 26.7

GPU 20.2 17.5 41.9 38.2

• Percent of pixels at wrong disparity

• Threshold: 1

28

Evaluation

Algorithm Tsukuba Venus Teddy Cones

SAD: 9x9 9.38 8.38 27.9 28.5

SSD: 9x9 10.1 8.37 23.7 23.7

GPU 7.4 8.8 27.4 22.0

• Percent of pixels at wrong disparity

• Threshold: 2

29

PerformanceUndistortion,

Rectification, ResizingSAD

Matchinglinear NN C++ with SSE3.80 1.64 26.12 11.04

GPU Implementation Confidence Mapintegrated

GPUdedicated

GPUC++ with SSE

1100 + CPU load

12.1 3.70 1.04

30

Outlook• Use of disparity map:

• object avoidance

• possibly SLAM

• visualization

• Camera framework with synchronization

• Realistic testing when quadrotor is finished

31

Acknowledgements

• Marc Pollefeys

• Friedrich Fraundorfer

• Lorenz Meier

• The whole team

32

DEMO