harris corner detector on fpga rohit banerjee jared choi 15-418: parallel computer architecture and...

Harris Corner Detector on FPGA Rohit BanerjeeJared Choi

15-418: Parallel Computer Architecture and Programming

Introduction to the problem

Corner detection

Corner Detection

• Corners provide a lot of information

• Spending time to detect corners can significant reduce computation time* Image matching* Motion tracking* Robot Navigation

Application Specific Need

Pixel Stream

OutputCORNERDETECT

OR

• Want to be able to process images in real time.

• Corner detection is very memory intensive.

• CPU cannot process data fast enough because it had to store data to DRAM first.

Exploit arbitrary hardware parallelism to process more within the given time window

Harris Detector Basic Idea

Hardware Implementation

Processing Pipeline

Processing Pipeline

Grayscale

Processing Pipeline

Processing Pipeline

Sobel Filter

2D convolution

Convolution Buffer

Convolution Buffer

Convolution Buffer

Convolution Buffer

Convolution Buffer

Convolution Buffer

Convolution Buffer

Convolution Buffer

Convolution Buffer

Convolution Buffer

After a while……

Convolution Buffer

This column is ready to be processed

0

1

N-2

N-1

Convolution Buffer

0

1

N-2

N-1

Convolution Buffer

0

1

N-2

N-1

Processing Pipeline

Processing Pipeline

Gaussian Filter

Processing Pipeline

Processing Pipeline

Harris Response

Harris Input

∑Gadxdx

∑Gadydy∑Gadxdy

∑Gadxdy

=Sxx

Sxy

Sxy

Syy

Harris Response

Results

OpenCV parallel software implementation SystemVerilog(50MHz)0

2

4

6

8

10

12

14

16

18

20

Execution Time for Different platforms

Platform

Executi

on t

ime/m

s

Platform Execution Time/ms

Naïve Serial Implementation 351.20

OpenCV 18.12

SystemVerilog(50MHz) 1.31

Results

Speedup over serial implementation Speedup over OpenCV implementaion0.00

50.00

100.00

150.00

200.00

250.00

300.00

Speedup

Speedup

PlatformSpeedup of

FPGA Implementation

Naïve Serial Implementation 268.09

OpenCV 13.83

Results

Intel i3-4330 Cyclone IVE FPGA0.00

5.00

10.00

15.00

20.00

25.00

30.00

35.00

40.00

45.00

Energy Efficiency

Platform

Appro

x.

Pow

er

Consum

pti

on/W

Hardware Approx. Power Consumption/W

Intel i3-4330 42.00

Cyclone IVE FPGA 6.30

Thank you for your attention