l12 introtracking - colorado state universitycs510/yr2017sp/more... ·...

2/19/17

1

Introduction to Tracking

CS 510 Lecture #12

February 15, 2017

PA2

• Any questions?– Yes, its due Monday.

2/17/17 CS510,ImageComputation,©RossBeveridge&BruceDraper 2

Moving Object Detection• Assuming a still camera• Two algorithms:

– Mixture of Gaussians (Stauffer & Grimson)– ViBE (Barnich & Van Droogenbroeck)

• But these algorithms only label pixels– Need to group foreground pixels into regions– Need to connect regions across images– To reason about moving objects


Connected Components• Goal: connect pixels into regions


https://en.wikipedia.org/w

iki/Connected-component_labeling

Pixel Grouping

• Connected components to group pixels• Put a bounding box around component• May chose to add additional requirements

– Minimum size– Minimum width/height– Merge regions that are “close enough”


After Pixel Grouping

• Once you have found the pixel groups in an image…

• ... How do you connect objects across frames?


2/19/17

2

Pearson’s Correlation


This is a very important equation…

A x, y( )− A( ) B x, y( )−B( )x,y∑

A x, y( )− A( )2

x,y∑ B x, y( )−B( )

2

x,y∑

New Goal: Find a small image within a larger one

• The image above is a small piece of the image to the right. But from where?


Brute-Force Translation Invariance

To find a small image in a large one, “slide” the small one across the large, computing Pearson’s correlation at every possible position.


Computing Cross-Correlation• In cross-correlation, the mask is correlated

repeatedly to image windows– zero-mean & unit length the mask– zero-mean & unit length the image– compute the sliding dot product

2/17/17

Thisisalmost convolvingtheimagewiththemask

CS510,ImageComputation,©RossBeveridge&BruceDraper 10

Naming conventions• In Engineering, convolving a normalized

mask with the source image is called correlation– Is this exactly the same as Pearson’s

correlation?– Why or why not?

• This is the most common definition of correlation in image processing texts


Statistical Cross-Correlation• The process of “slide & correlate” is called cross-correlation• Complexity is O(nm)

– N = # of pixels in image (w´h)– M = # of pixels in the template (w´h)

• Highly parallel (every position can be computed independently)• Still sensitive to

– Rotation• in-plane• out-of-plane

– Scale– Perspective


2/19/17

3

Simple Tracking

• So how do you track an object from one frame to the next?– Find the moving object in frame t– Cross-correlation the attention window

against frame t+1– The best-matching position is where the

object went• With a minimum threshold, in case the object left

the FOV


Can we do better?• Is there something better we can compare to

than the raw attention window?• Do we have to search all of frame t+1?

– Can we limit the search?– Predict where the object is headed?– Describe the object’s motion?– Exploit foreground information in frame t+1?

• What if we don’t find the target in frame t+1?


Three Ways of Thinking About Images….1. As 2D signals : f(x,y)

– For video, 3D signals : f(x,y,t)– E.g. correlation, cross-correlation

2. As points in image N-dimensional image space

– E.g. geometry of correlation space– To come: eigenvector analysis

3. As continuous surfaces– Today’s viewpoint


Surface in 1D


1Dcross-sectionofsimpleimagesurface

Imagefromhttp://ars.els-cdn.com/content/image/1-s2.0-S1077314204001675-gr1.jpg

Image as Surface• View the image as a 3D surface

– For every (x,y) pixel location, the intensity can be thought of as the z(height) value.

• Color images are 5D surfaces - too hard to think about.• Color can also be thought of as 3 3D surfaces

– Pretend the surface is continuous


Gradients on Surfaces• Every point on the image surface has a direction of maximum

change (remember your multivariate calculus?), and a magnitude of change in that direction


2/19/17

4

Image Edges • To find direction and magnitude of change,

compute the mag. in any 2 orthogonal directions and interpolate– Again, this assumes a continuous surface

• WLOG choose the X & Y directions:– dx(x,y) = I(x,y) - I(x-1,y)– dy(x,y) = I(x,y) - I(x,y-1)

• The edge magnitude and orientation is:


Δ = dx2 + dy2 θ = cos−1 dyΔ

#

$%%

&

'((

Estimating Edge Orientation

• Problem: images are not continuous surfaces– Estimates of dx, dy based on grid sampling– Here is the problem, do you agree:

– estimating derivatives from two values is highly error prone.


I x, y( )− I x +1, y( ) = I x −1, y( )− I x, y( )?

Accurate Edge Estimation• We want to compute a real-valued function

– The partial derivatives dx & dy• All we have to work with are samples at

equidistant points• So model the function in terms of its Taylor

series expansion:


f x + h( ) = f x( )+ h1

1!!f x( )+ h

2

2!!!f x( )+

Accurate (II)• Look at equations for f(x+h) and f(x-h):


f x + h( ) = f x( )+ h !f x( )+ h2

2!!f x( )+ (1)

f x − h( ) = f x( )− h !f x( )+ h2

2!!f x( )− (2)

subtract equation 2 from 1f x + h( )− f x − h( ) = 2h !f x( )+and solve for !f

!f x( ) =f x + h( )− f x − h( )

2h+

Accurate (III)

• So the best ±1 mask is [-1,0,1], from

• As an exercise, the best ±2 mask is [1,-8,0,8,-1]


ddx

f x, y( ) =f x +1, y( )− f x −1, y( )

2

d 2

dx2f x, y( ) =

− f x + 2, y( )+8 f x +1, y( )−8 f x −1, y( )+ f x − 2, y( )12

Stable Edges (III)• Of course, pixels are still noisy and pixels

are related to adjacent rows.• The Sobel Edge Masks

2/17/17CS510,ImageComputation,©RossBeveridge&

BruceDraper 24

Dx Dy

−1 0 1−2 0 2−1 0 1

"

#

$$$

%

&

'''

−1 −2 −10 0 01 2 1

"

#

$$$

%

&

'''

2/19/17

5

Sobel Explanation• In any row (dx) or column (dy), this is a [-

1,0,1] mask to estimate the derivative• [1,2,1] weights approximate a σ=1

Gaussian.– Over-constrained fit of a plane to 9 points– Minimizes the sum-of-squared error

• Multiply result by 1/4 : range -255<x< 255• Multiply results by 1/8 : 8-bit response

2/17/17 CS510,ImageComputation,©RossBeveridge&BruceDraper 25 2/17/17 CS510,ImageComputation,©RossBeveridge&BruceDraper 26

−1 0 1−2 0 2−1 0 1

"

#

$$$

%

&

'''

−1 −2 −10 0 01 2 1

"

#

$$$

%

&

'''

0 −1 0−1 4 −10 −1 0

Sobel Edge Magnitude

2/17/17

dx2 + dy2

CS510,ImageComputation,©RossBeveridge&BruceDraper 27

And why edges – Structure


3 Sources of edges• Surface discontinuity

– Where one surface ends and another begins• Illumination discontinuity

– Shadows– Sudden change in surface orientation

• Remember your surface reflectance equations– Specular reflections

• ditto• Surface marking

– Change in material and/or surface color


Related Concept - Laplacian• Second order derivatives• Zero crossing idea not widely used today


2/19/17

6

Say it with Code - Sobel

• Play with this Tutorial !



l12 introtracking - colorado state universitycs510/yr2017sp/more... ·...

Documents