dalv license object detection

7/26/2019 Dalv License Object Detection

1/31

MINISTRY OF EDUATION OF MOLDOVA

Free International University of Moldova

Falty of Infor!ati"s# En$ineerin$ and Desi$n

De%art!ent of Infor!ational Te"&nolo$ies and En$ineerin$

Accepted for defense Accepted for defense

Dean of the Faculty Head of the Department

Iuri Dubovehi, Dr, onf. Univ. Iuri Dubovehi, Dr, onf. Univ'

!"#$%& !" #$&

LI(ENSE )RO*E(T

O+,e"t dete"tion syste! +y o%ti"al "orrelator and intelli$ent analysis

Author

'lad (ordian, student )r. *+%

-roect /upervisor

'eaceslav0. -eru, Dr. Hab., *onf. Univ., aad.

&i-in. /012
http://vperju.ulim.md/http://vperju.ulim.md/


2/31

3 4ordian Vlad# /012

2


3/31

(ontents

/A1I2A........................................................................................................................................

AD234A15....................................................................................................................................6

0I/4A A(15'I51I031................................................................................................................7

Introdution.....................................................................................................................................8

%. 3bect 1eco)nition.................................................................................................................%%

%.%. Appearance (ased 9ethods............................................................................................%%

%.#. :eometry+(ased 9ethods..............................................................................................%

%.;. Feature+based methods....................................................................................................%6

%.. 1eco)nition as a *orrespondence of 0ocal Features......................................................%8

%.6. Invariant pattern reco)nition...........................................................................................##%..................................................................................................................................;$

3


4/31

?ef atedr@ 4ehnolo)ii Informaionale i In)inerie

Iuri Dubovehi, Dr, onf. Univ.

"" #$%&

SARINA

pentru teBa de lien@ a studentului )rupei *+#

(ordian 'lad

Te!a5 !Siste!l de dete"tare a o+ie"telor 6n +a7a "orelatorli o%ti" i anali7ei inteli$ente "

aprobat@ prin ordinul nr. din "" #$%&

on8intl notei e9%liative> %. AnaliBa metodelor i sistemelor e=istente de detectare a

obiectelor Cn baBa corelatorului optic si analiBei inteli)ente #. 5laborarea, realiBarea i eretarea

unui al)oritm nou de analiB@ inteli)ent@ a obiectelor ;. 5laborarea i eretarea sistemului de

detectare a obiectelor.

Lista !aterialli $rafi> %. lasifiarea al)oritmilor, metodelor i sistemelor e=istente de

e=tra)erea onturilor #. /trutura al)oritmului nou de e=tra)erea onturilor ;. /hema+blo a

softului elaborat de e=tra)erea onturilor . 1eBultatele eret@rilor al)oritmului de e=tra)erea

onturilor 6. /trutura sistemului de e=tra)erea onturilor "" #$&

5=eutant

(ordian 'lad,

/tudentul )rupei *+#

ondu@torul teBei

-eru0. 'eaceslav, Dr. Hab., *onf. univ., aademiian

4
http://vperju.ulim.md/http://vperju.ulim.md/


5/31

ADNOTARE

4ordian Vlad# Sistemul de detectare a obiectelor n baza corelatorului optic i analizei

inteligente,te7. de lien8. la s%eialitatea (al"latoare# &i-in.# /012'

Aest proiet uprinde introduerea, trei apitole, onluBii u reomand@ri, biblio)rafia

din #8 titluri. 5a este perfetat@ pe EE pa)ini, onine EE fi)uri, EE tabele i EE formule.

uvinte-heie:optic, corelator, reunoaterea, sistem, detectare, analiB@

Domeniul de studiual teBei este prelurarea semnalelor.

Sopul i obietivele lurriionstau Cn eretarea sistemului de detectare a obiectelor.

Noutatea i originalitatea lurriionst@ Cn folosirea unui al)oritm nou pentru a detecta

obiectele in baBa corelatorului optic.

Semnifiaia teoretia lur@rii onst@ Cn deBvoltarea unui softare are implementeaB@

al)oritme mai eficiente pentru detectarea obiectelor.

Valoarea apliativ a lurrii onst@ faptul @ aest sistem poate fi implementat Cn

diverse domenii preum ar fi Cn milit@rie, la ontrolul vamal i oriunde unde avem nevoie de

suprave)here video efectiva.

Implementarea reultatelor obinute. /istemul realiBat a fost onfi)urat i testat pe mai

multe alulatoare i pe mai multe ima)ini.

5


6/31

A4STRAT

4ordian Vlad# :Object detection system by optical correlator and intelligent analysist&esis

for s%eialty (o!%ters# &isina# /012'

4he thesis ontains the introdution, three hapters, onlusions, reommendations,

biblio)raphy of #8 titles. It onsists of EE pa)es, inludin) EE fi)ures, EE tables and EE

formule.

!e"#ords> optic, correlator, reco)nition, system, detection, analysis.

$ield of stud"of the thesis is information proessin).

%oals and ob&etivesinlude researhin) a obect detection system.

Novelt" and originalit" of this orG is the use of an ori)inal al)orithm for obect

detection by optical correlator.

'he theoretial signifianeis the developin) a softare that uses better al)orithms for

obect detection.

(ppliative valueof the orG is that this system an be used in various fields and areas.

Implementation results. 4he developed system has been onfi)ured and tested on multiple

omputers and multiple ima)es.

6


7/31

A;;O?

@BC GHC# :J# CKHoa Baoa Ko KePaHQo KQB# eW

/012'

oeJK LJMNOaeK LLePeQRe, KR SMaLT, LTLoPT R eJoeQPaVRR, WRWMRoSaXRN RY

#8 QaReQoLaQRZ. 3Q LT[oMQeQ Qa \\ KaQRVa], oPe^RK \\ XRS_T, \\ KaWMRVT R

\\ Xo_M.

)*+,e.e *oa> `[KROJRZ, J`MK`, [`YQ`LQR, RK, `WQ`_^QR,

QMRY.

/0*a12 3*e4oa536aWoKT `W`W`KJ RSQM`L.

7e*3 3 8a4a,3 9a0o1.oKoK L RMePoLaQR RKT `WQ`_^QR [PK`L.

o385a 3 o93;35a*25o12 9a0o1.YaJMNOaeK L R[oMYoLaQRe Q`LT] MS`RK`L

PM [`YQ`LQR [PK`L [``N [KROJ`S J`MK`.

o12 aWoKT YaJMNOaeK L aYaWoKJe [oSaQoSo

oWe[eOeQR W`M [KRMQTR MS`RKR PM WQ`_^QQR [PK`L.

?93=*a45a6 85a,3>o12 aWoKT LMeK Ko, OKo gKa RKea o^eK WTK

eaMRYoLaQa L aYMROQT] Jo[aQR] R oWMaK].

@e8A*21a1. 5e49e536. aYaWoKaQQa RKea WTMa QaKoeQa R R[TKaQa Qa

QeJoMJo Jo[NKeoL R RYoWa^eQRZ.

7


8/31

LISTA A4REVIERILOR

9/F 9atched spatial filters

3*1 3ptical character reco)nition

/09 /patial li)ht modulatorD9D Deformable mirror device

-/ oint poer spectrumFF4 Fast Fourier transforms

(-3F (inary phase+only filters

/21 /i)nal+to+noise ratio

Introdtion

:reat effort has been done in the research community of optical pattern reco)nition. /hort

time after the inventions of lasers in %8


9/31

and approach for improvin) the performance of the 9/F and findin) its ne applications have

been proposed by the pioneers in optical pattern reco)nition community. 4remendous pro)ress

has been made today in the area of pattern reco)nition, compared ith that in %8


10/31

1' O+,e"t Re"o$nition

In the field of computer vision obect reco)nition describes the tasG of findin) and

identifyin) obects in an ima)e or video sequence. Humans reco)niBe a multitude of obects in

ima)es ith little effort, despite the fact that the ima)e of the obects may vary somehat in

different vie points, in many different siBes and scales or even hen they are translated or

rotated. 3bects can even be reco)niBed hen they are partially obstructed from vie. 4his tasG

is still a challen)e for computer vision systems. 9any approaches to the tasG have been

implemented over multiple decades;%.

1'1'A%%earan"e 4ased Met&ods

+ Use e=ample ima)es jcalled templates or e=emplarsk of the obects to perform

reco)nition

10


11/31

+ 3bects looG different under varyin) conditions>

*han)es in li)htin) or color

*han)es in viein) direction

*han)es in siBe shape

+ A sin)le e=emplar is unliGely to succeed reliably. Hoever, it is impossible to represent

all appearances of an obect;%.

4he central idea behind appearance+based methods is the folloin). Havin) seen all

possible appearances of an obect, can reco)nition be achieved by ust efficiently rememberin)

all of them *ould reco)nition be thus implemented as an efficient visual jpictorialk memory

4he anser obviously depends on hat is meant by "all appearances". 4he approach has been

successfully demonstrated for scenes ith unoccluded obects on blacG bacG)round. (utrememberin) all possible obect appearances in the case of arbitrary bacG)round, occlusion and

illumination, is currently computationally prohibitive.

Appearance based methods typically include to phases. In the first phase, a model is

constructed from a set of reference ima)es. 4he set includes the appearance of the obect under

different orientations, different illuminants and potentially multiple instances of a class of

obects, for e=ample faces. 4he ima)es are hi)hly correlated and can be efficiently compressed

usin) e.). arhunen+0oeve transformation jalso Gnon as -rincipal *omponent Analysis +

-*Ak.

In the second phase, recall", parts of the input ima)e jsubima)es of the same siBe as the

trainin) ima)esk are e=tracted, possibly by se)mentation jby te=ture, colour, motionk or by

e=haustive enumeration of ima)e indos over hole ima)e. 4he reco)nition system then

compares an e=tracted part of the input ima)e ith the reference ima)es je.). by proectin) the

part to the arhunen+0oeve spacek.

A maor limitation of the appearance+based approaches is that they require isolation of

the complete obect of interest from the bacG)round. 4hey are thus sensitive to occlusion and

require )ood se)mentation. A number of attempts have been made to address reco)nition ith

occluded or partial data.

4he family of appearance+based obect reco)nition methods includes )lobal histo)ram

matchin) methods. /ain and (allard proposed to represent an obect by a colour histo)ram.

3bects are identified by matchin) histo)rams of ima)e re)ions to histo)rams of a model ima)e.

hile the technique is robust to obect orientation, scalin), and occlusion, it is very sensitive to

li)htin) conditions, and it is not suitable for reco)nition of obects that cannot be identified by

11


12/31

colour alone. 4he approach has been later modified by Healey and /later and Funt and Finlayson

to e=ploit illumination invariants. 1ecently, the concept of histo)ram matchin) as )eneralised

by /chiele, here, instead of pi=el colours, responses of various filters are used to form the

histo)rams jcalled then receptive field histo)ramsk.

4o summarise, appearance based approaches are attractive since they do not require

ima)e features or )eometric primitives to be detected and matched. (ut their limitations, i.e. the

necessity of dense samplin) of trainin) vies and the lo robustness to occlusion and cluttered

bacG)round, maGe them suitable mainly for certain applications ith limited or controlled

variations in the ima)e formation conditions, e.). for industrial inspection%%.

Ed$e !at"&in$

Uses ed)e detection techniques, such as the *anny ed)e detection, to find ed)es.

*han)es in li)htin) and color usually dont have much effect on ima)e ed)es

/trate)y>

ak Detect ed)es in template and ima)e

bk *ompare ed)es ima)es to find the template

ck 9ust consider ran)e of possible template positions

9easurements>ak :ood count the number of overlappin) ed)es. 2ot robust to chan)es in shape

bk (etter count the number of template ed)e pi=els ith some distance of an ed)e in

the search ima)eck (est determine probability distribution of distance to nearest ed)e in search ima)e

jif template at correct positionk. 5stimate liGelihood of each template position

)eneratin) ima)e

DivideXandX(oner sear"&

/trate)y>

ak *onsider all positions as a set ja cell in the space of positionskbk Determine loer bound on score at best position in cell

ck If bound is too lar)e, prune cell

dk If bound is not too lar)e, divide cell into subcells and try each subcell recursivelyek -rocess stops hen cell is small enou)h"

UnliGe multi+resolution search, this technique is )uaranteed to find all matches that meet the

criterion jassumin) that the loer bound is accuratek

Findin) the (ound>

12


13/31

ak 4o find the loer bound on the best score, looG at score for the template position

represented by the center of the cellbk /ubtract ma=imum chan)e from the center" position for any other position in cell

joccurs at cell cornersk

*omple=ities arise from determinin) bounds on distance

Zreys"ale !at"&in$

5d)es are jmostlyk robust to illumination chan)es, hoever they thro aay a lot of

information

9ust compute pi=el distance as a function of both pi=el position and pi=el intensity

*an be applied to color also

Zradient !at"&in$

Another ay to be robust to illumination chan)es ithout throin) aay as much

information is to compare ima)e )radients

9atchin) is performed liGe matchin) )reyscale ima)es

/imple alternative> Use jnormaliBedk correlation

[isto$ra!s of re"e%tive field res%onses

Avoids e=plicit point correspondences 1elations beteen different ima)e points implicitly coded in the receptive field responses

/ain and (allard j%88%k, /chiele and *roley j#$$$k, 0inde and 0indeber) j#$$, #$%#k

Lar$e !odel+ases

3ne approach to efficiently searchin) the database for a specific ima)e to use ei)envectors of

the templates jcalled ei)enfacesk

9odelbases are a collection of )eometric models of the obects that should be reco)niBed;$

1'/'Zeo!etryX4ased Met&ods

In )eometry+ jor shape+, or model+k based methods, the information about the obects is

represented e=plicitly. 4he reco)nition can than be interpreted as decidin) hether ja part ofk a

)iven ima)e can be a proection of the Gnon jusually ;Dk model of an obect.

:enerally, to representations are needed> one to represent obect model, and another to

represent the ima)e content. 4o facilitate findin) a match beteen model and ima)e, the to

representations should be closely related. In the ideal case there ill be a simple relation beteen

primitives used to describe the model and those used to describe the ima)e. ould the obect be,

13


14/31

for e=ample, described by a ireframe model, the ima)e mi)ht be best described in terms of

linear intensity ed)es. 5ach ed)e can be then matched directly to one of the model ires.

Hoever, the model and ima)e representations often have distinctly different meanin)s". 4he

model may describe the ;D shape of an obect hile the ima)e ed)es correspond only to visible

manifestations of that shape mi=ed to)ether ith false" ed)es jdiscontinuities in surface albedok

and illumination effects jshadosk.

4o achieve pose and illumination invariance, it is preferable to employ model primitives

that are at least somehat invariant ith respect to chan)es in these conditions. *onsiderable

effort has been directed to identify primitives that are invariant ith respect to viepoint chan)e.

4he main disadvanta)es of )eometry+based methods are> the dependency on reliable

e=traction of )eometric primitives jlines, circles, etc.k, the ambi)uity in interpretation of the

detected primitives jpresence of primitives that are not modelledk, the restricted modellin)

capabilities only to a class of obects hich are composed of fe easily detectable elements, and

the need to create the models manually %%.

1'\'FeatreX+ased !et&ods

+ a search is used to find feasible matches beteen obect features and ima)e features.

+ the primary constraint is that a sin)le position of the obect must account for all of the

feasible matches.

+ methods that e=tract features from the obects to be reco)niBed and the ima)es to be searched.

surface patches

corners

linear ed)es

Inter%retation trees

A method for searchin) for feasible matches, is to search throu)h a tree. 5ach node in the tree represents a set of matches.

ak 1oot node represents empty set

bk 5ach other node is the union of the matches in the parent node and one additional match.ck ildcard is used for features ith no match

2odes are pruned" hen the set of matches is infeasible.

ak A pruned node has no children

Historically si)nificant and still used, but less commonly

[y%ot&esi7e and test

:eneral Idea>

14


15/31

ak HypothesiBe a correspondence beteen a collection of ima)e features and a collection of

obect featuresbk 4hen use this to )enerate a hypothesis about the proection from the obect coordinate

frame to the ima)e frame

ck Use this proection hypothesis to )enerate a renderin) of the obect. 4his step is usually

Gnon as bacGproectiondk *ompare the renderin) to the ima)e, and, if the to are sufficiently similar, accept the

hypothesis

3btainin) Hypothesis>

ak 4here are a variety of different ays of )eneratin) hypotheses.bk hen camera intrinsic parameters are Gnon, the hypothesis is equivalent to a

hypothetical position and orientation pose for the obect.

ck UtiliBe )eometric constraints

dk *onstruct a correspondence for small sets of obect features to every correctly siBedsubset of ima)e points. j4hese are the hypothesesk

4hree basic approaches>

ak 3btainin) Hypotheses by -ose *onsistency

bk 3btainin) Hypotheses by -ose *lusterin)ck 3btainin) Hypotheses by Usin) Invariants

5=pense search that is also redundant, but can be improved usin) 1andomiBation andor

:roupin)

ak 1andomiBationi. 5=aminin) small sets of ima)e features until liGelihood of missin) obect becomes

small

ii. For each set of ima)e features, all possible matchin) sets of model features must be

considered.iii. Formula>

j % ckG w

the fraction of ima)e points that are )ood" j x mnk

c the number of correspondences necessary

G the number of trials

w the probability of every trial usin) one jor morek incorrect correspondences

bk :roupin)i. If e can determine )roups of points that are liGely to come from the same obect, e

can reduce the number of hypotheses that need to be e=amined

)ose "onsisten"y

Also called Ali)nment, since the obect is bein) ali)ned to the ima)e

*orrespondences beteen ima)e features and model features are not independent

:eometric constraints

15


16/31

A small number of correspondences yields the obect position the others must be consistent

ith this

:eneral Idea>

ck If e hypothesiBe a match beteen a sufficiently lar)e )roup of ima)e features and a

sufficiently lar)e )roup of obect features, then e can recover the missin) camera

parameters from this hypothesis jand so render the rest of the obectk

/trate)y>

ak :enerate hypotheses usin) small number of correspondences je.). triples of points for ;D

reco)nitionk

bk -roect other model features into ima)e jbacGproectk and verify additional

correspondences

Use the smallest number of correspondences necessary to achieve discrete obect poses

)ose "lsterin$

:eneral Idea>

ak 5ach obect leads to many correct sets of correspondences, each of hich has jrou)hlyk

the same pose

bk 'ote on pose. Use an accumulator array that represents pose space for each obectck 4his is essentially aHou)h transform

/trate)y>

ak For each obect, set up an accumulator array that represents pose space each element in

the accumulator array corresponds to a bucGet" in pose space.bk 4hen taGe each ima)e frame )roup, and hypothesiBe a correspondence beteen it and

every frame )roup on every obectck For each of these correspondences, determine pose parameters and maGe an entry in the

accumulator array for the current obect at the pose value.

dk If there are lar)e numbers of votes in any obects accumulator array, this can be

interpreted as evidence for the presence of that obect at that pose.ek 4he evidence can be checGed usin) a verification method

2ote that this method uses sets of correspondences, rather than individual correspondences

ak Implementation is easier, since each set yields a small number of possible obect poses. Improvement

ak 4he noise resistance of this method can be improved by not countin) votes for obects at

poses here the vote is obviously unreliableii. For e=ample, in cases here, if the obect as at that pose, the obect frame )roup

ould be invisible.

bk 4hese improvements are sufficient to yield orGin) systems

Invarian"e

4here are )eometric properties that are invariant to camera transformations

16
https://en.wikipedia.org/wiki/Backprojectionhttps://en.wikipedia.org/wiki/Pose_clusteringhttps://en.wikipedia.org/wiki/Hough_transformhttps://en.wikipedia.org/wiki/Hough_transformhttps://en.wikipedia.org/wiki/Invariant_(physics)https://en.wikipedia.org/wiki/Backprojectionhttps://en.wikipedia.org/wiki/Pose_clusteringhttps://en.wikipedia.org/wiki/Hough_transformhttps://en.wikipedia.org/wiki/Invariant_(physics)


17/31

9ost easily developed for ima)es of planar obects, but can be applied to other cases as ell

Zeo!etri" &as&in$

An al)orithm that uses )eometric invariants to vote for obect hypotheses /imilar to pose clusterin), hoever instead of votin) on pose, e are no votin) on

)eometry

A technique ori)inally developed for matchin) )eometric features juncalibrated affine vies

of plane modelsk a)ainst a database of such features

idely used for pattern+matchin), *AD*A9, and medical ima)in).

It is difficult to choose the siBe of the bucGets

It is hard to be sure hat enou)h" means. 4herefore there my be some dan)er that the table

ill )et clo))ed.

S"aleXinvariant featre transfor!]SIFT^

eypoints of obects are first e=tracted from a set of reference ima)es and stored in a

database

An obect is reco)niBed in a ne ima)e by individually comparin) each feature from the ne

ima)e to this database and findin) candidate matchin) features based on 5uclidean distance

of their feature vectors.

0oe j#$$k

S%eeded U% Ro+st Featres]SURF^

A robust ima)e detector descriptor

4he standard version is several times faster than /IF4 and claimed by its authors to be more

robust a)ainst different ima)e transformations than /IF4

(ased on sums of appro=imated #D Haar avelet responses and made efficient use of

inte)ral ima)es.

(ay et al j#$$7k ;$

1'_' Re"o$nition as a (orres%onden"e of Lo"al Featres

2either )eometry+based nor appearance+based methods discussed previously do ell as

defined by the requirements stated in the be)innin) of the paper, i.e. the generalit"B robustnessB

and eas" learningC:eometry+based approaches require the user to specify the obect models, and

can usually handle only obects consistin) of simple )eometric primitives. 4hey are not )eneral,

nor do they support easy learnin). Appearance+based methods demanded e=haustive set of

learnin) ima)es, taGen from densely distributed vies and illuminations. /uch set is only

17
https://en.wikipedia.org/wiki/Geometric_hashinghttps://en.wikipedia.org/wiki/Scale-invariant_feature_transformhttps://en.wikipedia.org/wiki/Speeded_Up_Robust_Featureshttps://en.wikipedia.org/wiki/Speeded_Up_Robust_Featureshttps://en.wikipedia.org/wiki/Geometric_hashinghttps://en.wikipedia.org/wiki/Scale-invariant_feature_transformhttps://en.wikipedia.org/wiki/Speeded_Up_Robust_Features


18/31

available hen the obect can be observed in a controlled environment, e.). placed on a

turntable. 4he methods are also sensitive to occlusion of the obects, and to the unGnon

bacG)round, thus they are not robust.

As an attempt to address the above mentioned issues, methods based on matchin) local

features have been proposed. 3bects are represented by a set of local features, hich are

automatically computed from the trainin) ima)es. 4he learned features are or)anised into a

database. hen reco)nisin) a query ima)e, local features are e=tracted as in the trainin) ima)es.

/imilar features are then retrieved from the database and the presence of obects is assessed in

the terms of the number of local correspondences. /ince it is not required that all local features

match, the approaches are robust to occlusion and cluttered bacG)round.

4o reco)nise obects from different vies, it is necessary to handle all variations in obect

appearance. 4he variations mi)ht be comple= in )eneral, but at the scale of the local features

they can be modelled by simple, e.). affine, transformations. 4hus, by alloin) simple

transformations at local scale, a si)nificant viepoint invariance is achieved even for obects

ith complicated shapes. As a result, it is possible to obtain models of obects from only a fe

vies, taGen e.). 8$ de)rees apart.

4he main advanta)es of the approaches based on matchin) local features are summarised

belo.

0earnin), i.e. the construction of internal models of Gnon obects, is done automatically

from ima)es depictin) the obects. 2o user intervention is required e=cept for providin)

the trainin) ima)es.

4he local representation is based on appearance. 4here is no need to e=tract )eometric

primitives je.). linesk, hich are )enerally hard to detect reliably.

/e)mentation of obects from bacG)round is not required prior reco)nition, and yet

obects are reco)nised on an unGnon bacG)round.

3bects of interest are reco)nised even if partially occluded by other unGnon obects in

the scene.

*omple= variations in obect appearance caused by varyin) viepoint and illumination

conditions are appro=imated by simple transformations at a local scale.

9easurements on both database and query ima)es are obtained and represented in an

identical ay.

-uttin) local features into correspondence is an approach that is robust to obect

occlusion and cluttered bacG)round in principle. hen a part of an obect is occluded by other

18


19/31

obects in the scene, only features of that part are missed. As lon) as there are enou)h features

detected in the unoccluded part, the obect can be reco)nised. 4he problem of cluttered

bacG)round is solved in a final step of the reco)nition process, hen a hypothesised match is

verified and confirmed, and false correspondences are reected.

/everal approaches based on local features have been proposed. :enerally, they follo a

certain common structure, hich is summarised belo.

Detetors. First, ima)e elements of interest are detected. 4he elements ill serve as

anchor locations in the ima)es + descriptors of local appearance ill be computed at these

locations. 4hus, an ima)e element is of interest if it depicts a part of an obect, hich can be

repeatedly detected and localised in ima)es taGen over lar)e ran)e of conditions. 4he challen)e

is to find such a definition of interest", that ould allo fast, reliable and precisely localised

detection of such elements. 4he brute force alternative to the detectors is to )enerate local

descriptors at every point. 4his course is obviously infeasible due to its computational

comple=ity.Desriptors. 3nce the elements of interest are found, the local ima)e appearance in

their nei)hbourhood has to be encoded in a ay that ould allo for searchin) of similar

elements.

hen desi)nin) a descriptor jalso called a feature vectork, several aspects have to be

taGen into account. First, the descriptors should be discriminative enou)h to distin)uish beteen

features of the obects stored in the database. ould e for e=ample ant to distin)uish beteen

to or three obects, each described by some ten odd features, the descriptions of local

appearance can be as simple as e.). four+bin colour histo)rams. 3n the other hand, handlin)

thousands of database obects requires the ability to distin)uish beteen a vast number of

descriptors, demandin) thus hi)hly discriminative representation. 4his problem can be partially

alleviated by usin) )roupin), i.e. simultaneous consistent matchin) of several detected elements.

Another aspect in desi)nin) a descriptor is that it has to be invariant, or at least in some

de)ree robust, to variations in an obects appearance that are not reflected by the detector. If, for

e=ample, the detector detects circular or elliptical re)ions ithout assi)nin) an orientation to

them, the descriptor must be made invariant to the orientation jrotational invariantsk. 3r if the

detector is imprecise in locatin) the elements of interest, e.). havin) fe pi=el tolerance, the

descriptor must be insensitive to these small misali)nments. /uch a descriptor mi)ht be based

e.). on colour moments jinte)ral statistics over hole re)ionk, or on local histo)rams.

It follos that the maor factors that affect the discriminative potential, and thus the

ability to handle lar)e obect databases, of a method are the repeatability and the localisation

precision of the detector.

19


20/31

IndeEing. Durin) learnin) of obect models, descriptors of local appearance are stored

into a database. In the reco)nition phase, descriptors are computed on the query ima)e, and the

database is looGed up for similar descriptors jpotential matchesk. 4he database should be

or)anised jinde=edk in a ay that allos an efficient retrieval of similar descriptors. 4he

character of suitable inde=in) structure depends )enerally on the properties of the descriptors

je.). their dimensionalityk and on the distance measure used to determine hich are the similar

ones je.). euclidean distancek. :enerally, for optimal performance of the inde= jfast retrieval

timesk, such combination of descriptor and distance measure should be sou)ht, that minimises

the ratio of distances to correct and to false matches.

4he choice of inde=in) scheme has maor effect on the speed of the reco)nition process,

especially on ho the speed scales to lar)e obect databases. *ommonly, thou)h, the database

searches are done simply by sequential scan, i.e. ithout usin) any inde=in) structure.

Fathing. hen reco)nisin) obects in an unGnon query ima)e, local features are

computed in the same form as for the database ima)es. 2one, one, or possibly more tentative

orrespondenesare then established for every feature detected in the query ima)e. /earchin)

the database, euclidean or mahalanobis distance is typically evaluated beteen the query feature

and the features stored in the database. 4he closest match, if close enou)h, is retrieved. 4hese

tentative orrespondenesare based purely on the similarity of the descriptors. A database obect

hich e=hibit hi)h jnonrandomk number of established correspondences is considered as a

candidate match.

Verifiation. 4he similarity of descriptors, on its on, is not a measure reliable enou)h to

)uarantee that an established correspondence is correct. As a final step of the reco)nition

process, a verification of presence of the model in the query ima)e is performed. A )lobal

transformation connectin) the ima)es is estimated in a robust ay je.). by usin) 1A2/A*

al)orithmk. 4ypically, the )lobal transformation has the form of epipolar )eometry constraint for

)eneral jbut ri)idk ;D obects, or of homo)raphy for planar obects. 9ore comple=

transformations can be derived for non+ri)id or articulated jpieceise ri)idk obects.

As mentioned before, if a detector cannot recover certain parameters of the ima)e

transformations, descriptor must be made invariant to them. It is preferable, thou)h, to have a

covariant detector rather than an invariant descriptor, as that allos for more poerful )lobal

consistency verification. If, for e=ample, the detector does not provide the orientations of the

ima)e elements, rotational invariants have to be employed in the descriptor. In such a case, it is

impossible to verify that all of the matched elements a)ree in their orientation.

Finally, tentative correspondences hich are not consistent ith the estimated )lobal

20


21/31

transformation are reected, and only remainin) correspondences are used to estimate the final

score of the match.

In the folloin), main contributions to the field of obect reco)nition based on local

correspondences are revieed. 4he approaches follo the aforementioned structure, but differ in

individual steps in the ay ho are the local features obtained jdetectorsk, and hat are the

features themselves jdescriptorsk %%.

1'`'Invariant %attern re"o$nition

O%ti"al invariant filter

4he invariance to rotation, scalin), vie an)le chan)e and partial occlusion of the input

ima)e remains as an important issue in all pattern reco)nition approaches. For optical correlator,

Hsu and Arsenault have proposed the circular harmonic matched filters in %87#, hich correlates

the input ima)e ith its one circular harmonic component. 4he synthetic discriminant function

may be desi)ned to produce equal correlation peaGs for all the 2input trainin) ima)es, includin)

all possibly distorted ima)es, for instance, 2 %# trainin) ima)es each rotated by ;$ de)rees#$.

Hoever, the number 2 of the distorted trainin) ima)es, hich may be composed into a sin)le

synthetic discriminant function, is limited. 9oreover, the ima)e distortions are often continuous,

such as the rotation an)le and scale of the ima)e are usually continuous. 4he discrete samplin)

of the ima)e continuous distortions can lead to a decrease of the output si)nal+to+noise ratio of

the composite filter.

S"ale Invariant Featre Transfor!

In the computer vision community, 0oe has proposed the /cale Invariant Feature

4ransform j/IF4k in #$$. 4he al)orithm is not desi)ned to e=tract and describe the Gey+points

in the ima)e. (ased on pairin) of the Geypoints detected in to ima)es of the same scene by to

calibrated cameras one can compute a ;D vision by the epipolar )eometry, or compute the

distortions beteen these to ima)es for ima)e re)istration and ima)e fusion. 4he /IF4 features

may be also used for obect classification and pattern reco)nition. 4he /IF4 is invariant to

translation, rotation, scalin) and illumination chan)es. It has been built by massive e=perimental

trial and testin), and proved to be minimally affected by noise and small distortion and

particularly efficient to process natural outdoor ima)es, compared ith other mathematically

based ima)e invariant features, such as the )eometrical ima)e moment invariants, hich are

mostly suitable for describin) #D )eometrical patterns. 4he /IF4 is based on the scale space

analysis ith the scale coordinate z discretiBed in lo)arithmic steps as G H GJiKsLSith the octave

21


22/31

scale inde= iand the sub+scale inde=s H MB J Sith the sub+scale resolution S, here G$H

MCOEJMLS is the basic scale. Firstly, the :aussian scale ima)e %PEB GQ and the Difference of

:aussian PDo%Qscale ima)esDPEB G PsBiQQof an input ima)eIPEQare computed as

:j=, zk j)zI kj=k and Dj=, z js,ikk :j=,z js {%,ikk | :j=, z js,ikk j%.6.%k

heregGis the :aussian Gernel of the idth G.

4he /IF4 Geypoints PEB GQare then e=tracted as local minimama=ima in the jDo:k scale

ima)es, compared ith their #< nei)hbour pi=els in the re)ions of ;=; pi=els at the current

subscale space s and the upscale PsKMQ and the donscale Ps-MQ adacent sub+scale spaces,

respectively. 4he Geypoints ith lo contrast and located alon) ed)es are ne=t discarded. 4hen,

the Geypoints PEBGQ is assi)ned by a dominant orientation as the ma=imum in an interpolated

histo)ram of the )radient orientations calculated in the :aussian scale ima)e %PEBGQithin a

:aussian support of idth MCRGaround the Geypoint. 4he contribution of each pi=el in the

:aussian support to the orientation histo)ram is ei)hted by its )radient ma)nitude and by the

:aussian function centered at the Geypoint. In addition to the location and the orientation, a /IF4

feature descriptor is computed for a Geypoint as a set of } histo)rams of )radient orientations.

5ach histo)ram is based on a mG mG pi=els re)ion ith mHTin the nei)hbourhood of the

Geypoint and consists of 7 orientation bins ith the ma)nitudes of that histo)ram entry. 4he

histo)rams are interpolated and ei)hted in the same ay as that for computin) the dominant

orientation. *onsequently, each /IF4 Geypoint is described by 7}} %#7 features, plus the

coordinates of its location, the scale of the ima)e and its dominant orientation.

(o!%onentX+ased a%%roa"&

4he component+based obect detection is another interestin) technique of pattern

reco)nition#;, hich can be not sensitive to ima)e distortions. As a hole obect appearance can

chan)e easily by chan)es in vie an)le, obect scale, orientation and illumination and by

occlusion, the component+based approach can provide a robust detection by first detectin) small

and characteristic parts of the obect and then combinin) the obect parts usin) an obect model.

Detection of small obect patches may be faster and less sensitive to the distortions, than that of

the hole obect. Also, the description of the obect components can be based on the invariant

ima)e features##. Furthermore, once the obect components are detected the obect model to

combine the obect parts to the obect can be deformable and is not sensitive to ima)e distortions.

4he al)orithm for re)roupin) the components to the obect accordin) to the unique spatial

relations amon) the parts can be robust to ima)e distortion and to the uncertainties in the

detection of the obect parts. 9ore importantly, the component+based obect detection does not

22


23/31

aim to detect one specific obect based on its appearance, but to detect a class of obects,

described by the same obect model. In addition, the approach is of cause not sensitive to the

occlusion of the obect. 4he component+based obect detection has been particularly successful

in the face reco)nition applications. Fi)ure % shos the detection of cars in the aerial ima)es by

the component+based approach. All the cars ere correctly classified re)ardless their orientations

and colors, e=cept a hite /U' hose components ere not present in the trainin) set for the

/'9 ##.

$igure MCRCM SiEt" eight ars in parUing-lot are deteted #ith onl" one SV missing

1'2' E9a!%les of o+,e"t dete"tion syste!s

T&e A%%roa"& of David LoeDavid 0oe has developed an obect reco)nition system, ith emphasis on efficiency,

achievin) real+time reco)nition times. Anchor points of interest are detected ith invariance to

scale, rotation and translation. /ince local patches under)o more complicated transformations

then similarities, a local+histo)ram based descriptor is proposed, hich is robust to imprecisions

in ali)nment of the patches.

Detetor. 4he detection of re)ions of interest proceeds as follos>

ak Detection of scale+space peaGs. *ircular re)ions ith ma=imal response of the difference+of+)aussians jDo:k filter, are detected at all scales and ima)e locations. 5fficient

implementation e=ploits the scale+space pyramid. 4he initial ima)e is repeatedly

convolved ith a :aussian filter to produce a set of scale+space ima)es. Adacent scale+

space ima)es are then subtracted to produce a set of Do: ima)es. In these ima)es, local

minima and ma=ima ji.e. e=trema of the Do: filter responsek are detected, both in spatial

and scale domains. 4he result of the first phase is thus a set of tripletsEB"and aBima)e

locations and a characteristic scales.

bk 4he location of the detected points is refined. 4he Do: responses are locally fitted ith

23


24/31

;D quadratic function and the location and characteristic scale of the circular re)ions are

determined ith subpi=el accuracy. 4he refinement is necessary, as, at hi)her levels of

the pyramid, a displacement by a sin)le pi=el mi)ht result in a lar)e shift in the ima)e

domain. Unstable re)ions are then reected, the stability is )iven by the ma)nitude of the

Do: response. 1e)ions ith the response loer than a predefined threshold are removed.

Further re)ions are discarded hich ere found alon) linear ed)es, hich, althou)h

havin) hi)h Do: response, have unstable localisation in one direction.

ck 3ne or more orientations are assi)ned to each re)ion. 0ocal histo)rams of )radient

orientations are formed and peaGs in the histo)ram determine the characteristic ori+

entations.

'he SI$' Desriptor. 0ocal ima)e )radients are measured at the re)ions characteristic

scale, ei)hted by the distance from the re)ion centre and combined into a set of orientation

histo)rams. Usin) the histo)rams, small misali)nments in the localisation does not affect the

final description. 4he construction of the descriptors allos for appro=imately #$~ ;D rotations

before the similarity model fails. At the end, every detected re)ion is represented by a %#7+

dimensional vector.

IndeEing. 4o support fast retrieval of database vectors, a modification of the GD tree

al)orithm, called ((F jbest bin firstk, is adopted. 4he al)orithm is appro=imate in the sense that

it returns the closest nei)hbour ith hi)h probability, or else another point that is very close in

distance to the closest nei)hbour. 4he ((F al)orithm modifies the UDtree al)orithm to search

bins in feature space in the order of their closest distance from the query location, instead of the

order )iven by the tree hierarchy.

Verifiation. 4he Hou)h transform is used to identify clusters of tentative

correspondences ith a consistent )eometric transformation. /ince the actual transformation is

appro=imated by a similarity, the Hou)h accumulator is +dimensionsional and is partitioned to

rather broad bins. 3nly clusters ith at least ; entries in a bin, are considered further. 5ach such

cluster is then subect to a )eometric verification procedure in hich an iterative least+squares

fittin) is used to find the best affineproection relatin) the query and database ima)es.

T&e A%%roa"& of Mibola,"7yb c S"&!id

(ased on an affine )eneralisation of Harris corner detector, anchor points are detected

and described by :aussian derivatives of ima)e intensities in shape+adapted elliptical

nei)hbourhoods.

Detetor. In their orG, 9iGolacByG and /chmid implement affine+adapted Harris point

24


25/31

detector. /ince the three+ parametric affine :aussian scale space is too comple= to be practically

useful, they propose a solution hich iteratively search for affine shape adaptation in

nei)hbourhoods of points detected in uniform scale space. For initialisation, appro=imate

locations and scales of interest points are e=tracted by standard multi+scale Harris detector. 4hese

points are not affine invariant because of the uniform :aussian Gernel used. :iven the initial

appro=imate solution, their al)orithm iteratively modifies the shape, the scale and the spatial

location of nei)hbourhood of each point, and conver)es to affine+invariant interest points. For

more details see.

Desriptors and Fathing. 4he descriptors are composed from :aussian derivatives

computed over the shape+ normalised re)ions. Invariance to rotation is obtained by "steerin)" the

derivatives in the direction of the )radient. Usin) derivatives up to th order, the descriptors are

%#dimensional. 4he similarity of descriptors is in first appro=imation measured by the

9ahalanobis distance. -romisin) close matches are then confirmed or reected by cross+

correlation measure computed over normalised nei)hbourhood indos.

Verifiation. 3nce the point+to+point correspondences are obtained, a robust estimation of

the )eometric transformation beteen the to ima)es is computed usin) 1A2/A* al)orithm.

4he transformation used is either a homo)raphy or a fundamental matri=.

1ecently, DorGo and /chmid e=tended the approach toards obect cate)orisation. 0ocal

ima)e patches are detected and described by the same approach as described above. -atches

from several e=amples of obects from a )iven cate)ory je.). carsk are collected to)ether, and a

classifier is trained to distin)uish them from patches of different cate)ories and from bacG)round

patches.

T&e A%%roa"& of Tytelaars# Ferrari c van Zool

0uc van :ool and his collaborators developed an approach based on matchin) of local

ima)e features. 4hey start ith detection of elliptical or parallelo)ram ima)e re)ions. 4he

re)ions are described by a vector of photometricaly invariant )eneralised colour moments, and

matchin) is typically verified by the epipolar )eometry constraint.

Detetor. 4o methods for e=traction of affinely invariant re)ions are proposed, yieldin)

)eometry+ and intensity+based re)ions. 4he re)ions are affine covariant, they adapt their shape to

the underlyin) intensity profile, in order to Geep on representin) the same physical part of an

obect. Apart from the )eometric invariance, photometric invariance allos for independent

scalin) and offsets for each of the three colour channels. 4he re)ion e=traction alays starts by

detectin) stable anchor points. 4he anchor points are either Harris points, or local e=trema of

25


26/31

ima)e intensity. Althou)h the detection of Harris points is not really affine invariant, as the

support set over hich is the response computed is circular, the points are still fairly stable under

viepoint chan)es, and could be precisely localised jeven to subpi=el accuracyk. Intensity

e=trema, on the other hand, are invariant to any continuous )eometric transformation and to any

monotonic transformation of the intensity, but are not localised as accurately. 3n colour ima)es,

the detection is performed three times, separately on each of the colour bands.

Desriptors and Fathing. In the case of )eometry+ based re)ions, each of the re)ions is

described by a vector of %7 )eneralised colour moments, invariant to photometric

transformations. For the intensity+based re)ions, 8 rotation+ invariant )eneralised colour

moments are used. 4he similarity beteen the descriptors is )iven by the 9ahalanobis distance,

correspondences beteen to ima)es are formed from re)ions ith the distance mutually

smallest. 3nce correspondin) re)ions have been found, the cross+correlation beteen them is

computed as a final checG before acceptin) the match. In the case of the intensity+based re)ions,

here the rotation is unGnon, the crosscorrelation is ma=imised over all rotations. :ood

matches are further fine+tuned by nonlinear optimisation> the crosscorrelation is ma=imised over

small deviations of the transformation parameters. Verifiation. 4he set of tentative

correspondences is pruned by both )eometric and photometric constraints. 4he )eometric

constraint basically reects correspondences contradictin) the epipolar )eometry. -hotometric

constraint assumes that there is alays a )roup of correspondin) re)ions that under)o the same

transformation of intensities. *orrespondences that have sin)ular photometric transformation are

reected. 1ecently, a )roin) fle=ible homo)raphy approach as presented, hich allos for

accurate model ali)nment even for nonri)id obects. 4he siBe of the ali)ned area is then used as a

measure of the match quality.

T&e LAF A%%roa"& of Matas et al'

4he approach of 9atas et al. starts ith detection of 9a=imally /table 5=tremal 1e)ions.

Affine covariant local coordinate systems jcalled 0ocal Affine Frames, 0AFsk are then

established, and measurements taGen relative to them describe the re)ions.

$igure MCOCM - WEamples of orrespondenes established bet#een frames of a database image

26


27/31

PleftQ and a Xuer" image PrightQC

Detetor. 4heFaEimall" Stable WEtremal Yegions j9/51sk ere introduced in. 4he attractive

properties of 9/51s are>

ak Invariance to affine transformations of ima)e coordinates,

bk Invariance to monotdonic transformation of intensity,

ck *omputational comple=ity almost linear in the number of pi=els and consequently near

real+time run time, and

dk /ince no smoothin) is involved, both very fine and coarse ima)e structures are detected.

/tartin) ith contours of the detected re)ion, local frames jcoordinate systemsk are

constructed in several affine covariant ays.

Affine covariant properties of covariance matri=, bi+tan)ent lines, and line parallelism are

e=ploited. As demonstrated in Fi)ure %, local affine frames facilitate normalisation of ima)e

patches into a canonical frame and enable direct comparison of photomet+ ricaly normalised

intensity values, eliminatin) the need for invariants.

Desriptor. 4hree different descriptors ere used. 4he first is directly the intensities of the local

patches. 4he intensities are discretised into %6 = %6 = ; rasters, yieldin)


28/31

;.#, second the 9a=imally /table 5=tremal 1e)ions from /ection ;.. 1epresentation of the

local appearance is realised by the /IF4 descriptors introduced by David 0oe jsee /ection ;.%k.

noin) that a motion video sequence is bein) processed, noisy and unstable re)ions can be

eliminated. 4he re)ions detected in each frame of the video are tracGed usin) a simple constant

velocity dynamic model and correlation. Any re)ion hich does not survive for more than three

frames is reected. 4he estimate of the descriptor for a re)ion is then computed by avera)in) the

descriptors throu)hout the tracG.

IndeEing and Fathing. 4he descriptors are )rouped into clusters, based on their

similarity. In analo)y to stop+lists in te=t retrieval, here common ords, liGe the, are i)nored,

lar)e clusters are eliminated. hen a ne ima)e is observed, each descriptor of the ne ima)e is

matched only a)ainst representants of individual clusters. /election of the nearest cluster

immediately )enerates matches for all frames of the cluster, throu)hout the hole movie. 4he

e=haustive comparison ith every descriptor of every frame is thus avoided. 4he similarity

measure, used for both the clusterin) and the closest cluster determination, is )iven by the

9ahalanobis distance of the descriptors.

Verifiation. 'ideo frames are first retrieved usin) the frequency of matched descriptors,

and then re+ranGed based on a measure of spatial consistency of the correspondences. 4he

matched re)ions provide affine transformation beteen the query and the retrieved ima)e, so a

point to point correspondence is locally available. A search area of each match is defined by fe

nearest nei)hbours. 3ther re)ions hich also match ithin this area casts a vote for that frame.

9atches ith no support are reected. 4he final ranG of the frame is determined by the total

number of votes %%.

4i+lio$ra%&y5

%. Astua *arlos, 1amon (arber, onathan *respo, Alberto ardon, 3bect Detection

4echniques Applied on 9obile 1obot /emantic 2avi)ation". http>.mdpi.com%#+

7##$% Fundamentals and *ase /tudies".

https > booGs .)oo)le .md booGs

id Fh-/(AA(A p) -A ;;


29/31

http>opticalen)ineerin).spiedi)itallibrary.or)article.asp=

articleid%$;%#result*licG%. Fen) -an, an) Eiaoun, an) eihon), 1esearch and 1estoration 4echnolo)y of 'ideo

9otion 4ar)et Detection (ased 3n ernel 9ethod".

http>s#is.or)Issuesvnpaperspaper.pdf

6. FernandeB oseph A., 'ishnu 2aresh (oddeti, Andres 1odri)ueB, (. '. . 'iaya umar,

wero+Aliasin) *orrelation Filters for 3bect 1eco)nition".

http>ar=iv.or)pdf%%%.#;%en.iGipedia.or)iGiFouriertransform7. :ranieri /er)io, 1icardo AriBa)a, 5nrique 5. /icre, 3ptical correlation based on the

fractional Fourier transform". https>.osapublishin).or)aoabstract.cfmuriao+;ieee=plore.ieee.or)=pllo)in.sp

tparnumber$7$#$urlhttp;A#F#Fieee=plore.ieee.or)#F=pls

#Fabsall.sp;Farnumber;D$7$#$%#. aliraan ., 9. /udha, 9ovin) 3bect Detection for 'ideo /urveillance".

http>.ncbi.nlm.nih.)ovpmcarticles-9*;6$;

%;. ypraios Ioannis, -erformance Analysis of the 9odified+Hybrid 3ptical 2eural 2etorG

3bect 1eco)nition /ystem ithin *luttered /cenes". http>cdn.intechopen.compdfs+m;proceedin)s.spiedi)itallibrary.or)proceedin).asp=articleid76#


30/31

% .ieee .li pdf vie)raphs tar)et detection usin) optical oint transform .pdf

%8.2aBrul Islam 9ohammed, 9ohammad Ataul arim, 9ohammad /hoGat Alam, .

'iayan Asari, 3ptical crypto)raphic system employin) multiple reference+based oint

transform correlation technique, 3pt. 5n). 6$j


31/31

#7. esley 5. Foor, Harold :. Andres II, *apt, U/AF, 9arG A. :etbehead, 3ptical

*orrelator 3ptimiBations And 5=tensions For Automated 3ptical 4ar)et 1eco)nition"

http>.dtic.milc)i+bin:et41DocADADA#8;$%;

#8. QW] KR ., [``W R RK PM `WQ_^QR KRM`L".

http > .findpatent .ru patent #66#66#%#

dalv license object detection

Documents