Chapter 4: Pattern Chapter 4: Pattern RecognitionRecognition

• Classification is a process that assigns a label to an object according to some representation of the object’s properties.

• Classifier is a device or algorithm that inputs an object representation and outputs a class label.

• Reject class is a generic class for objects that cannot be placed in any of the designated known classes.

ClassificationClassification

Error & Reject rateError & Reject rate

• Empirical error rate of a classification system is the number of errors made on independent test data divided by the number of classifications attempted.

• Empirical reject rate of a classification system is the number of rejects made on independent test data divided by the number of classifications attempted.

False Alarm & Miss False Alarm & Miss DetectionDetection

Two-class problem example: whether a person has disease or not

• False Alarm (False Positive): The system incorrectly says that the person does have disease.

• Miss Detection (False Negative): The system incorrectly says that the person does not have disease.

Receiver Operating Curve Receiver Operating Curve (ROC)(ROC)

Precision & RecallPrecision & Recall

• Example: the objective of document retrieval (image retrieval) is to retrieve interesting objects and not too many uninteresting objects according to features supplied in a user’s query.

• Precision is the number of relevant documents retrieved divided by the total number of documents retrieved.

• Recall is the number of relevant documents retrieved by the system divided by the total number of relevant documents in the database.

ExampleExample

• Suppose an image database contains 200 sunset images.• Suppose an automatic retrieval system retrieves 150 of those 200 relevant images and 100 other images.

Features used for Features used for representationrepresentation

• Area of the character in units of black pixels• Height and Width of the bounding box of its pixels• Number of holes inside the character• Number of strokes forming the character• Center (Centroid) of the set of pixels• Best axis direction (Orientation) through the pixels as the axis of least inertia• Second moments of the pixels about the axis of least inertia and most inertia

Example FeaturesExample Features

Classification using nearest Classification using nearest meanmean

Euclidean DistanceEuclidean Distance

ExampleExample

Classification using Classification using nearest neighborsnearest neighbors

• A brute force approach computes the distance from x to all samples in the database and remembers the minimum distance. Then, x is classified into the same category as its nearest neighbor.

• Advantage new labeled samples can be added to the database at any time.

• A better approach is the k nearest neighbors rule.

Structural Pattern Structural Pattern RecognitionRecognition

Graph matching algorithm can be used to perform structural pattern recognition.

Two characters with the same Two characters with the same global features but different global features but different structurestructure

Lid : a virtual line segment that closes up a bay.Left : specifies that one lid lies on the left of another.Right : specifies that one lid lies on the right of another.

Confusion MatrixConfusion Matrix Reject Rate =Error Rate =

Decision Tree 1Decision Tree 1

Decision Tree Decision Tree 22

Automatic Construction Automatic Construction of a Decision Treeof a Decision Tree

Information content I(C;F) of the class variable C with respect to the feature variable F is defined by

The feature variable F with maximum I(C,F) will be selected as the first feature to be tested.

m

i

d

j ji

jiji fFPcCP

fFcCPfFcCPFCI

1 12 )()(

),(log),();(

ExampleExample

I(C,X) =

I(C,Y) =

I(C,Z) =

General CaseGeneral Case

At any branch node of the tree At any branch node of the tree when the selected feature does not when the selected feature does not completely separate a set of completely separate a set of training samples into the proper training samples into the proper classes classes the tree construction the tree construction algorithm is invoked recursively for algorithm is invoked recursively for the subsets of training samples at the subsets of training samples at each of the child nodes.each of the child nodes.

BayesiBayesian an DecisioDecision n MakingMaking

Bayesian classifierBayesian classifier

• Bayesian classifier classifies an object into the class to which it is most likely to belong based on the observed features.

• In other words, it makes the classification decision wi for the maximum

• p(x) is the same for all the classes, so compare p(x|wi)P(wi) is enough.

• Poisson, Exponential, and Normal (Gaussian) distributions are commonly used for p(x|wi).

)(

)()|()|(

xp

wPwxpxwP ii

i