image analysis lecture 9.3 - introduction to machine learning€¦ · machine learning (pattern...

16
Image Analysis Lecture 9.3 - Introduction to Machine Learning Idar Dyrdal

Upload: others

Post on 23-Aug-2020

5 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Image Analysis Lecture 9.3 - Introduction to Machine Learning€¦ · Machine learning (Pattern recognition) • Recognition of individuals (instance recognition) • Discrimination

Image Analysis

Lecture 9.3 - Introduction to Machine Learning

Idar Dyrdal

Page 2: Image Analysis Lecture 9.3 - Introduction to Machine Learning€¦ · Machine learning (Pattern recognition) • Recognition of individuals (instance recognition) • Discrimination

Machine learning (Pattern recognition)

• Recognition of individuals (instance recognition)

• Discrimination between classes (pattern recognition, classification)

Training Labels

Training

ImagesClassifier Training

Supervised learning

Image Features

Trained Classifier

2

Page 3: Image Analysis Lecture 9.3 - Introduction to Machine Learning€¦ · Machine learning (Pattern recognition) • Recognition of individuals (instance recognition) • Discrimination

Pattern recognition in practice

Working applications of Image Pattern recognition:

• Reading license plates, postal codes, bar codes

• Character recognition

• Automatic diagnosis of medical samples

• Fingerprint recognition

• Face detection and recognition

• …

3

Page 4: Image Analysis Lecture 9.3 - Introduction to Machine Learning€¦ · Machine learning (Pattern recognition) • Recognition of individuals (instance recognition) • Discrimination

Classification system

4

Feature extractor Classifier

Pine

Birch

Camera

Page 5: Image Analysis Lecture 9.3 - Introduction to Machine Learning€¦ · Machine learning (Pattern recognition) • Recognition of individuals (instance recognition) • Discrimination

Image features for object recognition

5

Page 6: Image Analysis Lecture 9.3 - Introduction to Machine Learning€¦ · Machine learning (Pattern recognition) • Recognition of individuals (instance recognition) • Discrimination

Feature vector and feature space

6

Page 7: Image Analysis Lecture 9.3 - Introduction to Machine Learning€¦ · Machine learning (Pattern recognition) • Recognition of individuals (instance recognition) • Discrimination

Training of classifiers

7

Learn a function to predict the class from

the given featuresRegion 1

Region 2

Decision

boundary

Page 8: Image Analysis Lecture 9.3 - Introduction to Machine Learning€¦ · Machine learning (Pattern recognition) • Recognition of individuals (instance recognition) • Discrimination

Classifiers and training methods

• Bayes classifier

• Nearest-neighbors and K-nearest-neighbors

• Parzen windows

• Linear and higher order discriminant functions

• Neural nets

• Support Vector Machines (SVM)

• …

8

Page 9: Image Analysis Lecture 9.3 - Introduction to Machine Learning€¦ · Machine learning (Pattern recognition) • Recognition of individuals (instance recognition) • Discrimination

Class conditional probability density functions

9

Page 10: Image Analysis Lecture 9.3 - Introduction to Machine Learning€¦ · Machine learning (Pattern recognition) • Recognition of individuals (instance recognition) • Discrimination

Bayesian decision theory

Overview

10

x

Class conditional densities:

Prior probabilities:

Posterior probabilities given by Bayes rule:

Minimum error rate classification:

Assign the unknown object to the class with maximum posterior probability!

Page 11: Image Analysis Lecture 9.3 - Introduction to Machine Learning€¦ · Machine learning (Pattern recognition) • Recognition of individuals (instance recognition) • Discrimination

Density estimation

Parametric methods:

• Assume a given shape of the density function

• Use the training set to estimate the unknown parameters.

Non-parametric (distribution free) methods:

• Point estimation of the density using the training set directly

• Parzen windows

• Nearest neighbor estimation (leads directly to the nearest-neighbor and k-nearest-neighbor classifiers).

11

Example – Gaussian distribution:

Parameters:

Page 12: Image Analysis Lecture 9.3 - Introduction to Machine Learning€¦ · Machine learning (Pattern recognition) • Recognition of individuals (instance recognition) • Discrimination

Parameter estimation

12

Parameter estimates:

Page 13: Image Analysis Lecture 9.3 - Introduction to Machine Learning€¦ · Machine learning (Pattern recognition) • Recognition of individuals (instance recognition) • Discrimination

Discriminant functions

13

Estimate of the density in a given point:

From Bayes rule:

Examples of discriminant functions:

Decision rule:

Choose the class with maximum discriminant function value.

Page 14: Image Analysis Lecture 9.3 - Introduction to Machine Learning€¦ · Machine learning (Pattern recognition) • Recognition of individuals (instance recognition) • Discrimination

Example - linear classifier

14

Uncorrelated features and

common covariance matrices

Linear decision boundaries

Page 15: Image Analysis Lecture 9.3 - Introduction to Machine Learning€¦ · Machine learning (Pattern recognition) • Recognition of individuals (instance recognition) • Discrimination

Dimensionality reduction - linear transformations

• PCA, ICA, LLE, Isomap

• PCA (Principal Components Analysis)

is one of the most important techniques

for dimensionality reduction

• It takes advantage of correlations

between the features to produce the

best possible lower dimensional

representation of the data with respect

to reconstruction error

• The eigenvectors of the lumped

covariance matrix defines the new

features in the transformed feature

space.

15

Page 16: Image Analysis Lecture 9.3 - Introduction to Machine Learning€¦ · Machine learning (Pattern recognition) • Recognition of individuals (instance recognition) • Discrimination

Summary

Machine learning:

• Pattern classification

• Training of classifiers (supervised learning)

• Parametric and non-parametric methods

• Discriminant functions

• Dimensionality reduction

More information: Szeliski 14.1

16