machine vision - intelligence: lab description vision • the goal of machine vision is to create a...

TECHNICAL UNIVERSITY OF CRETEDEPARTMENT OF ELECTRONIC AND COMPUTER

ENGINEERING

MACHINE VISION

E.G.M. Petrakis Machine Vision (Introduction) 1

MACHINE VISIONEuripides G.M. Petrakis

Michalis Zervakis

http://www.intelligence.tuc/~petrakis

http://courses.ece.tuc.gr

Chania 2010

Machine Vision

• The goal of Machine Vision is to create a

model of the real world from images

– A machine vision system recovers useful


– A machine vision system recovers useful

information about a scene from its two

dimensional projections

– The world is three dimensional

– Two dimensional digitized images

Machine Vision (2)

• Knowledge about the objects (regions) in a scene and projection geometry is required.

• The information which is recovered differs depending on the application


• The information which is recovered differs depending on the application

– Satellite, medical images etc.

• Processing takes place in stages:

– Enhancement, segmentation, image analysis and matching (pattern recognition).

Illumination

2D Image

Image

Acquisition

Machine

Vision System

Scene2D

Digital Image

Image

Description

Feedback

The goal of a machine vision system is to compute a

meaningful description of the scene (e.g., object)

Machine Vision Stages

• Analog to digital conversion

• Remove noise/patterns, improve contrast

• Find regions (objects) in

Image Acquisition

(by cameras, scanners etc)

Image Processing

Image Enhancement

Image Restoration


• Find regions (objects) in the image

• Take measurements of objects/relationships

• Match the above description with similar description of known objects (models)Model Matching

Pattern Recognition

Image Analysis

(Binary Image Processing)

Image Segmentation

Image Processing

Image Processing

Input Image Output Image


• Image transformation

– image enhancement (filtering, edge detection, surface detection,

computation of depth).

– Image restoration (remove point/pattern degradation: there exist a

mathematical expression of the type of degradation like e.g. Added

multiplicative noise, sin/cos pattern degradation etc).

Image Segmentation

Image Segmentation

Input Image Regions/Objects


• Classify pixels into groups (regions/objects of interest) sharing common characteristics.

– Intensity/Color, texture, motion etc.

• Two types of techniques:

– Region segmentation: find the pixels of a region.

– Edge segmentation: find the pixels of its outline contour.

Image Analysis

Image Analysis

Input Image

Segmented Image

(regions, objects)

Measurements


• Take useful measurements from pixels, regions, spatial

relationships, motion etc.

– Grey scale / color intensity values;

– Size, distance;

– Velocity;

(regions, objects)

Pattern Recognition

Model Matching

Pattern Recognition

Image/regions �

•Measurements, or

•Structural description

Class identifier


• Classify an image (region) into one of a number of known classes

– Statistical pattern recognition (the measurements form vectors which are classified into classes);

– Structural pattern recognition (decompose the image into primitive structures).

•Structural description

Digital Image Representation

• Image: 2D array of gray level or color values

– Pixel: array element;

– Pixel value: arithmetic value of gray level or color

intensity.


intensity.

• Gray level image: f = f(x,y)

- 3D image f=f(x,y,z)

• Color image (multi-spectral)

f = {Rred(x,y), Ggreen(x,y), Bblue(x,y)}

What a computer “sees” is very different from what

a human sees. A computer sees pixels (arithmetic values)

while a human sees shapes, structures etc.


Relationships to other fields

• Image Processing (IP)

• Pattern Recognition (PR)

• Computer Graphics (CG)


• Computer Graphics (CG)

• Artificial Intelligence (AI)

• Neural Networks (NN)

• Psychophysics

Image Processing (IP)

• IP transforms images to images

– Image filtering, compression, restoration

• IP is applied at the early stages of machine


• IP is applied at the early stages of machine

vision.

– IP is usually used to enhance particular

information and to suppress noise.

Pattern Recognition (PR)

• PR classifies numerical and symbolic data.

– Statistical: classify feature vectors.

– Structural: represent the composition of an


– Structural: represent the composition of an

object in terms of primitives and parse this

description.

• PR is usually used to classify objects but

object recognition in machine vision usually

requires many other techniques.

Statistical Pattern Recognition

• Pattern: the description of an an object

– Feature vector

– (size, roundness, color, texture)


• Pattern class: set of patterns with similar characteristics.

• Take measurements from a population of patterns.

• Classification: Map each pattern to a class.

Structure of PR Systems

input

Sensor

Processing


Measurements

Processing

Classification

class

Example of Statistical PR

• Two classes:

I. W1 Basketball players

II. W2 jockeys

• Description: X = (X1, X2) = (height, weight)

X


X1

X2

. . .

. . .. .

. … ..

… … .. ..

.. ……W2

W1

D(X) = AX1 + BX2 + C = 0

Decision function-

+

Syntactic Pattern Recognition

• The structure is important

• Identify primitives

– E.g., Shape primitives

• Break down an image (shape) into a sequence of


• Break down an image (shape) into a sequence of such primitives.

• The way the primitives are related to each other to form a shape is unique.

– Use a grammar/algorithm

– Parse the shape

•Primitives


•G1,L(G1) : submedian Grammar

•G2,L(G2) : telocentric Grammar

•Each digit is represented by a waveform representing

black/white, white/black transitions (scan the image from

Left to right.


Computer Graphics (CG)

• Machine vision is the analysis of images

while CG is the decomposition of images:

– CG generates images from geometric primitives


– CG generates images from geometric primitives

(lines, circles, surfaces).

– Machine vision is the inverse: estimate the

geometric primitives from an image.

• Visualization and virtual reality bring these

two fields closer.

Artificial Intelligence (AI)

• Machine vision is considered to be sub-field of AI.

• AI studies the computational aspects of intelligence.

• CV is used to analyze scenes and compute


• CV is used to analyze scenes and compute symbolic representations from them.

• AI: perception, cognition, action– Perception translates signals to symbols;

– Cognition manipulates symbols;

– Action translates symbols to signals that effect the world.

Psychophysics

• Psychophysics and cognitive science have

studied human vision for a long time.

• Many techniques in machine vision are


• Many techniques in machine vision are

related to what is known about human

vision.

Neural Networks (NN)

• NNs are being increasingly applied to solve

many machine vision problems.

• NN techniques are usually applied to solve


• NN techniques are usually applied to solve

PR tasks.

– Image recognition/classification.

• They have also applied to segmentation and

other machine vision tasks.

Machine Vision Applications

• Robotics

• Medicine

• Remote Sensing


• Remote Sensing

• Cartography

• Meteorology

• Quality inspection

• Reconnaissance

Robot Vision


• Machine vision can make a robot manipulator

much more versatile.

– Allow it to deal with variations in parts position and

orientation.

Remote Sensing

• Take images from

high altitudes (from

aircrafts, satellites).

• Find ships in the aerial


• Find ships in the aerial

image of the dock.

– Find if new ships have

arrived.

– What kind of ships?

Remote Sensing (2)

• Analyze the image

– Generate a description

– Match this descriptions

with the descriptions of


with the descriptions of

empty docs

• There are four ships

– Marked by “+”

Medical Applications

• Assist a physician to

reach a diagnosis.

• Construct 2D, 3D

anatomy models of the


anatomy models of the

human body.

– CG geometric models.

• Analyze the image to

extract useful features.

Machine Vision Systems

• There is no universal machine vision system

– One system for each application

• Assumptions:


– Good lighting;

– Low noise;

– 2D images

• Passive - Active environment

– Changes in the environment call for different actions

(e.g., turn left, push the break etc).

Vision by Man and Machine

• What is the mechanism of human vision?

– Can a machine do the same thing?

– There are many studies;


– Most are empirical.

• Humans and machines have different

– Software

– Hardware

Human “Hardware”

• Photoreceptors take measurements of light signals.

– About 106 Photoreceptors.

• Retinal ganglion cells transmit electric and


chemical signals to the brain

– Complex 3D interconnections;

– What the neurons do? In what sequence?

– Algorithms?

• Heavy Parallelism.

Machine Vision Hardware

• PCs, workstations etc.

• Signals: 2D image arrays gray level/color values.

• Modules: low level processing, shape from


• Modules: low level processing, shape from

texture, motion, contours etc.

• Simple interconnections.

• No parallelism.

Course Outline

• Introduction to machine vision, applications,

Image formation, color, reflectance, depth,

stereopsis.

• Basic image processing techniques (filtering,


• Basic image processing techniques (filtering,

digitization, restoration), Fourier transform.

• Binary image processing and analysis, Distance

transform, morphological operators.

Course Outline (2)

• Image segmentation (region segmentation, edge segmentation).

• Edge detection, edge enhancement and linking. Thresholding, region growing, region


linking. Thresholding, region growing, region merging/splitting.

• Relaxation labeling, Hough transform.

• Image analysis, shape analysis. Polygonal approximation, splines, skeletons. Shape features, multi-resolution representations.

Course Outline (3)

• Image representation, image - shape recognition

and classification. Attributed relational graphs,

semantic nets.

• Image - shape matching (Fourier descriptors,


• Image - shape matching (Fourier descriptors,

moments, matching in scale space).

• Texture representation and recognition, statistical

and structural methods.

• Motion, motion detection, optical flow.

• Video

Bibliography

• “Machine Vision”, Ramesh Jain, Rangachar Kasturi, Brian G. Schunck, Mc Graw-Hill, 1995 (highly recommended!).

• "Image Processing, Analysis and Machine


• "Image Processing, Analysis and Machine Vision", Milan Sonka, Vaclav Hlavac, Roger Boyle, PWS Publishing, Second Edition.

• "Machine Vision, Theory, Algorithms, Practicalities'', E. R. Davies, Academic Press, 1997.

• "Practical Computer Vision Using C'', J.

R. Parker, John Wiley & Sons Inc., 1994.

• Selected articles from the literature.


• Selected articles from the literature.

• Lecture notes

(http://www.intelligence.tuc/~petrakis)

• Webcourses (http://courses.ece.tuc.gr)

Grading Scheme

• Final Exam (F): 40%, min 5

• Assignments (Α): 40%

• Two assignments


• Two assignments

– Obligatory

machine vision - intelligence: lab description vision • the goal of machine vision is to create a...

Documents