ferrier_visionsys

8/7/2019 Ferrier_VisionSys

1/25

11

Introduction toIntroduction toMachine Vision SystemsMachine Vision Systems

Professor Nicola Ferrier Professor Nicola Ferrier Room 3128, ECBRoom 3128, ECB

[email protected]@engr.wisc.edu


2/25

21

Machine VisionMachine Vision

To become familiar with technologies usedfor machine vision as a sensor for robots. Camera and lighting technology (obtaining a

digital representation of an image)

Software (computational techniques to processor modify the image data)

Analysis/decisions: using the results of theprocessing in robot control

Additional material in CS766, ECE 533, ME739


3/25

31

Machine Vision in AutomationMachine Vision in Automation

Use a camera to inspect parts to : Guide a robot or control automated equipment

Support statistical analysis in a computer-assisted-manufacturing (CAM) system

Ensure quality in manufacturing process:

dimensions/alignment Determine if all components are present Other quality issues: color, placement,


4/25


5/25

51

Why avoid Vision?Why avoid Vision?

Computation must process images

data = information

Calibration Sensitivity to lighting

conditions

/ B ecause the lighting is different, these 3 imagesappear substantially different to a computer to

a human we easily adapt our perception for variations in illumination and recognize that allthree images are of the same object.

Images (arrays of pixel data) must be processed

to provide information


6/25

61

Example Application:Example Application:MicroMicro--manipulationmanipulation

Micro Object handlingwith Micro gripper

Postech Robotics Lab Micro gripperMicroscope Table


7/25

71

A machine vision system often includesA machine vision system often includesthe following elements:the following elements:

Image Acquisition (generally from a cameraplaced above the production line),

Image Pre-Processing (e.g. increasing the

contrast, motion de-blur, etc),Feature Extraction (e.g. measuring a distance,checking a screw is in place etc),

Decisions (i.e. is the part OK to a tolerance, is alabel in the correct position), and,

Control (e.g. give the result to a ProgrammableLogic Controller (PLC) or robot controller).


8/25

81

Image AcquisitionImage Acquisition

Transforms the visual image of a physicalobjects into a set of digitized data Illumination

Image formation (including focusing) Image detection or sensing

Formatting camera output signal


9/25

91

Image Formation and DetectionImage Formation and Detection

Image is formed by: Illumination flux

from object

Optics (lens)

Photosensitivedetectors(photodiodes onsolid statecameras)

Vision systems have an optical-electro device thatconverts electromagnetic radiation from the image of the physical object into an electric signal used by thevision processing unit


10/25

101

VisionVision Image FormationImage Formation

Sh apeL ig ht ingR ela t ive Posi t ionsS ensor sensi t ivi ty

S ame s h ape ver y differen t images!


11/25

111

LightingLighting Structured Lighting

Diffuse B acklighting

Directional backlighting

Fiber-optic/LED ringlights


12/25

121

LightingLighting

Polarized lighting

Oblique lighting

Direct front lighting

Cross polarization


13/25

131

LightingLighting

Diffuse front lighting

Dark field illumination

Fibre optic near in-lighting


14/25

141

Image Formation and DetectionImage Formation and Detection

Light source


15/25

151

Digitization of Camera SignalDigitization of Camera Signal

Analog image data (voltage) is sampled andquantized (often to 8 bits greyscale or 24 bits of color)


16/25

161

Software: Processing the DataSoftware: Processing the Data

The software allows the image to beprocessed, analyzed, and stored. Different types of software packages are available, ranging from

easy-to-use packages with pre-defined tools, to SDKs (softwaredevelopment kits) that allow programmers to build custom imagingapplications.

Matlab has an image processing tool box Image Pre-processing Feature Extraction


17/25

171

Image PreImage Pre- -processingprocessing

What to do with the image? May need to preprocess the image in order to

analyze it

Remove motion blur (ECE 533/738) Enhance contrast


18/25

181

I Can See ItI Can See It Why cant the Computer?Why cant the Computer? Minimize possible problems The human eye and brain are elaborate and

versatile systems, capable of identifying objects in a wide variety of conditions.For example, we are able to identify familiar people even when they arewearing different clothes, and recognize familiar landmarks when driving on afoggy day. A PC-based imaging system is not as versatile; it can only performwhat it has been programmed to perform. Knowing what the system can andcannot "see" are important points to keep in mind to obtain the results youwant, and reduce errors and incorrect measurements. Common variablesinclude:

y Changes in objects color

y Changes in surrounding lighting

y Changes in camera focus or position

y

Improperly mounted cameray Environmental vibration

A vibration-free environment with all extraneous light removed will eliminatemany common problems.


19/25

191

Find the man.Find the man.

V isual t asks can be made difficul t !


20/25

201

Distractors

N a t ural s ys t ems t akeadvan t age of th e fac t th a t

visual t asks can be madedifficul t !


21/25

211

I Can See ItI Can See It Why cant the Computer?Why cant the Computer?

Minimize possible problems

Knowing what the system can and cannot "see"are important points to keep in mind to obtainthe results you want, and reduce errors and

incorrect measurements.

Eng i nee r th e

enviro

nment!

G rea t examples includecommercial mo t ion cap t ures ys t ems


22/25

221

Feature Extraction/AnalysisFeature Extraction/Analysis

2D Geometric Analysis: Must have high contrast to separate (segment)

part from background

In practice back lighting is often used The silhouette is used to determine:

part dimensions: Width, height, orientation, etc

Part features (e.g. number of holes)

Relationships between parts


23/25

231

Controlled EnvironmentControlled Environment

E as y t o segmen t image


24/25

241

Measurements from ImagesMeasurements from Images

Must have relationshipbetween the imagepixels and the world

2D imaging the image plane and the

world plane are in 1-1correspondence

3D harder


25/25

251

Goals for ME 4 39 and ME 739Goals for ME 4 39 and ME 739

Modeling Cameras B asic of pinhole

Kinematics of Vision Coordinatetransformations

Processing Images Some simple features

(sections 8.13 - 8.25)

2D problems

Modeling Cameras Pinhole model

Projective mapping

Calibration Procedures

Kinematics of Vision Coordinate transformations

Motion field equations

Processing Images Feature detection (lines,

blobs)

Visual Servoing (Eye-HandCoordination) in 3D

ferrier_visionsys

Documents