machine vision sample

LabVIEWTM Machine Visionand Image ProcessingCourse Manual

Course Software Version 8.5April 2008 EditionPart Number 321998F-01

LabVIEW Machine Vision and Image Processing

Copyright

© 1998-2008 National Instruments Corporation. All rights reserved. Under the copyright laws, this publication may not be reproduced or transmitted in any form, electronic or mechanical, including photocopying, recording, storing in an information retrieval system, or translating, in whole or in part, without the prior written consent of National Instruments Corporation.

National Instruments respects the intellectual property of others, and we ask our users to do the same. NI software is protected by copyright and other intellectual property laws. Where NI software may be used to reproduce software or other materials belonging to others, you may use NI software only to reproduce materials that you may reproduce in accordance with the terms of any applicable license or other legal restriction.

TrademarksNational Instruments, NI, ni.com, and LabVIEW are trademarks of National Instruments Corporation. Refer to the Terms of Use section on ni.com/legal for more information about National Instruments trademarks.

Other product and company names mentioned herein are trademarks or trade names of their respective companies.

Members of the National Instruments Alliance Partner Program are business entities independent from National Instruments and have no agency, partnership, or joint-venture relationship with National Instruments.

PatentsFor patents covering National Instruments products, refer to the appropriate location: Help»Patents in your software, the patents.txt file on your CD, or ni.com/legal/patents.

Sam

ple

Worldwide Technical Support and Product Informationni.com

National Instruments Corporate Headquarters11500 North Mopac Expressway Austin, Texas 78759-3504 USA Tel: 512 683 0100

Worldwide Offices

Australia 1800 300 800, Austria 43 662 457990-0, Belgium 32 (0) 2 757 0020, Brazil 55 11 3262 3599, Canada 800 433 3488, China 86 21 5050 9800, Czech Republic 420 224 235 774, Denmark 45 45 76 26 00, Finland 358 (0) 9 725 72511, France 01 57 66 24 24, Germany 49 89 7413130, India 91 80 41190000, Israel 972 3 6393737, Italy 39 02 41309277, Japan 0120-527196, Korea 82 02 3451 3400, Lebanon 961 (0) 1 33 28 28, Malaysia 1800 887710, Mexico 01 800 010 0793, Netherlands 31 (0) 348 433 466, New Zealand 0800 553 322, Norway 47 (0) 66 90 76 60, Poland 48 22 3390150, Portugal 351 210 311 210, Russia 7 495 783 6851, Singapore 1800 226 5886, Slovenia 386 3 425 42 00, South Africa 27 0 11 805 8197, Spain 34 91 640 0085, Sweden 46 (0) 8 587 895 00, Switzerland 41 56 2005151, Taiwan 886 02 2377 2222, Thailand 662 278 6777, Turkey 90 212 279 3031, United Kingdom 44 (0) 1635 523545

For further support information, refer to the Additional Information and Resources appendix. To comment on National Instruments documentation, refer to the National Instruments Web site at ni.com/info and enter the info code feedback.

Sam

ple

© National Instruments Corporation iii LabVIEW Machine Vision and Image Processing

Contents

Student GuideA. Course Description ...............................................................................................vB. What You Need to Get Started .............................................................................viC. Installing the Course Software..............................................................................viiD. Course Goals.........................................................................................................viiE. Course Conventions ..............................................................................................viii

Lesson 1Introduction to Machine Vision

A. National Instruments Machine Vision ..................................................................1-2B. NI Vision Products................................................................................................1-2C. Measurement & Automation Explorer .................................................................1-6

Lesson 2Preparing Your Imaging Environment

A. Preparing Your Imaging Environment .................................................................2-2B. Selecting a Camera ...............................................................................................2-11

Lesson 3Acquiring and Displaying Images

A. Acquisition Modes ................................................................................................3-2B. Property Nodes .....................................................................................................3-35C. Triggering .............................................................................................................3-45

Lesson 4Processing Images

A. NI Vision VIs........................................................................................................4-2B. Prototyping Applications with NI Vision Assistant .............................................4-3

Lesson 5Enhancing Acquired Images

A. Using Spatial Calibration......................................................................................5-2B. Calibrating Images with NI Vision.......................................................................5-3C. Calibrating Your Imaging Setup...........................................................................5-4D. Using Spatial Filters..............................................................................................5-13Sa

mpl

e

Contents

LabVIEW Machine Vision and Image Processing iv ni.com

Lesson 6Measuring Features

A. NI Vision Machine Vision VIs .............................................................................6-2B. Regions of Interest ................................................................................................6-2C. Nondestructive Overlays.......................................................................................6-8D. Edge Detection......................................................................................................6-9

Lesson 7Using Machine Vision Techniques

A. Pattern Matching...................................................................................................7-2B. Geometric Matching .............................................................................................7-6C. Coordinate Systems ..............................................................................................7-20

Lesson 8Processing Binary Images

A. Collecting Image Information with Histograms ...................................................8-2B. Thresholds.............................................................................................................8-4C. Morphology ..........................................................................................................8-11D. Making Particle Measurements ............................................................................8-15E. Using the Golden Template ..................................................................................8-25

Lesson 9Identifying Images

A. Binary Particle Classification ...............................................................................9-2B. Optical Character Recognition..............................................................................9-5C. 2D Barcode Functions ..........................................................................................9-17

Appendix AUsing Color Tools

A. Introduction to Color ............................................................................................A-2B. Using Color Tools.................................................................................................A-3

Appendix BAdditional Information and Resources

Glossary

Course Evaluation Sam

ple

© National Instruments Corporation 5-1 LabVIEW Machine Vision and Image Processing

5Enhancing Acquired Images

In this lesson, you will learn about calibration and filtering. Spatial calibration is the process of computing pixel to real-world unit transformations while accounting for errors inherent to the imaging setup. Calibrating your imaging setup is important when you need to make accurate measurements in real-world units.

Spatial filters serve a variety of purposes, such as detecting edges along a specific direction, contouring patterns, reducing noise, and detail outlining or smoothing. Filters smooth, sharpen, transform, and remove noise from an image so that you can extract the information you need.

Topics

A. Using Spatial Calibration

B. Calibrating Images with NI Vision

C. Calibrating Your Imaging Setup

D. Using Spatial Filters

Sam

ple

Lesson 5 Enhancing Acquired Images

LabVIEW Machine Vision and Image Processing 5-2 ni.com

A. Using Spatial CalibrationAn image contains information in the form of pixels. Spatial calibration allows you to translate a measurement from pixel units into physical units. This conversion can be a simple linear conversion between pixels and real-world units. For example, if the pixel to inch ratio is 1:1, a length measurement of ten pixels is equivalent to ten inches.

However, this conversion may be nonlinear because of perspective errors and lens distortion. In Figure 5-1a, the camera is in the ideal position: perpendicular to the image plane. If the camera is not perpendicular to the image plane, as shown in Figure 5-1b, the image results can have perspective errors and lens distortion errors.

Figure 5-1. Reasons for Calibrating Images

Perspective errors and lens errors cause images to appear distorted. This distortion misplaces information in an image, but it does not necessarily destroy the information in the image. Calibration accounts for possible errors by constructing mappings that you can use to convert between pixel and real-world units. You can also use the calibration information to correct perspective errors and nonlinear distortion errors in image displays and shape measurements.

Use the NI Vision calibration tools to perform the following operations:

• Calibrate your imaging setup automatically by learning a standard pattern (calibration template) or by providing reference points. A calibration template is a user-defined grid of circular dots.

• Apply a learned calibration mapping to correct an acquired image.

1 Lens Distortion 2 Perspective Error 3 Known Orientation Offset

3

2

1

a. b.

Sam

ple



• Assign an arbitrary coordinate system to measure positions in real-world units.

• Convert measurements (lengths, widths, areas) from real-world units to pixel units and back.

B. Calibrating Images with NI VisionYou can use NI Vision VIs to convert pixel coordinates to real-world coordinates in a calibrated image. In addition, you can transform a distorted image into an image in which distortions are corrected. NI Vision also allows you to save and load calibrated images for processing.

NI Vision has two types of image calibration: perspective calibration and nonlinear calibration. Perspective calibration corrects for perspective errors and nonlinear calibration corrects for perspective errors and nonlinear distortion.

Figure 5-2 illustrates the types of errors your image can exhibit. Figure 5-2a shows a grid of dots with no errors. Figure 5-2b illustrates perspective errors caused by a camera imaging the grid from an angle. Figure 5-2c illustrates the effect of lens distortion on the grid of dots. A typical camera lens introduces radial distortion, which causes points that are away from the lens’s optical center to appear further away from the center than they really are.

Figure 5-2. Perspective and Distortion Errors

Use perspective calibration when your system exhibits perspective errors only. Use nonlinear calibration when your system exhibits nonlinear lens distortion. If your system exhibits perspective errors and nonlinear distortion, use nonlinear calibration to correct for both. Applying perspective calibration is less computationally intensive than nonlinear calibration. However, perspective calibration is not designed to handle highly nonlinear distortions.

a. No Distortion c. Nonlinear Distortionb. Perspective Projection

Sam

ple



Perspective calibration computes one pixel to real-world mapping for the entire image. You can use the mapping to convert the coordinates of any pixel in the image to real-world units.

Nonlinear calibration computes pixel to real-world mappings in a rectangular region centered around each dot in the calibration template. NI Vision estimates the mapping information around each dot based on its neighboring dots. You can convert pixel units to real-world units within the area covered by the grid dots. Because NI Vision computes the mappings around each dot, only the area in the image covered by the grid dots is calibrated accurately.

C. Calibrating Your Imaging SetupThe following general steps explain how to calibrate your imaging setup using a calibration template.

1. Create a calibration template appropriate for your field of view.

Note National Instruments provides a calibration template that you can use to calibrate your image. However, this template may not be appropriate for all applications. Consider the size of your object under inspection, as well as whether or not you need a calibration template that has a certificate of accuracy. You can purchase highly accurate calibration templates from optics suppliers, such as Edmund Optics.

2. Acquire an image of the calibration template using your current imaging setup.

3. Enter the acquired image, the distances between the dots on the calibration template, and the location and orientation of the coordinate system to the IMAQ Learn Calibration Template VI. This VI produces a calibrated image.

4. Acquire an image of the object of interest without the calibration template.

5. Apply the calibration information to the acquired image by copying it from the calibrated image. The IMAQ Set Calibration Info VI provides the new image with the calibration transform equations.

6. Apply the calibration information to the pixel measurements using one of these three methods:

• Use the IMAQ Convert Pixel to Real World VI to correct individual pixels for distance or edge locations.

• Use the IMAQ Particle Analysis VI to return real-world measurements on the calibrated image.Sam

ple



• Use the IMAQ Correct Calibrated Image VI to correct the calibrated image by applying a calibration template. This produces a spatially correct image that you can use for particle or area analysis.

You have the option of generating an error map. An error map returns an estimate of the worst-case error when a pixel coordinate is transformed into a real-world coordinate.

Use the calibration information obtained from the calibration process to convert any pixel coordinate to its real-world coordinate and back.

Common Calibration MisconceptionsYou cannot calibrate images under poor lighting or insufficient resolution conditions. Also, calibration does not affect image accuracy, which is subject to your camera and lens selections. The following are some common calibration misconceptions:

• Calibration fixes any measurement to an arbitrary accuracy.

• Calibrated images always need to be corrected.

• Calibration can compensate for poor lighting or unstable conditions.

Sam

ple



Exercise 5-1 Calibration and Perspective Correction

GoalUse NI Vision calibration and correction tools to solve a perspective or lens distortion problem.

ScenarioMany machine vision applications are completely useless if they cannot report information in real-world units. NI Vision calibration functions can calibrate pixel separation in your images to a real-world distance.

Lens distortion and perspective distortion are also common problems found in image acquisition. If careful consideration is not taken, measurement accuracy will vary according to the location of the object in your image. NI Vision calibration functions can account for distortion factors and correction functions can adjust the image accordingly.

DescriptionIn this exercise, you will create a script in Vision Assistant to correct lens distortion and examine an example program to observe the perspective calibration process in LabVIEW.

ImplementationComplete both parts of this exercise.

Correcting Lens Distortion using the Vision Assistant Express VI

1. Open a blank VI.

2. Save the VI as Lens Distortion Calibration.vi in the <Exercises>\LabVIEW Machine Vision\Calibration directory.

3. Acquire an image.

❑ Place the Vision Acquisition Express VI (Vision and Motion»Vision Express»Vision Acquisition) on the block diagram.

❑ In the NI Vision Acquisition Express configuration window, select Simulated Acquisition»Folder of Images in the left-hand pane.

❑ Click Next.

❑ Select Single Acquisition with processing for the acquisition type.

❑ Click Next.

Sam

ple



❑ Click the browse button next to the Image Path textbox.

❑ Navigate to <Exercises>\LabVIEW Machine Vision\Calibration and Perspective Correction\ELP mug.png and click OK.

❑ Click Test to test the acquisition.

❑ Click Finish to finish building the express VI.

❑ On the front panel, right-click the Image Display indicator and select Snapshot.

4. Use the ELP cal template grid to calibrate the image to account for nonlinear lens distortion.

❑ Place a Vision Assistant Express VI (Vision and Motion»Vision Express»Vision Assistant) on the block diagram.

❑ In the NI Vision Assistant configuration window, select File»Open Image.

❑ Browse to <Exercises>\LabVIEW Machine Vision\Calibration and Perspective Correction, open the file ELP mug.png and click OK. If the Vision Assistant prompts you to remove previously acquired images, select Yes.

Sam

ple



Figure 5-3. Vision Assistant Express VI Configuration Window

❑ Select Processing Functions: Image»Image Calibration. This opens the Choose a calibration type window.

❑ Select Grid Calibration and click OK. The Grid Calibration Setup window opens.

❑ Click Open Image and double-click the file ELP cal template.png in the <Exercises>\LabVIEW Machine Vision\Calibration and Perspective Correction directory.

❑ Click the Zoom Out button in order to see the entire image.

❑ Select Nonlinear for the Distortion type.

❑ Click Next.

❑ Enter 0 for the Threshold Range Min and enter 110 for the Max. This setting allows the algorithm to find most of the grid dots without letting noise particles through.

Sam

ple



❑ Click Next.

❑ Enter 0.375 for the X-Spacing and enter 0.375 for the Y-Spacing.

❑ Set Unit to centimeter.

❑ Click Next.

❑ In the Axis Origin parameter, enter 0 for X and enter 0 for Y.

❑ Set the Axis Reference to Indirect, as shown in Figure 5-4a.

The calibration procedure automatically determines the direction of the horizontal axis. The vertical axis direction can either be indirect or direct as shown in Figure 5-4.

Figure 5-4. Axis Direction

Figure 5-5. Grid Calibration Setup

X

YX

Y

a. Indirect b. Direct

Sam

ple



❑ Click OK.

❑ Save the calibrated image as ELP Calibrated.png in the <Exercises>\LabVIEW Machine Vision\Calibration and Perspective Correction directory and click OK.

❑ Click OK in the Image Calibration Setup section.

Note Although the image perspective has not been corrected, the image perspective is fully calibrated at this point to accommodate lens distortion. You can take measurements in real-world units, and the results will be spatially correct.

5. Correct the image perspective. The text in the image will appear without curvature.

❑ Select Processing Functions: Image»Image Correction.

❑ Click OK in the Image Correction Setup section.

❑ Click Finish in the NI Vision Assistant window.

Figure 5-6. Calibrated Image

Sam

ple



6. Finish building the block diagram shown in Figure 5-7.

Figure 5-7. Lens Distortion Calibration VI Block Diagram

7. Add image management and error handling to the VI.

❑ Place a Flat Sequence Structure (Programming»Structures»Flat Sequence Structure) around everything on the block diagram.

❑ Right-click the Flat Sequence Structure and select Add Frame After.

❑ Place an IMAQ Dispose VI (Vision and Motion»Vision Utilities»Image Management»Image Dispose) and Simple Error Handler VI (Programming»Dialog & User Interface»Simple Error Handler) in the second frame of the Flat Sequence Structure.

❑ Wire the VI as shown in Figure 5-8.

Figure 5-8. Lens Distortion Calibration VI with Sequence Structure

8. Save the VI.Sam

ple



Testing1. Test the VI.

❑ Go to the front panel.

❑ Run the VI. You should see the corrected image in the image display.

2. Examine the code generated by the Vision Assistant Express VI.

❑ Go to the block diagram.

❑ Right-click the Vision Assistant Express VI and select Open Front Panel.

❑ Click Convert when prompted to convert to a subVI.

❑ View the code generated by the Vision Assistant Express VI.

Note The IMAQ Read Image and Vision Info VI reads an image file, including any extra vision information saved with the image. This includes calibration information. The IMAQ Set Calibration Info VI sets calibration information from the calibrated image to an uncalibrated image. The IMAQ Correct Calibrated Image VI corrects a calibrated image by applying a calibration to create a spatially correct image.

❑ Close the subVI when finished. Click Defer Decision.

3. Close the VI. Do not save changes.

End of Exercise 5-1

Sam

ple



D. Using Spatial FiltersSpatial filters alter pixel values with respect to variations in light intensity in their neighborhood. The neighborhood of a pixel is defined by the size of a matrix, or mask, centered on the pixel itself. These filters can be sensitive to the presence or absence of light-intensity variations.

Filters are divided into two types: linear (also called convolution) and nonlinear. A linear filter replaces each pixel by a weighted sum of its neighbors. The matrix defining the neighborhood of the pixel also specifies the weight assigned to each neighbor. This matrix is called the convolution kernel. A nonlinear filter replaces each pixel value with a nonlinear function of its surrounding pixels. Like the linear filters, the nonlinear filters operate on a neighborhood.

Linear and nonlinear filters are divided into two categories:

• Highpass filters—Emphasize significant variations of the light intensity usually found at the boundary of objects. Highpass frequency filters help isolate abruptly varying patterns that correspond to sharp edges, details, and noise.

• Lowpass filters—Attenuate variations of the light intensity. Lowpass frequency filters help emphasize gradually varying patterns such as objects and the background. They have the tendency to smooth images by eliminating details and blurring edges.

Table 5-1. Spatial Filter Types

Lowpass Highpass

Linear GaussianSmoothing

GradientLaplacian

Nonlinear LowpassMedianNth Order

DifferentiationGradientPrewittRobertsSigmaSobelSa

mpl

e



Convolution KernelsA convolution kernel defines a 2D filter that you can apply to a grayscale image. A convolution kernel is a 2D structure whose coefficients define the characteristics of the convolution filter that it represents. In a typical filtering operation, the coefficients of the convolution kernel determine the filtered value of each pixel in the image. NI Vision provides a set of convolution kernels that you can use to perform different types of filtering operations on an image. You can also define your own convolution kernels, thus creating custom filters.

Refer to Chapter 5, Image Processing, of the NI Vision Concepts Manual for more information about filtering and convolution kernels.

Sam

ple



Exercise 5-2 Concept: Using Filters

GoalUse filters to manipulate an image.

ScenarioSome images require filtering before they can be analyzed or displayed. NI Vision provides multiple filters.

DesignIn this exercise, you will acquire an image and use the Vision Assistant express VI to apply smoothing and sharpening filters.

Flowchart

Figure 5-9. Flowchart of Using Filters VI

Snap an Image Apply Filters

Display theOriginal Image

Generate Code Using Vision Assistant

Display theFiltered Image

Dispose ofthe Images

Sam

ple



Implementation1. Open Snap and Display.vi (or Snap and Display 1394.vi),

located in the <Exercises>\LabVIEW Machine Vision directory.

Figure 5-10. Snap and Display VI Block Diagram

2. Save the VI as Using Filters.vi in the <Exercises>\LabVIEW Machine Vision directory.

3. Apply a Smoothing Filter to your image.

❑ Place a Vision Assistant Express VI (Vision and Motion»Vision Express»Vision Assistant) on the block diagram.

❑ In the NI Vision Assistant configuration window, select File»Open Image.

❑ Navigate to the <Exercises>\LabVIEW Machine Vision directory and select the file Acquired Image.jpg and click Open. If the Vision Assistant prompts you to remove previously acquired images, select Yes.

Note This image will not be used when the VI runs, but it will be displayed while configuring the Vision Assistant Express VI so that the effects of the processing steps can be visualized.

❑ Select Processing Functions: Grayscale»Filters in the bottom left window of Vision Assistant.

❑ Select Smoothing – Low Pass.

❑ Increase the Filter Size to 5.

❑ Click OK.Sam

ple



Figure 5-11. Applying a Smoothing – Low Pass Filter

Tip You can double-click the Smoothing – Low Pass step to edit the filter. Vary the size of the filter to see the effect on the image.

4. Apply a Convolution Filter to your image to make details in the image stand out.

❑ Select Processing Functions: Grayscale»Filters in the bottom left window of Vision Assistant.

❑ Select Convolution – Highlight Details from the list of filters.

❑ Increase the Kernel Size to 5 × 5.

❑ Click OK.

❑ Click Select Controls.Sam

ple



❑ Place a checkmark in the Convolution - Highlight Details 1»Kernel checkbox.

❑ Click Finish.

Figure 5-12. Applying a Convolution – Highlight Details Filter

Sam

ple



5. Build the block diagram shown in Figure 5-13.

Figure 5-13. Block Diagram of Using Filters VI

Note Create the array constant by right-clicking the Kernal input of the Vision Assistant Express VI and selecting Create»Constant. Display multiple elements of the array constant by clicking and dragging the bottom right corner of the array constant.

6. Examine and run the VI.

❑ Run the VI and view the result in the Image Display indicator on the front panel.

❑ Run the VI with the different array constant values to see the results.

Challenge1. Modify the block diagram to display both the filtered and original

images on the front panel at the same time.

2. Save and close the VI.

End of Exercise 5-2

Sam

ple



Notes

Sam

ple

machine vision sample

Documents

national instruments

national instruments

ni software

support information

product information

course software version

additional information

labview machine vision