© 2

003-

201

5 M

VT

ecS

oftw

are

Gm

bH.

1

MVTec Software GmbH is a leading international manufacturer of software for machine vision used in all demanding areas of imaging: semi-conductor industry, web inspection, quality control and inspection applications in general, medicine, surveillance etc.

MVTec's innovative work is driven by a commitment to be the number one supplier for sophisticated technologies in machine vision. MVTec is engaged in sponsoring various activities in universities, thus participating in the challenging process of understanding how machines can be taught to see.

HALCON is the comprehensive standard software for machine vision with an integrated development environment (IDE) that is used worldwide. It leads to cost savings and improved time to market: HALCON's flexible architecture facilitates rapid development of machine vision, medical imaging, and image analysis applications. HALCON provides outstanding performance and a comprehensive support of multi-core platforms, MMX and SSE2, as well as GPU acceleration. It serves all industries with a library of more than 1600 operators for blob analysis, morphology, matching, measuring, identification, and 3D vision, to name just a few. HALCON secures your investment by supporting a wide range of operating systems and providing interfaces to hundreds of industrial cameras and frame grabbers, including GenICam, GigE Vision, and IIDC 1394.

© 2

003-

201

5 M

VT

ecS

oftw

are

Gm

bH.

2

MVTec is a leading international manufacturer of standard software products for machine vision.

Our industrially proven software products are used in a large variety of application areas, such as semiconductor industry, inspection applications, medicine, and surveillance. HALCON is a comprehensive and powerful standard machine vision software optimized for the needs of OEMs and system integrators. MERLIC – which will be available in 2015 – is an all-in-one software product for quickly building machine vision applications without programming.

Besides, MVTec builds customized solutions - from consultancy, studies, and prototypes up to integrated products. Moreover, MVTec offers services, such as technical support and customized trainings.

MVTec's innovative work is driven by a commitment to be the number one supplier for sophisticated technologies in machine vision.

© 2

003-

201

5 M

VT

ecS

oftw

are

Gm

bH.

3

© 2

003-

201

5 M

VT

ecS

oftw

are

Gm

bH.

4

Electrode sheet inspection machine - Ayaha Engineering

For web inspection, a high resolution line scan camera is basically used. To stably process large images in high speed, a fast hardware processing such as by FPGA (Field Programmable Gate Array) is needed. However, application development using FPGAs takes a lot of effort and time. This can be an obstruction against a customer's request. Therefore, powerful software processing for time to market is a trend in web inspection.

Ayaha Engineering, is one of the major web inspection machine manufacturers in Japan. Ayahaalso uses mainly FPGA for multiple line scan cameras system. But for some systems, which have a small number of cameras, they use HALCON by MVTec to meet their customers' requests in time.

As an example, their machine is used for inspection of electrode sheet, a material for lithium ion batteries which is essential for state-of-the-art electric vehicle technology. Continuous running of several thousand meters is needed in most production lines of electrode material. Thus, it is necessary to stabilize the infinite length running for a long time.

In Ayaha's web inspection machine, only one line scan camera captures the electrode sheets which is coated by lithium. Then it inspects some factors such as coated width, defects, or foreign matters. The machine vision application is based on the HALCON software. HALCON's support vector machine (SVM) technology classifies the kind of defects. Defects have large variation for size, angle, intensity, shape, to name just a few. This defect classification using SVM is adopted as standard function of their machine.

HALCON's very fast region processing enables to process large data in time. Furthermore, the latest software features, e.g., defect classification increase the value of Ayaha's machine. Moreover, Ayaha significantly reduced the development time using HALCONs integrated development environment HDevelop. Thus, Ayaha realized a more flexible response to their customer's needs.

Article kindly provided by LinX.

© 2

003-

201

5 M

VT

ecS

oftw

are

Gm

bH.

5

Optima Group Consumer (Schwäbisch Hall, Germany) is a leader in themanufacture of coffee capsule filling systems, supplying to major coffeeproducers worldwide. For a leading North Italian coffee roaster, Optima deliverspackaging stations that incorporate machine-vision systems from Seidenader Automation (Markt Schwaben, Germany). In these capsule filling systems, twouEye GigE and four uEye USB cameras from IDS Imaging Development Systems (Obersulm, Germany) are used to check the capsules before filling andto ensure they are properly sealed after.

Within less than a tenth of a second, HALCON machine-vision software fromMVTec Software is used to detect defective capsules and reject them from theconveyor belt using pneumatic actuators.

Each UI-1540RE-M camera is used for imaging two coffee capsules at a time. At throughput rates just above one second, captured images are once againprocessed using MVTec's HALCON machine-vision library. As a result, a row ofeight capsules is inspected in only 600 ms. Capsules with a defective seal areseparated immediately, resulting in no scrap at the end of the line.

Author: Andy WilsonArticle kindly provided by Vision Systems Design.

© 2

003-

201

5 M

VT

ecS

oftw

are

Gm

bH.

6

As energy prices continue to rise, the call for regenerative energy sources is growing louder. One of the fastest growing sectors among them is photovoltaics, in other words the generation of electric power from solar energy. Rapid technological progress has contributed to the fact that a widespread use of solar thermal systems is becoming increasingly affordable. Government funded projects, such as the German "100,000 Roofs" program, additionally stimulate the solar market. The key criterion in the use of solar modules is their efficiency factor. To achieve maximum power output, high demands are placed on the quality of each wafer produced for solar cells. The German company ECKELMANN AG meets this challenge with a newly developed wafer inspection system. High-resolution Gigabit Ethernet cameras from IDS and a powerful image processing application based on HALCON form the backbone of this system.

© 2

003-

201

5 M

VT

ecS

oftw

are

Gm

bH.

7

NP Plastics, a Dutch injection-molded plastics manufacturer, produces small plastic rings–among its many other products. These plastic rings are used in the closing mechanism of ring binders found in every office from Amsterdam to Hong Kong.

To meet volume demand while improving quality, NP Plastics recently decided to install a machine-vision system to inspect the small plastic rings. The company turned to automation solution provider Radine with the support of Data Vision, Allied Vision Technologies' (AVT) sales partner in the Benelux. Data Vision developed a custom machine-vision solution using two AVT FireWire cameras, strobe and LED illumination, and MVTec Software's HALCON image-processing software running on a standard Dell PC to check the rings for burrs, undershots, and other spatial defects.

Author: Winn HardinArticle kindly provided by Vision Systems Design.

© 2

003-

201

5 M

VT

ecS

oftw

are

Gm

bH.

8

Since more than 15 years, ROBOWORKER Automation GmbH (Weingarten, Germany) is specialized in developing and producing systems for automatic handling, assembly and checking of workpieces as well as robot-aided loading and unloading of production machines. The company now has developed a robot feeder system that has automated the handling, inspection and packaging of indexable inserts.

To properly inspect and then pack vending boxes of indexable inserts, stacks of trays are automatically unloaded row by row from an RL 20 linear robot machine manufactured by ROBOWORKER. Every two seconds the conveyor is forwarded one part. However, because the demands of the manufacturer required a greater throughput of 120.000 parts per day, four separate conveyors are loaded in parallel from the same depalletizing machine. This results in a throughput of approximately one part every 0,5 seconds.

Parts placed on all four conveyor belts approximately 20 mm apart (depending on the number of rows of the trays) by the depalletizer are then moved under a machine vision system where they are inspected for position, rotation and geometric properties as incircle, outer circle, hole diameter and included angle. As the parts move under the station, they are backlit so that the system can more easily determine the orientation of each part. To digitize images of the parts, several monochrome cameras are interfaced to a PC.

© 2

003-

201

5 M

VT

ecS

oftw

are

Gm

bH.

9

The images are analyzed using the HALCON machine vision library from MVTec Software GmbH. By using the pattern matching tool within HALCON, the position and orientation of each part can be extracted from the image even if parts are rotated or partly occluded. The software uses HALCON’s contour based pattern matching and locates objects based on their shape. This leads to a reliable detection even under changing lighting conditions.

Dimensions such as incircle, outer circle and hole diameter of each part are measured with a precision of 50 µm. When measuring the cutting edge radius and the part angle, subpixel accurate edge extraction algorithms are used. These procedures work reliably also under rough production conditions.

© 2

003-

201

5 M

VT

ecS

oftw

are

Gm

bH.

10

The system checks also, if a part is visible in the camera, detects wrong parts and can measure the dirt on the conveyor belt. Once the dimensions have been calculated, they are compared with the dimensional limits and tolerances of known good parts from an ISO-standard database. In this way, the system determines whether the parts can be transferred to the next inspection station or have to be discharged.

After the software has determined both the x-y position and orientation of the indexable inserts, the coordinates of each part are transmitted from the PC to a programmable logic controller. With the help of HALCON's 3D camera calibration, the system establishes a relationship between each pixel and the robot coordinates. The determined position and orientation data can be handed over directly to the robot. Tilt of the camera, distortions of the lens as well as not right-angled robot coordinates are corrected. The height of the indexable insert is also considered during the measurements.

© 2

003-

201

5 M

VT

ecS

oftw

are

Gm

bH.

11

A descriptive example for modernizing an automated packet identification system is the warehouse storage of the mail-order house WITT Weiden, Germany. The OCR has to identify standard labels with handwritten characters by about 400 suppliers around the world. The identification rate must be higher than 96%. ECKELMANN AG, Mainz, developed this application based on HALCON that perfectly fits to the existing environment and the existing proprietary inventory control system.

The goods are delivered via motor trucks, packed in packages of only three standard dimensions. These boxes are guided trough several registration conveyor belts operating automatic weighting, measurement, and identification of the single boxes. The ascertained data are subsequently transferred to the proprietary warehouse management systems. Identification and measurement of the boxes is performed by an imaging system which was substituted within a modernization process. The challenge was to identify the boxes through arabic numerals (item number, quantity, and size) written on standardized labels. That means that the system must be able to identify and classify printed, stamped, and handwritten digits. Each line must handle 5,000 packages a day with a capacity of about 40 packages each minute.

Author: Winn Hardin.

Article and images kindly provided by Vision Systems Design.

© 2

003-

201

5 M

VT

ecS

oftw

are

Gm

bH.

12

The imaging software is based on the standard software library HALCON from MVTec. The application building is supported by HALCON's integrated development environment HDevelop. Affected by a long transport, the labels often are polluted or damaged. Thus, the label localization and the segmentation of the characters can be extremely complex.

The classification of the handwritten characters is partly multi-stage processed by using different methods, for example the support vector machine provided by HALCON. "The implementation of the image processing as we planned it was only possible by several exclusively special HALCON attributes", says Arno Dewald, ECKELMANN's responsible project engineer.

After a first run of classification for a character sequence under the appliance of neural networks, a plausibility check occurs in adjustment with the total quantity of allowable results. If necessary, a second classification process runs with an alternative method (particularly the support vector machine has proved to be very powerful).

Author: Winn Hardin.

Article and images kindly provided by Vision Systems Design.

© 2

003-

201

5 M

VT

ecS

oftw

are

Gm

bH.

13

Furthermore, the machine vision software detects the size of packages. Because only three different packages are permitted, the image software must only detect one side of a package to conclude the size. Based on this data, the system decides if two small packages or one bigger package are stacked on one storage tray and location.The result of the new application exceeds all expectations of WITT. A first evaluation over the time-frame of one month showed that the identification rate was 100%. The processes of the OCR as well as the entire status of the plant are visualized through clearly arranged user interfaces. By data exchange with the warehouse management system, different plausibility checks are processed. Against this plausibility check, the automatic labeling of the package happens by a newly printed plain written label."We are very content regarding the project settlement and the performance of the application", says Sabine Schaumberger, project manager at WITT Weiden. "Even during the fast-paced Christmas business, the launch of the application without interrupting operations was no problem."

Author: Winn Hardin.

Article and images kindly provided by Vision Systems Design.

© 2

003-

201

5 M

VT

ecS

oftw

are

Gm

bH.

14

© 2

003-

201

5 M

VT

ecS

oftw

are

Gm

bH.

15

Introduction

There exists such a wide range of offers of 3D methods for industrial machine vision that it is hardly possible for the user to look at the big picture. Those methods can be classified according to the measuring principles they are based on.

For triangulation, two rays of light, that meet on one point of the surface, have to be linked. Depending on the object’s features this task can be solved differently, e.g. using the binocular stereo analysis, the sheet of light method or photometric stereo.

Information about heights and distances can also be gained through focus determination. A small depth from focus area is thereby utilized to detect the distance from the surface to the camera. Depth from focus, depth from defocus and confocal methods are the most common representatives of this family.

Another measure principle compares the modulation of an outgoing reference signal to the modulation of a received signal, that was changed by the object’s surface (e.g. time of flight and interferometry).

All of the methods that are shown here are able to measure 3D surfaces and return 3D scatterplots or distance images.

A variety of tasks can be solved without comprising 3D scatterplots or 2½D images especially if only the objects’ pose, their position and orientation in space, is of interest.

If the object’s geometry is known, the 3D pose of arbitrarily oriented objects can be directly determined. Known geometric characteristics like points, contours (e.g. circles or rectangles) or a CAD – model of the object are therefore needed.

The advantage of these direct methods is that they are significantly faster than the determination and registration of 3D scatterplots or 2½D images and that one camera is sufficient.

Typical application areas of these methods range from pose recognition for robot guidance to measuring simple geometric primitives (distances, radii, coplanarity and so on) of complex 3D objects.

© 2

003-

201

5 M

VT

ecS

oftw

are

Gm

bH.

16

Time of flight:

MESA Imaging SwissRanger SR4000 (via SwissRanger interface)

Bluetechnix Argos 3D P100 (via Argos3D-P1xx interface)

Fotonic E- and C-series

Laser triangulation (without laser):

SICK Ranger / Color Ranger (via SICK-3DCamera interface)

Automation Technology C4 (via GigEVision interface)

Photonfocus MV-D1024E-3D01 (via Camera Link board)

Laser-Triangulation (incl. laser):

SICK Ruler E (via SICK-3DCamera interface)

SICK ScanningRuler (via SICK-ScanningRuler interface)

Leuze LPS36 (via LPS36 interface)

(Micro Epsilon scanCONTROL (via 1394IIDC or GigEVision interface))

SmartRay Ecco

LMI Gocator 2000 and 2300 (via GenICamTL interface)

Fringe projection:

VRmagic AreaScan3D (via GenICamTL interface)

Shape Drive SD-1 (via planned ShapeDrive interface)

Structured light

Microsoft Kinect (via OpenNI interface)

Ensenso N10 (via NXLib interface)

LMI Gocator 3100 (via GenICamTL interface)

© 2

003-

201

5 M

VT

ecS

oftw

are

Gm

bH.

17

Besides other technologies HALCON offers a wide range of highly sophisticated matching technologies:

HALCON allows to locate objects with arbitrary orientation in 3D (3D alignment), it provides the well established shape-based matching – working even with color images, the unique component based matching and the well proven normalized cross correlation.

© 2

003-

201

5 M

VT

ecS

oftw

are

Gm

bH.

18

Application example

One of the biggest challenges in robotic currently is bin picking. Objects often lie arbitrarily oriented, reflect metallically and are irregularly shaped.

Additionally the illumination is hard to control.

Even though the metal cylinders in this example have a very simple shape, a pure 3D matching approach is not sufficient due to the large number of objects. The cylinders are deformed due to their manufacturing process and show many additional edges due to rust on their surface. Therefore the detection would take too long.

By combining different approaches, one can get a hold on the problem. First, the highest point is detected by binocular stereo analysis and the search area for 3D matching is limited around this area.

Therefore e.g. binocular stereo analysis or sheet of light measurement are suitable for detecting the approximate position of the cylinders. In particular this allows to quickly and easily determine the uppermost object. This is usually exactly the part to be picked up. Using the approximate location, the robot can approach the object, then take a 2D image with a higher resolution and subsequently determine the accurate 3D position of the cylinder using 3D matching.

© 2

003-

201

5 M

VT

ecS

oftw

are

Gm

bH.

19

In order to check whether the die is tilted or not, the height of the die's corners is measured in relation to the housing.

The measurement of the die height is also important in other applications, e.g. when using flexible PCBs.

In this example, a depth from focus measurement is performed for each of the four corners of the die to determine the die's height at every corner and therefore the tilt of the sensor.

For all four corners a DFF measurement has been performed which results in a complete 3D reconstruction.

After a height segmentation In both die and PCB a planar fit is being performed.

The distance of the two planes yields the relative height of the die on the PCB.

© 2

003-

201

5 M

VT

ecS

oftw

are

Gm

bH.

20

© 2

003-

201

5 M

VT

ecS

oftw

are

Gm

bH.

21