2007 IEEE International Conference on Control and Automation WeC5-3Guangzhou, CHINA - May 30 to June 1, 2007

An Integrated Inspection Method based on MachineVision for Solder Paste Depositing

Shenglin Luthe College of Mechanical EngineeringSouth China University of Technology

Guangzhou, Chinajoolu@163.com

Xianmin Zhangthe College of Mechanical EngineeringSouth China University of Technology

Guangzhou, Chinazhangxm@ scut.edu.cn

Yongcong Kuangthe College of Mechanical EngineeringSouth China University of Technology

Guangzhou, China

Abstract-The solder paste depositing inspection is veryimportant in the process of Surface Mounting for print circuitboard (PCB). An integrated inspection approach based on themachine vision was presented in the study. The methoddeveloped in this paper can identify the major defects of thesolder paste depositing, such as displacement, deficiency, excess,bridge and overflow. Firstly, position compensation is applied toimprove the inspection accuracy. Secondly, an imageenhancement algorithm based on the texture is developed, whichcan enlarge the differences of the grayscale values between thePCB images and the solder paste images. Thirdly, the images ofthe printed PCB are analyzed by using the particle analysismethod and the two-dimensional (2D) inspection results areobtained. A pseudo three-dimensional (3D) inspection approachis further proposed to identify the defects in the 3D state. Finally,experiment results are presented, which illustrate the validity ofthe approach.

Keywords- machine vision; inspection; solder paste depositing

I. INTRODUCTION

The screen printer is one of the key equipments of the SMTindustry [1]. Its performances directly affect the subsequenthandling and the product quality. Thus, it is necessary toinspect the Printing Circuit Board (PCB) after screen printing.It was reported that 50-70% PCB assembling defects werefrom the stage of solder paste depositing [2]. The inspection ofthe solder paste depositing is one of the most important stagesin the process control of SMT production line [3]. The defectsof solder paste depositing include displacement, deficiency,excess, bridge, overflow, and so on. The typical inspectionmethods for the solder paste depositing include manualinspection, the laser scan inspection [4] and the inspectionbased on machine vision [5]. The manual inspection is apopular method because of its simple operation; however, ithas many serious disadvantages, such as low efficiency andhigh probability of miss judgment. The laser scan approachprovides more detailed defect information because of its 3Dinspection function. However, the 3D model of the solder pasteis acquired by point or line scans of a laser [6], so theinspection very slowly. Furthermore, the laser scan device is

expensive. The method based on machine vision has the meritsof high inspection accuracy and speed, thus it can be used inPCB solder paste depositing inspection of the screen printer.Moreover, no extra hardware cost is needed, because theapproach can make full use of the existing hardware resourceof the vision system of the screen printer.

The main procedures of solder paste inspection based onmachine vision include image acquisition, image enhancementand defect analysis. In order to improve the performance of thisinspection approach, the position compensation method isutilized to determine precise position before acquiring PCBimages. An image enhancement method based on texture isdeveloped d, which can enlarge the differences of the grayscalevalues between PCB base images and solder paste images. Inaddition, the printed PCB images are analyzed by using the 2Dinspection algorithm based on the particle analysis method. Apseudo 3D inspection approach is applied to inspect the defectsof solder paste depositing in volume. Finally, the experimentresults illustrate the validity of the proposed method.

II. THE SCREEN PRINTER WORKFLOW

As one of the key equipments, the main function of thesolder paste screen printer is to deposit the solder paste on PCBprecisely. The screen printer is composed of conveyer,squeegee system, machine vision system and planar 3 degreesof freedom (DOF) adjusting unit, etc. The machine visionsystem is used to align the PCB and the stencil, and to inspectsolder paste depositing. A cycle of work flow of the screen isshown in Fig. 1. The position and pose of the PCB is not fixedafter being loaded by the conveyer. Thus, the machine visionsystem need to measure the marks positioning by patternmatching so that the printer can find the differences of theposition and pose of the PCB and the stencil. Then, thesedifferences can be compensated by use of the planar 3-DOFadjusting unit.

Manuscript received November 1, 2006. This work was supported by theTeaching and Research Award Program for Outstanding Young Teacher inHigher Education Institutions of MOE, P.R.C., the Guangdong Hong KongTechnology Cooperation Funding (Dongguan Project 20061682) and theResearch Project of Ministry of Education and Guangdong Province(2006D90304001). Shenglin Lu is with the South China University ofTechnology, Guangzhou, GuangDong 510640 China

137

1-4244-0818-0/07/$20.00 (¢ 2007 IEEE

Authorized licensed use limited to: Brigham Young University. Downloaded on December 6, 2009 at 00:56 from IEEE Xplore. Restrictions apply.

V~~~ +y

x

Figure 2. The position compensation of the PCB

IV. IMAGE ENHENCEMENT

Compared with the solder paste and the base of the PCB,the grayscales value of the pad is much higher. The differencecan be distinguished easily by using a threshold function. Butthe grayscales of solder paste and the base of PCB are verysimilar, and their histograms are even partially overlapped. Toaccurately distinguish the image of solder paste and the base ofPCB, an algorithm based on texture was presented in thissection.

Figure 1. The Workflow of the Screen Printer.

III. POSITION COMPENSATION

As shown in Fig. 1, the inspection program needs to beconfigured prior to inspection. The inspection programconfigurations include the setting of inspection sites, thresholdvalues, image enhancement parameters, and so on. Theinspection performance will be affected by the differences ofthe position and the pose between the PCB for configurationand the inspecting one. Here, a position compensation methodis used to compensate the differences of the position and posecomparing with the PCB prior to the images acquisition. Thesedifferences are approximately equal to coordinates differenceof the two marks of the PCBs, as shown in Fig. 2. The 6 and scan be calculated by the following equation:

Ml'=Tm1+s

M21=,Tm2 +s(1)

where ml and m2 represent the coordinates of markl andmark2 in the inspection program configuration, respectively.mlI' and M2' are the coordinates of markl and mark2 ininspection, respectively. s is the shift vector and s=(Alx, Jly)T.T is the rotation matrix which can be expressed as:

cos6 - sin6T =

sin6 cos )

From equation (1), one can find that there are 3 unknownvariables and 4 equations. The 0 can be calculate easily byusing the trigonometric function, then, the over constrainedproblem can be solved by using the least square method. Theapplying of the method of position compensation method canimprove the inspection accuracy

Figure 3. Image of solder paste. The (a) is the solder paste image bymagnified 4 times, pitch=0.3mm, and the (b) is the solder paste image acquire

by the vision system, pitch=l.Omm.

As shown in Fig.3 (a), the solder paste is mainly composedof little balls and its image has pit texture. However, the imageof the PCB base and pad is very smooth. According to thisfeature, the images can be enhanced by increasing the solderpaste grayscale value.

It is well known that the texture of the image can evaluatedby an n-order statistical moment of the histogram which can beexpressed as [7]

L-1

Alny (Zi _-m)nP(Zi)i=O

where zi is a random variable indicating intensity, p(zi) isthe histogram of the intensity levels in a region, L is thenumber of possible intensity levels, and m is the mean value ofgrayscale.

However, the histogram does not externalize the position ofpixel information. Thus, the images need to be separated into xXy parts before using n-order statistical moment, kin (X, y) iSthe n-older value of row x, column y part of the image. Theenhancement algorithm base on n-order statistical moment canbe written as

138

Authorized licensed use limited to: Brigham Young University. Downloaded on December 6, 2009 at 00:56 from IEEE Xplore. Restrictions apply.

{ g(i, j) f (i, j) + h.u,, (x, y) u, > h

g(i, j) f (i, j) u, < h

where J(i, j) is the image without enhancement, g(i, j) is theimage with enhancement, and h is a weight.

This algorithm can enhance the images by increasing thegrayscale of the solder paste part, so the solder paste and thebase of PCB can be distinguished easily. The enhanced imagescan be separated into pad particle, solder paste particle anddark loop around solder paste particle by using a multi-thresholds function.

V. INSPECTION METHOD

A. 2D Inspection Based on Particle Analyze methodThe Particle analysis method has many advantages, such as

simple and high speed. Furthermore, many particle factors areshift, scaling and rotation invariant. Thus, the particle analysisapproach is suitable for 2D inspection.

The defects of solder paste depositing can be inspected byanalyzing the position, area and shape of the particles of solderpaste and pads, which can be expressed by the followingfactors [8],

Center: center position of the particle.

A: area of the particle.

Ah: area of the hole in the particle.

ACH Area of the particle Convex Hull.

P: Length of the outer boundary of the particle.

Hydraulic Radius: The particle area divided by the particleperimeter.

Heywood Circularity Factor: HC=P/(2sqrt( 7T A)),Perimeter divided by the circumference of a circle with thesame area. The closer the shape of the particle is to a disk, thecloser the Heywood circularity factor is to 1.

I,,: moment of inertia with respect to the x axis.

Iyy moment of inertia with respect to they axis.

Type Factor: TF = A2 I(4ff. sqrt(I I,)), the type of theparticle.

Packing Ratio: PR= AI(A + Ah) it is used to judgewhether deficiency of solder paste exist or not.

The values of the factor mentioned above are inputted tothe defect analyzer after being calculated. The defect analyzeris used to determine the defect existing or not by using asynthesis weight method. The defect analyzer can find out thedefects of the solder paste, such as displacement, deficiency,excess, bridge and overflow. Though the 2D inspection methodcan determine most of the solder paste depositing defects, itcan not inspect the volume defects of solder paste deposition.

B. Pseudo-3D inspectionNote that the 3D inspection algorithm can find out the

defects of paste dose in volume, however, it needs a laserscanner and it is relatively slowly. To overcome thesedisadvantages, a pseudo-3D inspection approach was used in

order to calculate the volume ratio by creating a simple 3Dmodel of the solder paste depositing.

As shown in Fig. 3 (b), there is a dark loop around thesolder paste. One reason for the dark loop is that the PCB isilluminated by the coaxial illuminator of the vision system,thus the ray is perpendicular to the PCB according to theprinciple of reflection. If only the ray is perpendicular to thesurface, the reflected ray is captured by the camera, so theimage is bright; otherwise the image is dark, as shown in Fig.4(a).

ray

FaSolder Paste

Base of PCB

(a)

Vu)

Figure 4. 3D Model of Solder paste. The (a) is the side view and the (b) isthe top view.

As shown in Fig. 4(b), Ap is the area of particle of the pad,A, is the area of particle of the solder paste and Ad is the area ofparticle of the dark loop around the solder paste. The volumeratio of the solder paste deposit can be obtained by

Va VVt = 1 l2(A, + (A, + Ad) h /(AP .h) = (I/ 2Ad + A,) I AP(2)

where Va represents the actual volume of the solder pasteand V, is the theoretical volume of the solder paste.

The defects can be found out by a threshold according tothe volume ratio. This pseudo-3D inspection method can becarried out fast compare with the laser scanner inspection.

VI. EXPERIMENT AND ANALYSIS

In order to verify the performance of the integrated method,some experiments have been carried out. The machine visionsystem was shown in Fig. 5, the vision system includes a NM-PC1I409 image acquire card, two Teli-Cs8630i cameras, twoAllyVision-COR30 coaxial illuminators. The solder paste islead free.

139

Authorized licensed use limited to: Brigham Young University. Downloaded on December 6, 2009 at 00:56 from IEEE Xplore. Restrictions apply.

Fig. 7 shows the misalarm rates of solder paste depositingdefects for four kinds of electronic component From thisfigure, one finds that the misalarm rates reduced by usingposition compensation and texture-based image enhancementmethod.

Misalarm Rate

8

Figure 5. The hardware of the vision system.

The histogram of the images of the solder paste and PCBbase is shown in Fig. 6. Fig. 6 (a) is the histogram of the imageof the PCB base. Fig. 6 (b) is the histogram of the image of thesolder paste before image enhancement. Fig. 6(c) is thehistogram of image of solder paste after image enhancement. Itcan be seen that the histogram value of solder paste image wasincreased after image enhancement, and the difference of themean grayscale values between the two images varies from 4.6(109.8-105.2) toI3.7 (118.9-105.2). The increase value wasrelated to the weight factor h.

(a) Histogram of the image of the PCB base

ChiplO05 ChipO4O2 QPF128 BGA128P=0.5 P=l.0

* Without PositionCompensation & ImageEnhancement

U With PositionCompensation

C With ImageEnhancement

E With PositionCompensation & ImageEnhancement

Figure 7. Misalarm rates comparison.

TABLE IMISALARM RATES

Defect type of solder pasteChip Chip IC IC

1005 0402 QFP BGADisplacement 3 9 5 7Deficiency 1 6 7 5Excess 2 3 2 2Bridge, 0 0 1 0Overflow 0 1 0 2

Table I shows the misalarm rates of five defects of solderpaste depositing for four electronic components, two kindschip and two kinds of ICs. One can finds that the misalarmrate of the displacement defect is the highest and is related tothreshold value set by user. Whilst, the defect misalarm rate ofthe small and round pads is higher.

The results of pseudo-3D inspection are shown in Fig. 8.Using pseudo-3D inspection algorithm measure the chiplO05and chip 0402 fifteen times. The results show that thefluctuation range of the volume ratio described in equation (2)is from 50 to 110. Therefore, it can be concluded that the 3Dinspection method is stable.

120

(b) Histogram of the image of the solder paste before image enhancement

100

> 80

60

40(C) histogram of image of solder paste after image enhancement 0 2 4 6 8

Times10 12 14 16

Figure 6. Histogram of the image of the solder paste and PCB Figure 8. Pseudo-3D Inspection Results

-u-Chip 1005* Chip 0402

140

Authorized licensed use limited to: Brigham Young University. Downloaded on December 6, 2009 at 00:56 from IEEE Xplore. Restrictions apply.

VII. CONCLUSIONAn integrated inspection method of solder paste depositing

based on the machine vision was presented. To improve thepositioning precision of the inspection locations, the positioncompensation method was performed. A texture-basedalgorithm is also proposed to enlarge the differences of thegrayscale values between the solder paste and the PCB base,which makes the multi-threshold method more stable andprecise. Furthermore, the 2D and pseudo-3D inspectionapproach can be used to inspect the solder paste depositingdefects effectively. The experiment results demonstrated thefeasibility and robustness of the integrated method.

REFERENCES

[2] Glenn Wyllie, Mark Norris, "2D or Not 2D, That Is TheQuestion!" ,Conference: SMTA International, 2003

[3] D. Burr, "Solder paste inspection: process control for defect reduction,"Test Conference, 1997. Proceedings., International, Nov. 1997,pp.1036.

[4] T. Okura, M. Kanai, S. Ogata, T. Takei, and H. Takakusagi,"Optimization of solder paste printability with laser inspectiontechnique," Electronics Manufacturing Technology Symposium, 1995.Manufacturing Technologies - Present and Future' Seventeenth

IEEEICPMT International, 1995, pp.361 - 365.[5] Omron Inc. "AOI Inspection in Pb Free Craft," Equipment for

Electronic Products Manufacturing, 2003, Vol64.[6] J. Mahon, "Automatic 3-D inspection of solder paste on surface mount

printed circuit boards ," Journal of Materials Processing Technology,Volume 26, Issue 2, June 1991, pp. 245-256.

[7] Rafael C. Gonzalez Richard E. Woods Digital Image Process PublishingHouse of Electronics Industry, 2004.

[1] Nepcon South China 2007, [Online]. Available: [8] National lnstruments, IMAQ Vision Concepts Manual,2003, pp.10-1.

141

Authorized licensed use limited to: Brigham Young University. Downloaded on December 6, 2009 at 00:56 from IEEE Xplore. Restrictions apply.