669 978-1-4673-0199-2/12/$31.00 ©2012 IEEE

2012 International Conference on Systems and Informatics (ICSAI 2012)

Study on the 3D Non-contact Measurement Method of Hyaloid and Closed Container

Wei Pei Environmental Science and Engineering College

Dalian Maritime University Dalian, Liaoning Province, China

Zheng Xu Key Laboratory for Micro/Nano Technology and System

of Liaoning Province Dalian University of Technology Dalian, Liaoning Province, China

Yong Ying Zhu Ocean and Civil Engineering department

Dalian Ocean University Dalian, Liaoning Province, China

Chen Xiang Wang Environmental Science and Engineering College

Dalian Maritime University Dalian, Liaoning Province, China

Abstract—The research of microfluidic chip is a vital domain and the forefront of micro total analysis systems (μTAS). Compared with the development of microfluidic chip, the research of detecting systems for microfluidic chip is backward. The micro stereo vision system is a good solution for non-contact measurement of microfluidic chip. However, it often meets refraction of light and occlusion. In this paper, based on optical theory and digital image processing, the micro stereo imaging principle of stereo light microscope (SLM) is studied and the refraction correction and micro stereo occlusion correction algorithms are proposed to achieve microfluidic chip detection. These algorithms quantize the influence of cover plate and occlusion on the measurement accuracy, and realize non-destructive, non-contact and precise 3D measurement of a hyaloid and closed container.

Keywords- Pattern recognition & intelligent systems; Micro stereo vision; Refraction; Occlusion

I. INTRODUCTION The cross section dimensions of the micro channel in the

microfluidic chip are generally measured with contourgraph [1-3] or scanning electron microscope (SEM) [4-6]. To be measured with contourgraph, the substrate and the cover plate of the chip bonding must be separated forcibly. To be measured with SEM, the microfluidic chip need to be cut off at the measurement position, and the cross section need to be spurted with gold as conducting layers from ion sputtering instrument. Because the two methods are expensive and inefficient, they have not been used in mass production.

With visual feedback, the binocular micro stereo vision system based on SLM makes it possible to achieve 3D high accuracy auto-positioning, 3D information extraction, 3D shape reconstruction and measurement. It has been extensively used in micromanipulation, micro assembly, micro robot navigation and bioengineering [7, 8].

One of the problems encountered with basic light microscope is the refraction of light. Refraction occurs when a

ray of light passes through an object that bends the light to another direction. The optical process of refraction is due to the theory of light that when it slows down in a medium of different refractive index, it consequently changes direction. Greater refraction means less focus for the same microscope lens and a blurred image. How can we reduce the refraction influence when looking at samples under a microscope? One solution is to use oil immersion lens. However, depth of field and focal length of SLM are larger than those of biological/fluorescence microscope, which will blur image and decrease the measurement accuracy if oil immersion lens is used. Furthermore, oil may pollute samples. Therefore, the solution of using oil immersion lens is not ideal for the measurement of microfluidic chip with SLM.

The other of the problems encountered with basic light microscope is occlusion. The processing quality of occlusion is an important indicator to evaluate a stereo matching algorithm. If the occlusion is not processed, it is very easy to produce the wrong match, miss match, even the error disparity map and depth map. It also may have a wrong match illusion.

In the early study, the occlusion detection was delayed to stereo matching [9], which is likely to give the wrong disparity value to the block area. The current study tends to deal match with the occlusion detection simultaneously [10-13]. The blocked area does not exist in the corresponding region so it often gets lower matching values. Based on the fact of lower similarity, most of these algorithms use Bayesian criteria [11], variable window method [14], two-way matching method [15], etc. The key of these methods is to use a similar criterion to minimize the similarity of the block area [10]. The testing is not ideal for complex scenes.

Some algorithms have already begun to consider the essential characteristics of occlusion [12, 16-18]. The basic idea is to detect the occlusion with the disparity gradient limit constraint, the order constraint, the epipolar constraint and so on to restrict stereo matching in disparity space based on the pinhole model.

670

The rest of the paper is organized as follows. The refraction correction algorithm based on SLM is presented in Section Ⅱ. Section Ⅲ describes the micro stereo occlusion correction algorithm. Our experimental studies are shown in section Ⅳ. Finally, sectionⅤ concludes this paper.

II. REFRACTION CORRECTION BASED ON SLM Refraction correction in vision measurement is divided into

two schemes according to the environments of calibration and measurement: 1) If environments are same, refraction correction is performed in calibration [19, 20]. 2) If environments are different, refraction correction is performed in measurement. In general, the refractive index of intermediate medium is known and the position of refraction is random in the measurement based on SLM. Obviously the first scheme is not suitable for such situation. Consequently, the second scheme is adopted in this paper.

In general, calibration of stereo vision model and 3D measurement are completed in the air. In other words, the whole process is completed in the same medium and there is no refraction. But in practice, the micro channel of microfluidic chip is enclosed with a cover plate, which causes at least two refractions. One occurs between the cover plate and air. The other occurs between cover plate and gas/liquid in micro channel, as shown in Fig. 1.

Fig. 2 shows the twice refraction excursion of micro stereo vision base on SLM. The space point PW(X,Y,Z) conforms to the linear propagation law of light in the same homogeneous medium. PL and PR are the left and right image point corresponding to PW respectively.

When the light propagates in different kinds of mediums, it will not propagate along a straight line but bend the propagation direction on the interface of two mediums. If refraction is neglected, PL

’ and PR’ are the left and right pseudo

image points corresponding to PW respectively, and the pseudo space point Pw

’(X’,Y’,Z’) will be obtained. The measurement error which is the difference between Pw

’and Pw will change with the position of the space point and the thickness of the cover plate. The twice refraction can cause visual field error, optical aberration, magnification decrease, the chromatic aberration and distortion increase. It may also result in non-uniformity of the lens magnification in the entire field of view [21-23]. These problems will affect image quality and make

Figure 1.Twice refractions in microfluidic chip

Figure 2. Twice refraction excursion of micro stereo vision base on SLM

3D measurement becomes very difficult.

The double optical paths of SLM make the twice refraction model very complex. Our SLM-based refractive index correction model using twice refraction will be introduced.

When there are homogeneous medium on both sides of the cover plate, the refractive indexes are equal. Namely,

,R R L Lβ γ β γ= = . The incident ray and the emergent ray are

parallel. The unknown variable d (the distance from cover plate to objective lens) can be eliminated. However, when R Rβ γ≠ , L Lβ γ≠ , the incident ray and the emergent ray

are not parallel and the unknown variable d exists. Therefore the key of the twice refraction correction with variable refractive index is to eliminate d.

SLM based micro stereo vision is passive and its measurement aim is to obtain accurate 3D space coordinates of each point. After the ray of measured space point passes through a series of mediums, it will arrive at the image plane eventually. No matter what intermediate medium it is and how complex the ray propagation path is, the position of space point and the linear propagation law in the same medium are unchanged. The variation of the refraction indexes just affects the ray propagation direction, the position of refraction point in object space and the coordinate value of corresponding image point in image space.

In the parallel stereo vision, the ray PwD from space point Pw projects to cover plate vertically, and its propagation direction is unchanged. Pseudo space point Pw

’ (X’,Y’,Z’) and real space point Pw(X,Y,Z) are collinear, which are derived from the coaxial spherical imaging principle. Namely, X’=X.

In SLM based micro stereo vision model, as shown in Fig.

2, R Rβ γ≠ , L Lβ γ≠ , the increase of the effective front focal

length caused by refraction and angle of the SLM twice optical path.

Two refractions and angle of the SLM two optical paths lengthen the effective front focal. It results that the rays of the two optical path can not converge at the point Pw

’(X’,Y’,Z’) or Pw(X,Y,Z), but form a blurred circle in the front of pseudo space point Pw

’(X’,Y’,Z’). Fortunately, pseudo space point Pw

’(X’,Y’,Z’) and real space point Pw(X,Y,Z) are collinear in the parallel stereo vision, which is known from refraction principle. That is, the formula X’=X=XC is right.

Cover plate

Substrate

Micro channel

Emergent

Pw(Xc, Yc, Zc)

P

A

B

C D E

F

G

OL OR

PR P RPLP L

γ

β

γ

βα

α

O

d

wp

O RO L

HI

K

L

Z

X

Y

671

With (2) the coefficients Rη and Lη for variable refractive

index correction with twice refraction can be obtained as (1) [24].

( ) ( )( ) ( )( ) ( )( ) ( )

/ 2 cos cos sin 1/ 2 sin cos cos tan( )

/ 2 cos cos sin 1/ 2 sin cos cos tan( )

C R C R R RR

C R C R R R R R

C L C L L LL

C L C R L L L L

b X Z fb X Z f

b X Z fb X Z f

θ θ θη

θ θ θ β θθ θ θ

ηθ θ θ β θ

− − −= ×

− + − −

+ − −= ×

+ + − −

(1)

( )( )( )( )

12

'1

1

2

cos tan cos tan/

tan tan

/ 2 cos tan tan tan tan tan /

tan tan tan tantan tan tan tan tan tan

tan tan tan tan

θ β θ ββ γ

θ β β γ β γ

β γ β γβ γ β γ α γα β α γ

+ × + ×⎡ ⎤= ⎢ ⎥

− − ×⎢ ⎥⎣ ⎦

− − + − + −

= −= − +

+ −

L L L R R RC

R R p

R R R L L R R

R L L R

L R R L R L

L R L R

b f fZ T

w T

b X f T

TT

tan tanα β− R L

(2)

III. MICRO STEREO OCCLUSION CORRECTION ALGORITHM BASED ON SLM

Fig.3 is the sketch map of occlusion detection. In order to describe it more specifically, the rectangular box in vision model (a) is enlarged to (b).

The half-angles of two optical paths, Lθ and Rθ are known for the calibrated micro stereo vision system based on SLM.

In Fig. 3(b), LF is the left focus, the line AB is panned right, while the point B coincides with the point F, it will be occluded by the point A. The point B locating on the left side of the point F is occluded, on the other side, it is not occluded. The point F is called occlusion critical point of the point B. Similarly, the point E is called occlusion critical point of the point C.

After the edge maps of the pair images are matched and reconstructed, the cross section of measurement object can be obtained and shown in Fig. 3(b).

By the relative position of AB and CD, it is known that the measured object is convex or concave, and then the candidate occlusion regions are determined. This paper selects the concave object to study the occlusion problem. The occlusion point H results that the left and right incident ray CLFL and CRFR wrong intersect at the point F. In fact, they should intersect at the original space point C.

The left neighborhood of the left image point CL imaged with the space point C is occluded and the right neighborhood is not occluded. Therefore, in the right neighborhood the 3D line BG is obtained by the region-based stereo matching and fitted with the cubic spline function. The right point of the line BG is extended along the tangent direction to intersect with the left incident ray CLF at the point CL. The space point C is obtained.

If we set threshold ( 0)T T > and C FZ Z ZΔ = − , then

Figure 3. The sketch map of occlusion detection

Z TZ T

Δ ≥⎧⎨Δ <⎩

occl udedNot occl uded

(3)

Here, it supposes that the occlusion is detected. In order to restore the true spatial structure, the image points CL and CR will be adjusted to converge at the space point C, and then the location of occlusion point H can be found.

If the point C is known, the relationship between disparity and depth in the weak disparity micro stereovision model [7] can be expressed as

1 2

1 2

3

3

1 2

1 2

3

3

s i ns i n

c o sc o s

s i ns i n

c o sc o s

L LM L m m M L

L LC m m C

LM L m

LC m

R RM R m m M R

R RC m m C

RM R m

RC m

x d f xx d f x

x fx f

x d f xx d f x

x fx f

θθ

θθ

θθ

θθ

⎧ + + =⎪ + +⎪⎪ +

=⎪+⎪

⎨− −⎪ =⎪ − −

⎪+⎪ =⎪ +⎩

(4)

From (4), the right image point RC of the occluded point can be obtained.

In order to solve (4), two different situations are considered as weak disparity micro stereovision model.

1) 2 0Lw ≠ or 2 0Rw ≠

（a）

θP

Z

WRO1R R

W R

W

RRM RC

rT

2R

FR

Hr

rH

RM

FR2

W

LO

W

R

L1L

W

lH

LC MlH

L

lT

F L

O

F L

MLL

Y

X

L

D i

' '

' '

ω L

ω R

A

FL FR

OL OR

OC

B C

D

F

G

H

DL

DRH C CRHRL L O X

Z

（b）

672

The point C and CL are known, therefore the weak disparity micro stereo vision model formula can be expressed as

3'3

2 2

2 2

2 2

cos

( sin sin )sin sin

L

L

L RR R L L L R

L RL R R L

xxx x

w ww w

θ

β θ β θθ θ

−Φ=

Φ + Φ=−Φ − Φ

(5)

1 2 1

2 2 2

2 1

2 2

sin sec( sin sin )

sin sec( sin sin )

L RR R L R

L RR R L L L R

R LL L R L

L RR R L L L R

x w wx w w

w ww w

β θ θβ θ β θβ θ θ

β θ β θ

Φ=Φ + Φ

Φ+Φ + Φ

(6)

From (5) and (6), the following relation can be obtained.

( )

( )

3 11

2 2

1 2 1

1 2

2 2 2

3

sin cossin sec

( sin sin )sin sec

( sin sin )sin cos

L R L LR

R R L RL

R R L L L RL

L L R LL

L R R L L L R

R L L L

xw

x

w xw x

x wx

θ θ ϕβ θ θ ϕ

ϕ β θ β θβ θ θ

ϕ β θ β θθ θ

Φ − Φ=

Φ

= Φ + Φ

− Φ

= Φ Φ + Φ−Φ − Φ

(7)

Then, the right image point RC can be expressed as

( )

( )

( )

'

3 1

2 2

2 3

2 2

3

sin cossin sec

sin cos[ ( sin sin )

sin cos ]

L R L L

R R L RR L

L R L LL

L R R L L L R

R L L L

xx

w xx w

x

θ θ ϕβ θ θ ϕ

θ θβ θ β θθ θ

Φ − Φ⎡ ⎤⎢ ⎥Φ⎢ ⎥⎢ ⎥= Φ − Φ⎢ ⎥

Φ Φ + Φ⎢ ⎥⎢ ⎥−Φ − Φ⎣ ⎦

W (8)

2) 2 2 0L Rw w= =

Similarly, the right image point RC can be expressed as

( )'

L R 3

( sin sin )coscos sec

0

L R L L R LR

L R L L Rxβ θ β θ θ

β β θ θΨ −Φ +⎡ ⎤⎢ ⎥Φ Φ −Φ= ⎢ ⎥⎢ ⎥⎣ ⎦

W (9)

where

( )1

L R 3

sec ( sin sin )

cos

LL L R L R R L

L L

w

x

θ θ θ

β β θ

⎡ ⎤Ψ = Φ Φ − Φ + Φ⎣ ⎦− Φ

After the right image point RC of the spatial point C is

obtained, in the range of DH, there are no points occluded.

When the left image is as source image and the point DL is as a starting point, the region based stereo matching is accomplished to obtain the left image point HL of the spatial point H.

In the range of HRCR which is occluded, the smooth restraint is not satisfied because a point corresponds to multi points. However, we find a key to recover the range occluded according to the imaging law. It is that the range occluded obeys reversing correspondence and the other range obeys sequential restraint.

Consequently, in the range of HC where the points are occluded, when the point HL is as a starting point, the region based reversing stereo matching and 3D reconstruction is accomplished to obtain 3D information of occluded part.

IV. EXPERIMENTAL RESULTS To illustrate the effectiveness of the micro stereo correction

algorithm, the microchannel is measured. The measurement results are shown in Fig. 4. To highlight the block position, the data points are not shown. Fig. 4(a) is the reconstruction points cloud without refraction correction and occlusion correction. Fig. 4(b) is the reconstruction points cloud with refraction correction and without occlusion correction. Fig. 4(c) is the reconstruction points cloud with refraction correction and occlusion correction.

The middle line in Fig. 4(c) is the detected occlusion position. It is not the lower left line in Fig. 4 (b) and it is larger than Fig. 4 (b) in the depth direction. It is consistent with the previous theoretical analysis.

From Fig. 4 we can see that the effect of the refraction and occlusion in measurement is mainly reflected in the lower bottom and high of the micro-channel, it less impact on the upper bottom. The actual channel is generally symmetrical, but the lower bottom of the measurement results without occlusion correction is side higher as shown in Fig. 4(b).

V. CONCLUSIONS In the microfluidic chip detection, twice refraction in cover

plate can lead to the non-uniform of magnification in the entire field of view and cause focus error, chromatic aberration and visual field error. This can result in great measurement error for the micro channel. The refraction correction algorithm combines twice optical path structure of SLM to present twice refraction variable refractive index correction algorithm based on SLM.

The occlusion correction algorithm can realize occlusion detection, occlusion correction and correction verification in real 3D space. This method draws essential character of occlusion and weak nonlinear character of the weak disparity micro stereovision model, adopts occlusion critical points in variable neighborhood to position, matches the gray feature points forward and reverse local areas determined by edges and verifies orthogonality of 3D space occlusion critical points to excavate depth information which is transformed from 2D image space to 3D object space. Moreover, the depth

673

information of occluded part can also be recovered successfully.

(a)

(b)

(c)

Figure 4. The comparison of the measurement results between fore and after correction

Relaxed iterative method can increase the stability of the algorithm and improve matching and measurement accuracy.

These algorithms quantize the influence of cover plate and occlusion on the measurement accuracy, and realize non-destructive, non-contact and precise 3D measurement of a hyaloid and closed container.

ACKNOWLEDGMENT The work is supported by the National Natural Science Foundation of China under grant no 61001158 and Specialized Research Fund for the Doctoral Program of Higher Education (SRFDP no 200801511027) and the Fundamental Research Funds for the Central Universities (no 2011JC019).

REFERENCES [1] Y. H. Chen and S. H. Chen, “Analysis of DNA fragments by microchip

electrophoresis fabricated on poly (methylmethacrylate) substrates using a wire-imprinting method,” Electrophoresis, 2000, 21(1) , pp.165-170.

[2] H. Y. Wang, A. K. Bhunia and C. Lu, “A microfluidic flow-through device for high throughput electrical lysis of bacterial cells based on continuous DC voltage,” Biosensors and Bioelectronics, 2006, 22, pp.582-588.

[3] M. Abdelgawad and A. R. Wheeler, “Low-cost, rapid-prototyping of digital microfluidics devices,” Microfluid Nanofluid, 2008, 4, pp.349-355.

[4] L. Martynova, L. E. Locascio, M. Gaitan, G. W. Kramer, R. G. Christensen and W.A. Maccrehan, “Fabrication of plastic microfluid

channels by imprinting methods,” Anal. Chem, 1997, 69(23) , pp.4783-4789.

[5] W. W. Y. Chow, K. F. Lei, G. Shi, W. J. Li and Q. Huang, “Microfluidic channel fabrication by PDMS-interface bonding,” Smart Mater. Struct. 2006,15, pp. 112-116.

[6] K. S. Ryu and C. Liu, “Precision Patterning of PDMS Thin Films:A New Fabrication Method and Its Applications,” Sixth International Symposium on Micro Total Analysis System (mTAS), Nara, Japan. 2002.

[7] C. Liu, W. Pei, S. Niyokindi, J. C. Song and L. D. Wang, “Micro stereo matching based on wavelet transform and projective invariance,” Meas. Sci. Technol. 2006,17, pp.565–71.

[8] C. Liu, W. Pei, Z. Y. Xia, S. Niyokindi, J. C. Song and L. D. Wang, “Wavelet transform based 3D scattered data processing in binocular micro stereovision system,” Key Engineering Materials, 2005, 291-292, pp.673-683.

[9] U. R. Dhond and J. K. Aggarwal, “Structure from stereo - A review,” IEEE Trans on Systems Man and Cybernetics, 1989, 19 (6), pp.1489-1510.

[10] P. S. Toh and A. K. Forrest, “Occlusion detection in early vision,” International Conf on Computer Vision, Osaka, Japan, pp.126–132, December 1990.

[11] G. A. Triantafyllidis, D. Tzovaras and M. G. Strintzis, “Occlusion and visible background and foreground areas in stereo: A Bayesian approach,” IEEE Trans on Circuits and Systems for Video Technology, 2000,10 (4), pp.563-575.

[12] C. L. Zitnick and T. Kanade, “A cooperative algorithmfor stereo matching and occlusion detection,” IEEE Trans on Pattern Analysis and Machine Intelligence, 2000, 22 (7), pp.675-684.

[13] S. Birchfield and C. Tomasi, “Depth discontinuities by pixel-to-pixel stereo,” International Journal of Computer Vision, vol. 35(3), pp.269-293, December 1999.

[14] T. Kanade and M. Okutomi, “A stereo matching algorithm with an adaptive window: Theory and experiment,” The IEEE Int’l Conf on Robotics and Automation, St. Louis, USA, 1991(2), pp.1088-1095.

[15] P. Werth and S. Scherer. “A novel bidirectional framework for control and refinement of area based correlation techniques,” IEEE 15th Int’l Conf on Pattern Recognition, Barcelona, Spain, vol. 3, pp. 730–733, 2000.

[16] O. Watanabe and K. Fukushima, “Stereo algorithm that extracts a depth cue from interocularly unpaired points,” Elsivier Neural Networks, vol. 12(425), pp.569-578,1999.

[17] Q. M. Luo, J. L. Zhou,S. S. Yu and J. Li, “Stereo matching and occlusion detection with integrity and illusion-sensitivity,” Journal Of Computer Research And Development, Vol.40(2), pp.297-303, 2003.

[18] S. S. Intille and A. F. Bobick, “Disparity-space images and large occlusion stereo,” Proc. ECCV94, pp.179-186, 1994.

[19] N. Morris and K. N. Kutulakos, “Dynamic refraction stereo,” Proceedings of the 10th international conference on computer vision , 2005, pp.1573-1580.

[20] R. Ferreira, J. P. Costeira and C. Silvestre, “Using stereo image reconstruction to survey scale models of rubble-mound structures,” Proceedings of the First International Conference on the Application of Physical Modelling to Port and Coastal Protection, 2006, pp.1-10.

[21] Z. Y. Wang, D. P. Li and Y. Shan, “Image distortion analysis for underwater laser scanning system,” OCEANS, Proceedings of MTS/IEEE, 2005, 3, pp.17-23.

[22] S. M. Soloff, R. J. Adrian and Z. C. Liu, “Distortion compensation for generalized stereoscopic particle image velocimetry,” Measurement Science and Technology, 1997, 8, pp.1441-1454.

[23] A. K. Prasad and K. Jensen, “Scheimpflug stereocamera for particle image velocimetry in liquid flows,” Appl. Opt. 1995, 34, pp.7092-7099.

[24] W. Pei and Y. Y. Zhu, “The variable refractive index correction algorithm based on stereo light microscope,” Measurement Science and Technology, 2010, 21, pp.1-7.