use of high resolution real-time image processing techniques in generation and analysis of espi...

Optics and Lasers in Engineering 8 (1988) 109-121

Use of High Resolution Real-time Image Processing Techniques in Generation and Analysis of

ESPI Fringe Patterns

D. Kerr and J. Tyrer

Loughborough University of Technology, Loughborough, Leicestershire, UK

(Received 19 January 1987; revised version received 8 July 1987; accepted 10 July 1987)

A B S T R A C T

A high resolution (1024 × 1024) image processing system has been employed in the analysis of ESPI fringe patterns. This machine utilises real-time, high bandwidth digitisation and recursive video techniques to produce high quality surface displacement fringes requiring a minimum of enhancement.

Operational parameters of the system are discussed and a basic processing methodology is outlined. A flexible set of software algorithms has been developed and results from typical image fields are presented.

A technique of phase stepping has been evaluated and the effects of reduced modulation and signal-to-noise ratio on the resulting phase difference maps are examined.

1 I N T R O D U C T I O N

Electronic Speckle Pattern Interferometry (ESPI) is by now a well documented and established technique in the field of engineering surface metrology. 1'2 Akin to holographic interferometry, ESPI is a very powerful whole-field surface displacement measurement technique which is finding application within the engineering industry in a variety of areas ranging from quality assurance through NDT to research and development.

One major constraint which is at present preventing a more widespread industrial application of this technique is the inherent

109 Optics and Lasers in Engineering 0143-8166/88/$03-50 © Elsevier Applied Science Publishers Ltd, England, 1988. Printed in Northern Ireland

110 D. Kerr, J. Tyrer

complexity of the data produced, creating a requirement for specialiscd analysis skills in interpretation of fringe patterns.

In order to serve the needs of the engineering community some sort of automated analysis system is required which will produce easy to assimilate, accurate data in a variety of forms.

Much work has already been done using various processing techniques ~4 with varying degrees of success, towards a suitable system. However, there is still a degree of essential expertise involved at some stage to interpret any ambiguous data which may arise, for example, displacement directions or the relative phases of vibrational modes (see Fig. 1).

It is intended in this paper to describe a processing "methodology" which addresses the whole spectrum of a potential automatic analysis system, and to outline some of the hardware and software constraints placed on the system designer.

A wide range of analogue and digital processing techniques is now available, and as a starting point in the methodology, a system capable of

Fig. 1. Time averaged fringes on a vibrating car engine showing two modes in anti-phase.

Use of high resolution real-time image processing techniques 111

implementing some of these is described, and one particular approach is explored more closely.

Maoy techniques for fringe enhancement and position determination are already well developed and have been applied successfully in some systems, but this represents only part of the solution. Interpretation of the fringe data and reduction of these results into a format which accurately shows the displacement or vibration characteristics of the target surface has to be the final goal of any automated system. The end-user of the analysis equipment must be able to assimilate the results from a non-specialist viewpoint to facilitate application to real world problems.

The methodology is extended this far with a consideration of some possible methods of data presentation.

2 PROCESSING METHODOLOGY

Fringes produced by an ESPI system are not solid and continuous, and not always of good visibility (modulation depth). They are 'speckled' with high spatial frequency noise on which is superimposed the lower frequency pattern of interest. A typical pattern and some stages during its processing are shown in Fig. 2.

Although interpretation of the fringe pattern to yield a displacement vector is relatively straightforward once a fringe is located in space, such accurate location of a fringe contour, which may be quite complex, is a problem due mainly to noise.

Many of the established techniques of noise reduction, enhancement and marking ESPI fringes tend to be complicated and time consuming, and also something of an 'intuitive process' requiring intervention by skilled operators. Any avenue of approach can be reduced to a broad-ranging processing methodology as outlined below:

(1) Optical, analogue and digital pre-processing to achieve high visibility fringes.

(2) Digitisation of images into some sort of image processing system.

(3) Digital noise reduction and image enhancement if required. (4) Fringe position and marking. (5) Extraction of 2-D displacement data. (6) Combination with surface form data to provide 3-D

information. (7) Presentation of data at appropriate level.

112 D. Kerr, J. 7~vrer

Fig. 2. Stages in fringe processing: top left, original ESPI fringes. Bottom left, after low pass filter. Top right, after inversion and peak detecting. Bottom right, after

centre marking peaks and troughs.

Obviously, the more processing stages that exist, the more complex and time consuming the data reduction becomes. The slowest processes arc those performed in the image processing system and for this reason, work has tended to concentrate on producing the best possible quality fringe information in a high signal-to-noise ratio format to reduce digital processing to a minimum.

The images in Fig. 2 are typical of some of the above stages of ESPI fringe processing, the final stage being the result of a simple fringe marking routine.

The provision of unambiguous displacement data is essential and in particular during stages (5) and (6) the direction of any deduced displacement vectors must be taken into account. There are several variations of the normal ESPI method which can produce such data in a suitable format, and one of these, Digital Phase Stepping, was chosen as a starting point for investigation and a feasibility study.

Other methods include Heterodyning, 5 and Fourier Transform


processing 6 both of which are as yet to be evaluated under the above methodology analysis.

Work has been undertaken by Hariharan et al. 4, Robinson and Williams 7 and Creath 8 using the phase stepping approach to fringe analysis, in which displacement information in terms of intensity representation is converted to a phase representation. Originally developed for holography, the technique involves phase shifting of the reference beam and subsequent processing of the resulting images to reduce the intensity distribution to a phase displacement.

The technique has been adapted for ESPI, where each speckle point on the object's surface may be considered to reflect light with an amplitude and phase given by:

Ea cos (Wt + dpa)

By phase shifting the reference beam of the interferometer in a discrete 'step', this will be altered to:

E6 cos (Wt + dpd)

Three such equal steps will produce three different intensity values associated with each surface speckle; 11---> 13. The phase of the reflected light at each point is then given by (for 3 steps of 2:r/3 radians):

q~ = TAN-~(X/3(I1 - 13)/(212 - 11 - 13)) (1)

By producing phase 'maps' of all the surface speckle points before and after a deformation and then subtracting these maps, an image of the original object superimposed by saw-tooth profile fringes is produced. These fringes relate to surface out-of-plane displacement in exactly the same way as normal ESPI fringes, the phase difference at any point being proportional to the displacement.

When the phase difference across the fringe field exceeds an integer multiple of 2:r radians, a discontinuity occurs (see Fig. 5). These may be processed out after deducing the displacement direction.

Robinson and Williams 7 have identified the advantages of the phase approach over conventional speckle intensity correlation methods as follows:

(1) An inherently less noisy image results. (2) Areas of low modulation can be identified and removed. (3) Displacement direction interpretation is unambiguous. (4) A resolution of up to 1/30 of a fringe should be possible.

The application of this method to ESPI fringe processing and its place within the overall methodolgy will be considered later.


3 THE IMAGE PROCESSING SYSTEM

The ESPI device will produce data in digital format and so is ideal for interfacing to a digital computer for much of the analysis process. The fringe pattern may be observed most easily by display on a television monitor after passing the output signal through a suitable digital to video converter stage. In this way, an operator may at all times be able to view and compare the processed and unprocessed images.

A typical image processor system is shown in the layout drawing of Fig. 3. This represents a somewhat idealised system, and any proprietary device will vary according to specifications offered by the manufacturer. The layout represents a model from the top end of the range with high resolution and real time capability. It may consist of a 'stand alone' package or a custom-made modular system built up to specific user requirements. However it is configured, there are several basic areas of importance in relation to the processing of ESPI fringes.

The data capture system, or 'front end' processor consists of a digital or video input stage and analogue to digital converter. It is of particular importance that incoming video signals from ESPI are

INTERFACE MI CROPROGRAH 1 CONTROL CONTROL

or" ~ l RG8

II ~ II I ~ I VIDEO REALTIME

l IHAGE BUS SYSTEH

LDMA |NTERFACE I HE~ORY / IP,ta, GE PROC~SOR

DISK DISK

Fig. 3. A typical image processing system with real-time video capability.


sampled at an adequate rate (according to the Nyquist criterion) to avoid aliasing of the high frequency speckle information. This means that good resolution and linearity of response of this stage should extend out to at least 5MHz. Failure to provide such a high quality input stage will result not only in spurious aliasing frequencies due to undersampling, but also in inaccuracies from quantitative software routines such as level slicing, histogramming and non-linear filtering.

The ideal ESPI image processor should be able to cope with input data at TV (i.e. CCIR standard) flame rates combined with the desired resolution. Most systems currently available are stretched to their limits at 512 x 512 pixels with 8-bit video digitisation (i.e. 256 grey levels).

It is further desirable that the input processor has access to a recursive loop, so that processed data may be fed back to the input to enable integration, subtraction or filtering techniques to be employed. Such techniques will require the provision of a fast ALU to carry out the necessary computation.

It is of great importance that the system image memory is controlled and structured to an optimum for speed and ease of handling. It should be remembered that for processes such as phase stepping or heterodyning, several processed versions of the original image may have to be stored simultaneously. Allowances should be made for adequate storage, unless such images can be conveniently dumped to a mass storage device.

If the system is based around a pipeline or array processor, consideration should be given to providing adequate hardware and software control at a level appropriate to the operator. It may be necessary to provide a 'host' processor to run the system and allow programming at a high language or interactive level. Great care should be taken to ensure compatibility between host and system, particularly with regard to processing speed and communication bus architecture.

4 EXPERIMENTAL WORK

Work has initially concentrated on the digital phase stepping approach outlined in Section 2. In order to investigate possible ways of applying this method to ESPI fringe processing, it was decided to produce a tool kit of computer algorithms which would simulate an easily interpretable set of raw 'data' and the corresponding processed results. Variation of certain parameters of this data would enable the effects of modulation depth, noise and phase step errors to be examined.


As an extension to this work, a basic ESPI device was built (see Fig. 4) with a phase stepping facility incorporated into its reference beam. This consisted of a mirror mounted on a piezo-electric crystal (PZT) with appropriate control circuitry to enable it to be moved in discrete steps along its normal axis. The second part of the work involved the use of 'real' data in the form of phase-stepped fringes.

The software tool kit comprised algorithms for implementation of the phase mapping process, i.e. calculation of eqn (1) above, a digital sine wave generator with variable wavelength, amplitude, step length and signal-to-noise ratio (SNR) to create simplistic data images, a line by line averaging routine to measure modulation depth and a line histogram to interrogate the resulting images.

Initial tests of the digital phase stepping algorithm consisted of processing a series of computer generated sine wave patterns in order to produce a phase map which was easily verifiable. Two such maps were produced at 128 and 256 pixel wavelengths respectively and a difference map calculated by subtraction. Because of the one- dimensional, cyclic nature of the raw data, results could be easily predicted and interpreted. Phase maps from this stage are shown in Fig. 5.

The next stage involved investigation of the effects of low modulation

He Ne LASER ] ,.~

TELEVISLON

CAMERA

I FILTER & RECTIFY

F FRAME STORE

,3BJEETIVE LENS

v i ~ PLANE

t

i 0

FZT

1500v

COMPUTER IMAGE

PROCESSOR

Fig. 4. Experimental ESPI apparatus with phase stepping facility and image processor.


(a) (b) Fig. 5. (a) Sine waves and their corresponding phase maps with horizontal line profiles. Left hand pair, 256 pixel wavelenth. Right hand pair, 128 pixel wavelenth. (b) Phase difference maps of those in Fig. 5(a) showing discontinuity points; horizontal

line profile is shown in lower part of picture.

and r e d u c e d S N R on the phase maps; again at two wave leng ths values for ease of i n t e rp r e t a t i on .

Noise on the sine wave signal was g e n e r a t e d by supe rpos i t i on o f an a t t e n u a t e d speckle field image , se lec ted at r a n d o m on a l ine-for- l ine basis. T h e effects o f m o d u l a t i o n and S N R levels can be seen on images of Fig. 6 ( (a) and (b)) .

phase |

pixel n (a)

Fig. 6. (a) Effect of low modulation (signal amplitude) on phase difference map from Fig. 5(b) with horizontal line profiles shown below each map. From left to right; 6, 4, 3 and 2 bits of signal level. (b) Effect of low signal to noise ratio on phase difference map from Fig. 5(b) with horizontal line profiles shown below each map. From left to right;

1, 2, 3 and 4 bit levels of additive speckle noise.


(b) Fig. 6~(contd. )

The final stage of this investigation involved the use of the ESPI apparatus to measure and compare intensity (grey-level) modulat ion depth for small phase displacements at various points in the processing chain.

Modula t ion of signals direct from the TV camera were compared with those levels obta ined after passing the signal through the ESPI

,l =o

g r e y l e v e l

(a)

Fig. 7. (a) Grey level modulation histograms for various phase shifts of the speckle pattern produced from the frame store. Top left to bottom right, increasing in steps of

Jr/3. (b) As for Fig. 7(a) but image taken directly from the television camera.


(b) Fig. 7--(contd.)

rectifier, the modulat ion being measured and composed on a line-for- line basis such that the average intensity (grey level) value for a line of n pixels was given by:

n

1/n~I~ - (I + 6I)i (2) i = 1

where/,. = grey level of ith pixel

(I + 61) = grey level of ith pixel after a small phase step.

Such information was obtained over a range of I values by application of a range of voltage intervals to the PZT mirror. Results from this stage are illustrated in Fig. 7 ((a) and (b)).

5 DISCUSSION

Experimental data from use of the software tool kit has so far identified several areas of interest which are currently under closer investigation.

(1) The digitised output from the store or TV camera has a greatly reduced grey-scale range; sometimes as low as only ~ of the available levels.

(2) Modulation depth produced by phase stepping is much lower when measured directly from TV camera images than after


rectification in the ESPI store, highlighting the need tot analogue video processing before digitisation.

Figure 7 (a) shows a series of grey level histograms of speckle patterns from the ESPI rectifier. The quadrants represent increasing phase modulation from top left to bottom right.

Comparison with a similar histogram set of speckle images taken directly from the camera (Fig. 7 (b)) shows up the large difference in effective use of the total grey scale range.

(3) The digital phase stepping method is relatively insensitive to reduced signal amplitudes, and reasonable results are obtained with only 4 bit modulation. Figure 6 (a) shows phase difference maps from 128 and 256 pixel wavelength sine waves at various levels of signal modulation; from 6 bits (top left) to only 2 bits (bottom right). A reasonably good saw-tooth phase representation is retained (upper right) even at 4 bits or 0.0625 of the original amplitude level.

The algorithm is very sensitive to noise, however, as shown by Fig. 6 (b) where speckle noise has been added to the signals before processing in the same way as those in Fig. 6 (a). Signal distortion is acute and unacceptable for fringe analysis purposes at 5 bits of maximum amplitude with 3 bits speckle noise (lower left).

(4) The need for rapid processing algorithms has been highlighted by diffÉculties experienced when attempting to implement the phase calculation algorithms of eqn (1) (see Section 2). Restriction to integer calculation within the pipe-line processor together with minimised floating print arithmetic served to increase speed to a more acceptable level, but future routines will be re-written in machine code.

Returning to the methodology of Section 2, it may be seen that, even after the application of techniques such as digital phase stepping to produce high resolution, unambiguous fringe data, there still remains the problem of presenting thi:~ in a form useful to the engineer or skilled technician.

The task of locating fringe centres and so extracting two-dimensional displacement data is greatly eased by the provision of phase stepped fringes, or by such techniques as speckle decorrelation and integration." Several difficulties still, however, remain. Among these, perhaps the most exacting task is that of combining displacement data with information about the equilibrium state of the target surface, i.e. its contour shape before excitation or displacement.

It is possible to use ESPI to produce a contour map of the surface of


the object of interest before it is displaced or vibrated. This information may then be combined with fringe data obtained during excitation of the object to produce a three-dimensional surface plot. Work already done at Loughborough University of Technology by Bergquist and Montgomery l° has shown the feasibility of using the ESPI apparatus to generate 'tilt' fringes which contour the surface of the object in its rest position.

Combined with suitable perspective correction techniques, which may be implemented by a computer interation, the surface contour map can be used to produce an equilibrium 'reference' surface modelled within the host processor. By superposition on the reference surface of the displacement vectors from fringe analysis, a three-dimensional plot of the object with its characteristic modal or stress patterns can be generated and interrogated for accurate, quantitative data.

The three-dimensional plot can become part of another, larger data base regarding the object, thus allowing real time computer modelling or analysis by CAD machines---digital information generated in this way can be manipulated by a wide range of devices associated with the computer analysis field.

Whatever the format of the final data, the ESPI automated fringe analysis 'package' must be flexible enough to meet a large range of applications. The choice of how to apply the system must be decided by the user; whether the requirement is for inspection or quality assurance on a simple comparative basis, quantitative non-destructive testing, or complex research and development involving computer modelling and CAD/CAM.

As part of an on-going research program, the system designer must always be aware of the changing needs of industry and we are constantly seeking such feedback from users and potential users of the ESPI technique.

REFERENCES

1. J. N. Butters and J. A. Leendertz, J. Meas. Control, 4, (1971) 344. 2. R. Jones and C. Wykes, Optica Acta, 28, (1981) 7. 3. P. Hariharan et al., Appl. Opt. 22, (1983) 6. 4. P. Hariharan et al., Opt. Comms., 41 (1982) 6. 5. G. T. Reid, SPIE Proc., 376, 1982. 6. M. Takeda et al., J. Opt. Soc. Amer., 72 (1981) 1. 7. D. W. Robinson and D. C. Williams, Opt. Comm., 57 (1986) 26-30. 8. K. Creath, Appl. Opt., 24 (1985) 18. 9. P. C. Montgomery, Forward looking innovations in ESPI, PhD Thesis,

Loughborough University of Technology, 1987, p. 100. 10. B. D. Bergquist and P. Montgomery, SPIE Proc., 599-630, 1986.

use of high resolution real-time image processing techniques in generation and analysis of espi...

Documents