191

AN APPROACH TO THE POD BASED ON REAL DEFECTS USING DESTRUCTIVE TESTING AND BAYESIAN STATISTICS

Daniel Kanzler1, Christina, Müller1, Martina Rosenthal1, Uwe Ewert1,

Jorma Pitkänen2 1 BAM Federal Institute for Materials Research and Testing; Berlin, Germany;

daniel.kanzler@bam.de, christina.mueller@bam.de, uwe.ewert@bam.de 2 Posiva Oy; Eurajoki, Finland; jorma.pitkanen@posiva.fi

Keywords: Reliability, Probability of Detection, real defect testing, Bayesian statistics, Radiographic Testing ABSTRACT The assessment of the Probability of Detection (POD) is used to evaluate the reliability of the non-destructive testing (NDT) system. The POD is required in industries, where a missed flaw might cause grave consequences; the POD could lead to wrong conclusion or be even invalid if only the artificial defects are evaluated. The POD based on real flaws is needed. But handling of real flaws for the evaluation differs from handling of artificial defects and a small amount of real flaws can lead to a not significant result or even to incorrect results. The objective of the work is on one hand to present an approach to describe the real flaws and on the other hand to obtain a significant result. Two steps are necessary to assess a NDT system based on real flaws. First we evaluated the correlation between the NDT signal and the real size of the flaw from the destructive testing. Secondly we used a statistical approach based on the Bayesian statistics to assess a POD in spite of the small amount of data. The approach allows including information of the POD evaluation of artificial defects in the assessment of the POD of real flaws. INTRODUCTION The Finnish company POSIVA is developing a concept for the final deposit of high-level radioactive waste. The deposit is planned as the so-called engineering barrier system. A multi-barrier system encases the nuclear waste and protects the nature and the human being from the danger of the radioactive material. One important element of this system is the copper canister, which represents the barrier against corrosion.

POSIVA introduces and applies various non-destructive testing (NDT) methods to ensure, that the canister is safely sealed and no flaws jeopardize its functionality. It is essential to evaluate the functionality and the reliability of the used NDT methods. The most widespread procedure is to calculate the probability of detection (POD) depending on flaw parameters. Using the function of dependence on flaw size and its 95% confidence bound, the value “a90/95” can be defined which indicates the flaw size which will be found with 90% POD in 95 from 100 cases when the experiment is repeated. In this way the POD evaluates the largest size of the parameter, e.g. the size, which cannot be found with sufficient probability.

Most of the evaluations are based on artificially made reference defects. Such reference-defect-based PODs are required by various standards. Nevertheless, along with POD based on artificially made defects, a POD based on real flaws, which may appear in the production process, is particularly vital for highly sophisticated systems and in case of processes whose failure may lead to serious consequences. However, on the one hand producing real flaws might produce high costs and sometimes is hardly possible. On the other hand a big amount of data on various kinds of flaws is needed for the statistical analysis, which is the mathematical basis the POD operates upon.

In this work we consider an approach how to handle real flaws for the evaluation of the NDT system with POD and discuss a procedure for obtaining data from both metallography and data from radiographic testing. Another aim is to introduce a statistical method, which would support the small amount of data on real flaws with the knowledge obtained from the evaluation of artificial defects.

192

This statistical approach is based on the Bayesian statistics. We used radiographic and metallographic

data from the electron beam welded copper canister, which are part of the Finish project mentioned

above.

We may not mix the data of experiments on real flaws and the data of experiments on artificial

defects to create a common distribution for the calculation of the POD. But in the Bayesian approach

we consider to support the data of the experiments of real flaws with the prior knowledge which we

possess before these experiments. The prior knowledge is achieved from experiments with artificial

defects. The so-called likelihood knowledge is built up of the data from experiments with real flaws.

Both prior knowledge and likelihood knowledge can be combined in the posterior information. The

Bayesian approach allows us to express the posterior information and the prior and likelihood

knowledge through different distribution functions.

The calculation of the POD is based on the normal distribution. The POD curve can be

expressed by a normal distribution and a parameter for the confidence band. We may calculate the

distribution of the artificial flaws as the prior knowledge and combine it with the distribution of the

likelihood knowledge to one posterior function in form of a distribution as a basis for the POD

calculation. The expected benefit is the smaller confidence band and the better estimation of the

distribution parameters. In this work we discuss the procedure to build a POD data pool for real flaws

out of metallographic and radiographic data, before we use the Bayesian approach to create a useful

posterior POD evaluation. To verify the obtained results we calculate the POD based on a bigger

amount of real flaws without applying the Bayesian statistical approach.

EVALUATION OF NON DESTRUCTIV TESTING SYSTEMS

Long-term storage for high active nuclear waste

The question how to handle the high radioactive waste, which is produced by the nuclear power plants,

remains open so far. The aim is to isolate the waste until the radioactivity has decreased to level safe

for the environment. One approach for the isolation is the engineering barrier system proposed by the

Finnish company POSIVA [1]. The barrier system consists of multiple independent technical and

natural barriers: the first two are Finnish bedrock and bentonite clay around disposal canisters (Figure

1a) [1]. Another important barrier is the canister in which the nuclear fuel will be encapsulated (Figure

1b). The core of the canister is made of cast iron has to withstand mechanical loads imposed on the

canister. The shell of the canister, made of copper, has to prevent the contact of the radioactive fuel

with the environment.

Figure 1: Multiple technical and natural barriers and the canister for encapsulation of nuclear fuel

(POSIVA)

After the waste is put inside the canister, the copper lid is welded to the copper tube by electron-

beam welding (EBW) or friction stir welding. EBW is a fusion joining process that produces a weld by

impinging a beam of high energy electrons to heat the weld joint. Even though the beam is tightly

a b

193

focused and zone affected by the heat is narrow, some flaws might develop in the weld and as well as

in the surrounding area [2].

For this reason the weld has to be inspected with several NDT methods, to ensure there are no

flaws, which might jeopardize the function of the canister. This testing is done before the canister is

put into the final repository. The estimation of the detection limit of the NDT system is an important

step within the assessment of the total reliability of the deposit system.

To make sure that the NDT system will successfully detect all flaws, it is important to identify

types which might occur in the weld during the welding process. Especially the pores are a typical

flaw for the welding of copper canisters. These flaws should be found by radiographic testing.

Moreover s caused by gun discharge, cavities and internal root s can be a problem for the functionality

of the copper canisters, too. These types of s can arise from the welding process and need to be

detected by at least one NDT method [2].

Non-destructive testing of the electron-beam weld

To assure the functionality of the canister and to be certain that no leakage occurs, NDT

methods are applied. The testing process is split in three steps. In the first testing trial we take artificial

reference s, which have a structure to similar that of the real s. Afterwards, the s are created by

intentionally changing parameters of the welding process. Finally, there is the inspection of a complete

welded copper shell in the production of the future waste canister. To ensure the detection of all types

of s, inspection with the four different NDT methods is planned. The focus of this work is on the

radiographic testing (RT). Besides ultrasonic testing, eddy current testing and visual testing with a

remote camera are used [3].

The practice showed that inspection systems, when applied at the extreme of their capabilities,

do not detect all flaws of the same size [4]. Repeated inspection of the same defect does not

necessarily detect it either. This is the reason to introduce the concept of reliability in applications

where every missed flaw can lead to severe consequences.

Probability of Detection

In the 70th Berens 5) created a POD approach to estimate the reliability of eddy current testing

in finding cracks. It is the best known and widespread method and is also used since then in a lot of

different industrial branches as a characteristic magnitude for quality. The maximum amplitude is

plotted versus the significant defect parameter causing the signal (Figure 2a) – which is in this case the

depth of the crack (size) 5). The plotted graph is called â vs. a graph. For an easy interpolation and

evaluation of the relation between the chosen parameter and the measured amplitude, both axes can be

transformed to get a linear relation – for example through the logarithm of the axes.

With the deviation and the mean of the data it is possible to calculate the POD for every

parameter status. It is summed up in the POD curve with a confidence band (Figure 2b).

Figure 2: â vs. a-graph (a) and probability of detection curve (b)

The POD curve with the lower 95% confidence band is a typical way to present the capability of

the NDT system to detect a flaw [5]. Those types of curves were applied for simple cases where it is

a b

194

assumed that the POD of the defect depends only on the defect size and no other influence of the

defect or the component is taken into account.

To guarantee the integrity of the canister, the size of the defect that is detected with 90%

probability and 95% confidence, has to be determined. This defect size is also called a90/95. If the

critical defect, which is necessary to be found, is much bigger than the a90/95 then the NDT method is

acceptable. If the critical defect is smaller than a90/95 then it is necessary to improve the method or to

use another method.

BAYESIAN APPROACH

However, the evaluation of the reliability of NDT methods for highly developed production processes

is much more complicated. In these processes the comprehension of real defects, which are generated

during production, is essential. But the existing data are too few to create a statistically correct

evaluation for a repeatable evaluation while using only the real data.

The experiments with the artificial defects provide the evaluation of the reliability with

important basic information, i.e. general suitability of the NDT method for the inspection task and

trend behavior concerning size, angle and shape, which are right now seen as a separate analysis. It is

not correct to mix both data of different kind of defects because of the different structure, varying

geometry and, therefore, the different behavior of the NDT system. But there is no objection to create

an evaluation of the system based on both data.

In this work, the combination will be made by the Bayesian approach. How to combine real

defects and artificial defects shall be shown in the section “method”. The section “data” uses the

approach to handle RT data of real and artificial defects. In the section “discussion” an evaluation of

the results and an overview over the future steps will be presented.

General idea of the Bayesian statistics for reliability

The following equation describes the Bayesian statistics, which will be applied to our approach. In our

case the detection of a defect is important. The detection should be called A. We would like to know

the probability of detection under the condition, that there is a defect with specific parameters B. Thus,

the result we want to get is P(A|B).

This equation is solving the challenge of how to improve the knowledge of A under the

condition B. There is a prior knowledge P(A), which was collected before the experiments and

additional knowledge from the results of the experiments (P(B|A)), which were done to “learn” more

about the A. In a nutshell, the pre-knowledge P(A) combined with additional knowledge P(B|A) leads

to improved knowledge of A, P(A|B) 6).

For the use in the reliability evaluation the P will be a normal distribution function. So the

whole approach combines two different distribution functions to obtain a combined posterior

distribution P( A | B ). The reliability of the NDT systems should be mainly based on the real defects,

since this is the reality under production conditions. Therefore, the prior function P(A), which is also

often referred to as previous knowledge, consists of knowledge, which is known before the testing of

real defects. The artificial defects are useful to evaluate the basic response of the method and of the

equipment to flat or volumetric defects in the given material represented by flat bottom holes, notches

or side drill holes. These type of basic responses, determined by the mentioned experiments with

artificial flaws, but also computed modeling experiments or an estimation from the management,

which is based on facts 6) are well suited to compose the prior function. The assessment of well

defined artificial defects, according to the approach from Berens 5), provides with the estimator of the

mean and the variance of the reliability of the NDT system in detecting artificial defects.

The likelihood function P(B | A) will contain an amount of information from the experiments

with real defects. Similar to the artificial defects, a response of the NDT system to real defects will

(1)

195

create a distribution function of the likelihood function. The posterior function P( A | B ) will provide

then the response behavior from the real defects, supported by the behavior of the same NDT system,

while testing artificial defects. Through this combination the number of experiments and the amount of

state of the art information rises. This is an advantage over consideration of only real defects.

RESULTS: EVALUATION OF REAL DEFECTS

For the POD evaluation of the radiographic testing results the adequate parameters for the defect (a)

are necessary. In this work the parameter a for radiographic testing is defined as the penetrated length

of the defect 3). For the evaluation of real defects the determination of a is a challenge. Therefore

every real defect was evaluated with metallography 7). For the reconstruction of the defect the slices of

the metallography where rebuild in a spatial model. Figure 3a shows an example of a spatial model of

one of the defects. On the one hand the aim is to determine the real penetrated length of the defect

from the slices, on which the spatial model is based (Figure 3a (â)). On the other hand the aim is to

determine the indication of the radiographic picture which is caused by the defect (Figure 3b (â)).

Figure 3b shows one area of the indication after splitting them into parts, which matches with the

slices of Figure 3a.

Figure 3: a: Spatial model of the metallographic slices. b: radiographic indication for the each slide

For the detailed comparison we chose as an example one slide in the article, which is marked

with blue color in Figure 3. In Figure 4 the slice 11 of defect D5 is shown. The first diagram shows a

histogram (profile of amount of the pixels) from the metallography while the last part of the picture

shows a profile of the grey values of the radiographic indication based on a median grey value for

noise which has the value 0. Because of the comparison between the amount of white pixels in the

metallographic picture and a mixture between grey values and amount of pixels of the RT indication

the y-axis cannot be interpreted easily. Nevertheless it can be understood as the correspondence of the

penetrated length of the defect and the X-ray intensity behind the defect, which is expressed by the

absorption law). But it is easy to see that metallographic indication and the slice of metallography fits

together from the position and form of the profile although there are still small variances.

196

In the common POD model the highest grey values and the adequate penetrated length are

evaluated 3). Therefore, the areas of the largest penetrated length in the metallographic picture should

be defined. It is the area, which corresponds to the signal, detected of one RT detector element (size of

0.4 mm) and hence one image pixel. Figure 4b shows as an example of the largest penetrated length

which can be detected by one pixel of the detector. This value (a) will be evaluated with the adequate

highest grey value in this area (â). In this procedure the data for POD for a (the penetrated length) and

for â (the grey value intensity) can be calculated for every slice of the metallography for every defect.

Figure 4: a: Comparison between profiles of metallographic and radiographic data.

b: Estimation of the penetrated length for the highest grey value

RESULTS: BAYESIAN APPROACH FOR THE POD

We searched for a procedure to create a POD with the small amount of available data. Since the

mixture of the data is not an option, we applied the Bayesian approach as mentioned before and

combined the small amount of data from the real defects with the data from experiments with artificial

defects (prior knowledge).

First we checked the linearity of the relation between a (defect size) and â (signal height),

expressed in the data points for the likelihood function and for the prior function. For the data used in

this work, a transformation to a double logarithmic coordinate system appeared adequate. The

maximum likelihood estimation (MLE) of the â versus a relation created a distribution of the data

points that relate to â in dependence of a.

Secondly we used the following two parameters of the MLE to define the distribution for the

Bayesian approach: One parameter is the mean value, which is defined as the linear function, and the

other parameter is the variance, which describe the scattering of the experimental values around the

linear function.

Furthermore, the calculated covariance matrix of the MLE provided estimation into the

goodness of the mean value and the variance value [9]. The confidence bound is defined by the

covariance matrix, by the amount of data depending, and by the level of confidence, which was

defined as 95%.

197

The estimated parameters and their confidence bound created a bivariate normal distribution for

the prior function and another one for the likelihood function. Each of the distribution functions

describes the behavior of the NDT system. We assume that the combined distribution with a scaled

amount of data provides a statistically significant result, which was used for the evaluation of the

testing method.

We combined the prior bivariate distributions of the artificial defects with the likelihood

bivariate distribution of the real defects to the bivariate posterior normal distribution function,

regarding the Bayesian theorem. With the combined distribution the further calculation of the POD

curve and the confidence bound were performed, as described in the literature for the POD calculation

(see [5], etc.). Therefore, the number of data for the posterior function, which has a large influence on

the width of the confidence band, was the sum of a scaled amount of prior data plus the amount of

likelihood data. For the calculation of the scaled amount of the prior data the equation of reference

[11] was used.

Due to the good understanding of the correlation of the signals and the defect size, we applied

the procedure on the data of radiographic testing. The data for real defects and artificial defects can be

seen in the double logarithmic graph in Figure 5. We supposed and justified the use of a linear

correlation between the â, which is the contrast of the RT indication and a, which is the penetrated

length of the defect, which is proportional to its size.

Figure 5: Data of artificial defects and real defects for RT

The amount of data and the level of confidence are summed up in the parameter γ which

describes the goodness of the estimators for the distribution model. Therefore, the calculation of the

adequate weight of the information for the posterior function was necessary (for the detailed

calculation see [11]). We calculated the value γ with the amount of posterior experiments and defined

confidence bound at 95% (see [9]). According to Cheng [9], we calculated the corresponding γ-value,

which equaled 5.211 for the amount of posterior experiments, which equaled 29. The posterior POD of

the combined data in comparison to the POD of only few real defects is drawn in the Figure 6.

198

Figure 6: POD of real defects with and without the Bayesian approach

As a comparable key parameter we chose the size of the defect that is detected with 90%

probability and 95% confidence, so called a90/95. The number of experiments increased from 24 of

the real defects to 29 of the combined amount of defects. Through this increased number of data and

the joint values of the mean and the variance value, the a90/95 decreased from 1.2 mm to 1.0 mm.

The comparison with the POD calculated with a three times greater pool of real defects which

was in this investigation available – figure 7 - shows the POD curve – shows that the Bayesian

approach provided a better result as only the few real defects. The a90/95 from the Bayesian approach

of 1.03 mm was verified by the result from bigger data pool of real defects of 1.05 mm.

Figure 7: POD with a bigger amount of data.

CONCLUSION

Our approach reveals the characteristic a90/95 based on a small amount of real data that is comparable

with the evaluation of a bigger amount of data. The approach improves the estimation of the statistical

parameters and provides the confidence bound with a scaled amount of data from artificial defects and

real defects. The Bayesian statistics offer an approach for combining the data in a statistical correct

way. Instead of having a small amount of data, we have to pay close attention to the requirements and

to deal with a more sophisticated approach.

199

We broaden the description of the linear relation between the defects and the signal in this work,

due to the importance of the relation of the â vs. a in the evaluation process. We provide a framework

for the radiographic testing to consider the relation to transform it into a linear correlation. Two useful

tools are the metallographic examination of the real defects to define the real size of the defects and

the reconstruction to get an idea of the structure of the defect. Furthermore, the requirements

concerning the normal distribution, written in MIL-HDBK-1823A [12], are essential and should be

met. Especially the above-mentioned requirement of the bivariate normal distribution is an important

fact, which should be tested.

In conclusion, we provide a useful method in the case of only a small amount of data to create a

useful statistical correct evaluation of the often used POD. A reasoned combination of the data obtains

a remarkable improvement of the result without changing the usual basics of the POD.

At the future work we will focus on the testing of statistical requirements, on using other

distributions, beside the ones previously used, on other NDT methods (e.g. Eddy Current and

Ultrasonic testing [13][14]), and on the application of the Bayesian approach with multi-parameter-

and data-field-POD [15][16].

REFERENCES

1) Pitkänen J, Salonen T, Sandlin S, Ronneteg U, “Defect detectability in EB-welded copper disposal

canister with 9 MeV accelerator”, 6th

International Conference on NDE in Relation to Structural Integrity

for Nuclear and Pressurized Components, Budapest 2007 2) Pitkänen J. Posiva Report 2010-04, Inspection of bottom and lid welds for disposal canisters, September

2010, 98p.

3) Kanzler D., Milsch S., Pavlovic M., Müller C., Pitkänen J., “Concept of total reliability of NDT methods

for inspection of the EB weld of the copper canister used for a long-term storage of spent nuclear fuel”,

Proceedings of the Eighth International Conference on NDE in Relation to Structural Integrity for Nuclear

and Pressurized Components, Berlin, 2010

4) Rummel W. D., “Recommended Practice for a Demonstration of Nondestructive Evaluation (NDE)

Reliability on Aircraft Production Parts”, Materials Evaluation, Vol. 40 August 1982

5) Berens A P, NDE Reliability Data Analysis – Metals Handbook, Volume 17, 9th Edition: Nondestructive

Evaluation and Quality Control, ASM International, OH, 1989

6) Bury, Karl V. John Wiley & Sons, J. W. (Ed.) Statistical Models in Applied Science Wiley Series in

Probability and Mathematical Statistics, 1975

7) Pitkänen, J.; Paussu, R.; Pohjanne, P.; Virkkunen, I.; Kemppainen, M.; Lipponen, A.; Sarkimo, M.;

Simola, K. & Reddy, K.-M. Metallographic study of Detected Indications in EB-Copper welds for

Verifying the NDT Reliability of Inspection 8th International Conference on NDE in Relation to

Structural Integrity for Nuclear and Pressurized Components September 29 - October 1 2010

8) Pavlovic, M., Müller C., Takahashi K., Pitkänen J., Ronneteg U., 2009, Multiparameter influence on the

response of the flaw to the phased array ultrasonic NDT system., 4th European-American Workshop on

Reliability of NDE, 2009

9) Cheng R.C.H., Iles T. C., “Confidence Bands for Cumulative Distribution Function of Continuous

Random Variables”, Technometrics, Vol. 25, No. 1. 1983

10) Cheng R.C.H., Iles T. C., “One-Sided Confidence Bands for Cumulative Distribution Functions”,

Technometrics, Vol. 30, No. 2. 1988

11) Stange K., “Bayes-Verfahren, Schätz- und Testverfahren bei Berücksichtigung von Vorinformationen”,

Springer-Verlag, Berlin, 1977

12) MIL-HDBK-1823A, “Nondestructive evaluation system reliability assessment”, Department of Defense

Handbook, 2009

13) Pitkänen, J., Salonen, T., Sandlin, S. & Ronneteg. U., “Defect detectability in EB-welded copper disposal

canister with 9 MeV accelerator “ , 2008

14) Pitkänen J. “Inspection of Bottom and Lid Welds for Disposal Canisters”, Posiva 2010-04, September

2010

15) Pavlovic, M.. Müller, ,C., Takahashi:, K.., Pitkänen, J., Ronneteg, U., 2009, Multiparameter influence on

the response of the flaw to the phased array ultrasonic NDT system., 4th European-American Workshop

on Reliability of NDE, 2009.

16) Takahashi, K., Pavlovic, M., Ronneteg, U., Pitkänen, J., Müller, C., 2009, POD from C-Scan Data for

Reliability Analysis of Ultrasonic NDT, 4th European-American Workshop on Reliability of NDE, 2009.