Application of POD Analysis at Airbus

Ulf SCHNARS, Andreas KÜCK, Airbus, Bremen, Germany

Abstract. Within in the scope of NDT technique verification projects performed at Airbus, POD analysis normally aims at determination of a90|95. This is the size of a flaw that has a probability of detection of 90% at a confidence level of 95%. In such projects, reliability of an NDT technique is to be shown, usually by an experiment. POD analysis at Airbus is based upon following POD analysis methods: a) hit/miss POD analysis (also called pass/fail POD analysis); b) signal response POD analysis: also the signal amplitudes are taken into account; c) alternative analysis approaches (e. g. the "29/29 method"). Applications of these methods are described in this paper.

1. Introduction and Overview

At Airbus, new NDT methods / applications for production or in-service have to be qualified. It has to be demonstrated, that the methods fulfil requirements concerning flaw detection probability. The ”Probability of Detection“ (POD) curve is a plot of the detection probability versus flaw size (e. g. crack length). The usual Airbus requirement is to detect flaws with a probability of 90% at a confidence level of 95%. This flaw size is abbreviated as a90Ι95.

POD is based on Airbus standard AITM 6-0014 (Probability of Detection). Mainly three statistical methods are used to calculate POD: - the “29 / 29”-method - hit/miss analysis according to Berens [1], which is an adaptation of Cheng, Iles [2,3] - signal response analysis according to Berens [1]

The aircraft structure has to be designed in such a way, that missing of flaws <a90Ι95 is tolerable.

2. The 29/29 Method

The 29/29-method can be applied to determine a90|95 if there are n inspection trials of which there are maximum d failures in detecting a90|95. This method is based upon a binomial approach (corresponding equations can be found e. g. in ISO 14560:2004 Annex B):

( ) %100110

⋅⎥⎦

⎤⎢⎣

⎡−⎟⎟

⎠

⎞⎜⎜⎝

⎛−= −

=∑ ini

d

iU pp

in

C

where CU is the desired confidence interval, and p is 1-POD. For CU=95%, POD=90%, and d=0 (no miss), n must be 29; this explains the name of the method. For CU=95%, POD=90%, and d=1 (one failed detection), n must be 46; etc. Further figures are given in table 1.

4th European-American Workshop on Reliability of NDE - We.3.A.1

1

ww

w.ndt.net/index.php?id=

8320

Number of trials n Number of successes y Number of failures n-y=d 29 29 0 46 45 1 61 59 2 76 73 3 89 85 4 103 98 5 116 110 6 129 122 7 142 134 8 154 145 9 167 157 10

Table 1. Number of trials versus permissible number of failures

As an example, crack detection in laser beam welded structures by eddy current testing for in-service applications is discussed [4], see figures 1 and 2. The defects were introduced artificially by a special mechanical tool. The samples were prepared with in total 22 artificial flaws. The inspections were performed by four inspectors. The result was that for defects with crack lengths between 2.0 mm and 3.0 mm, there were 22 x 4 = 88 hits and no misses; there were also no misses of flaws >3.0 mm. Therefore a90|95 = 3.0 mm.

Longitudinal crack

in welding joint

Longitudinal crack

in welding joint

C

mark of crack start and end

Crack length

C

mark of crack start and end

Crack length

C

mark of crack start and end

Crack length

Figure 1. POD example: crack detection in laser beam welded components

2

Figure 2. Qualification specimens

3. Hit/Miss Analysis

Hit/miss-analysis is applicable if • there is the tendency that the greater the defect size a, the greater its POD • only hits and misses (no signal amplitudes) are taken into account • there are both hits and misses in a sufficient number. (What is “sufficient“, cannot be stated in general.) • defect size a covers a sufficiently broad range

Calculation is based on a cumulative standard normal distribution approach published by P. Alan Berens (taking recourse to earlier papers from Cheng and Iles). Mathematical details are not discussed here, we refer to the literature, see [1] to [3].

As an example, detection of debonding between honeycomb and monolithic skin (opposite to the inspection surface) in rudder shells is discussed. Manual ultrasonic testing is used. Special qualification specimens were prepared from rudders that had been in service at an airline for several years. Defined debonded areas of varying sizes were introduced into these qualification specimens in defined areas. The inspectors tasked with the execution of the qualification tests came from different plants. The inspectors had been trained in ultrasonic testing and had all obtained the level 2 qualification as per the requirements of EN 4179. The position of the debondings and the size of the debonded areas in the qualification specimens were unknown to the inspectors.

The rudder shell with some typical findings is shown in figure 3. The POD curve calculated with the hit/miss approach is shown in figure 4; a90|95 = 42 mm. The calculation is performed by means of an Excel spreadsheet realising the necessary algorithms.

3

E

E

230

435

785

ø 25

ø200

ø 15 ø10

420

ø 70

ø 15

ø20

ø 30

ø 30

ø 25

250

350

450

D1

15 30

180 280

330

480

270 245

400 200

Figure 3. rudder shell used for POD trials

0%10%20%30%40%50%60%70%80%90%

100%

0 10 20 30 40 50 60 70 80

defect size a [mm]

POD

POD (confidence level) a 90| V

Figure 4. Excel spreadsheet with POD curve as result of hit/miss analysis

a90|950%

10%20%30%40%50%60%70%80%90%

100%

0 10 20 30 40 50 60 70 80

defect size a [mm]

POD

POD (confidence level) a 90| V

a90|95

4

4. Signal Response Analysis

Signal response analysis is applicable if • there is the tendency that the greater the defect size a, the greater its POD • defect size a covers a sufficiently broad range • the signal response â is taken into account Calculation is based on an approach published by Berens [1] to [3], not discussed in detail here.

As an example, high frequency eddy current inspection of suspected areas in castings is discussed here. The defect extension in depth was the important parameter to be quantified. Therefore a signal response analysis was performed using defect extension in depth as defect characteristic; result: a90|95 = 1.2 mm (defect extension in depth). Also for signal response analysis, the calculation is performed by means of an Excel spreadsheet realising the necessary algorithms.

5. False Alarm Probability

False alarm probability pf at a confidence level of 95% is calculated by an equation given in Airbus standard AITM 6-0014 (this equation is a kind of “inverse” presentation of a binomial equation):

xnx

xnxf Fxxn

Fxp

22;22%;95

22;22%;95

)1()1(

2

−+

−+

⋅++−

⋅+=

where n is the number of inspections on flawless areas, x is the number of false

calls, and F is the quantile of the F-distribution. In the example experiment described in paragraph 2, there were n = 156, i. e.

4 inspection turns on 39 flawless areas, and one false alarm call, so x = 1. Therefore the false alarm probability at a confidence level of 95% is pf = 3.0%. At Airbus, there is a recommendation to consider revision of an inspection technique whose false alarm probability is greater than 3% at a confidence level of 95%.

6. Conclusions

In general, new NDT methods or applications have to be qualified at Airbus. The calculation of POD curves is part of the qualification process. The usual Airbus requirement is to detect flaws with a probability of 90% at a confidence level of 95%. In such projects, reliability of an NDT technique is to be shown, usually by an experiment. POD analysis at Airbus is based upon following POD analysis methods: a) hit/miss POD analysis (also called pass/fail POD analysis); b) signal response POD analysis: also the signal amplitudes are taken into account; c) alternative analysis approaches (e. g. the "29/29 method"). Applications of these methods are described in this paper.

5

7. References

[1] Berens, Alan P. (1989): NDE Reliability Data Analysis. In: Metals Handbook, Vol. 17, AMS International, Metals Park, Ohio, 689-701. [2] Cheng, R. C. H; Iles, T. C. (1983): Confidence Bounds for Cumulative Distribution Functions of Continuous Random Variables. In: Technometrics, Vol. 25 (No. 1), 77-86. [3] Cheng, R. C. H; Iles, T. C. (1988): One-Sided Confidence Bounds for Cumulative Distribution Functions. In: Technometrics, Vol. 32 (No. 2), 155-159. [4] T. Meier (2007): Eddy Current Laser Beam Welded Procedure Qualification Investigation (Airbus document)

6