1 evaluation of safety critical software david l. parnas, c acm, june 1990
TRANSCRIPT
![Page 1: 1 Evaluation of Safety Critical Software David L. Parnas, C ACM, June 1990](https://reader030.vdocuments.net/reader030/viewer/2022032703/56649f4f5503460f94c71af6/html5/thumbnails/1.jpg)
1
Evaluation of Safety Critical Software
David L. Parnas, C ACM, June 1990
![Page 2: 1 Evaluation of Safety Critical Software David L. Parnas, C ACM, June 1990](https://reader030.vdocuments.net/reader030/viewer/2022032703/56649f4f5503460f94c71af6/html5/thumbnails/2.jpg)
Software Reliability
Nonetheless, our practical experience is that software appears to exhibit stochastic properties. It is quite useful to associate reliability figures such as MTBF (Mean Time Between Failures) with an operating system or other software product. Some software experts attribute the apparently random behavior to our ignorance. They believe that all software failures would be predictable if we fully understood the software, but our failure to understand our own creations justifies the treatment of software failures as random.
2841f07hamlet19oct30
![Page 3: 1 Evaluation of Safety Critical Software David L. Parnas, C ACM, June 1990](https://reader030.vdocuments.net/reader030/viewer/2022032703/56649f4f5503460f94c71af6/html5/thumbnails/3.jpg)
MTTF
Mean is an average.
What is the relationship to Expected Value?
3841f07hamlet19oct30
![Page 4: 1 Evaluation of Safety Critical Software David L. Parnas, C ACM, June 1990](https://reader030.vdocuments.net/reader030/viewer/2022032703/56649f4f5503460f94c71af6/html5/thumbnails/4.jpg)
![Page 5: 1 Evaluation of Safety Critical Software David L. Parnas, C ACM, June 1990](https://reader030.vdocuments.net/reader030/viewer/2022032703/56649f4f5503460f94c71af6/html5/thumbnails/5.jpg)
5
Table 1
Table I shows that, if our design target
was to have the probability of failure be
less than 1 in 1000, performing between
4500 and 5000 tests (randomly chosen
from the appropriate test case distribution)
without failure would mean that the
probability of an unacceptable product
passing the test was less than 1 in a
hundred.
5841f07hamlet19oct30
![Page 6: 1 Evaluation of Safety Critical Software David L. Parnas, C ACM, June 1990](https://reader030.vdocuments.net/reader030/viewer/2022032703/56649f4f5503460f94c71af6/html5/thumbnails/6.jpg)
6
Table II
6841f07hamlet19oct30
![Page 7: 1 Evaluation of Safety Critical Software David L. Parnas, C ACM, June 1990](https://reader030.vdocuments.net/reader030/viewer/2022032703/56649f4f5503460f94c71af6/html5/thumbnails/7.jpg)
7
Practical ultra-reliability forabstract data types
Borislav Nikolik and Dick Hamlet
Softw. Test. Verif. Reliab. 2007; 17:183–203
7841f07hamlet19oct30
![Page 8: 1 Evaluation of Safety Critical Software David L. Parnas, C ACM, June 1990](https://reader030.vdocuments.net/reader030/viewer/2022032703/56649f4f5503460f94c71af6/html5/thumbnails/8.jpg)
8
Term Redundancy Method (TRM)
8841f07hamlet19oct30
![Page 9: 1 Evaluation of Safety Critical Software David L. Parnas, C ACM, June 1990](https://reader030.vdocuments.net/reader030/viewer/2022032703/56649f4f5503460f94c71af6/html5/thumbnails/9.jpg)
9
Boolean Stack ADT
9841f07hamlet19oct30
r1: pop(push(s, b))→sr2: top(push(s, b))→bFigure 1. Stack of boolean values TRS.
![Page 10: 1 Evaluation of Safety Critical Software David L. Parnas, C ACM, June 1990](https://reader030.vdocuments.net/reader030/viewer/2022032703/56649f4f5503460f94c71af6/html5/thumbnails/10.jpg)
10
Rewriting
10841f07hamlet19oct30
pop(push(pop(push(x, y)), b)) = pop(push(x, y))
![Page 11: 1 Evaluation of Safety Critical Software David L. Parnas, C ACM, June 1990](https://reader030.vdocuments.net/reader030/viewer/2022032703/56649f4f5503460f94c71af6/html5/thumbnails/11.jpg)
11
Post-release testing
11841f07hamlet19oct30
The relatively poor reliability estimate from the pre-release testing phase can now be used to obtain ultra-reliable term evaluations. In the self-checking phase the additional equivalent terms are drawnfrom the test-phase distribution for which the 10−4 bound was obtained. Therefore, if three randomly chosen terms agree, the probability that all of them are failures is less than (10−4)3 = 10−12.
![Page 12: 1 Evaluation of Safety Critical Software David L. Parnas, C ACM, June 1990](https://reader030.vdocuments.net/reader030/viewer/2022032703/56649f4f5503460f94c71af6/html5/thumbnails/12.jpg)
12
Reliability
12841f07hamlet19oct30
Suppose a constant failure rate θ of π, and n random terms drawn from π, executed on δ without failure. The probability that δ fails on a randomly chosen term from π is θ, and 1 − θ that it will succeed. Given that the n terms are independent, the probability that δ succeeds on all the terms is (1 − θ)n. The confidence bound α on θ is defined as the probability that the failure rate of δ is below θ.The confidence bound is related to the testset size n and the failure rate θ by
α ≤ 1 − (1 − θ)n
![Page 13: 1 Evaluation of Safety Critical Software David L. Parnas, C ACM, June 1990](https://reader030.vdocuments.net/reader030/viewer/2022032703/56649f4f5503460f94c71af6/html5/thumbnails/13.jpg)
13
Failure rate
13841f07hamlet19oct30
The confidence bound of Equation (1) is used to quantify the probability of failure of a majority values of a self-check. Equation (1) could be used to estimate the confidence bound on the failurerate of δ on a majority of N random terms generated by RBTR. Suppose a successful test (no failures occurred during the test) of δ on n terms is conducted at test time. Half or more of N terms (majority) falsely agreeing at run-time gives a failure rate of at least N/2n. Therefore, substituting N/2n for θ in Equation (1) yields
α ≤ 1 − 1 − N 2nn
![Page 14: 1 Evaluation of Safety Critical Software David L. Parnas, C ACM, June 1990](https://reader030.vdocuments.net/reader030/viewer/2022032703/56649f4f5503460f94c71af6/html5/thumbnails/14.jpg)
![Page 15: 1 Evaluation of Safety Critical Software David L. Parnas, C ACM, June 1990](https://reader030.vdocuments.net/reader030/viewer/2022032703/56649f4f5503460f94c71af6/html5/thumbnails/15.jpg)
15
Equation 2
15841f07hamlet19oct30
Equation (1) yieldsα ≤ 1 − 1 − N 2nn (2)
The meaning of the confidence bound is the probability that the failure rate is below N/2n for a repetition of the test. For example, 1 − α = 6.0 × 10−8 with N = 33 and n = 104.
![Page 16: 1 Evaluation of Safety Critical Software David L. Parnas, C ACM, June 1990](https://reader030.vdocuments.net/reader030/viewer/2022032703/56649f4f5503460f94c71af6/html5/thumbnails/16.jpg)
16
How do we evaluate Hamlet?
16841f07hamlet19oct30