56 IEEE TRANSACTIONS ON RELIABILITY, VOL. R-33, NO. 1, APRIL 1984

Maintainability: A Historical Perspective

Bernard L . Retterer, M e m b e r I E E E A R I N C Resea rch Corpo ra t i on , Annapo l i s

Richard A. K o w a l s k i , Senior M e m b e r I E E E A R I N C Resea rch Corpo ra t i on , Annapo l i s

Key Words—Maintainability, History, Military standards, Military Handbook.

Abstract—This paper reviews the evolution of the principal military standards and handbooks for maintainability engineering over the past 25 years. The growth and development of those documents are related to developments in both technology and equipment.

1. I N T R O D U C T I O N

In 1960, the Reliability Training Text [22] advised that "Recently more emphasis has been placed on maintenance. Equipment is becoming more difficult to maintain due to complexity, high-level skills required, and the generally in­adequate consideration of maintenance needs in the original design. ' ' Unfortunately, these comments continue, in one form or another, to appear regularly in the technical press and in maintainability discussions. Considering the increase in the complexity of electronic equipment over the past 25 years, these complaints may actually be a very reserved form of compliment. It has required considerable progress in maintainability design analysis, prediction, and test to stay only a little behind the technology curves of the past 25 years.

Maintainability is defined by MIL-STD-721B as " a characteristic of design and installation which is expressed as the probability that an item will be retained in or restored to a specified condition within a given period of time, when the maintenance is performed in accordance with prescribed procedures and resources." This definition resulted from ef­forts to characterize, define, specify, and measure the facility with which a trained maintenance technician could perform the required equipment maintenance.

This paper reviews the evolution of the principal main­tainability military standards and handbooks over the past 25 years. It relates the growth and development of those documents to developments in both technology and equip­ment, dividing the evolution into three periods.

Many activities occurred before 1966 when the initial issues of MIL-STD-470, Maintainability Program Requirements', MIL-STD-471, Maintainability Demonstration', and MIL-HDBK-472, Maintainability Prediction appeared. During that period of rapid technology growth, tubes were replaced by transistors and then by integrated circuits (ICs), and com­puters progressed from the ENIAC and M A R K I to the IBM system 360.

The interval from 1966 to 1978 saw a consolidation and further growth of the technologies that would form the basis of military electronics. Innovations included the minicom­puter, the microprocessor chip, the programmable calculator, and the personal computer. Digital designs touched many new areas for the first time, and testability became a new word in our technical vocabulary.

À third period began in 1979 and continues today. Maintainability has become a mature discipline that will adapt and grow to meet the problems that are a part of con­tinuing technological progress. VHSIC technology, remote maintenance monitoring, and design for testability are only a few of the concepts that have been and will be incorporated into maintainability documents.

2. T H E BEGINNINGS (PRIOR T O 1966)

The initiatives to address both reliability and main­tainability stemmed from the growing complexity of elec­tronics used primarily by the US military. Typically, the complexity of radar, sonar, and communication equipment ranged from 1000 to 100 000 parts. The equipments con­sisted of vacuum tubes, transistors, resistors, and capacitors, all hand-soldered into large chassis frames. In the commer­cial and private sectors, computer systems, air traffic control systems, and entertainment systems were increasing in sophistication. These systems were maintained through a series of tests with general purpose test equipment (eg, voltmeter, ohmmeter, power meter) to locate the defective part. The defective part was removed and replaced, restor­ing the system to operational status. Except for vacuum tubes, which in most cases were plug-in, most other com­ponents were mechanically fastened and soldered in place. Generally, there was little provision for self-testing, and few test points were provided. In one document [22] a rule of thumb applicable to shipboard maintenance manpower planning was that one technician was required for every 250 vacuum tubes.

The reliability of equipment in this era also suffered due to the growing complexity. It was not uncommon to en­counter equipment whose mean time to failure ranged be­tween 20 and 100 hours. This frequency of failure, combined with the difficult maintenance task presented by the lack of proper design consideration, created a considerable maintenance burden. A number of the early researchers quantified the impact of this maintenance burden [1-3, 10, 16, 20, 21].

Early efforts to address the maintenance problem generally took the form of design guidance material. As an example, on the basis of work performed under a US Air Force contract for the Psychology Branch, Aero Medical Laboratory, Wright Air Development Center, J . D. Folley and J . W. Altman prepared a 12-part series of articles that

0018-9529/84/0400-0056$01.00 © 1984 IEEE

RETTERER/KOWALSKI: MAINTAINABILITY: A HISTORICAL PERSPECTIVE 57

appeared in Machine Design in 1956. The articles covered topics such as:

• Designing Electronic Equipment for Maintainability • Design Recommendations for Separating Electronic

Equipment into Units, Assemblies, and Subassemblies • Design of Covers and Cases • Selecting and Applying Wiring, Cables, and Con- '

nectors • Recommendations for Designing Maintenance Ac­

cess in Electronic Equipment • Design Recommendations for Test Points • Design of Maintenance Controls • Factors to Consider in Designing Displays • Designing for Installation • Maintenance Auxiliaries (Test Equipment, Mock-

Ups, Tools) • A Systematic Approach to Preparing Maintenance

Procedures • Recommended Method of Presenting Maintenance

Instructions

These efforts, although useful, suffered from an inability to enforce their use. Further, there were no means to judge how well they had been followed. These problems thus set the stage for developments in the 1960s.

The vacuum tube technology of the early 1950s had given way to discrete transistors, integrated circuits, metal-oxide semiconductors (MOS), and other solid-state devices. With the increased recognition of the maintenance problem, use of module boards, subassemblies, some self-test, and test points were the norm rather than the exception. Although solid-state devices brought reliability gains, such gains were often offset by further growth in performance-complexity. In addition, the new environment of space operation brought new challenges.

Efforts to bring more control to the maintainability design process began with a search to develop measures of quantifying this equipment attribute. Maintenance time was an obvious choice, but its measurement was complicated by a number of factors. The skill and experience of the maintenance technician were the major factor, together with issues of the environment in which the maintenance was per­formed and the support facilities provided. Notwithstanding these problems, researchers concluded that the qualitative rating systems used with design guidelines were unsatisfac­tory and that time measurement represented a logical parameter by which to express equipment maintainability [6, 7 , 9 , 15, 17, 18, 23].

A major issue of time specification was the realization of the different types of maintenance activity (eg, preparation, testing, securing of repair parts, and administrative delays). If maintenance time was to be used as a specification parameter, it was important to identify those time elements which were a consequence of maintenance design. Those ef­forts led to the maintenance time definitions (downtime, delay, corrective) now contained within MIL-STD-721B. These definitions were brought to fruition through the com­

bined efforts of researchers in maintenance processes [5-7, 10, 12, 14] and industry committees, such as thé Electronic Industries Association Committee, working with cognizant government agencies.

Wi th the identification of selected elements of maintenance time as a control parameter for maintainabili­ty, it became possible to develop control specifications. The US Navy and US Air Force, respectively, developed Specifications MIL-M-23313 and MIL-M-26512 (Navy BUWEPS also developed Maintainability Specification XWR-30). Those maintainability program specifications re­quired the establishment of a maintainability assurance pro­gram, the application of maintainability design criteria, and the evaluation of achieved maintainability. The specifica­tions were later combined, becoming the basis for MIL-STD-470, Maintainability Program Requirements [8, 23].

Early researchers into ma in tenance (downtime) characteristics observed in a number of cases that downtime was distributed log-normally. Typically, a number of fre­quently occurring tasks requiring modest amounts of downtime would be offset by infrequent lengthy tasks. Since the log-normal distribution requires two parameters for com­plete description (typically mean and standard deviation), the choice of parameters became an issue. Some researchers favored the use of geometric mean as the principal measure, since for the log-normal distribution the geometric mean is also the median value. Others felt that the arithmetic mean for a given data sample could be estimated more accurately, and that a percentile (90th, 95th) represented a more effec­tive method of characterizing log-normally distributed data. The latter combination has become the generally accepted method [4, 5, 19, 26].

The cited military documents required the development and performance of maintainability demonstration testing. Air Force Specification MIL-M-26512 contained as an ap­pendix a suggested method for planning a maintainability demonstration test. The basis for the test was to randomly select parts that would be considered failed. If the selected part could be placed in a failed mode in the equipment to be tested, a trained technician would be timed as he performed the diagnosis and repair. If properly planned and conducted, a series of simulated failed tasks performed by several techni­cians provided the basis for estimating the maintenance time distribution for the equipment being evaluated. The appen­dix to Specification MIL-M-26512 dealt with issues of sam­ple size, sample selection, data evaluation, and control of the test. The appendix formed the original basis for MIL-STD-471 on maintainability testing [11, 27].

The final area addressed by these formative efforts was maintainability prediction, ie, the process of evaluating a proposed equipment design for maintainability and express­ing the results in quantitative terms such as mean time to repair ( M T T R ) . Four major procedures that were developed as a result of these early efforts formed the basis of the original MIL-HDBK-472.

Procedure I is a time-element synthesis technique designed to predict flight-line maintenance time of airborne

58 IEEE TRANSACTIONS ON RELIABILITY, VOL. R-33, NO. 1, APRIL 1984

electronic and electromechanical systems entailing modular replacement. Elemental maintenance activities and frequen­cy of occurrence are identified. The procedure provides time data for various types of activities that are mathematically combined with the occurrence rates to yield a total maintenance time estimate and distribution [24, 25].

Under Procedure II, maintenance time is estimated by systematically reviewing the functional breakdown of the equipment, defining the functional levels at which maintenance is performed, assigning appropriate time fac­tors provided by the procedure for levels of maintenance per­formed, and combining those factors with associated failure rate information. This procedure was developed using data observed for shipboard electronic equipment [29].

Procedure III uses equipment design parameters to predict maintenance time. These parameters are related to the physical configuration of the system, the facilities pro­vided for maintenance by the design, and the degree of maintenance skill required by the design. The maintenance times are estimated for a representative sample of maintenance tasks randomly selected from the total set of potential tasks. This procedure was developed using data derived from general electronic equipment maintenance [28].

Finally, Procedure IV provides a strategic approach to estimating corrective and preventive maintenance times. It recognizes that requirements for maintenance are driven by the operation of the end system in association with the reliability of equipment elements. The procedure combines these maintenance drivers with either user-supplied maintenance task times or expert opinion to estimate total maintenance labor hours on equipment for weapon systems such as aircraft [13].

3. T H E ADOLESCENT P E R I O D (1966 T O 1978)

Following the appearance in 1966 of the initial issues of MIL-STD-470, MIL-STD-471, and MIL-HDBK-472, developments in the component and computer industries continually caused maintainability engineers to reassess and redefine their roles in the design, analysis, and test of elec­tronic equipment. The 12 years from 1966 to 1978 (just before the introduction of VLSI) were a period during which the maintainability discipline was hard-pressed to keep up with technology advances.

Technology: Components and Computers. During this period, the miniaturization process that replaced the tube with the transistor with the IC was extended to other com­ponents. Resistors and capacitors were shrunk to the size of IC chips, and many types of components were fitted into dual-in-line packages. In 1971 Intel Corporation introduced the 4-bit microprocessor, the I N T E L 4004, which was followed shortly by the 8-bit INTEL 8008. Now a computer on a chip could be made an integral part of equipment design. The availability of high-speed digital processing and memory devices led to the digitization of many signal pro­cessing and analysis functions in communication, navigation,

and radar systems. In the maintainability discipline, the con­cepts of self-test, built-in test (BIT), diagnostic test, and failure would need to be reevaluated and redefined. As for discrete computers, the DEC PDP-8 minicomputer, originally introduced in 1965, was available in an IC version in 1967. Hewlett-Packard Company introduced the HP-35 electronic slide rule in 1973 and the HP-65 programmable calculator in 1974. By 1977, Radio Shack had introduced the TRS-80 personal computer for under $600. Powerful digital test capabilities could be had at reasonable prices.

Testability: A New Need. The first appearance of the word * 'testability' ' was in a 1965 paper, "Designing Equipment for Automatic Testability, ' ' by F. Liguori [30, 45]. In subse­quent years, several other papers discussed factors that pro­duced designs with easy-to-diagnose circuitry (generally digital) and reduced test time or cost. In 1975, some specific testability guidelines appeared [37]. The topic continued to grow throughout the 1970s (eg, the AUTOTESTCON Pro­ceedings). The Naval Electronics Laboratory Center pro­duced a Built-in Test Design Guide in 1976; the Naval Avionics Facility sponsored a study of standard BIT circuits in 1977; and the Joint Logistics Commanders (JLC) established a J L C Panel on Automatic Testing in 1978 to coordinate and guide the Joint Services Automatic Testing program. Both the Navy and Rome Air Development Center (RADC) issued BIT design guides in 1979.

Impacts on Maintainability Documents. During this period, no formal changes were made to MIL-STD-470. However, several maintainability planning and engineering design documents were developed to assist program managers in planning, managing, and monitoring program main­tainability plans [34-36, 38-40].

MIL-STD-471, Maintainability Demonstration underwent several changes that reflected the variety of requirements needed for various equipments. After being updated to 471A in 1973 March on the basis of RADC-sponsored work [31, 32], it was expanded with Notice 1 in 1975 January and was given the title Maintainability Verification/Demonstration/Evalua­tion. Notice 1 added four tests for mean/percentile combina­tions and for preventive maintenance demonstrations. In 1978 December, an addendum to 471A was issued entitled, Demonstration and Evaluation of Equipment/System Built-In-Test/External Test/Fault Isolation/Testability Attributes and Re­quirements. This title itself demonstrates the new main­tainability characteristics that had developed during this period.

MIL-HDBK-472, Maintainability Prediction was not for­mally changed during this period; however, studies were conducted to develop and evaluate design-sensitive predic­tion techniques [31-33]. The resulting methods used fault-symptom matrices that linked system failure characteristics to its fault signals. Several measures of entropy were used in regression analyses to develop subsequent prediction equa­tions. Although those methods were not formally incor­porated into MIL-STD prediction methods, they did foreshadow the types of analysis that would be needed in the next generation of prediction techniques.

RETTERER/KOWALSKI: MAINTAINABILITY: A HISTORICAL PERSPECTIVE 59

4. MATURATION (1978 TO PRESENT)

By the end of the 1970s, a substantial collection of maintainability design analysis, prediction, and testing procedures and tools had been developed for application to electronics designs. In contrast to the 1950s, maintainability has become a mature discipline that can adapt and grow to meet the problems that are a part of continuing technological evolution of electronic equipment.

Technology Growth Continues. The era of very-large-scale integration began in 1979 with the announcement of 64K random-access memories. These components continued technology's march toward increasing the densities of single devices. The next milestone in this direction will be achieved by the US Department of Defense Very High Speed Integrated Circuits (VHSIC) program, begun in 1980, which calls for six major VHSIC contractors to fabricate prototype chips in 1984. (Three contractors had completed chips by the end of 1983.) The goal of the program is the pilot production in 1986 of processors containing 250 000 gates and operating at or above 25 MHz. A VHSIC Working Group Report was issued in 1983 as part of the Institute for Defense Analysis/Office of the Secretary of Defense (IDA/OSD) reliability and maintainability (R&M) study [48] to identify high payoff actions in this area. A VHSIC chip is sufficiently complex that it can include both circuits and software for chip testing of functions or circuit parameters, for diagnosing detected faults, and, possibly, for monitoring (and storing) device history or trend data. As production devices are developed, the maintainability engineer will have new opportunities for planning and implementing diagnostic and status tests and for identifying the need for corrective or preventive repairs for VHSIC-based equipment.

Impact on Maintainability Documents. In 1983 January, after nearly 16 years of service, MIL-STD-470 was superseded by MIL-STD-470A, Maintainability Program for Systems and Equipment, a substantial expansion of the original document. MIL-STD-470A contains detailed task descriptions for various maintainability program elements and identifies specific details that must be added by the contracting activity. A comprehensive appendix provides additional guidance to be used in tailoring maintainability program requirements to individual efforts.

No formal changes have been made to MIL-STD-471 since 1978.

Notice 1 to MIL-HDBK-472 was published in 1984 January. It adds a new maintainability prediction technique with two levels of application: 1) an early procedure for use when preliminary design data are available, and 2) a detailed procedure for use when detailed design and support data are available. The detailed procedure includes the analysis of system failure symptoms to relate them to potential failure candidates. Full details of the techniques are presented in [41] and summarized in [44].

5. NEW FRONTIERS OF RESEARCH

Testability. Since 1978, Rome Air Development Center has conducted several studies to develop guidance on specify­ing and demonstrating testability in electronics equipments and conducting design and cost trade-offs associated with testability issues [42, 43]. The results of these efforts were consolidated into the RADC Testability Notebook in 1982 [46]. Analytic techniques from several branches of operations research were studied for application to testability, and the results were published in Analytical Procedures for Testability [47]. A "Testability Technology Working Group Report" was issued in 1983 as part of the IDA/OSD R&M study to identify high payoff actions in this area [49]. Further in­vestigations in the testability arena are expected to provide payoffs by reducing support problems and enhancing system readiness when deployed [52].

Artificial Intelligence. Modern military systems use a variety of automated maintenance tools. One possibility for creating quantum improvements in maintenance is the ap­plication of artificial intelligence (AI) techniques. These techniques can include natural language processing, inferen­tial retrieval from data bases, combinational/scheduling problems, machine perception, expert consulting systems, and robotics. An "AI Applications to Maintenance Technology Working Group Report" was issued in 1983 as part of the IDA/OSD R&M study to identify potential ap­plications of AI techniques to the DoD maintenance environ­ment [50].

6. SUMMARY

As maintainability engineering developed in the 1960s, equipment repair times were driven by disassembly, inter­change, and reassembly tasks. As electronics technology evolved from tube to transistor to various levels of integrated circuits, electronic systems became modular, modules were easy to remove and replace, and little or no alignment was required after replacement. As a result, maintainability designs now emphasize diagnostic capabilities and equip­ment testability; computers have become an integral part of the maintenance process.

This paper has traced the origins and evolution of the principal maintainability standards and handbooks over the past three decades and has identified many of the basic studies that contributed to the development of maintainabili­ty analysis tools.

The following sources of maintainability research infor­mation can help researchers keep up with new policies, findings, and results.

Reference [51] is a guide to government reliability, maintainability, and quality assurance (RM&QA) organiza­tions. It lists agencies with RM&QA responsibilities in all three services and NASA, FAA, NSA, and NBS.

Since 1983, RADC has been delegated the authority to act for the Secretary of Defense in the areas of reliability and maintainability standardization. RADC's duties include

60 IEEE T R A N S A C T I O N S O N RELIABILITY, V O L . R-33, N O . 1, A P R I L 1984

working with the preparing activities in all three services to maintain the currency of R&M standards and handbooks. RADC will publish biennial standardization plans for both reliability and maintainability.

In 1983 September, a Reliability Maintainability Technology Transfer-Fact Sheet was published to report progress in R&M standardization activities. The Fact Sheet's editor is Mr. Preston MacDiarmid; RADC/RBE-2; Grifflss AFB, New York 13441 USA. \phone: (315) 330-7095]. Further issues will be printed as warranted.

REFERENCES

Ralph L. Madison, " A n analysis of the effects of maintenance on part replacements ," Proc. Fourth National Symp. Reliability and (Quality Con­trol, pp 19-29, 1957. George A. Raymond, "Reliability vs. the cost of fai lure," Proc. Fourth National Symp. Reliability and Quality Control, pp 287-288, 1957. R. L. McLaughlin, H . D. Voegtlen, " G r o u n d electronic equipment support cost vs. reliability and maintainabi l i ty ," Proc. Fifth National Symp. Reliability and (Quality Control, pp 36-42, 1958. E. L. Welker, "Relat ionship between equipment reliability, preven­tive maintenance policy, and operating cos ts ," Proc. Fifth National Symp. Reliability and duality Control, pp 270-280, 1958. R. R. Howard, W. J . Howard, F. Hadden , "S tudy of down time in military equ ipmen t , " Proc. Fifth National Symp. Reliability and (Quality Control, pp 402-408, 1958. J . Schechtel, S. Seltzer, "Maintainabil i ty index study on shipboard electronic equipment and sys tems," Proc. Sixth National Symp. on Reliability and duality Control, pp 335-342, 1959. M . P. Feyerherm, H . W. Kennedy, J r . , "Practical maintainability numer ics , " Proc. Sixth National Symp. Reliability and Quality Control, pp 343-346, 1959. Michael P. Forte, Frank D. Mazzola, " R o m e Air Development Center 's maintainability p r o g r a m , " Proc. Sixth National Symp. Reliabili­ty and duality Control, pp 347-352, 1959. B. D . Smith et al . , "Industr ia l approach to maintenance engineering and maintainabi l i ty ," Electronic Maintainability, 1960. W. R. Gibson, " W h a t price maintainabil i ty?" Electronic Maintainabili­ty, 1960. John W. Lewis, "Integrat ion of maintainability requirements into the research and development phase of weapon sys tems," Electronic Main­tainability, 1960. E. D. Hart , "Maintainability design for the airlines avionics marke t , " Electronic Maintainability, 1960. Wright Air Development Center. Maintainability and Supportability Evaluation Techniques, vols. I and II, 1960 May and J u n e . James B. Nickels, "Us ing the human element to predict equipment maintainability, ' ' Electronic Maintainability, 1960. Mark L. Hinkle, "Availability as a parameter in system evaluat ion," Electronic Maintainability, 1960. M . J . Marcus , " T h e maintainability problem in systematic perspec­tive , ' ' Electronic Maintainability, 1960. T . B. Slattery, "Rat ionale for a fundamental measure of system maintainabi l i ty ," Electronic Maintainability, 1960. H . Kennedy, B. L. Retterer, "Maintainabi l i ty measurement and predict ion, ' ' Electronic Maintainability, 1960. R. C. H o m e , "Measurement and prediction of systems maintainabili­t y , " Electronic Maintainability, 1960. D. Kenneth Richardson, "Increased motivation for maintainability design, ' ' Electronic Maintainability, 1960. Lloyd R. Collins, "Accessibility in shipboard electronic systems main tenance , " Electronic Maintainability, 1960. F. Gryna et al . , eds. Reliability Training Text, 2nd ed., Institute of Radio Engineers, 1960 March. G. Margulies, J . Sacks, "Bureau of Ships maintainability specifica-

[24

[25

[26

[27

[28

[29

[30

[31

[32

[33

[34

[35

[36

[37

[38

[39

[40

[41

[42

[43

[44

[45

[46

[47

[48

[49

t i on , " Proc. Ninth National Symp. Reliability and Quality Control, pp 84-92, 1962. A. Drummon et al. Maintainability Prediction: Theoretical Basis and Prac­tical Approach. A R I N C Research Corp . , 207-1-275, 1962 February (AD-273014).

George T . Harrison, " A practical approach to maintainability predic­t i o n , " Proc. Ninth National Symp. Reliability and (Quality Control, pp 439-447, 1962. E. L. Welker, "Predict ing reliability and maintainability of complex systems by non-parametric me thods , " Proc. Ninth National Symp. Reliability and duality Control, pp 582-593, 1962. B. L. Retterer, G. H . Griswold, "Main tenance time specification," Proc. Tenth National Symp. Reliability and (Quality Control, pp 1-9, 1963. B. Retterer et al. Maintainability Engineering. Rome Air Development Center , TDR-63-85 , 1963 February (AD-404898). Federal Electric Corp . A Maintainability Prediction Procedure for Designers of Shipboard Electronic Equipment and Systems. 1964 (AD-431269). F. Liguori, "Designing equipment for automatic testabili ty," Automatic Support Systems Symposium, 1965 J u n e . G. Griswold, B. Retterer, H . Balaban. Maintainability Prediction and Demonstration Techniques, Volume 1: Development of Prediction Techniques. Rome Air Development Center , TR-69-356, 1970 J a n u a r y (AD-869331). G. Griswold, B. Retterer, H . Balaban. Maintainability Prediction and Demonstration Techniques. Rome Air Development Center, TR-70-89, 1970 J u n e (AD-872873). R. Kowalski, W. Stevenson. Application, Evaluation, and Refinement of Maintainability Prediction and Demonstration Techniques. Rome Air Development Center , TR-74-112, 1974 May (AD-919972L). R. Anderson et al. Reliability and Maintainability Planning Guide for Army Aviation Systems and Components. I IT Research Institute, 1974 July . L. Greenman. Maintainability Engineering Design Notebook, Revision II, and Cost of Maintainability. Rome Air Development Center, TR-74-308, 1975 J anua ry (AD-A009043, A009044, A009045). Army Materiel Command . Engineering Design Handbook: Maintenance Engineering Techniques, AMCP-706-132, 1975 J u n e (AD-A021390). P. Writer, "Design for testabil i ty," Automatic Support Systems Sym­posium, 1975. Army Materiel Command . Engineering Design Handbook: Maintainability Engineering Theory and Practice, AMCP-706-133, 1976 January (AD-A026006). P. Mihalkanin. Update to Reliability and Maintainability Planning Guide for Army Aviation Systems and Components. I IT Research Institute, I ITRI-E6337-FR, 1976 August (AD-A037446). Naval Sea Systems Command . Reliability (R) and Maintainability (M) Design Checklist, NAVSEA-S0300-AC-MMA-010-r /m, 1977 October (AD-A094250). T . Pliska, F. J ew , J . Angus. Maintainability Prediction and Analysis Study. Rome Air Development Center , TR-78-169-Rev-A, 1978 July (AD-A059753). T . Pliska, F. Jew, J . Angus. BIT/External Test Figures of Merit and Demonstration Techniques. Rome Air Development Center, TR-79-309, 1979 December (AD-A081128). W. Consollo, F. Danner . An Objective Printed Circuit Board Testability Design Guide and Rating Systems. Rome Air Development Center, TR-79-327, 1980 J anua ry (AD-A082329). J . Lipa, J r . , " A modern maintainability prediction technique ," IEEE Trans. Reliability, vol R-30, 1981 Aug, pp 218-221. M . Thiel, ed. Productivity Improvement Through Testability, Volume I: Management. Giordano Associates, Inc. , 1982. J . Byron, L. Deight, G. Stratton. RADC Testability Notebook. Rome Air Development Center , TR-82-198, 1982 J u n e (AD-A118881L). A. Aly, J . Bredeson. Analytical Procedures for Testability. Rome Air Development Center, TR-83-4, 1983 Janua ry (AD-A126167). E. Maynard , chair 'n . VHSIC Technology Working Group Report. In­stitute for Defense Analysis, D-42, 1983 November (AD-A137720). G. Newman, chair 'n . Testing Technology Working Group Report. Institute for Defense Analysis, D-41 , 1983 August (AD-A137526).

RETTERER/KOWALSKI: MAINTAINABILITY: A HISTORICAL PERSPECTIVE 61

[50] A. Coppola, chair'n. Artificial Intelligence Application to Maintenance Technology Working Group Report. Institute for Defense Analysis, D-28, 1983 August (AD-A137329).

[51] P. MacDiarmid, G. Lyne. A Guide to Government Reliability, Main­tainability, and Quality Assurance Organizations. Rome Air Development Center, TR-83-49, 1983 February (AD-A130465).

[52] William R. Simpson, Harold S. Balaban, "The ARINC Research system testability and maintenance program (STAMP)," 1982 IEEE AUTOTESTCON Conference, 1982 October.

AUTHORS Bernard L. Retterer; ARINC Research Corporation; 2551 Riva Road; Annapolis, MD 21401 USA.

Mr. Retterer (M'58) graduated from Ohio Northern University in 1952 with a BS in electrical engineering. He received an MS in system ad­ministration from The George Washington University in 1972. Mr. Retterer currently is Vice President of Engineering at ARINC Research Corporation. He participated in the research and development efforts that led to the formulation of MIL-STD-470, MIL-STD-471, MIL-STD-721, and MIL-HDBK-472.

Richard A. Kowalski; ARINC Research Corporation; 2551 Riva Road; Annapolis, MD 21401 USA.

Richard A. Kowalski (M'70, SM'82) received a BS degree in Mathematics from Northeastern University in 1962, and MS and PhD degrees in Mathematics from Case Institute of Technology in 1963 and 1967 respectively. He is a member of the technical staff of ARINC Research Corporation's Aircraft and Vehicles Division. He is responsible for planning and directing projects to support military program managers in developing, applying, and improving innovative system procurement methods. Typical programs are the F-16 aircraft, the US Navy cruise missile, and US Army avionics systems.

In 1980, Dr. Kowalski was a member of the Air Force/Industry Com­mittee which developed the Product Performance Agreements Guide that was published as a joint USAF Systems Command and Logistics Com­mand document. In 1978 and 1979 he was a member of the Availability and Life-Cycle Cost Panels at the USAF Avionics Planning Conferences.

Dr. Kowalski has given several papers at the Annual R&M Symposia, and is a member of the American Mathematical Society, the IEEE, the Mathematical Association of America, and Sigma Xi. He was chair 'n of the Baltimore Chapter of the Reliability Society in 1972-73, and is a member of the Reliability Society Administrative Committee and chair'n of its Main­tainability Committee. * * *