applying do178b

Applying DO178B for

IV & V of Safety critical Software

Wipro TechnologiesInnovat ive Solut ions, Qual i ty Leadership

Software for airborne application is highly safety critical as any failures may resultin loss of human life. Government agencies like FAA and JAA in the US andEurope respectively, enforce stringent software development practices to ensurethe safety of life. RTCA DO-178B provides the guidelines for all the phases of thesoftware development life cycle for airborne applications and equipment certifica-tion. The aviation community as a whole and the FAA endorse these guidelines.

The safety criticality of airborne software poses a lot of challenges for its V&Vbecause the airborne system as a whole should not fail and result in damage orthreat to human life. This paper aims at explaining these in more detail from apractitioner’s perspective. The first few sections define the activities involved in theverification of safety critical software and describe the processes recommendedby DO178B. In last couple of sections, a small case study is presented. It isconcluded with a brief description of Wipro Technologies’ focus in this area.

Author : Sreekumar Panicker

WHITE PAPER

© Wipro Technologies

Applying DO178B for IV & V of Safety critical SoftwareWhite Paper

Page : 2 of 11

Table of Contents

Introduction ............................................................................................................................ 3

Verification and Validation ..................................................................................................... 3

Failure Categorization and Software Levels ......................................................................... 4

Categorization of Software Failure Conditions ..................................................................... 4

V&V and Other Software Life Cycle Processes .................................................................... 5

DO-178B V&V Process ..........................................................................................................6

Testing ................................................................................................................................... 7

V&V – A typical case .............................................................................................................. 9

Simulated Testing ............................................................................................................... 10

About the Authors ................................................................................................................ 11

About Wipro Technologies .................................................................................................. 11

Wipro in Embedded Systems ............................................................................................. 11



IntroductionThe guidelines from RTCA for implementation of safety critical software for airborneequipment are fairly elaborate covering all the phases of development and testing.However, from a practitioners perspective or for an organization entering into this busi-ness, it was felt that these guidelines should be explained further for a good understand-ing. This paper focuses on the verification and validation part of DO178B process recom-mended by RTCA.

Verification and ValidationFor non-critical software or normal application software, verification and validationactivities, though important, may not be safety critical. Where as, in the case of safetycritical software, verification and validation activities are crucial for ensuring reliablesoftware. And, these activities assume lot of significance.

- Validation involves assessing the degree to which a software system actually fulfills thesystem requirements or the user needs. Validation activities refer primarily to the overallsystem specification and the final code. A system that is consistent with its specificationsis dependable.

- Verification refers to the work product with respect to its immediate predecessor. Forexample, this may involve checking the consistency of an implementation with a specifica-tion that forms the reference for the check. The implementation is the “item being verified”and the specification is the “reference”.

For example, an overall design could be the specification and a more detailed designcould be the implementation. In this case, checking whether the detailed design isconsistent with the overall design would then be the verification of the detailed design.The same detailed design could play the specification with respect to source code, whichwould be verified against the design. In every case, verification is a check of consistencybetween two descriptions, in contrast to validation that compares a description againstactual needs.

The diagram on the following page, illustrates the verification and validation activities atvarious phases of development. As seen in the diagram, verification activities checkconsistency between descriptions at adjacent levels of outputs and between descriptionsand implementation.

Validation activities refer primarily to the overall system specification and the final code.With respect to overall system specification, validation checks for discrepancies betweenactual needs and the system specification as laid out by the analysts, to ensure that thespecification is an adequate guide to building a product that will fulfill its goals. Withrespect to final code, validation aims at checking discrepancies between actual need andthe final product (the final code), to reveal possible failures of the development processand to make sure that the product meets the actual end-user expectation. Validationchecks between the specification and the final product are primarily checks of decisionsthat were left open in the specification, e.g., details of the user interface or productfeatures.

Validation against actual requirements involves a good deal of human judgment. Thechances for ambiguity, misunderstanding and disagreement are high. Specificationsshould be sufficiently precise and unambiguous so that there can be no disagreementabout whether a particular system behavior is acceptable to aid validation.

Page : 3 of 11



Page : 4 of 11

Verification and Validation

Properties related to dependability include correctness, reliability, robustness, and safety.Correctness is absolute consistency with a specification, always and in all circum-stances.Reliability is a statistical approximation to correctness, expressed as the likelihood ofcorrect behavior in the expected use. The following sections describe how DO-178Bobjectives and guidelines attempt to facilitate the development of reliable softwaresystems for airborne systems.

Failure Categorization and Software LevelsDO-178B categorizes and defines system failure conditions. The failure conditioncategorization is based on Federal Aviation Administration AC25-1309-1A and/or the JointAviation Authorities AMJ 25-1309, as amended. Failure condition categorization isfundamental to determination of software level as well as Verification and Validation.

Categorization of Software Failure ConditionsFive types of failure conditions are defined in DO-178B guideline document. This failurecategorization is based on the effect of such a failure on the airborne system.

Catastrophic - Failure conditions that would prevent continued safe flight and landing.

Hazardous/Severe-Major - Failure conditions that would reduce the capability of theaircraft or the ability of the crew to cope with adverse operating conditions resulting in� a large reduction in safety margins or functional capabilities� physical distress or higher workload such that the flight crew cannot perform their

tasks accurately or completely or� adverse effects on occupants including serious or potentially fatal injuries to a small

number of those occupants.

Sub System/ Designspecifications

Units/ ComponentSpecifications

Analysis/Review

Analysis/Review

SystemRequirementspecification

Legends

Module Test

Validation

Verification

Units/ Components

Integration Test

Sub Systems

SystemIntegration

SoftwareSolutionDelivered

Acceptance Tests/Conformity Review

SystemRequirements and

Constraints



Page : 5 of 11

Major - Failure conditions that reduce the capability of the aircraft or the ability of the crewto cope with adverse operating conditions, to a significant extent. For example, the failuremay result in a significant reduction in safety margins or functional capabilities, a signifi-cant increase in crew workload, in conditions impairing crew efficiency or discomfort tooccupants, possibly including injuries.

Minor - Failure conditions which would not significantly reduce aircraft safety, and whichwould involve crew actions that are well within their capabilities. Minor failure conditionsmay include, for example, a slight reduction in safety margins or functional capabilities, aslight increase in crew workload, such as, routine flight plan changes, or some inconve-nience to the occupants.

No Effect - Failure conditions that do not affect the operational capability of the aircraft orincrease crew workload.

Software LevelsCorresponding to the failure conditions described above, software levels from A to E aredefined – Level A indicates that the failure of software results in catastrophic results, whilelevel E will not impact the safety anyway. Software level assessment is done as part ofsystem safety assessment process. The software level also indicates the efforts that arerequired to demonstrate the compliance for certification and it varies with the failurecondition category.

V&V and Other Software Life Cycle ProcessesDO-178B defines Software Verification as an Integral process within the software Lifecycleprocess. The V&V requirements as per DO-178B address the development process (TheSoftware Requirement Process, Software Design Process, Software Coding and Integra-tion Process and Software Integration Process) and the V&V process itself.

The Software Planning Process and other Integral Processes viz. Software ConfigurationManagement Process, Software Quality Assurance Processes and Software LiaisonProcess do not fall under the purview of Software Verification and Validation Process.However reviews and audits respectively ensure the correctness of these processes.

The diagram shows the V&V requirements assuming waterfall model. The same can beextended to iterative development.

Development Process

IntegrationProcess

RequirementProcess

DesignProcess

Coding andIntegration Process

Verification Process

DO-178B V & V Process and Development Process



Page : 6 of 11

Annexes A3 through A7 of RTCA document on DO178B provide the list of outputs of theDevelopment process and the V&V process itself. It is necessary that these are verifiedand the objectives applicable to the level of the software are satisfied through the verifica-tion process. For example, for level A software, many of the objectives have to be met withindependence, whereas for level B and C, less number of objectives need to be satisfiedindependence.

DO-178B V&V ProcessDO-178B based V&V shall be as defined in the Software Verification Plan (DO-178BSection 11.3), which is prepared in the planning stage of the software development. DO-178B distinguishes between testing and verification. The process of testing per se,doesn’t ensure absence of errors. Verification on the other hand is a generic term foractivities viz., Reviews, Analyses, and Testing.

Verification / Test Results

RequirementProcess

DesignProcessVerification Verification

Verification VerificationIntegration

Coding andIntegration

Verification Plan

Defect -Reporting

andManagementVerification of

ReviewResults Test Cases for

StructuralCoverage

Templates /Checklists

Test Cases forRequirement

Coverage

DO-178B V & V Process

The figure above represents the Verification Process as required by DO-178B andindicates the verification activities at the end of each of the processes like RequirementProcess, Design Process, Coding and Software Integration Process and the HardwareIntegration Process.

Verification of requirement process involves review/analysis of Software RequirementData. The review comments from this process are fed back to the previous life cycleactivities as appropriate.

Verification of Design process involves review and analysis of the design that is providedin the Software Design Data. The review comments from this process are fed back toprevious life cycle activities as appropriate.



Page : 7 of 11

Verification of coding and integration process involves review and testing of the sourcecode implemented as per the Software Design Data. The review comments and errorsidentified from this process are fed back to previous life cycle activities as appropriate.

Verification of integration process involves testing of the object code on Instruction SetSimulator/ Target Emulator, Target board for compliance. The test results from thisprocess are fed back to previous life cycle activities as appropriate. In general all errorsthat are reported are managed and tracked to closure.

Software Verification Cases and Procedures as well as the Software Verification Resultsare verified for completeness and correctness in the Verification of Verification ProcessResults.

Reviews are conducted with respect to a reference. For example the review of the designdocument is done either with the Software Requirement Data or a suitable checklistprepared based on the same. Analyses are expected to provide repeatable evidence ofcorrectness. Detailed examination of functionality, performance, traceability and safetyimplications of a software component are typical subjects of analysis.

Indirect Path

End ofTesting

S/WRequirement

H/W – S/WIntegration testsS/W Integration

Tests

Low leveltests

SoftwareRequirements

Software StructureCoverage Analysis

AdditionalVerification

Direct Path

TestingTesting is the verification of source code generated at unit level, module level and thesystem level. It has two objectives:� To demonstrate that the software satisfies requirements� To have a high degree of confidence in the software – i.e., the errors leading to failure

conditions as determined by the system safety assessment process have beenremoved.

Testing Process



Page : 8 of 11

The above diagram (reproduced from DO-178B) depicts the testing process.Three types of testing are indicated:� Hardware Software Integration Testing: - This is to verify that the software functions

correctly in the target environment.� Software Integration Testing:� To verify that the interrelationships and the software components are and the software

requirements.� To verify the implementation of the software requirements and the software compo

nents within the software architecture.� Low-Level Testing: - This is to verify the implementation of the software low level

requirements.

DO-178B makes it clear that the requirement based coverage and structural coverageobtained by hardware/ software Integration testing need not be duplicated in low leveltesting. However, no low-level test can provide the effectiveness of the high-level tests.

The DO-178B stresses that the software requirements are the primary basis for test casegeneration. Primary objective of test cases shall be to test functionality and to bring outpotential errors.

Requirement coverage analysis is aimed at determining the requirements that are nottested. Structural coverage analysis is aimed at identification of code sequences that is notcovered structurally.

Test EnvironmentFor comprehensive testing of the software, more than one test environment may berequired. A recommended test environment for the software is the software loaded ontothe target computer. The target computer shall interact with a high fidelity simulation of itsactual environment. This may be considered as an integrated target environment. Testingon the target computer environment assumes importance as some errors are detectedonly in this kind of environment.

The requirement-based coverage and structural coverage are achieved only with the helpof precise control and monitoring of the input. Such testing may call for different test setupwherein small software components get tested in isolation. Test Drivers and test toolscan be used in this kind of testing.

Requirement Based TestingDO-178B emphasizes the requirement based testing as it is found to be very effective inrevealing errors. Test cases shall consider normal range as well as abnormal range testdata. Abnormal range testing is for robustness. Test cases shall be prepared also foruncovering errors common in software development process.

The requirement based testing methods consist of requirement based hardware/software integration testing, requirement based software integration testing and require-ment based low-level testing.



Page : 9 of 11

Testing for Structural CoverageWhereas the Requirement based testing focuses on the compliance of the software withthe system requirements, structural coverage focuses on testing the source codestructurally to ensure that there is no unnecessary code in the implementation. Therequirement based testing will cover the code structures that are directly responsible formeeting the system requirements. The uncovered code structures in the software aftermeeting the requirement based testing would be mainly the code for handling errorconditions and dead code if any. Unit test/ Software integration tools indicate the codestructure that is covered and not covered and also provide structural coverage statistics.

Test Coverage AnalysisTest Coverage analysis involves Requirement based Test Coverage analysis andStructural Coverage analysis. The purpose here is to confirm necessity and sufficiency ofthe selected test cases to cover the stated software Requirements. It also looks into thecode coverage aspects achieved/ not achieved by tests for requirement coverage.

Structural Coverage Analysis is expected to uncover shortcomings in the structuralcoverage. Such shortcomings are generally attributed to inadequate requirement basedtesting, inadequacies in requirements, presence of dead code.

V&V – A typical caseTo further help in the understanding of the activities involved in V&V, let us assume thatsoftware for a Flight Control System needs to be validated. Typically, this involves a)generation of software verification plan b) verification c) test description and d) testing –unit and integration.

Software Verification Plan essentially addresses issues like validation of requirementsand verification activities throughout the life cycle of the project execution. Information onthe environments for verification, stages at which verification has to be carried out, etc. areto be clearly addressed here. It could be considered as the master plan regarding allaspects related to verification and validation. This document should be reviewed toensure that it is complete with respect to all aspects of V&V that are relevant. Additionallythe test plan should cover issues like V&V independence.

Verification of sequential work products with respect to a reference (previous work productin the life cycle) is an ongoing activity as the software development progresses. Require-ments document, design document, source code are typical work products. For examplethe source code has to be reviewed with respect to the design document for functionalityand with respect to the coding standards for adherence to the same. Checklists preparedusing the design document and coding standards can be used for the respective reviews.Wherever the review with a checklist is felt to be inadequate as in the case of design,analysis has to be done with respect to modularity, architecture, algorithms, design offunctionality, etc.

Test Description for Functional Testing should be prepared to act as the guide for func-tional testing. This document should list all the test cases, test objective, test input/expected output, test setup, pass fail criteria, traceability, etc. It is essential that this testdescription document be derived from the requirements document.



Test Description for Unit Testing should explain the unit testing methods for achievingstructural coverage and decision coverage. It may be difficult to generate the test casesmanually and a tool, such as AdaTest, for example may be used to facilitate easy scriptingand execution of test cases. It also provided correct feedback on the test statistics likecoverage and others like complexity. Other tools such as RTRT, AdaTest, Cantata, etc.,can be used for testing. The tools can be used for software integration testing in additionto unit testing. Test cases can be generated in parallel with testing, based on the cover-age information provided by the tool. Practically, for documentation purpose, the set of testcases that are developed by using the tool for the required coverage can be documentedto form the test description document for unit testing.

Testing: Testing being an important verification activity should be a combination ofdifferent testing methods – black box and white box testing, for example. While black boxshould focus on product function, unit level/white box testing should focus on testing Cfunction or an Ada function/ procedure. White box testing should focus on structural/decision coverage. This kind of testing ensures that redundant code is discovered and alldecisions including those that are encountered during rare events are exercised and thesoftware is proven to be bug free. Sophisticated tools are available that can be used forunit level testing cum software integration. A bottom up integration strategy can befollowed.

Integration Testing is the last step in the testing activity and may be done in differentways. For example, in this case, this can be done in the host development environmentwith simulated input conditions. The tests can be repeated on target emulator and lateron the target board itself. Such tests often form part of acceptance tests if the develop-ment activity is out-sourced from a vendor.

Simulated TestingFor reliable testing, the test environment has to be close to what the real scenario wouldbe. In practice sophisticated simulators are used. In less complex cases, as in this case,a test harness that can provide predetermined input values at regular intervals from datafiles to the software under test can be used for testing. The output data can be stored andfurther analyzed.

References1. RTCA Guidelines on DO-178B2. Software Testing and Analysis: Process, Principles, and Techniques

By Mauro Pezze and Michal Young

Page : 10 of 11



About the AuthorSreekumar K Panicker has over ten years of experience in the development of mission/safety critical software for defense and aerospace systems.He also has a strong background in the Verification & Validation as per Software develop-ment standards like DOD-STD-2167A. On the DO-178B side his contributions to defineWipro’s DO-178B process model meant for Avionics/safety critical software is significant.He works with Wipro as a Software specialist in the avionics group.

About Wipro TechnologiesWipro is the first PCMM Level 5 and SEI CMMi Level 5 certified IT Services Companyglobally. Wipro provides comprehensive IT solutions and services (including systemsintegration, IS outsourcing, package implementation, software application developmentand maintenance) and Research & Development services (hardware and softwaredesign, development and implementation) to corporations globally.Wipro’s unique value proposition is further delivered through our pioneering OffshoreOutsourcing Model and stringent Quality Processes of SEI and Six Sigma.

Wipro in Embedded SystemsWipro Technologies, leading global provider of IT solutions and services has wideranging expertise in Datacom, Telecom, System Software and VLSI Systems. We haveexecuted projects using diverse real-time operating systems, networking technologiesand associated tools. The areas in which we offer services include Consumer electron-ics, Automotive electronics, Real time OS, Semiconductors, Mobile computing, Devicedrivers, Process control systems and Multimedia.

© Copyright 2002. Wipro Technologies. All rights reserved. No part of this document may be reproduced, stored in a retrieval system, ortransmitted in any form or by any means, electronic, mechanical, photocopying, recording, or otherwise, without express written permission fromWipro Technologies. Specifications subject to change without notice. All other trademarks mentioned herein are the property of their respective

owners. Specifications subject to change without notice.

Page : 11 of 11

America Europe1995 EI Camino Real, Suite 200 137, Euston RoadSanta Clara, CA 95050, USA London NW12AA,UKPhone:+1 (408) 2496345 Phone:+ 44 (020) 73870606Fax: +1 (408) 6157174/6157178 Fax: + 44 (020) 73870605

Japan India-Worldwide HDSaint Paul Bldg, 5-14-11 Doddakannelli, Sarjapur RoadHigashi-Oi, Shinagawa-Ku, Bangalore-560 035, IndiaTokyo 140-0011Japan Phone:+ 91 (80) 8440011 -15Phone:+ 81 (03) 54627921 Fax: + 91 (80) 8440254Fax: + 81 (03) 54627922

www.wipro.comeMail: [email protected]

For more white papers log onto www.wipro.com/shortcuts/downloads.htm

applying do178b

Documents