requirement engineering for dependable systems

Requirements engineering for dependable systems

Tutorial

Joint Meeting of DC/Baltimore SIGAda Chapter DC Chapter of ACM

11 October 2005

Dr. William [email protected]

The MITRE CorporationThe authors’ affiliation with The MITRE Corporation is provided for identification purposes only, and is not intended to

convey or imply MITRE's concurrence with, or support for, the positions, opinions or view points expressed by these authors.

2MITRE

Requirements engineering for dependable systems – SIGAda – 11 Oct 2005

Agenda Introduction to tutorial content Dependability and requirements Nature of requirements Developing requirements Quality factors for requirements Common pitfalls Summary

3MITRE


What we will cover The nature of dependability The nature and role of requirements The various types of requirements, their role in

development, their impact on system success Important characteristics of requirements (quality

attributes) The different ways that requirements need to be

handled A set of common challenges and strategies for

how to manage them What we will not cover

» Particular approaches to requirements definition (such as specific tools and techniques) – this is beyond the scope (and time)

4MITRE



5MITRE


Focus is on dependability As defined by IFIP WG-10.4

"The trustworthiness of a computing system which allows reliance to be justifiably placed on the service it delivers“ http://www.dependability.org/

Dependability is partially a concern of the systems’ specification which defines the services to be delivered

The specification defines (or should define) what aspects of the system need to be dependable (what services)

» and what dependable means in the context of that specific system

The nature of reliance must be defined» Total conformance to spec, or» Degraded levels allowed, or» Optional feature, or….

6MITRE


Some aspects of reliance The system does not fail when a service is requested Software users, selfishly so, expect the systems they

use to be completely dependable» But may be unwilling to pay the cost of developing such

systems In practice, experienced users of specific systems

learn the undependable features of the system and work around them

Sometimes, this is unacceptable When developing the requirements for a system, the

nature of the reliance must be clearly defined» What do the users expect?» What do they need?» What will they tolerate?» Where are the risks and dangers?

7MITRE


Dependability Characterizing dependability has been an important focus for

years e.g., NASA’s HDCP – High Dependability Computing Project

» http://hdcp.org/» HDCP “investigates and evaluates new approaches for improving

NASA's ability to create dependable software for mission applications”» Participants:

> Carnegie Mellon University > University of Washington > Massachusetts Institute of Technology > NASA Ames Research Center > University of Maryland > University of Wisconsin-

Milwaukee> University of Southern California

1970 – IEEE-CS TC on Fault-Tolerant Computing 1980 – IFIP WG 10.4 - Dependable Computing and Fault

Tolerance 1985 - J.-C. Laprie dependability synthesis 1989 – IFIP Working Conference on Dependable Computing for

Critical Applications

8MITRE


Laprie Dependability Tree

Dependability

Attributes

Means

Threats

AvailabilityReliabilitySafetyConfidentialityIntegrityMaintainability

Fault PreventionFault ToleranceFault RemovalFault Forecasting

FaultsErrorsFailures

Avižienis, Algirdas, Jean-Claude Laprie, and Brian Randell. “Fundamental Concepts of Computer System Dependability”. IARP/IEEE-RAS Workshop on Robot Dependability: Technological Challenge of Dependable Robots on Human Environments. May 21-22, 2001

9MITRE


Dependability attributes (page 1 of 2)

Availability – the ability of the system to be ready for operation at any point in time

» Usually defined as the proportion of time that the system is ready for use, to the total time period of interest

» e.g. if the goal is for the system to be available for use for all but 4 seconds per year, then the availability is 0.9999999 over the year.

Reliability – the ability of the system to operate for a period of time without failure

» Defined as either the failure rate (failures per time period) or as the average operational time between failures (MTBF). A typical reliability requirement would be 100 hours, meaning that the system is required to operate continuously for 100 hours on the average before failing.

Safety – the ability of the system to avoid actions that result in catastrophic actions that result in human injury, large financial loss, or damage to important assets

» Includes actions that are required as well as actions that must be avoided.

10MITRE


Dependability attributes (page 2 of 2)

Confidentiality – the ability of the system to avoid disclosure of information to unauthorized recipients

Integrity – the ability of the system to avoid being corrupted, including:

» the correctness and consistency of the software code and the data structures that contain the information being processed

» the system’s ability to avoid corruption while it is executing, including protection against unauthorized modification or destruction of the information and the code.

Maintainability – the ability of the system to be easily repaired, enhanced, or modernized

11MITRE


Information Assurance A newer thrust regarding dependability – IA –

expands on the Dependability Tree Availability Integrity Confidentiality Authentication – A security measure designed to

establish the validity of a transmission, message, or originator, or a means of verifying an individual's authorization to receive specific categories of information.

Non-repudiation – An assurance that the sender of data is provided with proof of delivery and the recipient is provided with proof of the sender's identity, so neither can later deny having processed the data.

Department of Defense. Information Assurance - DoD Directive.8500.1. Oct 2002.

as previously discussed

12MITRE


Acceptability Overall, key focus is on what the stakeholders

want and need Defined through the requirements Generally willing to make tradeoffs among various

system attributes» Higher cost in exchange for more functionality» Late delivery just to get the system operational while it

is still useful Tradeoffs can be expressed in an Acceptability

Framework» A classification of requirements and system attributes

organized to permit making trade-offs to determine whether system is acceptable for use

13MITRE


Acceptability Framework

Acceptability

Dependability

AvailabilityReliabilitySafetyConfidentialityIntegrityMaintainability

Fitness for purpose

Adherence to requirements

Cost

Delivery Schedule

UsabilityEase of learningEfficient to useEasy to remember

AuthenticationNon-repudiation

InformationAssurance

14MITRE



15MITRE


What are requirements? IEEE Std. 610.12-1990, IEEE Standard Glossary

of Software Engineering TerminologyRequirement:(1) A condition or capability needed by a user to solve a

problem or achieve an objective.(2) A condition or capability that must be met or

possessed by a system or system component to satisfy a contract, standard, specification, or other formally imposed documents.

(3) A documented representation of a condition or capability as in (1) or (2).See also: design requirement; functional requirement; implementation requirement; interface requirement; performance requirement; physical requirement.

16MITRE


Analysis of definition IEEE definition is broad – term often used carelessly Popular use often ignores this definition

» Anything that is “required”» Ranges from hopes to dreams to budgets to schedules...» “This schedule is required”…”This budget is required”…“These

software (SW) components must be used”…”These algorithms must be used”…

Different types of requirements need to be handled differently» because they affect system/SW development in different ways

This section of the tutorial focuses on differentiating different types of requirements » and on recommending how to handle them appropriately

Remember that all types of requirements are important in some way

Just because something is a requirement, does not mean

that it is a requirement



Huh?

17MITRE


Could you repeat that?

I’m sorry – I must have missed something Just because someone says ”I require xxx” does

not mean that this is a software requirement» That is, it might not be a behavioral specification

“I require this product to be developed within 6 months”» A constraint on development period but says nothing

directly about product behavior» Important, but plays different role

Our focus is on keeping different types of requirements separate» and managed appropriately





“You get what you spec, not what you expect”

18MITRE


IEEE Std 830-1998 – IEEE Recommended Practice for Software Requirements Specifications: “A requirement specifies an externally visible function or attribute of a system”» We can see inputs and the outputs, but not what happens

inside For any product (SW, HW, total system), the

behavioral requirements for that product specify its externally visible behavior» as seen by other systems outside

IEEE Std 830-1998

System

19MITRE


Levels of systems But each such system could be part of a larger system

» Which has its own requirements (externally visible behavior)

System

20MITRE


Component A Requirements

A B

C D E

Input Output

Context of requirements All requirements are defined in context of a specific

component (e.g., black box)» Which may consist of additional constituent components

(e.g., subsystem, modules,...)» Hence there are multiple levels of requirements based on

level of component> System level, subsystem level, software configuration item

(SCI) level, component level, software unit level,... Component design (its architecture) consists of:

» The requirements for behavior of each constituent component

» The interrelationships between the components

Interaction of components produces the behavior of parent component

OutputInput

21MITRE


Major categories of requirements Behavioral - externally visible behaviors of an item (aka

functional specifications, functional requirements) Quality-of-construction - qualitative attributes of an item,

such as maintainability and portability» Often not directly observable» Usually deal with how product can be handled

Programmatic - terms and conditions imposed as a part of a contract exclusive of behavioral requirements (e.g., costs, schedules, organizational structures) aka contractual » Addresses development of product

Implementation - aka implementation constraints, design constraints – restrictions placed on developers that limit design space» e.g., Use of specific software components (DII COE)» e.g., Imposition of specific algorithms» e.g., Customer-mandated architectures (e.g., Joint Technical Architecture (JTA))

22MITRE


Types of behavioral requirements (1 of 3)

Functional - input-output behavior in terms of responses to stimuli

» Output = fn (input), e.g., x x2

Interface - characteristics of component’s interfaces» e.g., interfaces with other components (peer-to-peer)» e.g., appearance of operator screens (user)» e.g., computing infrastructure attributes / APIs (infrastructure)

Temporal - speed, latency, and throughput of functional behaviors

» e.g., display refreshed screen every 0.5 sec, e.g., x x2 in 0.5 sec

» e.g., transmit filtered data within 2 sec of receiving unfiltered data» e.g., process 10,000 database requests per hour

Capacity - amount of information that can be handled» e.g., 25 simultaneous users» e.g., 20,000 employee records

23MITRE



Resource utilization - limitations on computer resources that can be used

» Memory and processor usage» e.g., limit of 5 gb disk memory for scratch storage» e.g., limit of 20% of total processor cycles (loading)

Trustworthiness - degree of confidence in product’s delivery of functions

» Reliability - MTTF = 30 hrs» Availability – 99% over 30 days» Safety – e.g., actions to avoid» Confidentiality – “the system shall not allow access to

system features unless operator first enters correct ID and password.”

» Integrity (partial) – “the system shall not be vulnerable to spyware”

24MITRE



Usability - how easy it is for an operator/user to make use of the system

» For system-to-system interfaces – deals with the complexity of the interfaces, their ease of implementation, and their efficiency of operation

> “Use of system resources by other systems shall be facilitated by minimum messaging and handshaking”

» For human operators – deals with the complexity of the interfaces relative to the how operators can operate with them, the ease of learning, and the efficiencies with which operators can exploit the system services

> “Operators shall be able to use system services with minimum number of actions.”

25MITRE


Characteristics of behavioral requirements Often, behavioral requirements can be directly tested

» Functional, interface, temporal, capacity» Apply input, observe input, compare with oracle

For some requirements, analysis of test results required» “Every 10 minutes the average of the room temperatures over

the previous 10 minutes will be displayed” For others, direct testing might not be feasible (or

possible)» “System will have 0.9999999 availability (down 4 sec/year)”» “System shall run for 100 years without failure”

or be practical» “System shall be able to handle 1000 users simultaneously”

Some are qualitative – cannot be tested directly» Trustworthiness and usability

26MITRE


Quality-of-construction requirements Qualitative attributes of an item

» Deal with how the product can be handled» Not usually directly measurable or observable» We have measures that can give us insight into these qualities

> to help us to infer level of quality> Based on related quantitative attributes of systems

» Evaluation of these requirements tend to be based on subjective and heuristic criteria.

Examples:» Portability – ease with which component can be ported from one

platform to another» Maintainability – ease with which product can be fixed when

defects are discovered» Extensibility – ease with which product can be enhanced with

new functionality» Integrity (shared with behavioral)– resistance to unauthorized change

27MITRE


Programmatic (contractual) requirements Terms and conditions imposed as a part of a contract

exclusive of behavioral requirements Address development aspects of product Examples

» Costs» Schedules» Organizational structures» Key people» Locations

While these are required characteristics of development effort, they are not characteristics of the product

Such requirements may directly affect ability to achieve desired levels of dependability

» e.g., not enough money or time

28MITRE


Implementation requirements (1 of 2)

Restrictions placed on developers that limit design space Two important types:

» Product design and implementation constraints – restrictions on design styles and coding

» Process and development approach constraints – restrictions on processes and techniques

Examples» Use of specific software components (can also be viewed as

interface requirements depending on context)» Imposition of specific algorithms

> But sometimes algorithms can be used to define functionality» Required use of specific designs» Imposition of specific coding styles» Requirement to use specific fault tolerance mechanisms» Requirement to employ an object-oriented development approach

29MITRE


Implementation requirements (2 of 2)

Important note - an implementation constraint to a system may be a requirement to a SW component within that system

» Paul Simon: “One man’s ceiling is another man’s floor.” Implementation constraints are (in one sense)

independent of the required system behavior Generally, there are many different ways that a specific

set of requirements could be implemented Implementation constraints narrow this design freedom

» Reduce the degrees of freedom available to the developers Sometimes, for dependable systems, constraints are

essential» Constrain design to certain patterns known for enhancing

dependability (e.g., fault tolerance, exception handling)» Enforce use of certain algorithms which have been verified to

provide effective solutions

30MITRE



31MITRE


Developing requirements Now we now what requirements are But where do they come from

» and how do we define them?

Requirements grow out of user needs These needs are analyzed, and approaches to

satisfying them are defined Approaches are refined into specific requirements for

a system that will provide needed services Note that a system can consist of

» Software – that is our job» Hardware – we have yet to figure out how to get rid of it» People – the hardest piece to the puzzle – they are soooo

unpredictable

32MITRE


Dec

isio

n P

roce

ss

Dec

isio

n P

roce

ss

Decision points for requirements definitionEach step in progression involves deciding between alternative approaches

CapabilitiesSet A

CapabilitiesSet C

SystemReqs

SystemReqs

SystemReqs

SystemReqs

SystemDsn

SystemDsn

SWReqs

SWReqs

SWReqs

SWReqs

Dec

isio

n P

roce

ss

SystemConcept

Set AD

ecis

ion

Pro

cess

SystemDsn

CapabilitiesSet B

SystemConcept

Set C

Dec

isio

n P

roce

ss

MissionNeeds

SystemConcept

Set B

System designSoftwaredesign

33MITRE


SW requirements analysis Requirements allocation to SW components is not

the same as defining SW requirements Once requirements allocated to components, SW

requirements analysis (SRA) needed to:» Derive SW requirements» Place into form suitable for implementation

Involves trade-off analyses

DraftSRSs

“Final”SRSs

FinalSCIs

SW Reqs AnlSW Dsn SW Impl SW Test

Software Engineering

Systems Engineering

Sys Reqs AnlSys Dsn

Skipping SRA is very risky

34MITRE


SW requirements analysis techniques Different techniques/processes are used

» ad hoc, functional, object-oriented, formal New processes arrive every day

» Agile Unified Process, Extreme Programming, Cleanroom Software Engineering

Many tools exist» DOORS, Analyst Pro, Rational Rose, …

Many notations exist with which requirements may be expressed» UML, Z, State charts, English, Latin, …

Whatever technique is used, goal is to develop a set of well-defined requirements that clearly describe what the system is to do» or the component, or the unit,…

All should produce the same result – a description of behavior of the system

Important to select technique to be appropriate to system

35MITRE


When requirements are defined Often, requirements are not complete until product is

complete During systems definition phase

»Usually at a general level During system design phase

»Detail added, capabilities become functions

During SW requirements analysis»More detail added, more specific

behaviors and formats During SW design/implementation

»Refinement / clarification / more detail> e.g., GUI details often deferred until later

»Changes / adaptations »Additions (e.g., evolutionary development)

0

20

40

60

80

100

120

140

160

180

200

36MITRE



37MITRE


Requirements quality factors So, how do I know that my system requirements

are good enough? IEEE Std 830-1993 (IEEE Recommended Practice

for Software Requirements Specifications) provides a set of quality factors for requirements specifications for systems

Contains nine criteria against which a requirements set can be assessed

Absence of these qualities is strongly correlated to subsequent problems in software development, i.e., ignore at your own risk

These factors are particularly crucial for developing dependable systems:

CompleteUnambiguousCorrectConsistentVerifiable

Modifiable TraceableRanked for importanceRanked for stability

38MITRE


Description of quality factors (page 1 of 2)

A requirements specification for a system is complete if all external behaviors are defined

A requirement is unambiguous if it has one and only one interpretation

A requirements specification is correct if every requirement stated in the specification is one that the software shall meet

A requirements specification is consistent if» It agrees with its baseline, foundation specification (e.g., system

specification)» No subset of requirements within the specification conflict with each

other A requirements specification is verifiable if every

requirement contained in the specification can be verified A requirements specification is modifiable if changes can

be made to the requirements without major disruption of the structure of the specification

39MITRE


Description of quality factors (page 2 of 2)

A requirements specification is traceable if » the origin of each requirement is clear, and » the structure of the specification facilitates the

referencing each requirement within lower-level documentation

A requirement is ranked for importance if it is assigned a rating of its criticality to the system, based on negative impact should requirement not be implemented or fail during execution

A requirement is ranked for stability if its likelihood to change is identified, based on changing expectations or level of uncertainty in its description

40MITRE



41MITRE


Common pitfalls Lack of prioritization of requirements Over-specified / over-constrained / unbounded Volatile and late-defined requirements Unknown “physics” Mixing requirements and design

42MITRE


Lack of prioritization of requirements Not all requirements are equal

» Some are more firm than others» Some are more important than others» Others may be guesses and have flexibility about final

behavior Need to provide info to developers about where

» Requirements may change» Less firm requirements exist» Areas of flexibility» Areas of firmness

Recommendations» Define rigidity of requirements and ranges of acceptance

> Thresholds and objectives> “Must haves” versus “wanna haves” versus “wouldn’t it be

nice”

43MITRE


Over-specified / over-constrained / unbounded Sometimes requirements are too ambitious, too

restrictive, or too general Too ambitious – results in gold-plating

» Unneeded capabilities created, unattainable functions defined Too restrictive – results in narrow, point solutions

» System rapidly becomes outdated when mission changes Too general – results in inefficient system that does

everything but not well Result is wasted resources Recommendations (these are very general but

important)» Focus on prioritization of requirements» Ensure what is needed is emphasized» Build system in a series of increments

> With most critical functions completed early

44MITRE


Volatile and late-defined requirements (1 of 4)

Requirements always change» Some don’t change, but are defined late

Not necessarily bad but careful management necessary to avoid

» expensive rework (and cost and schedule impact) » compromises to functionality

Crucial to associate levels of risk to levels of change» Some changes are low-risk» Other may be high risk» Related to amount of rework require

Developers better able to design defensively if they know

» Which requirements are likely to change» Degree of change that could be expected

45MITRE



Depends on attributes of the requirement and its linkage to design

» Some can be defined early or late» Some must be defined early» Some should be defined later

Important attributes (i.e., how to decide…)» If level of understanding of desired behavior is low

(exact behaviors not well understood or unknown) – delay in definition may reduce risk

> If defined and frozen early, later changes may impact design and cause rework

» If high likelihood that requirement will change – delay in definition may reduce risk

> Avoids rework due to late changes

46MITRE



Important attributes (cont’d)» If a requirement has high or complex external

component dependencies – early resolution may reduce risk

> Late changes likely to affect external systems/components

» If a requirement has strong internal design dependencies – early resolution may reduce risk

> Late changes may force extensive rework due to design dependencies

Level of understanding of desired behavior

Likelihood that requirement will change

External component dependencies

Internal design dependencies

Early definition

lowhigh

highlow

complex simple

weakstrong

Late definition

47MITRE



The following are recommendations to address requirements volatility: » Define requirements with priorities and likelihood to change

> Allows designers to insulate themselves from unexpected change

» Ensure design accommodates expected changes» Where possible, allow run-time reconfiguration to allow

changing behavior without changing requirements> e.g., screen color options

» Correlate with assessment of late definitions» Assess dependencies between requirements and design

> Some requirements deeply affect design globally> Others have limited design impact (GUI formats)

» Ensure requirements dependencies are well understood» Define and monitor requirements stability with metrics

> Track immature requirements, undefined requirements, and changing requirements

48MITRE


Unknown “physics” (1 of 2) i.e., the outside world Sometimes we don’t know what the behavior of the

software should be» Because of our lack of knowledge of real world» Typical for embedded systems

Results in changing or late-defined requirements» Also may need to change internals of systems (algorithms,

design structures) Sometimes need to use the software system to

explore its operational environment» and determine its correct behavior» and update SW based on discoveries

Frequently, discoveries are at fine grain level» Adjustments to constants

49MITRE


Unknown “physics” (2 of 2) Sometimes, changes are to algorithms In such a case, requirements statements must

explicitly recognize situation» Design (and process) must allow for experimentation /

exploration» Product is also a prototype used for requirements

elicitation Mitigation strategies include:

» Use of executable models and prototypes» Use of simulations to depict external environment» Use of data logging functions to collect relevant data» Iterative development of system, and use of iterations to

probe and explore the environment

50MITRE


Mixing requirements and design (1 of 2)

Because of complex interrelationships among types of requirements, it is crucial to identify them and keep them separate

Why? Behavioral specs deal with external entities (mutual

dependencies)» Other systems» Hardware» People

Changing them affects external entities» Not always possible to make changes» May involve negotiation

Implementation requirements deal with internal design» No direct external dependencies

Typical implementation constraints» Algorithms» Flow charts

51MITRE


Mixing requirements and design (2 of 2)

What might happen...» Inefficient and ineffective testing

> Software testing is based on SRS> If SRS contains design as well as behavior, either

- Testers must separate design from behavior before testing, or- Testers must test for design as well as behavior, requiring

breaking into internals of SW» Inefficient processing

> If algorithm is specified as part of SRS, designers might not have flexibility to optimize

» Excessive CM effort – design changes require CM authority at SRS level

Recommendations:» Place all design information (including. algorithms) into separate

volume> e.g., Algorithm Design Document (ADD)

» Ensure all requirements are externally visible and can be tested without examining design/construction

52MITRE



53MITRE


Summary Proper management and control of requirements

is essential to the development of dependable systems

requirements form the foundation of all systems, and are especially crucial for systems that must be dependable

Attention must be placed on capturing the areas of required dependability in the specification

There are many requirement-based techniques that can be applied to mitigate risks and enhance dependability of system

54MITRE


End Any questions?....

Contact informationDr. William Bail

[email protected](202) 781-1945

55MITRE


References and Suggested Readings (1 of 3)

Booch, Grady, James Rumbaugh, Ivar Jacobson. The Unified Modeling Language User Guide. Addison-Wesley. 1999

Capability Maturity Model® Integration (CMMISM), Version 1.1. Software Engineering Institute. December 2001

Davis, Alan M. Software Requirements - Objects, Functions, & States. Prentice Hall. 1993

IEEE Standard for Software Verification and Validation Plans. IEEE Standard 012-1986, IEEE Computer Society. 1968

IEEE Recommended Practices for Software Requirements Specifications. IEEE Std 830-1998, IEEE Computer Society. October 20, 1998

Institute of Electrical and Electronics Engineers. Glossary of Software Engineering Terminology. IEEE Std. 610.12-1990.

56MITRE


References and Suggested Readings (2 of 3)

Kovitz, Practical Software Requirements - A Manual of Content & Style. Manning Publications. 1999.

Military Standard Specification Practices. MIL-STD-490A, Department of Defense, USA. June 4, 1985

Prowell, Stacy J., Carmen J. Trammell, Richard C. Linger, Jesse H. Poore. Cleanroom Software Engineering - Technology and Process. The SEI Series in Software Engineering. 1999.

Risky Requirements. CrossTalk, The Journal of Defense Software Engineering. April 2002

Rosenberg, Doug. Use Case Driven Object Modeling with UML - A Practical Approach. Addison-Wesley. 1999

Dr. Young, Ralph, Effective Requirements Practices, Addison-Wesley, 2001.

requirement engineering for dependable systems

Technology

computing system

nature of dependability

role of requirements

requirements definition

dependability attributespage

robot dependability

systems ability

requirements quality