nonbehavioral specifications 0810. non-behavioral characteristics portability portability...
TRANSCRIPT
Nonbehavioral Specifications
08100810
Non-behavioral Characteristics
PortabilityPortability ReliabilityReliability EfficiencyEfficiency Human Engineering Human Engineering TestabilityTestability UnderstandabilityUnderstandability ModifiabilityModifiability
Portability
Degree to which software running on one Degree to which software running on one host computer environment can be host computer environment can be converted to run on another.converted to run on another.
Not necessary for all applications Not necessary for all applications (embedded systems, single-use systems).(embedded systems, single-use systems).
Some applications, it is essential.Some applications, it is essential.
Problems with specification
May be impossible to quantify: “The May be impossible to quantify: “The maximum time to port to host system X maximum time to port to host system X shall be …”shall be …” We don’t know what the next generation We don’t know what the next generation
will be;will be; Does the maximum time affect the design Does the maximum time affect the design
of the system?of the system?
Approaches to Specifying Portability
Source languageSource language Java: JVM ported to lots of platformsJava: JVM ported to lots of platforms Ada: DoD certifies – no extentions, no subsetsAda: DoD certifies – no extentions, no subsets Ideally, language is a design issue, but if its effect on Ideally, language is a design issue, but if its effect on
portability is critical, it’s a requirementportability is critical, it’s a requirement Language selection tends to be political in organizationsLanguage selection tends to be political in organizations
Host operating systemsHost operating systems Say which ones up front, if you knowSay which ones up front, if you know
Compiler selectionCompiler selection Ansi Standard compilersAnsi Standard compilers
Reliability
This is difficult in software:This is difficult in software: ““The system shall be 99.999% reliable.”The system shall be 99.999% reliable.”
What does this mean?What does this mean? It could mean that the phone system may lose a It could mean that the phone system may lose a
call now and again, but the entire system must call now and again, but the entire system must not fail for more than 5 minutes a year.not fail for more than 5 minutes a year.
In a patient monitoring system, it may mean In a patient monitoring system, it may mean that if it does go down, it alerts staff. It must that if it does go down, it alerts staff. It must not make a mistake in monitoring more than not make a mistake in monitoring more than one patient in 100,000.one patient in 100,000.
Traditional reliability (hardware) MTTFMTTF
MTTF of system is well defined in terms of MTTF of components.MTTF of system is well defined in terms of MTTF of components. Redundant components improve reliability because failure of components Redundant components improve reliability because failure of components
is independent.is independent. Hardware degrades in its environment.Hardware degrades in its environment. Bathtub curve for electronicsBathtub curve for electronics
time
failures
Burn-in
MTTF and Software
Software doesn’t degrade over time.Software doesn’t degrade over time. Suppose you run a program for 10 years Suppose you run a program for 10 years
without failure, then it suddenly fails. without failure, then it suddenly fails. Why?Why?Software was changed.Software was changed.Software was used a new way.Software was used a new way.
Bugs
Failure: Software does not do what is required Failure: Software does not do what is required (specified). Behavior is different from that needed.(specified). Behavior is different from that needed.
Fault: A cause of a failure. Fault: A cause of a failure. Not all faults result in failure.Not all faults result in failure. All failures result from faults.All failures result from faults. A state in which there is a fault without a A state in which there is a fault without a
failure is a failure is a hazardhazard state. state. Error: A design or implementation flaw.Error: A design or implementation flaw.
Testing
The purpose of testing is not to demonstrate The purpose of testing is not to demonstrate correct execution of the program.correct execution of the program.
The purpose of testing is to discover faults.The purpose of testing is to discover faults.
Problems specifying reliability in terms of bugs. Assume quality is designed in from the start:Assume quality is designed in from the start:
The software testing shall require no more than two The software testing shall require no more than two months.months.
Software testing shall discover no more than 10 bugs.Software testing shall discover no more than 10 bugs. The software shall have no more than one bug per The software shall have no more than one bug per
thousand lines of code.thousand lines of code. We want zero bugs, so this must be better than 5 per We want zero bugs, so this must be better than 5 per
1000.1000. How do we know how many there are? Wait until How do we know how many there are? Wait until
software is retired, then count them.software is retired, then count them.
Fault Seeding
Before testing, insert some bugsBefore testing, insert some bugs Track how many of these are found in Track how many of these are found in
testing.testing. Total bugs in system = Total bugs in system =
(#seeded * total_detected)/seeded_detected(#seeded * total_detected)/seeded_detected
Example
Secretly seed 10 bugs.Secretly seed 10 bugs. Test team discovers 120 bugs, 6 of which Test team discovers 120 bugs, 6 of which
are seeds.are seeds. Bugs = 10 * 120 / 6 = 200 bugsBugs = 10 * 120 / 6 = 200 bugs # bugs remaining = 200 – 120 = 80, 4 of # bugs remaining = 200 – 120 = 80, 4 of
which are “known”.which are “known”.
Notes
Not all bugs are equalNot all bugs are equal Equally difficult to findEqually difficult to find Equally difficult to repairEqually difficult to repair
Seeding is harder than it looks.Seeding is harder than it looks.
IntuitiveIntuitive Measure of testing effectivenessMeasure of testing effectiveness
Reliability
Levels of criticalityLevels of criticality Cause mild inconvenienceCause mild inconvenience Cause minor financial lossCause minor financial loss Cause major embarrassmentCause major embarrassment Cause major financial lossCause major financial loss Injure peopleInjure people Kill a few peopleKill a few people Kill many peopleKill many people Destroy humankindDestroy humankind
Example Reliability Requirement
No more than five bugs per 10K lines of executable No more than five bugs per 10K lines of executable code may be detected during integration and code may be detected during integration and system testing. No more than ten bugs per 10K system testing. No more than ten bugs per 10K lines of executable code may remain in the system lines of executable code may remain in the system after delivery, as calculated by the Monte Carlo after delivery, as calculated by the Monte Carlo seeding technique defined in Appendix III. The seeding technique defined in Appendix III. The system must be 100% operational 99.9% of the system must be 100% operational 99.9% of the calendar time during its first year of operation.calendar time during its first year of operation.
Efficiency
The utilization of scarce resources.The utilization of scarce resources. MemoryMemory CPUCPU DiskDisk CommunicationCommunication
Easy to specify if limits are givenEasy to specify if limits are given Example: ATC system: Example: ATC system: The system shall trace The system shall trace
the movements of up to fifty aircraft.the movements of up to fifty aircraft.
Need to say how it will degrade:
What if there are 51 aircraft? Possibilities:What if there are 51 aircraft? Possibilities: Software fails.Software fails. Track first 50, ignore 51Track first 50, ignore 51stst.. Software notifies 51Software notifies 51stst pilot to leave area. pilot to leave area.
Human Engineering: Levels of Specification The system shall have an easy-to-use The system shall have an easy-to-use
human interface.human interface. The system shall be menu driven.The system shall be menu driven. The system shall be menu-driven. Appendix The system shall be menu-driven. Appendix
A shows sample menus.A shows sample menus. The system shall be menu-driven. Appendix The system shall be menu-driven. Appendix
A shows all menus to be used.A shows all menus to be used.
Human Engineering: Error Messages Unless there is a sound understanding of the Unless there is a sound understanding of the
types of error messages the system can types of error messages the system can generate, there is insufficient knowledge of generate, there is insufficient knowledge of the system’s expected normal behavior.the system’s expected normal behavior.
It is a good idea to have an appendix that It is a good idea to have an appendix that specifies the text of all error messages.specifies the text of all error messages.
Testability, Modifiability, Understandability Very difficult to quantify.Very difficult to quantify. These are important contributors to cost These are important contributors to cost
(maintenance).(maintenance). One suggestion is to specify conformity to a One suggestion is to specify conformity to a
set of programming standardsset of programming standards
Programming standards specify
Naming conventionsNaming conventions Invocation conventionsInvocation conventions
Calls, interrupts, synchronizationCalls, interrupts, synchronization Message formatsMessage formats
Component headersComponent headers Format and contentFormat and content
In-line documentation styleIn-line documentation style Use of global constructs and variablesUse of global constructs and variables Use of named constructsUse of named constructs Modularity standardsModularity standards
Where to put these?
Be sure you really have these requirementsBe sure you really have these requirements Sections 5 and 6 in SRSSections 5 and 6 in SRS
5. NON-BEHAVIORAL REQUIREMENTS5.1. PERFORMANCE REQUIREMENTS
5.2. SECURITY
5.3. QUALITATIVE REQUIREMENTS
5.3.1. Availability
5.3.2. Maintainability
5.3.3. Portability
5.3.4. Design and Implementation Constraints
6. Other Requirements
6.1. DATABASE
6.2. OPERATIONS
6.3. SITE ADAPTATION