obstacle driven development: extending requirements analysis

Obstacle Driven Development

Extending Requirements Analysis

©odd.enterprises

19/11/2014

Obstacle Driven Development

19/11/2014 ©odd.enterprises 2

ODD Process


ODD Traffic Light Model


Background

Ideas of Obstacle Driven Development (ODD) are based on numerous development processes including:

• ISO V-model

• Test Driven Development

• ISO specifications

• Requirements analysis

• Agile principles


About Requirements Analysis

Requirements analysis encompasses tasks that determine the needs or conditions necessary for a new or altered product.

Tasks necessary for requirements analysis include:

• Analysing, documenting, validating and managing software or system requirements

• Identify and resolve conflicting requirements of stakeholders

• Identifying business needs or opportunities using testable and traceable processes


Requirements Analysis 1

Requirements analysis is performed in numerous ways

• Spiral model

• Use case analysis

• Safety integrity levels

This presentation is used to show how a requirements analysis spiral and SILs were adapted to give ODD processes.



Comparing, mapping and adapting the processes from the Spiral model to ODD led to this model.

• Agreed Behaviours replaces Agreed Requirements due to extended Specification

• Quality Assurance equivalent to Testing

• Negotiation similar to Verification



Further adapting the model leads to an Obstacle Driven Development model.

• Verification replaces Negotiation

• Validation replaces Evaluation

• Testing replaces Quality Assurance

Consolidated Requirements and Documents can be thought of as checkpoints for development19/11/2014 ©odd.enterprises 9

Requirements Checkpoint

The Consolidated Requirements can be considered the checkpoint for the Analysis stage.

• Consolidation using SILs is used to ensure the most important requirements are identified

• Most important requirements for each situation must be dealt with for a successful product


Documents Checkpoint

When using a product there should be sufficient documentation to describe all the behaviours expected of it.

• Documents should describe everything the product does

• Decomposed from high level behaviours into components


ODD M-model Checkpoints 1

Adding solution and production stages of development results in an ODD M-model.

• ODD process is linked from start to finish, and beyond

• Verification and validation between stages is performed

• Tests are ran as additions and editions are made



For the solution and production stages there has also been Checkpoints determined.

• Prototype should be created from an integrated solution

• Product should be the result of production

• Linked to other checkpoints horizontally


Prototype Checkpoint

Once a solution is fully integrated and tested then a prototype of the product should be achieved.

• Created from the integrated solutions at various levels

• Used to ensure the requirements are covered by the solution

• A working model or template of the product is achieved


Product Checkpoint

Finally once production is complete then working products should be achieved.

• Decomposition is used to ensure all products are produced according to solution

• Product Assembly at high level to demonstrate decomposition process



Checkpoints allow linking and testing of the results of previous stages.

• Each checkpoint is used to link another

• Prototype should fulfil identified requirements

• Product should behave as described in documents


ODD Analysis

The following slides are used to demonstrate how the requirements analysis is adapted to allow for ODD.

• Safety Integrity Levels used to measure and process hazards

• Decision tree approach used to create situations

• Verification and validation of specification


Fire Triangle

A fire triangle is a well known educational tool for understanding and preventing fires.

• If the fire triangle is completed then a fire will occur

• Preventing one of the situations will prevent a fire

• Requirements will often regard preventing fires


Fire Triangle 2

By reordering the fire triangle we can create a diagram which demonstrate the components interaction for a fire.

• Situations can be constructed using a tree diagram

• Constructing situations from components allow greater range to be considered


Fire Triangle 3

Using the reordered fire triangle it can be seen that the components combine to create a hazard.

• Process is adaptable to all fire hazards and environments

• Can be extended to any number of situations

• Each of the components can be given SIL ratings for Probability, Severity and Controllability


Fire Prevention 1

A simple fire prevention example to demonstrate how the requirements may be linked through the process.

• Fuel and oxygen are present

• Fire is prevented by preventing heat sources

• No smoking chosen as a solution, more are possible

Note this diagram has been simplified significantly.19/11/2014 ©odd.enterprises 21

Fire Prevention 2

The diagram shows how a behaviour has been identified which prevents fire.

• Fuel and oxygen are present so preventing fire through preventing heat is chosen

• Adaptable to different situations and fire prevention strategies.


Probability Tree 1

Probability Trees are used to measure the probability of a situation occurring that is based on a number of interacting events.

• Common examples are the probabilities of situations occurring from a coin toss or a drawing from a deck of cards

• Can be used to model complex interactions


Heads 50%

Heads 50%

Tails 50%

Tails 50%

Heads 50%

Tails 50%

Binomial Distribution 1

Binomial Distributions can be used to determine the probability for any number of events with 2 possible outcomes.

• Binomial process illustrates the concept of decision trees can be extended

• Can be used to model complex interactions


Heads 50%

Heads 50%

Tails 50%

Tails 50%

Heads 50%

Tails 50%


Binomial Distributions can be used to determine the probability for any number of events with 2 possible outcomes.

• Binomial process illustrates the concept of decision trees can be extended

• Can only be used to model yes or no experiments

𝑃 𝑋 = 𝑘 =𝑛𝑘

𝑝𝑘(1 − 𝑝)𝑛−𝑘


Probability Tree 2

We can extend the coin toss example into engineering by replacing the coin tosses with system components.

• Working component replaces heads

• Failing component replaces tails

• Potential hazards of a series of failures can be determined


Component 1Pass 99%

Component 2Pass 98%

Component 2Fail 2%

Component 1Fail 1%

Component 2Pass 98%

Component 2Tails 2%


Binomial Distributions can be used to determine the probability for components failing when creating a product.

• Process is a common example of using probabilities in manufacturing

• Can be extended to model failure in the field


Component 1Pass 99%

Component 2Pass 98%

Component 2Fail 2%

Component 1Fail 1%

Component 2Pass 98%

Component 2Tails 2%

Probability Tree 3

Probability Trees can be easily extended to other situations.

• Coins may also land on their side

• The event has been assigned a probability of 0.017 % (1 per 6000 tosses)

• Probability tree has exponentially more branches due to another outcome


Head ≈ 50%

Head ≈ 50%

Tails ≈ 50%

Side ≈ 1 / 6000

Tails ≈ 50%

Head ≈ 50%

Tails ≈ 50%

Side ≈ 1 / 6000

Side ≈ 1 / 6000

Head ≈ 50%

Tails ≈ 50%

Side ≈ 1 / 6000

Probability Tree 4

Probability Trees can be used to ensure all possible situations are modelled.

• Systems may have unknown states between pass and fail

• Unknown states can include loss of communication or wear

• Unknown states then investigated for effects


Component 1Pass 98%

Component 2Pass 96%

Component 2Fail 2%

Component 2Unknown 2%

Component 1Fail 1%

Component 2Pass 96%

Component 2Fail 2%



Component 2Pass 96%

Component 2Fail 2%


Safety Integrity Levels 1

Safety Integrity Levels (SILs) are used to provide a process for measuring the potential hazards of a situation.

• Situation is analysed and measures for Probability, Severity and Controllability determined

• From these measures for the risk of hazard occurring from the situation

𝑆𝐼𝐿 = 𝑃 𝐸 ∗ 𝑆 𝐸 ∗ 𝐶 𝐸



Safety Integrity Levels may be used for a wide range of safety critical analysis.

• Probability is measured by how likely the situation will occur

• Severity is measured by the potential damage of the situation

• Controllability is measured by the ability to effect change in the situation


Component𝑆𝐼𝐿 = 𝑃 𝐸 ∗ 𝑆 𝐸 ∗ 𝐶 𝐸


Coin toss example will be extended to provide example of SILs.

• Probability of outcome is slightly different for different coins

• Severity of an outcome can be measured by what is lost or gained

• Controllability of probability and severity determined by who flips the coin and the stakes



If a Probability Tree and SILs are processed in the same way then why not combine them in a decision tree?

• Measures are added for severity and controllability

• Each branch is a situation which can be processed to find SIL ratings and determine requirements


Component 1Pass

P(P1)*S(P1)*C(P1)

Component 2Pass

P(P2)*S(P2)*C(P2)

Component 2Fail

P(F2)*S(F2)*C(F2)

Component 1Fail

P(F1)*S(F1)*C(F1)

Component 2Pass

P(P2)*S(P2)*C(P2)

Component 2Fail

P(F2)*S(F2)*C(F2)

ODD Decision Tree 1

It is theorised that a Decision Tree can be used to find requirements.

• Severity and Controllability are added to each event

• Requirements are found with SIL processes using the branches

• Facilitates a unit testing approach for situations


Component 1Pass

P(P1)*S(P1)*C(P1)

Component 2Pass

P(P2)*S(P2)*C(P2)

Component 2Fail

P(F2)*S(F2)*C(F2)

Component 1Fail

P(F1)*S(F1)*C(F1)

Component 2Pass

P(P2)*S(P2)*C(P2)

Component 2Fail

P(F2)*S(F2)*C(F2)

ODD Decision Tree 2

Adding the SIL components to a Probability Tree allows requirements to be found directly from a decision tree.

• Structure is as a Probability Tree with processing including SILs

• SILs may be found by multiplying along the branches of the decision tree


Component 1Pass

P(P1)*S(P1)*C(P1)

Component 2Pass

P(P2)*S(P2)*C(P2)

Component 2Fail

P(F2)*S(F2)*C(F2)

Component 1Fail

P(F1)*S(F1)*C(F1)

Component 2Pass

P(P2)*S(P2)*C(P2)

Component 2Fail

P(F2)*S(F2)*C(F2)

ODD Decision Tree 3

Processing the resulting Decision Tree should be similar to a Probability Tree.

• SIL ratings processed by multiplying probability, severity and controllability along branches

• SIL result rating between 1 and 4, with 4 being the best


Component 1Pass

P(P1)*S(P1)*C(P1)

Component 2Pass

P(P2)*S(P2)*C(P2)

Component 2Fail

P(F2)*S(F2)*C(F2)

Component 1Fail

P(F1)*S(F1)*C(F1)

Component 2Pass

P(P2)*S(P2)*C(P2)

Component 2Fail

P(F2)*S(F2)*C(F2)

ODD Decision Tree 4

Using a Decision Tree and SILs gives numerous advantages.

• Branches can be used to ensure that situations are not missed

• Extendible and adaptable to new and differing situations

• Each branch gives a situation

• Situations can be found from combining their components


Component 1Pass

Component 2Pass

Situation A

Component 2Fail

Situation B

Component 1Fail

Component 2Pass

Situation C

Component 2Fail

Situation D

Processing Decision Tree

Situation A is used to define the situation where both components have passes.

Using the decision tree we can find the SIL rating for the situation.

𝑆𝐼𝐿 𝐴= 𝑃 𝑃1 ∗ 𝑆 𝑃1 ∗ 𝐶 𝑃1 ∗𝑃 𝑃2 ∗ 𝑆 𝑃2 ∗ 𝐶(𝑃2)

Situation D is used to define the situation where both components have failed.

Using the decision tree we can find the SIL rating for the situation.

𝑆𝐼𝐿 𝐷= 𝑃 𝐹1 ∗ 𝑆 𝐹1 ∗ 𝐶 𝐹1 ∗𝑃 𝐹2 ∗ 𝑆 𝐹2 ∗ 𝐶(𝐹2)


ODD Decision Tree 5

When a Decision Tree is used there is an infinite range of situations possible and most potential outcomes can be considered.

• Events can be comprehensively modelled and extended

• New situations are added as branches

• Models the complexity of the expected situations encountered


ODD Decision Tree 6

Component 1 Pass

Component 2 Pass

Component 3 Pass

Component 3 Fail

Component 3 Unknown

Component 2 Fail

Component 3 Pass

Component 3 Fail

Component 3 Unknown

Component 3 Unknown

Component 3 Pass

Component 3 Fail

Component 3 Unknown

Component 1 Fail

Component 2 Pass

Component 3 Pass

Component 3 Fail

Component 3 Unknown

Component 2 Fail

Component 3 Pass

Component 3 Fail

Component 3 Unknown

Component 2 Unknown

Component 3 Pass

Component 3 Fail

Component 3 Unknown

Component 1 Unknown

Component 2 Pass

Component 3 Pass

Component 3 Fail

Component 3 Unknown

Component 2 Fail

Component 3 Pass

Component 3 Fail

Component 3 Unknown

Component 2 Unknown

Component 3 Pass

Component 3 Fail

Component 3 Unknown


The following Decision Tree is used to illustrate the complexity of creating decision trees. This Decision Tree models just 3 components with 3 different states: Pass, Fail and Unknown.

Creating Unit Tests 1

ODD uses a unit testing approach throughout so tests must be created directly from situations to create behaviours.

• Each branch represents a situation and therefore a behaviour should cover each

• Unit tests should be created for each branch of the decision tree


Verify Behaviour

DescribeBehaviour

Validate Behaviour

Situation


1. Situation is chosen from a branch on the decision tree.

2. Behaviour verification test is first created and linked directly from the situation.

3. Behaviour is described to cover a situation.

4. Behaviour is validated.

5. Repeat for all situations.


Verify Behaviour

DescribeBehaviour

Validate Behaviour

Situation


Creating a specification from the analysis requires each situation will have a test created which is then passed.

• Specification used to describe all behaviours of the product

• Using an ODD Specification allows cross examination of the behaviours against situations


Verify Specification

DescribeSpecification

Validate Specification

Situation


The diagram shows an alternative form of ODD with the traffic light testing process between analysis and specification.

• Demonstrates the linking of stages with testing processes

• Traffic lights on the right indicate processes used to link the specification and solution (Design is

assumed to be tested)


odd.enterprises Service Model


Legal Stuff

ReferencesRequirements Analysis

http://en.wikipedia.org/wiki/Requirements_analysis

Safety Integrity Level

http://en.wikipedia.org/wiki/Safety_Integrity_Level

Decision Tree

http://en.wikipedia.org/wiki/Decision_tree

Requirements Spiral Model

www.engineering.uiowa.edu/~kuhl/SoftEng/Slides4.pdf

Unit Testing

http://en.wikipedia.org/wiki/Unit_testing

DisclaimerThe ODD M-model and associated processes are provided by odd.enterprises and may be used for any purpose whatsoever.

The names odd.enterprises and associated logos should not be used in any representation, advertising, publicity or other manner whatsoever to endorse or promote any entity that adopts or uses the model and/or associated processes.

odd.enterprises does not guarantee to provide support, consulting, training or assistance of any kind with regards to the use of the model and/or processes including any updates.

You agree to indemnify odd.enterprises and its affiliates, officers, agents and employees against any claim or demand including reasonable solicitors fees, related to your use, reliance or adoption of the model and/or processes for any purpose whatsoever.

The model is provided by odd.enterprises “as is” and any express or implied warranties, included but not limited to the implied warranties of merchantability and fitness for a particular purpose are expressly disclaimed.

In no event shall odd.enterprises be liable for any damages whatsoever, including but not limited to claims associated with the loss of data or profits, which may result from any action in contract, negligence or other tortious claim that arises out of or in connection with the use or performance of the model.


http://en.wikipedia.org/wiki/Requirements_analysis

http://en.wikipedia.org/wiki/Safety_Integrity_Level

http://en.wikipedia.org/wiki/Decision_tree

http://www.engineering.uiowa.edu/~kuhl/SoftEng/Slides4.pdf

http://en.wikipedia.org/wiki/Unit_testing