risk based testing - meetupfiles.meetup.com/3093192/risk based testing.pdfrisk based testing...
Post on 23-Mar-2020
6 Views
Preview:
TRANSCRIPT
Risk Based Testing
Ladislau Szilagyi
www.EuroQST.ro
-Why we need RBT ?
-Types of risks
-Managing risks
-Methods of evaluation & risk analysis
-Costs and benefits
Definitions (ISTQB glossary)
Risk = a factor that could result in future negative
consequences; usually expressed as impact and
likelihood.
Risk based testing = a testing strategy intended
to reduce the level of product risks and inform the
stakeholders of their status, starting in the initial
stages of a project. It involves the identification of
product risks and the use of risk levels to guide the
test process.
Safety = the capability of the software product to
achieve acceptable levels of risk of harm to
people, business, software, property or
environment in a specified context of use.
Risk outcome
Risk = negative outcome? … not always …
• Positive risks are opportunities, desired by both
the Project Manager and the stakeholders, and
may positively affect the project ( such as
increasing the ROI or finishing the project ahead
of time ).
• … But, positive risk may generate negative risks
( finishing a part of the project way before
schedule will create a lot of slack, as other
resources are not scheduled to work on the
project until much later ).
Risk types
• Product risk
– Functional
– Non-functional
• Project risk
– External
– Organizational
– Technical
Risk types
Project risk categories
• External:
– service provider related issues;
– client related issues;
…
Risk types
…
• Organizational: – skill and staff shortages;
– personal and training issues;
– political issues, such as: • problems with testers communicating their needs and
test results;
• failure to follow up on information found in testing and reviews (e.g. not improving development and testing practices).
– improper attitude toward or expectations of testing (e.g. not appreciating the value of finding defects during testing).
…
Risk types
…
• Technical:
– problems in defining the right requirements;
– the extent that requirements can be met given
existing constraints;
– the quality of the design, code and tests.
Risk dimensions
• Likelihood = the estimated probability that a risk
will become an actual outcome or event
• Impact = the damage that will be caused if the
risk become an actual outcome or event
Risk dimensions
Risk dimensions are depending on the context:
• Automotive: – Exposure (the relative expected frequency of the
operational conditions in which the damage can possibly happen)
– Control (the relative likelihood that the user can act to prevent the damage)
– Severity of the damage
• Avionics: – Threat
– Vulnerability
– Consequences
Risk factors
• Technical
– Complexity of technology and teams
– Personnel and training issues among the business analysts, designers, and programmers
– Conflict within the team
– Contractual problems with suppliers
– Geographical distribution of the development organization
– Legacy versus new approaches
– …
Risk factors
– …
– Tools and technology
– Bad managerial or technical leadership
– Time, resource and management pressure
– Lack of earlier quality assurance
– High change rates
– High earlier defect rates
– Interfacing and integration issues
Risk factors
• Business
– Frequency of use of the affected feature
– Damage to image
– Loss of business
– Potential financial, ecological or social losses or liability
– Civil or criminal legal sanctions
– Loss of license
– Lack of reasonable workarounds
– Visibility of failure leading to negative publicity
Risk options
• Ignore
• Assume
• Delegate
• Mitigate
• Contingency planning
RBT activities
• Establish the context
• Risk identification
• Risk analysis – Risk assessment
– Risks sorting
• Risk mitigation
• Risk contingency
• Risk reporting
Establishing the context
• Study the business domain
• Identify the stakeholders
• Define a framework
• Planning of the remaining activities
Product Risk Identification
• Risk statement
“Something may fail in some way due to some circumstances”
– Something – The component or feature where the
problem could occur
– Fail in some way – What potential failure
– Some circumstances – The reasons or vulnerabilities, why we are concerned
Product Risk Identification techniques
• Expert interviews
• Project matrix
• Independent assessment
• Use of risk templates
• Lessons learned
• Risk workshops
• Brainstorming
• Checklists
Product Risk Analysis techniques
• Informal – fast, cheap, but not accurate – Pragmatic Risk Analysis and Management (PRAM)
– Systematic Software Testing (SST)
– Product Risk Management (PRisMa)
Consequence
Pro
bab
ilit
y
Low
Low
High
High
1
2
3
4
Product Risk Analysis techniques
• Formal – accurate, but takes time, expensive – Cost-of-exposure
– Quality function deployment
– Fault tree analysis
– Hazard analysis
– Failure Mode Effect Analysis
FMEA - Example of formal Risk Analysis technique
• Failure Mode Effect Analysis – Define the System
– Identify Potential Failure Modes & Their Causes
– Evaluate the Effects on the System of Each Failure Mode
– Identify Failure Detection Methods
– Identify Corrective Measures for Failure Modes
3 factors:
– Severity = The criticality of the effects of bugs in this failure mode, should any exist, from 1 (most damaging) to 5 (least damaging),
– Likelihood = The probability of—and extent of impact associated with—bugs included in this failure mode, from 1 (most probable) to 5 (least probable).
– Priority = The importance of fixing bugs in this failure mode, should any exist, based primarily on the ability of the delivered system to meet customer needs, though also on logistical project issues, regulatory or standards compliance, or other business considerations, from 1 (most important to fix) to 5 (least important to fix).
Product Risk Analysis techniques
Product Risk Mitigation techniques
• Non-testing related
• Testing related
– Choosing an appropriate test design technique
– Reviews & inspection
– Reviews of test design
– Level of independence
– Most experienced person
– The way re-testing is performed
– Regression testing
RBT activities
Project Risk Mitigation techniques
• Non-testing related
• Testing related
– Early preparation of test ware
– Pre-testing test equipment
– Pre-testing earlier versions of the product
– Tougher entry criteria
– Requirements for testability
– Participation in reviews of earlier project results
– Participation in problem and change management
– Monitoring of the testing progress and quality
Questions?
top related