inf5530 : foundations of software testing 05. february ... · inf5530/ 05.02.2014 / © raluca...
TRANSCRIPT
INF5530/ 05.02.2014 / © Raluca Popescu 1
Static techniques
Lecturer: Raluca Popescu
INF5530 : Foundations of Software Testing
05. February 2014
Chapter 3
INF5530/ 05.02.2014 / © Raluca Popescu 2
Overview
1. Static techniques and the test process
2. Review process
3. Static analysis by tools
INF5530/ 05.02.2014 / © Raluca Popescu 3
1. Static techniques and the test process
INF5530/ 05.02.2014 / © Raluca Popescu 4
1. Static techniques and the test process
Reviews, static analysis and dynamic testing have the same objective – identifying defects. They are
complementary. Compared to dynamic testing, static techniques find causes of failures (defects) rather
than the failures themselves.
• Dynamic testing - requires the execution of software.
• Static testing - manual examination and automated analysis of the code or
documentation.
• Reviews = a way of testing software products (including code) and can be performed
well before dynamic test execution.
• Reason to make reviews
- Defects detected during reviews early in the life cycle are often cheaper to remove
than those detected while running tests.
- Reviews can find omissions, for example, in requirements, which are unlikely to be
found in dynamic testing.
• Tools (manual + tool support)
The main manual activity is to examine a work product and make comments about it.
INF5530/ 05.02.2014 / © Raluca Popescu 5
1. Static techniques and the test process
• Object of reviews: Any software work product can be reviewed:– requirements specifications
– design specifications
– code
– test plans, test specifications, test cases, test scripts
– user guides
– web pages.
• Benefits:– early defect detection and correction
– development productivity improvements
– reduced development timescales
– reduced testing cost and time
– lifetime cost reductions
– fewer defects and improved communication
• Typical defects (easier to find in reviews than in dynamic testing): – deviations from standards
– requirement defects
– design defects
– insufficient maintainability
– incorrect interface specifications
INF5530/ 05.02.2014 / © Raluca Popescu 6
2. Review process
• Background
• 2.1 Phases of a formal review
• 2.2 Roles and responsibilities
• 2.3 Types of review
• 2.4 Success factors for reviews
INF5530/ 05.02.2014 / © Raluca Popescu 7
Background
• Different types of reviews vary from:
– very informal (e.g. no written instructions for reviewers)
– to very formal (i.e. well structured and regulated).
• The formality of a review process is related to factors like:
– the maturity of the development process
– any legal or regulatory requirements
– the need for an audit trail.
• The way a review is carried out depends on the agreed objective of the
review:
– find defects
– gain understanding
– discussion and decision by consensus
INF5530/ 05.02.2014 / © Raluca Popescu 8
2. Review process
2.1 Phases of a formal review1. Planning • select the personnel
• allocate roles
• define the entry and exit criteria for more formal review types (e.g. inspection)
• select which parts of documents to look at
2. Kick-off • distributing documents
• explaining the objectives
• process and documents to the participants
• and checking entry criteria (for more formal review types)
3. Individual preparation Work done by each of the participants on their own before the review meeting,
noting potential defects, questions and comments.
4. Review meeting - Discussion or logging, with documented results or minutes (for more formal
review types).
- The meeting participants may simply note defects, make recommendations for
handling the defects, or make decisions about the defects.
5. Rework fixing defects found, typically done by the author.
6. Follow-up • check that defects have been addressed
• gather metrics
INF5530/ 05.02.2014 / © Raluca Popescu 9
2. Review process
2.2 Roles and responsibilities1. Manager • decides on the execution of reviews
• allocates time in project schedules
• determines if the review objectives have been met
2. Moderator • leads the review of the document or set of documents
• planning the review
• running the meeting
• and follow-up after the meeting
• If necessary, the moderator may mediate between the various points of view
and is often the person upon whom the success of the review rests
3. Author the writer or person with chief responsibility for the documents to be reviewed.
4. Reviewers - Discussion or logging, with documented results or minutes (for more formal
review types).
- The meeting participants may simply note defects, make recommendations for
handling the defects, or make decisions about the defects.
5. Scribe fixing defects found, typically done by the author.
INF5530/ 05.02.2014 / © Raluca Popescu 10
2. Review process2.3 Types of review1.Informal
review
•Purpose: inexpensive way to get some benefit.
• Form: pair programming or a technical lead reviewing designs and code
• no formal process
• optionally may be documented
2.Walkthrough • Purposes: learning, gaining understanding, defect finding.
• Form: meeting led by author
• scenarios, dry runs, peer group
• open-ended sessions
• may vary in practice from quite informal to very formal
3.Technical
review
•Purposes: discuss, make decisions, evaluate alternatives, find defects, solve technical
problems and check conformance to specifications and standards.
• Form: may be performed as a peer review without management participation
• ideally led by trained moderator
• documented, defined defect-detection process; includes peers and technical experts
•pre-meeting preparation
• optionally the use of checklists, review report, list of findings and management participation
4.Inspection •Purpose: find defects.
• Form: usually peer examination led by trained moderator (not the author)
• formal process based on rules and checklists with entry and exit criteria
• pre-meeting preparation
• defined roles
• includes metrics
• inspection report, list of findings
• formal follow-up process
INF5530/ 05.02.2014 / © Raluca Popescu 11
2. Review process
• Objectives: each review has a clear predefined objective.
• Roles: the right people for the review objectives are involved.
• Approach:
– Defects found are welcomed, and expressed objectively.
– Apply suitable review techniques for the type and level of software products.
– Use checklists or roles if appropriate to increase effectiveness of defect
identification.
– Management supports a good review process (e.g. by incorporating adequate
time for review activities).
• Training and learning:
– Training is given in review techniques, especially the more formal techniques,
such as inspection.
– There is an emphasis on learning and process improvement
2.4 Success factors for reviews
INF5530/ 05.02.2014 / © Raluca Popescu 12
3. Static analysis by tools
INF5530/ 05.02.2014 / © Raluca Popescu 13
• Objectives of static analysis: Find defects in:• software source code
• software models.
• Note! Static analysis finds defects rather than failures
• Static analysis is performed without actually executing the software being
examined by the tool;
• Static analysis tools analyze program code (e.g. control flow and data flow), as
well as generated output such as HTML and XML.
• Typical defects discovered by static analysis tools include:
– referencing a variable with an undefined value;
– inconsistent interface between modules and components;
– variables that are never used;
– unreachable (dead) code;
– programming standards violations;
– security vulnerabilities;
– syntax violations of code and software models.
3. Static analysis by tools
INF5530/ 05.02.2014 / © Raluca Popescu 14
Practical side:
Static analysis tools may produce a large number of warning
messages, which need to be well managed to allow the most effective
use of the tool.
Why is static analysis valuable:• Early detection of defects prior to test execution.
• Early warning about suspicious aspects of the code or design, by the
calculation of metrics, such as a high complexity measure.
• Identification of defects not easily found by dynamic testing.
• Detecting dependencies and inconsistencies in software models, such
as links.
• Improved maintainability of code and design.
• Prevention of defects, if lessons are learned in development.
Users When to be used
Developers Before and during:-Component testing
-Integration testing
Designers During software modeling
3. Static analysis by tools
INF5530/ 05.02.2014 / © Raluca Popescu 15
Please search on the internet (or use the ISTQB Syllabus)
Definition of:
- Static testing, dynamic testing
- Types of reviews
- Roles in the review process (moderator, scribe,
inspector)
- Compiler
- Complexity
- Cyclomatic complexity
Examples of:
- Control flow
- Data flow
To read
INF5530/ 05.02.2014 / © Raluca Popescu 16
Appendix (DITA documentation format ; Oxygen XML Author – tool for documentation)
INF5530/ 05.02.2014 / © Raluca Popescu 17
Appendix (bad documentation – shift gears, Toyota)
INF5530/ 05.02.2014 / © Raluca Popescu 18
Appendix (inelegant documentation imposed by legal formulations – Asus EEEE 701)
INF5530/ 05.02.2014 / © Raluca Popescu 19
Appendix (static analysis tools for .Net)
INF5530/ 05.02.2014 / © Raluca Popescu 20
Appendix (static analysis tools for Java)
INF5530/ 05.02.2014 / © Raluca Popescu 21
Appendix (example of static analysis tool – Coverity)
INF5530/ 05.02.2014 / © Raluca Popescu 22
Appendix (example of requirements specifications)
INF5530/ 05.02.2014 / © Raluca Popescu 23
Appendix (example of technical specifications)
INF5530/ 05.02.2014 / © Raluca Popescu 24
Appendix (example of test design specifications)
INF5530/ 05.02.2014 / © Raluca Popescu 25
Appendix (example of test cases specifications)