4th lecture on software testing given to ktu students
TRANSCRIPT
What is software testing about ?
Software testing is a process of executing a program or application with the intent of finding the software bugs. (ISTQB)
The principle objective of software testing is to give confidence in the software. (Unknown Author)
Software testing is an investigation conducted to provide stakeholders with information about the quality of the product or service under test. (Bertrand Meyer)
Remember that slide?
Test Case
"A test case, in software engineering, is a set of conditions under which a tester will determine whether an application, software system or one of its features is working as it was originally established for it to do." (Wikipedia)
"A sequence of one or more subtests executed as a sequence because the outcome and/or final state of one subtest is the input and/or initial state of the next. The word ‘test’ is used to include subtests, tests proper, and test suites." (Boris Beizer, 1995)
Testing Activities
Test PlanningTest AnalysisTest DesignTest ImplementationTest Execution -> here we testEvaluating Closure
Remember that slide?
Test DesignHow to test
(ISTQB Syllabus)
Remember that slide?
Test Planning and Test Execution
What + How + When = almost ready TESTING PLAN
0
50
100
150
200
250
Unit Integration Smoke System End-to-End Regession Acceptance
Module 1 Functionality Module 2 Functionality System Functionality System Perfomance System Usability
Remember that slide?
Example: System Tests (implemented)
System Tests
Functionality 200
Perforamance 10
Usability 50
200
10
50
0
50
100
150
200
250
200
10
50
Test Case LifeCycle
Example: System Tests (executed)
Not Executed Passed Failed
Functionality 20 130 50
Performance 10 0 0
Usability 5 35 10
20
130
50
10
0 05
35
10
0
20
40
60
80
100
120
140
Example: System Tests (executed)
35
165
60
Not Executed
Passed
Failed
Example: System Tests (progress)
Release #1 Release #2 Release #3 Release #4
Failed 60 40 35 20
Passed 165 165 200 240
NotExecuted 35 55 25 0
3555
250
165
165200 240
6040 35
20
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Defects
Error – a human action that produces an incorrect result.
Defect (“bug”, fault) – a manifestation of an error.
Failure – deviation of the software from the expectations. (Executed defect causes failure)
Defect LifeCycle
Some Defect Attributes
Severity (High, Medium, Low, etc.)Priority (High, Medium, Low, etc.)Product, ComponentVersion FoundVersion FixedStatus (Open, Reopened, Fixed, Verified, Closed)Resolution (Fixed, Cannot Reproduce, Not a Defect, Duplicate)Test Environment
Defect Metrics – Defects by Status
Defect Metrics – Open Defects by Priority
Defect Metrics – Created versus Resolved
Combined Metrics – Test Case execution and Defects
35
165
60
Test Cases
Not Executed
Passed
Failed
10
4
4
1
Defects
Open
In Progress
Resolved
Reopened
Open 10
In Progress 4
Resolved 4
Reopened 1
Not Executed 35
Passed 165
Failed 60
Combined Metrics – Test Case execution and Defects
Metrics – Bad Example
Writing Good Defect Report
“The point of writing problem report(bug report) is to get bugs fixed.” (By Cem Kaner)
One problem – one issueClear Title (mind “searchability”)ReproducibilityClear Description
Steps to ReproduceExpected ResultsAdd screenshots (movies)
Test EnvironmentCheck for duplicates before reportingDo not over-prioritizeAdd notes / comments
"Program testing can be used to show the presence of bugs, but never to show their absence!" (Edsger W. Dijkstra, 1970)