the core of testing – dynamic testing process – according to iso 29119 with anne mette hass
DESCRIPTION
Testers like to test! But sometimes it is difficult to get to grips with what testing really is; and it can be an overwhelming experience to be handed over a system and asked to test it. Where should you start? What should you test? How should you test it? In this webinar we will go through the dynamic testing process (as defined in ISO 29119 Standard for Software Testing) step by step. This will give an in-depth understanding of what the dynamic testing activities: Test Design and Implementation, Test Environment Set-up, and Test Execution, consist of. You can choose to perform the activities formally or informally, but no matter how you do it, understanding the activities and their purpose will enhance the quality of your testing.TRANSCRIPT
The Core of Testing- Dynamic Testing Process -
According to ISO 29119
EuroSTAR Webinar24th September 2014
Anne Mette [email protected]
Please test this!
Test basisTest item
The Example Test Item
Evolutionary development
Development of IT systems
RDCI
RDCI
RDCI
RDCI
RDCI
RDCI
RDCI
Agile development
Sequential development
Testing of IT systems
Test is a support process for development
Test sub-processes are designed according to development phases and requirements
Definitions of test
In Denmark we have a saying: ”Charished child has many names.”
Testing has many definitions, 2 of them are:
• ISO 29119: ”set of activities conducted to facilitate discovery and/or evaluation of properties of one or more test items”
• Lee Copeland: ”Testing is comparing what you have to what you expected to get”
Classification xxxx Author xxxx Approved by xxxx Version xx6
Test basis is indispensible
Test basis may for example be: • Requirements • User stories• Use cases• Functional descriptions• Behaviour descriptions• Rules • Standards• Design at various levels• Experience
Classification xxxx Author xxxx Approved by xxxx Version xx7
NEVER, EVER the code
Test Management
Testing must be planned, monitored and controlled like any other activity in the world
Classification xxxx Author xxxx Approved by xxxx Version xx8
Test management
Completion
Planning Monitoring and control
Be prepared -Be prepared -
Basic Test Types
Types can be Mixed
A test sub-process can for example be • a component test• a functional system test• a performance test
Dynamic Test
This webinar will concentrate on the dynamic testing process , especially the
Test design and implementation
activity
Paul Gerrard’s New Testing Model
Classification xxxx Author xxxx Approved by xxxx Version xx12
Test Design and Implementation
Feature Sets
- or divide and conquer
A feature set is a logical subset of the test item
• for a component test we might have just one feature set
• for a system test we might have dozens
Examples
Feature set 1: Use CasesFeature set 1.1 LoginFeature set 1.2 Earn PointsFeature set 1.3 Spend PointsFeature set 1.4 Benefits
Classification xxxx Author xxxx Approved by xxxx Version xx15
Feature set 2: NavigationFeature set 2.1 MenusFeature set 2.2 Keys
Feature set 3: Error HandlingFeature set 3.1 TextsFeature set 3.2 Severity
Test Condition
A test condition is a testable aspect of a test item such as a verifiable
• function
• transaction
• feature
• quality attribute
• structural element
There may be one or many for each feature set
Description of Member Levels
“There are 3 member levels: Basis, Silver, Gold. Your member level is determined by the number of Basis Points you earn within your personal 12 month period. You will automatically be upgraded to Silver Member if you earn 20.000 Basis Points during your earning period. If you earn 50.000 Basis Points in the period you a become Gold Member. The earning period runs for 12 months from the first day of joining.”
Now test
that!
Derived Test Conditions
• TCO-1.1: A member starts as a Basis member.
• TCO-1.2: A member will automatically be upgraded to Silver Member if he/she earns 20.000 Basis Points during an earning period. If he/she earns 50.000 Basis Points in a period he/she becomes Gold Member.
• TCO-1.3: The earning period runs for 12 months from the first day of joining.
This activity usually creates lots of questions.
Test Coverage Items
A test coverage item is an attribute or combination of attributes that
is derived from one or more test conditions
by using a test case design technique.
Each test case design techniques provides its own coverage element
Derived coverage item
• TCO-1.2: A member will automatically be upgraded to Silver Member if he/she earns 20.000 Basis Points during an earning period. If he/she earns 50.000 Basis Points in a period he/she becomes Gold Member.
• CI-1.2-1 0 <= earned points < 20.000
• CI-1.2-2 20.000 <= earned points <50.000
• CI-1.2-3 50.000 <= earned points
Classification xxxx Author xxxx Approved by xxxx Version xx20
Specification-based Test CD Techniques
• Equivalence partitioning• Classification tree method• Boundary value analysis• Domain analysis• Syntax testing• Combinatorial testing• Decision tables• Cause-effect graph• State transition testing• Scenario testing• Random testing
• Partition• Leaf• 2/3 boundary values• In- and Out-points• None• Combination • Combination• Combination• Chow’s n switch• Flow• None
Structure-based Test CD Techniques
• Statement testing• Decision testing• Branch condition testing• Branch condition
combination testing• Modified condition
decision coverage testing• Path testing• Intercomponent testing
• Statement• Decision outcome• Branch• Branch condition
outcome• Modified condition
• Path• Call
Experience-based Test CD Techniques
• Error guessing• Checklist-based• Exploratory testing• Attacks
• Defect taxonomies• Fault injection and
mutation
• None• List item• None• Registered attack
• Entry• None
Test Cases
Test cases are used to guide the dynamic test.
A test case is a set of input and the test item’s expected reaction.
Test Environment
Classification xxxx Author xxxx Approved by xxxx Version xx25
Detailed Test Environment Requirements ID Description Responsible
ER1 Hardware: Three PCs Production
ER2 Software: MS Windows 7 or later Production
…
ERn Security controls are identified in the Corporation Security Protocol.
Security manager
ERm Refer to the Test Plan for relevant testing tools.
Production and development.
…
Detailed Test Data Requirements ID Description Responsible Reset A/D
DR1 At least 20 members for various countries in the database.
Production Yes A
DR2 An ordinary user and a system administration user.
Production N/A D
…
DRn Table with 5 different trips with different points.
Production N/A A
DRm Table with membership status and necessary point to obtain each.
Production NA A
…
Test Data
Test set and test procedure
• A test set is a collection of test cases selected for execution for a specific purpose
• A test procedure is a recipe for the test execution where the test cases are sorted in a reasonable order
Test Procedure
Classification xxxx Author xxxx Approved by xxxx Version xx28
Test procedure: P11 (3)
Case Input Expected output Actual output
3.6 (17)
Enter… The sum …
3.8
(18)
Enter… The value..
Stop and wrap up: Run the “Clean 1” script.
The information about which test procedures have been executed and with
Objective: This test procedure tests …
Priority: 2
Start up: The form F1 must be current.
Relationships to other: None
Expected duration: 15 min.
Execution time: Initials:
System: Result:
Log result
Log when Log who
Identify item etc. Log overall test result
Test Execution - The moment of thruth!
• Don’t make a false start – check the start criteria
• Follow the test procedures, because:
• actual time spent should be comparable with estimates
• progress should be comparable with the plan
• the tests should be exactly repeatable for retesting and regression testing
• a complete audit of the test may be required
Identifying Failures
• Be very careful when comparing the expected result with the actual result
• in order not to miss failures (called false-positives)
• report correct behavior as failures (called false-negatives)
Actual result = Expected result => Test result = Passed
Actual result = Expected result => Test result = Failed
3
Test Execution Logging
Logging • is essential• is often done directly in the procedure • can also be done in a tool
Test procedure: P11 (3)
Case Input Expected output Actual output
3.6 (17)
Enter… The sum …
3.8
(18)
Enter… The value..
Stop and wrap up: Run the “Clean 1” script.
The information about which test procedures have been executed and with
Log result
√
I-5342
The End
Commercial
Classification xxxx Author xxxx Approved by xxxx Version xx33
Remember
• Test is difficult
• Test requires overview
• Test requires creativity
• Test requires systematic work
• Test requires imagination
• Test requires courage
• Test is fun11-04-202334