test plan simplicity
Post on 06-May-2015
508 Views
Preview:
DESCRIPTION
TRANSCRIPT
ConfidentialPA12013-11-121 ConfidentialPA12013-11-121
Test PlansSimplicity
ConfidentialPA12013-11-122
Introduction• In this presentation I will use James Bach’s
definition of a test plan [1]• Test Plan: the set of ideas that guide a test project
• Test Strategy: the set of ideas that guide test design
• Test Logistics: the set of ideas that guide the application of resources to fulfill a test strategy
• Every artifact created must add value
• The process of creating a test plan adds value, but the document itself is of limited value• The value of a test plan is the ability to
communicate the set of ideas that guides the test project to different stakeholders
ConfidentialPA12013-11-123
Simplicity• Simplicity is the state or quality of being
simple [2]
• There is a value in keeping the test plan simple which relates to the burden which the test plan puts on someone trying to explain or understand it
• We also want to limit the amount of effort we spend on creating the test plan, and minimize the amount of unnecessary artifacts created
• On Google and Microsoft there is a concept of a 10 minute test plan, which should in accordance with the name only take a short period of time to create [3]
ConfidentialPA12013-11-124
Simplicity
Complicated
Simple
Cost of Simplicity
Value of Simplicity
Valu
e Gai
n
ConfidentialPA12013-11-125
Dynamic Test Plan• The key to reducing the size of the test plan is
to remove anything that cannot change over time
• Static information can be moved from a test plan to a Test Strategy or a Test Logistics document
• Anything that does not add value to any stakeholder should also be removed
ConfidentialPA12013-11-126
Content of a Test Plan• Test Ownership
• System Configuration Under Test
• Definition of Done
• System Risk Matrix
• Test Activities
• Test Schedule
• Self-Learning Loop (SLL)
ConfidentialPA12013-11-127
Overview
Identify Risks
Plan Test Activities
Self-Learning Loop
Ownership - Test Configuration - Definition of Done - Activity Description
ConfidentialPA12013-11-128
Test Ownership• Which person is responsible for performing
what tests?
• This could be done on a overview scope level, for different test charters, or for specific test cases
• A test ownership for a specific person or group could be defined in the following ways:• 250 specific performance test cases
• All performance tests
• All or specific charters related to performance
• Performance
• Running performance tests during a specific time period or project phase
ConfidentialPA12013-11-129
System Configuration Under Test• Most systems can be configured in many
different ways
• It is necessary to describe the configuration under test in order to• Reproduce tests
• Write bug reports
• Divide test ownership between different configurations
• Let all testers know what to test on
• This can change over time depending on the configurability of the system
ConfidentialPA12013-11-1210
Definition of Done• It is necessary to have a Definition of Done [4]
for different phases of the test project
• This is used to define when testing stops as implied by the name
• Definition of Done should include:• Expected System Quality
• Expected Test Coverage
• Expected Risk Level
• Formal Documentation Needed
• Definition of Done can change over time dependant on stakeholder needs
ConfidentialPA12013-11-1211
System Risk Matrix• What risks threaten the system from reaching the
definition of done?
• These risks change continuously over the life cycle of the project
• Risks can include (but are not limited to):• Software changes
• New hardware builds
• Software delta between configurations
• Hardware delta between configurations
• Unknown system dependencies
• Unknown environment dependencies
• Changed stakeholder focus
ConfidentialPA12013-11-1212
Risk Quantification• The Risk Matrix needs to be filled quantifiable
risks
• There are many different models available• Severity - Occurrence - Detection [5] is one
• Low risk, Medium risk, High risk, is another
• If a system has many configurations, each configuration needs a column in the risk matrix
• Risks most likely also need to be categorized in a good way to make the matrix usable
ConfidentialPA12013-11-1213
Test Activities• All different test activities need to be explained
in the test plan
• The scope of these test activities should not be static, since it is seldom efficient to run static activities
• Different activities can be added and removed during the life cycle of the project
ConfidentialPA12013-11-1214
Test Schedule• A schedule or plan of when each test activity
should be executed in time
• The schedule should be dynamic [6] and based on input from what is happening in the project, and what has happened during previous test activities
ConfidentialPA12013-11-1215
Self-Learning Loop (SLL)• It is critical to describe how the test plan will
evolve over the project life cycle, and then continuously document what we have learned from this feedback loop
• Continuous improvement of gaps in test scope
• Continuous improvement of understanding of system dependencies
ConfidentialPA12013-11-1216
Conclusion• Test plans should add value and not include
any waste
• Test plans should be dynamic – static parts should be included in other documents which can be reused between projects, or not documented at all
• These 7 bullets help include critical parts in the test plan, and not introduce unnecessary waste
ConfidentialPA12013-11-1217
References[1] A question about test strategyhttp://www.satisfice.com/blog/archives/63
[2] Simplicityhttp://en.wikipedia.org/wiki/Simplicity
[3] 10 Minute Test Planhttp://googletesting.blogspot.se/2011/09/10-minute-test-plan.html
[4] Definition of Donehttps://www.scrum.org/Resources/Scrum-Glossary/Definition-of-Done
[5] Severity, Occurrence, and Detection Criteria for Process FMEAhttp://www.fmeainfocentre.com/guides/ProcessPktNewRatings.pdf
[6] Dynamic Test Planshttp://www.slideshare.net/JohanHoberg/dynamic-test-plans
top related