test missions as requirements

ConfidentialPA12013-12-131

Test Missions as RequirementsRequirements in a fast paced, dynamic environment


Introduction

▪ Creating detailed written requirements has been done in the past, but is no longer feasible in our current fast paced, dynamic environment

▪ Stakeholders have expectations which need to be fulfilled

▪ There needs to be some way to communicate those expectations to developers and testers

▪ These are some of my thoughts on the subject – the novelty of the thoughts can be debated


Perfect Requirements? [1]

Unitary (Cohesive)

Complete

Consistent

Atomic

Traceable

Current

Unambiguous

Specify Importance

Verifiable

Is this possible to achieve? Is it cost efficient?


Good Enough Requirements

Express stakeholder expectations in a way which both stakeholders and

developers/testers understand

Be continuously updated when discrepancies are discovered

Prioritized according to stakeholder expectations

Agile User Stories?


Test Missions as Requirements

Create Test Missions

Test Product

Develop Product

Stakeholder Expectations


Test Missions?

▪ Test missions can be something of the following:

▪ Test heuristics [2]

▪ Test tours [3]

▪ Use cases in form of tests

▪ User stories in form of tests [4]

▪ High level test cases that can be understood by stakeholders


Stakeholder Communication

▪ The stakeholders express their expectations

▪ The tester forms those expectations into test missions

▪ The stakeholders reviews the test missions and confirms that they match the expectations

▪ The test missions are then sent to the development and test team(s)

▪ Developers and testers review test missions and secure that they understand them

▪ If test missions need to be updated this is done together with the stakeholders


What about unit tests and test cases?

▪ A unit test or a detailed test case are in most cases not a requirement because it is not possible for the stakeholder to properly understand how it is connected to the stakeholder’s expectations

▪ A unit test can confirm that a requirement/test mission is met, but is in itself not a requirement

▪ Same thing goes for detailed test cases


What about standards and certifications?

▪ A stakeholder expectation when it comes to standards and certifications is that the standard/certification is fulfilled

▪ This can be a test mission/requirement

▪ Actual detailed test cases are only confirming the overall test mission that we are actually fulfilling the standard

▪ A detailed test case in a 3GPP standard is thus not a requirement, only confirming the requirement that we fulfill the 3GPP standard


Defect Reports

▪ If a defect report is escalated to the stakeholder and this is rejected or accepted, the defect report now becomes part of the requirements and should be linked to the appropriate test mission

▪ If many defect reports are rejected by the stakeholder, then the test missions should be reviewed as they may be incorrect or inconclusive


Benefits?

▪ One major problem with traditional requirement artifacts is that they are not keep up to date because the artifact in itself holds no value to anyone

▪ Test Missions on the other hand are actively used by testers and there is a purpose to keeping them updated other than because someone tells you to

▪ Compared to having unit tests as requirements the benefit is that you can actually use them in your communication with stakeholders

▪ No need to have separate user stories and test cases – they are now both the same entity

▪ Testers are involved early in the development process and testability gets more focus


Problems?

▪ One problem is that the development/test organization needs to embrace exploratory testing and at least partly abandon it’s reliance on scripted manual testing

▪ It requires close cooperation between stakeholders, developers and testers

▪ It requires higher test competence to formulate stakeholder expectations directly into test missions which can be understood by both developers and stakeholders


Conclusion

▪ There are problems with having detailed requirements, and there are problems with communicating expectations in form of detailed tests or unit tests

▪ Using test missions as requirements is one way to try to solve these problems

▪ However this requires high test competence, and a new way of thinking about testing and requirements for many organizations


References

[1] Requirement http://en.wikipedia.org/wiki/Requirement

[2] Test Heuristics Cheat Sheethttp://testobsessed.com/wp-content/uploads/2011/04/testheuristicscheatsheetv1.pdf

[3] Exploratory Testing Tourshttp://msdn.microsoft.com/en-us/library/jj620911.aspx#bkmk_tours

[4] User Storieshttp://guide.agilealliance.org/guide/user-stories.html

http://en.wikipedia.org/wiki/Requirement

http://testobsessed.com/wp-content/uploads/2011/04/testheuristicscheatsheetv1.pdf

http://msdn.microsoft.com/en-us/library/jj620911.aspx

http://guide.agilealliance.org/guide/user-stories.html

test missions as requirements

Software