taming the testing beast
TRANSCRIPT
![Page 1: Taming the Testing Beast](https://reader036.vdocuments.net/reader036/viewer/2022081517/58eeb4881a28ab1a298b46c1/html5/thumbnails/1.jpg)
Taming the Testing Beast
Marc J. BalcerChief ArchitectModel Compilers LLC
![Page 2: Taming the Testing Beast](https://reader036.vdocuments.net/reader036/viewer/2022081517/58eeb4881a28ab1a298b46c1/html5/thumbnails/2.jpg)
How do you knowwhat to test?
![Page 3: Taming the Testing Beast](https://reader036.vdocuments.net/reader036/viewer/2022081517/58eeb4881a28ab1a298b46c1/html5/thumbnails/3.jpg)
Test Case
TAME – Test Authoring Made Easy
Test Workshee
t TAME
Test CaseTest
CaseTest CaseTest
Case
Excel Open Office
![Page 4: Taming the Testing Beast](https://reader036.vdocuments.net/reader036/viewer/2022081517/58eeb4881a28ab1a298b46c1/html5/thumbnails/4.jpg)
Test Case
Employee
Add an expense to an expense report
![Page 5: Taming the Testing Beast](https://reader036.vdocuments.net/reader036/viewer/2022081517/58eeb4881a28ab1a298b46c1/html5/thumbnails/5.jpg)
Employee
Add an expense to an expense report
Test Case
Expense TypeExpense DateAmountNotes
InputsExpense AddedNo Date ErrorBad Date ErrorAmount Missing ErrorAmount Too Large Warning
Expected Results
“Stateless”
![Page 6: Taming the Testing Beast](https://reader036.vdocuments.net/reader036/viewer/2022081517/58eeb4881a28ab1a298b46c1/html5/thumbnails/6.jpg)
Test Case
Expense TypeExpense DateAmountNotes
InputsExpense AddedNo Date ErrorBad Date ErrorAmount Missing ErrorAmount Too Large WarningNo Project ErrorEmployee Error
Expected Results
Project ExistsEmployee CurrentOther Items Exist
Preconditions
“Stateful”
![Page 7: Taming the Testing Beast](https://reader036.vdocuments.net/reader036/viewer/2022081517/58eeb4881a28ab1a298b46c1/html5/thumbnails/7.jpg)
Test Case
TAME – Test Authoring Made Easy
Test Workshee
t TAME
Test CaseTest
CaseTest CaseTest
CaseInputsPreconditionsExpected Results
Test Suite
![Page 8: Taming the Testing Beast](https://reader036.vdocuments.net/reader036/viewer/2022081517/58eeb4881a28ab1a298b46c1/html5/thumbnails/8.jpg)
How do you geta good set oftest cases?
![Page 9: Taming the Testing Beast](https://reader036.vdocuments.net/reader036/viewer/2022081517/58eeb4881a28ab1a298b46c1/html5/thumbnails/9.jpg)
Example: A Login Page
Inputs
![Page 10: Taming the Testing Beast](https://reader036.vdocuments.net/reader036/viewer/2022081517/58eeb4881a28ab1a298b46c1/html5/thumbnails/10.jpg)
Generic Login Command
Input User ID FieldEmptyToo longHas invalid charactersValid Syntax
Input Password FieldEmptyToo longValid Syntax
Inputs CharacteristicValues
(“choices”)
![Page 11: Taming the Testing Beast](https://reader036.vdocuments.net/reader036/viewer/2022081517/58eeb4881a28ab1a298b46c1/html5/thumbnails/11.jpg)
Kinds of Input ValuesNumeric Symbolic Enumerated Composite
A number
Specified as• Range• Precision
(number of decimal places)
May include a unit of measure
A character string
Specified as• Size• Pattern
(regular expression or equivalent)
A list of values
Specified as the set of values
May be ordered or unordered
May have a corresponding numeric or symbolic value
A single value composed of other values
Examples:• Mailing
Address• Grid
Coordinate
![Page 12: Taming the Testing Beast](https://reader036.vdocuments.net/reader036/viewer/2022081517/58eeb4881a28ab1a298b46c1/html5/thumbnails/12.jpg)
Numeric Type ExamplesType Name Description ExamplesYear A four-digit integer between
1900 and one plus the current year
2012 is good1832 is not good342.34 is not good“huh” is not good
Money A US Dollar value between 0 and 1000.00 to a precision of 2 decimal places.
345.23 is good-60 is not good34.219 is not good
Count A nonnegative number used to indicate a count of items between 0 and 100.
46 is good245 is not good-10 is not good
Date A calendar date to the precision of one day
15-Jan-2010 is good4/3/2010 is good*June 6 is not good
* How this is interpreted depends upon whether you are in the US or Europe…
![Page 13: Taming the Testing Beast](https://reader036.vdocuments.net/reader036/viewer/2022081517/58eeb4881a28ab1a298b46c1/html5/thumbnails/13.jpg)
Symbolic Type ExamplesType Name Description ExamplesISBN Book Number
10 or 13 decimal digitsThe last character may be an X instead of a zero.
0-201-74804-5 is good4-87197-308-5 is good9781492217947 is good9321G4E3 is not good
ISO Country Code
2 capital letters US is goodGB is goodEUR is not good
Order Number 4 digits plus a dash plus 6 digits:[0-9]{4}\-[0-9]{6}(or \d{4}-\d{6})
2014-002432 is good1986-231321 is good503-3324-324 is not good
Title String 1 to 100 characters, including non-ASCII (foreign language) characters
Pretty much anything as long as it’s not more that 100 characters long.
![Page 14: Taming the Testing Beast](https://reader036.vdocuments.net/reader036/viewer/2022081517/58eeb4881a28ab1a298b46c1/html5/thumbnails/14.jpg)
Common Input Values• Errors
• Missing (empty, null, nothing, unchecked, …)
• Too short• Too long• Wrong characters (bad syntax)• Out of range
• Acceptable Values• Good syntax• Special characters
(e.g. quotes, < > characters)• Non-ASCII and non-Latin characters
![Page 15: Taming the Testing Beast](https://reader036.vdocuments.net/reader036/viewer/2022081517/58eeb4881a28ab1a298b46c1/html5/thumbnails/15.jpg)
Generate All Combinations
![Page 16: Taming the Testing Beast](https://reader036.vdocuments.net/reader036/viewer/2022081517/58eeb4881a28ab1a298b46c1/html5/thumbnails/16.jpg)
Results
Generic Login Command
Error message "Empty User ID" is displayed
User ID Too Long
User ID has invalid characters
Empty Password
Password Too Long
User ID or password incorrect
User ID or password incorrect
User logged in
Input User ID Field
Empty xToo long xHas invalid characters xValid Syntax x x x
Input Password Field
Empty xToo long xValid Syntax x x x
![Page 17: Taming the Testing Beast](https://reader036.vdocuments.net/reader036/viewer/2022081517/58eeb4881a28ab1a298b46c1/html5/thumbnails/17.jpg)
Preconditions
Generic Login Command
Input User ID FieldEmptyToo longHas invalid charactersValid Syntax
Input Password FieldEmptyToo longValid Syntax
Condition User
no user with the user IDuser with the user ID
Condition User Password
does not match the user IDmatches the user ID
Facts aboutthe environment(usually set up in advance)
![Page 18: Taming the Testing Beast](https://reader036.vdocuments.net/reader036/viewer/2022081517/58eeb4881a28ab1a298b46c1/html5/thumbnails/18.jpg)
Forming Combinations
Generic Login Command
Input User ID FieldEmptyToo longHas invalid charactersValid Syntax
Input Password FieldEmptyToo longValid Syntax
Condition User
no user with the user IDuser with the user ID
Condition User Password
does not match the user IDmatches the user ID
4 choices
3 choices
2 choices2 choices
4 × 3 × 2 × 2 = 48 test cases
![Page 19: Taming the Testing Beast](https://reader036.vdocuments.net/reader036/viewer/2022081517/58eeb4881a28ab1a298b46c1/html5/thumbnails/19.jpg)
All Combinations = Many Useless Test CasesRedundant (duplicate)
User ID emptyPassword too longUser: none with the user IDPassword: does not match the user ID Result: Empty User ID errorUser ID: emptyPassword: good syntaxUser: none with the user IDPassword: does not match the user ID Result: Empty User ID error
Make No Sense
User ID emptyPassword too longUser: user with the user IDPassword: does not match the user IDUser ID bad syntaxPassword too longUser: user with the user IDPassword: does not match the user ID
![Page 20: Taming the Testing Beast](https://reader036.vdocuments.net/reader036/viewer/2022081517/58eeb4881a28ab1a298b46c1/html5/thumbnails/20.jpg)
Define Results
Generic Login Command
Empty User ID error
User ID Too Long error
User ID has invalid characters error
Empty Password error
Password Too Long error
User ID or password incorrect error
User ID or password incorrect error
User logged in
Input User ID Field
Empty Too long Has invalid characters Valid Syntax
Input Password Field
Empty Too long Valid Syntax
Condition User
no user with the user ID user with the user ID
Condition User Password
does not match the user ID matches the user ID
Expected Results
![Page 21: Taming the Testing Beast](https://reader036.vdocuments.net/reader036/viewer/2022081517/58eeb4881a28ab1a298b46c1/html5/thumbnails/21.jpg)
Simple Result Marking
Generic Login Command
Empty User ID error
User ID Too Long error
User ID has invalid characters error
Empty Password error
Password Too Long error
User ID or password incorrect error
User ID or password incorrect error
User logged in
Input User ID Field
Empty xToo long xHas invalid characters xValid Syntax
Input Password Field
Empty xToo long xValid Syntax
Condition User
no user with the user ID user with the user ID
Condition User Password
does not match the user ID matches the user ID
![Page 22: Taming the Testing Beast](https://reader036.vdocuments.net/reader036/viewer/2022081517/58eeb4881a28ab1a298b46c1/html5/thumbnails/22.jpg)
Combination Results
Generic Login Command
Empty User ID error
User ID Too Long error
User ID has invalid characters error
Empty Password error
Password Too Long error
User ID or password incorrect error
User ID or password incorrect error
User logged in
Input User ID Field
Empty Too long Has invalid characters Valid Syntax x
Input Password Field
Empty Too long Valid Syntax x
Condition User
no user with the user ID user with the user ID x
Condition User Password
does not match the user ID matches the user ID x
![Page 23: Taming the Testing Beast](https://reader036.vdocuments.net/reader036/viewer/2022081517/58eeb4881a28ab1a298b46c1/html5/thumbnails/23.jpg)
Multiple Paths to the Same Result
Generic Login Command
Empty User ID error
User ID Too Long error
User ID has invalid characters error
Empty Password error
Password Too Long error
User ID or password incorrect error
User ID or password incorrect error
User logged in
Input User ID Field
Empty Too long Has invalid characters Valid Syntax x x
Input Password Field
Empty Too long Valid Syntax x x
Condition User
no user with the user ID xuser with the user ID x
Condition User Password
does not match the user ID xmatches the user ID
same
![Page 24: Taming the Testing Beast](https://reader036.vdocuments.net/reader036/viewer/2022081517/58eeb4881a28ab1a298b46c1/html5/thumbnails/24.jpg)
Fully Marked Worksheet
Generic Login Command
Empty User ID error
User ID Too Long error
User ID has invalid characters error
Empty Password error
Password Too Long error
User ID or password incorrect error
User ID or password incorrect error
User logged in
Input User ID Field
Empty xToo long xHas invalid characters xValid Syntax x x x
Input Password Field
Empty xToo long xValid Syntax x x x
Condition User
no user with the user ID xuser with the user ID x x
Condition User Password
does not match the user ID xmatches the user ID x
![Page 25: Taming the Testing Beast](https://reader036.vdocuments.net/reader036/viewer/2022081517/58eeb4881a28ab1a298b46c1/html5/thumbnails/25.jpg)
Forming Test Cases
Generic Login Command
Empty User ID error
User ID Too Long error
User ID has invalid characters error
Empty Password error
Password Too Long error
Input User ID Field
Empty xToo long xHas invalid characters xValid Syntax
Input Password Field
Empty xToo long xValid Syntax
Condition User
no user with the user ID user with the user ID
Condition User Password
does not match the user ID matches the user ID
Test Case 1Inputs: User ID Field : Empty Expected Results: Empty User ID error------------------------------------Test Case 2Inputs: User ID Field : Too long Expected Results: User ID Too Long error------------------------------------Test Case 3Inputs: User ID Field : Has invalid characters Expected Results: User ID has invalid characters error
![Page 26: Taming the Testing Beast](https://reader036.vdocuments.net/reader036/viewer/2022081517/58eeb4881a28ab1a298b46c1/html5/thumbnails/26.jpg)
No Mark – Don’t Care
Generic Login Command
Empty User ID error
User ID Too Long error
User ID has invalid characters error
Empty Password error
Password Too Long error
Input User ID Field
Empty xToo long xHas invalid characters xValid Syntax
Input Password Field
Empty xToo long xValid Syntax
Condition User
no user with the user ID user with the user ID
Condition User Password
does not match the user ID matches the user ID
Test Case 1Inputs: User ID Field : Empty Expected Results: Empty User ID error
Any choice for passwordwill result in this sameerror message.
![Page 27: Taming the Testing Beast](https://reader036.vdocuments.net/reader036/viewer/2022081517/58eeb4881a28ab1a298b46c1/html5/thumbnails/27.jpg)
Default Choice
Select a default if you want to put that “don’t care” category into the test case.
![Page 28: Taming the Testing Beast](https://reader036.vdocuments.net/reader036/viewer/2022081517/58eeb4881a28ab1a298b46c1/html5/thumbnails/28.jpg)
Combine All Choices
Select all choices if you want to combine all choices into test cases.
Alternatively check the category name.
![Page 29: Taming the Testing Beast](https://reader036.vdocuments.net/reader036/viewer/2022081517/58eeb4881a28ab1a298b46c1/html5/thumbnails/29.jpg)
Multiple Paths to the Same Result
Generic Login Command
User ID or password incorrect error
User ID or password incorrect error
Input User ID Field
Empty Too long Has invalid characters Valid Syntax x x
Input Password Field
Empty Too long Valid Syntax x x
Condition User
no user with the user ID xuser with the user ID x
Condition User Password
does not match the user ID xmatches the user ID
Test Case 6 Preconditions: User : no user with the user ID Inputs: User ID Field : Valid Syntax Password Field : Valid Syntax Expected Results: User ID or password incorrect-----------------------------------------------Test Case 7 Preconditions: User : user with the user ID User Password : does not match the user ID Inputs: User ID Field : Valid Syntax Password Field : Valid Syntax Expected Results: User ID or password incorrect
![Page 30: Taming the Testing Beast](https://reader036.vdocuments.net/reader036/viewer/2022081517/58eeb4881a28ab1a298b46c1/html5/thumbnails/30.jpg)
Combination Results
Generic Login Command
User logged in
Input User ID Field
Empty Too long Has invalid characters Valid Syntax x
Input Password Field
Empty Too long Valid Syntax x
Condition User
no user with the user ID user with the user ID x
Condition User Password
does not match the user ID matches the user ID x
Test Case 8 Preconditions: User : user with the user ID User Password : matches the user ID Inputs: User ID Field : Valid Syntax Password Field : Valid Syntax Expected Results: User logged in
![Page 31: Taming the Testing Beast](https://reader036.vdocuments.net/reader036/viewer/2022081517/58eeb4881a28ab1a298b46c1/html5/thumbnails/31.jpg)
Multiple CombinationsGeneric Login Command (With Special
Inputs)
User logged in
Input User ID FieldEmptyToo longHas invalid charactersValid Syntax xValid with non-ASCII chars x
Input Password FieldEmptyToo longValid Syntax xValid with quotes xValid with HTML chars xValid with non-ASCII chars x
Condition Userno user with the user IDuser with the user ID x
Condition User Passworddoes not match the user IDmatches the user ID x
Form all combinationsof these inputs.
2 × 4= 8 test cases
![Page 32: Taming the Testing Beast](https://reader036.vdocuments.net/reader036/viewer/2022081517/58eeb4881a28ab1a298b46c1/html5/thumbnails/32.jpg)
LimitsGeneric Login Command (With Special
Inputs)LIMITS
User logged in
Input User ID Field
Empty Too long Has invalid characters Valid Syntax xValid with non-ASCII chars 1 x
Input Password Field
Empty Too long Valid Syntax xValid with quotes 1 xValid with HTML chars 1 xValid with non-ASCII chars 1 x
Condition User
no user with the user ID user with the user ID x
Condition User Password
does not match the user ID matches the user ID x
Each choice will only beused in one test case.
![Page 33: Taming the Testing Beast](https://reader036.vdocuments.net/reader036/viewer/2022081517/58eeb4881a28ab1a298b46c1/html5/thumbnails/33.jpg)
Generating Test Cases
Test CaseTest
Worksheet TAME
Test CaseTest
CaseTest CaseTest
Case
Excel Open Office
Test Case List
Test SummaryTable
Test ProtocolDocument
![Page 34: Taming the Testing Beast](https://reader036.vdocuments.net/reader036/viewer/2022081517/58eeb4881a28ab1a298b46c1/html5/thumbnails/34.jpg)
Test Case ListTest Cases for Generic Login Command (With Special Inputs)----------------------------------------------------Test Case 1 Preconditions: Inputs: User ID Field : Empty Expected Results: Empty User ID error----------------------------------------------------Test Case 2 Preconditions: Inputs: User ID Field : Too long Expected Results: User ID Too Long error----------------------------------------------------Test Case 3 Preconditions: Inputs: User ID Field : Has invalid characters Expected Results: User ID has invalid characters error
![Page 35: Taming the Testing Beast](https://reader036.vdocuments.net/reader036/viewer/2022081517/58eeb4881a28ab1a298b46c1/html5/thumbnails/35.jpg)
Test Summary TableUser ID Field Password Field User User Password Results...
1Empty Empty User ID error2Too long User ID Too Long error
3Has invalid characters
User ID has invalid characters error
4 Empty Empty Password error5 Too long Password Too Long error6Valid Syntax Valid Syntax no user with the user ID User ID or password incorrect
error
7Valid Syntax Valid with quotes no user with the user ID User ID or password incorrect error
8Valid Syntax Valid with HTML charsno user with the user ID User ID or password incorrect error
9Valid Syntax Valid Syntax user with the user ID does not match the user ID User ID or password incorrect error
10Valid Syntax Valid with quotes user with the user ID does not match the user ID User ID or password incorrect error
11Valid Syntax Valid with HTML charsuser with the user ID does not match the user ID User ID or password incorrect error
12Valid Syntax Valid Syntax user with the user ID matches the user ID User logged in
13Valid Syntax Valid with quotes user with the user ID matches the user ID User logged in
14Valid Syntax Valid with HTML charsuser with the user ID matches the user ID User logged in
![Page 36: Taming the Testing Beast](https://reader036.vdocuments.net/reader036/viewer/2022081517/58eeb4881a28ab1a298b46c1/html5/thumbnails/36.jpg)
Default Protocol Document
![Page 37: Taming the Testing Beast](https://reader036.vdocuments.net/reader036/viewer/2022081517/58eeb4881a28ab1a298b46c1/html5/thumbnails/37.jpg)
Creating a Custom ProtocolHow to log in:• Click a link to go to the login page• Enter the user ID• Go to the password field• Enter the password• Click the login button• Observe whether
• an error message is displayed, or• the application indicates that the user is
logged in
![Page 38: Taming the Testing Beast](https://reader036.vdocuments.net/reader036/viewer/2022081517/58eeb4881a28ab1a298b46c1/html5/thumbnails/38.jpg)
Creating a Custom ProtocolHow to log in:• Click a link to go to the login page• Enter the user ID• Go to the password field• Enter the password• Click the login button• Observe whether
• an error message is displayed, or• the application indicates that the user is
logged in
We want real valuesfor each of these parameters.
![Page 39: Taming the Testing Beast](https://reader036.vdocuments.net/reader036/viewer/2022081517/58eeb4881a28ab1a298b46c1/html5/thumbnails/39.jpg)
Defining Values
Generic Login Command (With Special Inputs)
LIMITS
Empty User ID error
User ID Too Long error
User ID has invalid characters error
Empty Password error
Password Too Long error
User ID or password incorrect error
User ID or password incorrect error
User logged in
User ID:
User ID: abcdefghijklm
User ID: 56%#@!*(a
User ID: gary*
User ID: charlie
User ID: garcía
Password:
Password: 56%#@!*(a
Password: Good*
Password: Bad
Password: G"oo"d
Password:Go<o>d&
Password:¿sueÑo
Input User ID Field
Empty x xToo long x xHas invalid characters x xValid Syntax x x x x xValid with non-ASCII chars 1 x x
Input Password Field
Empty x xToo long x xValid Syntax x x x x xValid with quotes 1 x x x xValid with HTML chars 1 x x x xValid with non-ASCII chars 1 x x
Condition User
no user with the user ID x xuser with the user ID x x x x x x x
Condition User Password
does not match the user ID x xmatches the user ID x x x
![Page 40: Taming the Testing Beast](https://reader036.vdocuments.net/reader036/viewer/2022081517/58eeb4881a28ab1a298b46c1/html5/thumbnails/40.jpg)
Defining Values
User ID:
User ID: abcdefghijklm
User ID: 56%#@!*(a
User ID: gary*
User ID: charlie
User ID: garcía
Password:
Password: 56%#@!*(a
Password: Good*
Password: Bad
Password: G"oo"d
Password:Go<o>d&
Password:¿sueÑo
Parameter Name : value
![Page 41: Taming the Testing Beast](https://reader036.vdocuments.net/reader036/viewer/2022081517/58eeb4881a28ab1a298b46c1/html5/thumbnails/41.jpg)
Custom Protocol TemplateTest Case «TestCaseNumber» Preconditions
«TableStart:Precondition»«CategoryName»
«ChoiceName»«TableEnd:Precondition»
Inputs «TableStart:Input»«CategoryName»
«ChoiceName»«TableEnd:Input»
Results «TableStart:Result»«ResultName»«TableEnd:Result»
Procedure: (Warning: If you customize this section do not use numbered lists. Word MailMerge doesn’t have the sense to reset the numbering.)
Open the website Click the Login button Enter «User ID» into the User ID field. Hit tab Enter «Password» into the password field Click the Login button
Now verify that: «TableStart:Result»
«ResultName»
«TableEnd:Result»
WordMailmergeFields
User ID and Passwordvalues filled inhere
![Page 42: Taming the Testing Beast](https://reader036.vdocuments.net/reader036/viewer/2022081517/58eeb4881a28ab1a298b46c1/html5/thumbnails/42.jpg)
Simple ValuesGeneric Login Command (With Special
Inputs) User ID:
User ID: abcdefghijklm
User ID: 56%#@!*(a
User ID: gary*
User ID: charlie
User ID: garcía
Password:
Password: 56%#@!*(a
Password: Good*
Password: Bad
Password: G"oo"d
Password:Go<o>d&
Password:¿sueÑo
Input User ID Field
Empty xToo long xHas invalid characters xValid Syntax x xValid with non-ASCII chars x
Input Password Field
Empty xToo long xValid Syntax x xValid with quotes xValid with HTML chars xValid with non-ASCII chars x
Condition User
no user with the user ID xuser with the user ID x x x x x
Condition User Password
does not match the user ID xmatches the user ID x x
![Page 43: Taming the Testing Beast](https://reader036.vdocuments.net/reader036/viewer/2022081517/58eeb4881a28ab1a298b46c1/html5/thumbnails/43.jpg)
Complex ValuesGeneric Login Command (With Special
Inputs) User ID:
User ID: abcdefghijklm
User ID: 56%#@!*(a
User ID: gary*
User ID: charlie
User ID: garcía
Password:
Password: 56%#@!*(a
Password: Good*
Password: Bad
Password: G"oo"d
Password:Go<o>d&
Password:¿sueÑo
Input User ID Field
Empty xToo long xHas invalid characters xValid Syntax x xValid with non-ASCII chars x
Input Password Field
Empty xToo long xValid Syntax x xValid with quotes xValid with HTML chars xValid with non-ASCII chars x
Condition User
no user with the user ID xuser with the user ID x x x x x
Condition User Password
does not match the user ID xmatches the user ID x x
![Page 44: Taming the Testing Beast](https://reader036.vdocuments.net/reader036/viewer/2022081517/58eeb4881a28ab1a298b46c1/html5/thumbnails/44.jpg)
Complex ValuesGeneric Login Command (With Special
Inputs) User ID:
User ID: abcdefghijklm
User ID: 56%#@!*(a
User ID: gary*
User ID: charlie
User ID: garcía
Password:
Password: 56%#@!*(a
Password: Good*
Password: Bad
Password: G"oo"d
Password:Go<o>d&
Password:¿sueÑo
Input User ID Field
Empty xToo long xHas invalid characters xValid Syntax x xValid with non-ASCII chars x
Input Password Field
Empty xToo long xValid Syntax x xValid with quotes xValid with HTML chars xValid with non-ASCII chars x
Condition User
no user with the user ID xuser with the user ID x x x x x
Condition User Password
does not match the user ID xmatches the user ID x x
![Page 45: Taming the Testing Beast](https://reader036.vdocuments.net/reader036/viewer/2022081517/58eeb4881a28ab1a298b46c1/html5/thumbnails/45.jpg)
Default Values
Generic Login Command (With Special Inputs)
LIMITS
Empty User ID error
User ID Too Long error
User ID: abcdefghijklm
User ID: 56%#@!*(a
User ID: gary*
User ID: charlie
User ID: garcía
Password:
Password: 56%#@!*(a
Password: Good*
Password: Bad
Password: G"oo"d
Password:Go<o>d&
Password:¿sueÑo
Input User ID Field
Empty xToo long x xHas invalid characters xValid Syntax x xValid with non-ASCII chars 1 x
Input Password Field
Empty xToo long xValid Syntax x xValid with quotes 1 xValid with HTML chars 1 xValid with non-ASCII chars 1 x
Condition User
no user with the user ID xuser with the user ID x x x x x
Condition User Password
does not match the user ID xmatches the user ID x x
* indicates thedefault value
![Page 46: Taming the Testing Beast](https://reader036.vdocuments.net/reader036/viewer/2022081517/58eeb4881a28ab1a298b46c1/html5/thumbnails/46.jpg)
Values Applied to the ProtocolTest Case 7 Preconditions
User no user with the user ID
Inputs User ID Field Valid Syntax Password Field Valid with quotes
Results User ID or password incorrect error
Procedure: Open the website Click the Login button Enter charlie into the User ID field. Hit tab Enter G"oo"d into the password field Click the Login button
Now verify that: User ID or password incorrect error
![Page 47: Taming the Testing Beast](https://reader036.vdocuments.net/reader036/viewer/2022081517/58eeb4881a28ab1a298b46c1/html5/thumbnails/47.jpg)
Going Beyond the Basics• Create variability in procedures
• Action Choices
• Using TAME when bugs are found• Root Cause Analysis• Modify the Tests
• Scaling Up• Higher Level Tests• Results become Preconditions
![Page 48: Taming the Testing Beast](https://reader036.vdocuments.net/reader036/viewer/2022081517/58eeb4881a28ab1a298b46c1/html5/thumbnails/48.jpg)
Additional variabilityTest Case 7 Preconditions
User no user with the user ID
Inputs User ID Field Valid Syntax Password Field Valid with quotes
Results User ID or password incorrect error
Procedure: Open the website Click the Login button Enter charlie into the User ID field. Hit tab Enter G"oo"d into the password field Click the Login button
Now verify that: User ID or password incorrect error
- Enter a URL directly- Click a link on the company
homepage- Click a link on an unsecured
page- Tab to the next field- Click into the password
field- Click the Login button- Tab to the Login button
and press Enter
![Page 49: Taming the Testing Beast](https://reader036.vdocuments.net/reader036/viewer/2022081517/58eeb4881a28ab1a298b46c1/html5/thumbnails/49.jpg)
Action ChoicesTest Case 7 Preconditions
User no user with the user ID
Inputs User ID Field Valid Syntax Password Field Valid with quotes
Results User ID or password incorrect error
Procedure: Open the website Click the Login button Enter charlie into the User ID field. Hit tab Enter G"oo"d into the password field Click the Login button
Now verify that: User ID or password incorrect error
• Enter a URL directly• Click a link on the company
homepage• Click a link on an unsecured
page• Tab to the next field• Click into the password
field• Click the Login button• Tab to the Login button
and press Enter
For each of these• Create “action”
choices• Use the action
choices in the protocol
![Page 50: Taming the Testing Beast](https://reader036.vdocuments.net/reader036/viewer/2022081517/58eeb4881a28ab1a298b46c1/html5/thumbnails/50.jpg)
Unforseen Circumstances
A user ID that includes spaces causes a crash
Three failed logins lock the user’s account
• Add “contains spaces” as a choice
• Add a value definition that includes spaces
• Determine how to handle a user ID that includes spaces: is it allowed or is it an error?
• Add “Preceding failed logins” as a new preconditions with several choices
• Determine whether a locked account presents a different error message
• Add result selectors
![Page 51: Taming the Testing Beast](https://reader036.vdocuments.net/reader036/viewer/2022081517/58eeb4881a28ab1a298b46c1/html5/thumbnails/51.jpg)
Scaling Up
Employee hasexpenses to report
Log In
Create NewExpense Report
Add ExpenseItem
Submit toManager
Expense Report Submitted
OpenExpense App
already logged in
not logged in
Login is just one step in a larger process
![Page 52: Taming the Testing Beast](https://reader036.vdocuments.net/reader036/viewer/2022081517/58eeb4881a28ab1a298b46c1/html5/thumbnails/52.jpg)
Scaling UpEmployee hasexpenses to report
Log In
Create NewExpense Report
Add ExpenseItem
Submit toManager
Expense Report Submitted
OpenExpense App
already logged in
not logged in
Create separatetest specificationsfor each stepin the process
![Page 53: Taming the Testing Beast](https://reader036.vdocuments.net/reader036/viewer/2022081517/58eeb4881a28ab1a298b46c1/html5/thumbnails/53.jpg)
Scaling Up – Business Processes
Employee
Manager
Controller
Employee has expenses to report
RecordExpenses
ApproveExpenseReport
PayReimbursement
Reimbursement
RejectExpenseReport
ResubmitExpenseReport
Submitted Expense Report
Rejected Expense Report
Resubmitted Expense Report
Approved Expense Report
Review New Expense Report
Review Resubmitted Expense Report
CancelExpenseReport
Expense Reporthas been canceled
Consider whether to resubmit
Employee hasexpenses to report
Log In
Create NewExpense Report
Add ExpenseItem
Submit toManager
Expense Report Submitted
OpenExpense App
already logged in
not logged in
![Page 54: Taming the Testing Beast](https://reader036.vdocuments.net/reader036/viewer/2022081517/58eeb4881a28ab1a298b46c1/html5/thumbnails/54.jpg)
Generating Test Cases
Test CaseTest
Worksheet TAME
Test CaseTest
CaseTest CaseTest
Case
Excel Open Office
Test Case List
Test SummaryTable
Test ProtocolDocument
![Page 55: Taming the Testing Beast](https://reader036.vdocuments.net/reader036/viewer/2022081517/58eeb4881a28ab1a298b46c1/html5/thumbnails/55.jpg)
Comparative Testing Tools• Run Tests
• White-box, code-centricNUnit, JUnit
• UI-based, capture-playbackRanorex, Telerik
• Design by Contract• Write a specification, construct a single
test caseCucumber / Gherkin
![Page 56: Taming the Testing Beast](https://reader036.vdocuments.net/reader036/viewer/2022081517/58eeb4881a28ab1a298b46c1/html5/thumbnails/56.jpg)
Where to use TAME• Unit testing
• specify and generate test cases that can run against software modules (classes, functions).
• GUI testing• whether a traditional installed application, a
website, or a mobile app, use TAME to specify the contents of UI pages, then generate test cases to validate user-facing behavior.
• Regression testing• update the test specification and the test
cases whenever new features are deployed or bugs are fixed.
![Page 57: Taming the Testing Beast](https://reader036.vdocuments.net/reader036/viewer/2022081517/58eeb4881a28ab1a298b46c1/html5/thumbnails/57.jpg)
Where to use TAME• Project Planning
• create initial test specifications to estimate the number of test cases to be run and project the cost of the test effort.
• Retrospectives• use the test specifications to probe
features to better understand the root causes of failures.
![Page 58: Taming the Testing Beast](https://reader036.vdocuments.net/reader036/viewer/2022081517/58eeb4881a28ab1a298b46c1/html5/thumbnails/58.jpg)
TAME for All• TAME helps the analyst
to think through what needs to be tested.
• TAME gives the tester a platform for creating many test cases quickly.
• TAME gives the project manager a means for sizing and scoping the testing effort.
![Page 59: Taming the Testing Beast](https://reader036.vdocuments.net/reader036/viewer/2022081517/58eeb4881a28ab1a298b46c1/html5/thumbnails/59.jpg)
TAME software
Training Courses“Writing Testable
Requirementsand Effective Test Cases”
.
“Modeling Techniquesfor Business Analysts”
![Page 60: Taming the Testing Beast](https://reader036.vdocuments.net/reader036/viewer/2022081517/58eeb4881a28ab1a298b46c1/html5/thumbnails/60.jpg)
Thank you
Marc J. BalcerChief Architect
Model Compilers [email protected]
http://www.modelcompilers.com