Download - Black Box Testing
![Page 1: Black Box Testing](https://reader036.vdocuments.net/reader036/viewer/2022062314/56812ad4550346895d8eb6d6/html5/thumbnails/1.jpg)
BLACK BOX TESTING
Csci 565
Chapters 5-7
![Page 2: Black Box Testing](https://reader036.vdocuments.net/reader036/viewer/2022062314/56812ad4550346895d8eb6d6/html5/thumbnails/2.jpg)
Objectives
•Systematic Specification-based testing•Decision Table-Based Testing•Cause-Effect Graphs in Functional testing
•Equivalence Partitioning and Boundary value Analysis
•Input validation and Syntax-driven Testing
![Page 3: Black Box Testing](https://reader036.vdocuments.net/reader036/viewer/2022062314/56812ad4550346895d8eb6d6/html5/thumbnails/3.jpg)
Black box testing
•Black box testing•Also known as data-driven or input/output driven testing
•Views the software as a black box•Aims at finding cases in which the software does not behave according to its specifications or requirements•Test data are derived from the specifications
•(no need for taking advantage of knowledge of code)
•The main criterion is exhaustive input testing!!!!
![Page 4: Black Box Testing](https://reader036.vdocuments.net/reader036/viewer/2022062314/56812ad4550346895d8eb6d6/html5/thumbnails/4.jpg)
Decision table-based Testing (DTT)•Applicable to the software requirements written using “if-then” statements
•Can be automatically translated into code•Conditions = inputs•Actions = outputs•Rules = test cases
•Assume the independence of inputs
•Example• If cond1 AND cond2 OR cond3 then Action1
![Page 5: Black Box Testing](https://reader036.vdocuments.net/reader036/viewer/2022062314/56812ad4550346895d8eb6d6/html5/thumbnails/5.jpg)
Sample of Decision table
A decision table consists of a number of columns (rules) that comprise all test situations
Action ai will take place if c1 and c2 are true
Example: the triangle problem C1: a, b,c form a triangle C2: a=b C3: a= c C4: b= c A1: Not a triangle A2:scalene A3: Isosceles A4:equilateral A5: impossible
![Page 6: Black Box Testing](https://reader036.vdocuments.net/reader036/viewer/2022062314/56812ad4550346895d8eb6d6/html5/thumbnails/6.jpg)
r1 r2 … rn
C1 1 1 0
c2 1 1 0
C3 - 1 1
C4 - 1 0
A1 1 0 0
A2 0 0 1
A3 0 0 0
A4 0 1 1
A5 0 0
![Page 7: Black Box Testing](https://reader036.vdocuments.net/reader036/viewer/2022062314/56812ad4550346895d8eb6d6/html5/thumbnails/7.jpg)
Test cases from Decision TablesTest Case ID a b c Expected
output
DT1 4 1 2 Not a Triangle
DT2 2888 2888 2888 Equilateral
DT3 ? | ) Impossible
DT4
…
DTn
![Page 8: Black Box Testing](https://reader036.vdocuments.net/reader036/viewer/2022062314/56812ad4550346895d8eb6d6/html5/thumbnails/8.jpg)
Example: Simple editor
•A simple text editor should provide the following features•Copy•Paste•Boldface•Underline
![Page 9: Black Box Testing](https://reader036.vdocuments.net/reader036/viewer/2022062314/56812ad4550346895d8eb6d6/html5/thumbnails/9.jpg)
r1
r2 r3 r4 r5 r6 r7 r8 r9 r10 r11 r12 r13 r14 r15 r16
copy 1
paste 0
Undrln 0
Bold 0
Cpy-test
1
Pst-text
0
Und-text
0
Bld-text
0
In general, for n conditions, we need 2n rules
![Page 10: Black Box Testing](https://reader036.vdocuments.net/reader036/viewer/2022062314/56812ad4550346895d8eb6d6/html5/thumbnails/10.jpg)
Decision tables as a basis for test case designThe use of decision-table model to design test
cases is applicable The spec is given by table or is easily converted to one The order in which the conditions are evaluated does
not affect the interpretation of rules or the resulting action
The order in which the rules are evaluated does not affect the resulting action
Once a rule has been satisfied and an action is selected, no other rule need be examined
If multiple actions result from the satisfaction of a rule, the order in which the actions take place is not important
Inconsistency problem and nondeterministic table When two rules with same conditions elevated to two different actions
then the whole decisions tables is nondeterministic because it is difficult to decide which rule should be applied
![Page 11: Black Box Testing](https://reader036.vdocuments.net/reader036/viewer/2022062314/56812ad4550346895d8eb6d6/html5/thumbnails/11.jpg)
The implications of rules
The above conditions have the following implicationsRules are complete (i.e., every
combination of decision table values including default combinations are inherent in the decision table)
The rules are consistent (i.e., there is not two actions for the same combinations of conditions)
![Page 12: Black Box Testing](https://reader036.vdocuments.net/reader036/viewer/2022062314/56812ad4550346895d8eb6d6/html5/thumbnails/12.jpg)
Cause-effect graphs in black box testingCaptures the relationships between
specific combinations of inputs (causes) and outputs (effects)Deals with specific cases Avoids combinatorial explosionExplore combinations of possible inputs
Causes/effects represented as nodes of a cause effect
graphThe graph also includes a number of
intermediate nodes linking causes and effects
![Page 13: Black Box Testing](https://reader036.vdocuments.net/reader036/viewer/2022062314/56812ad4550346895d8eb6d6/html5/thumbnails/13.jpg)
Drawing Cause-Effect Graphs
A B
If A then B (identity)
AC
If (A=true and B=true)then C
B
![Page 14: Black Box Testing](https://reader036.vdocuments.net/reader036/viewer/2022062314/56812ad4550346895d8eb6d6/html5/thumbnails/14.jpg)
Drawing Cause-Effect Graphs
AC
If (A or B) then C
B
AC
If (not(A AND B)) then C
B
![Page 15: Black Box Testing](https://reader036.vdocuments.net/reader036/viewer/2022062314/56812ad4550346895d8eb6d6/html5/thumbnails/15.jpg)
Drawing Cause-Effect Graphs
AC
If (not (A OR B))then C
B
A B
If (not A) then B
![Page 16: Black Box Testing](https://reader036.vdocuments.net/reader036/viewer/2022062314/56812ad4550346895d8eb6d6/html5/thumbnails/16.jpg)
Constraint Symbols
a
b
at most, one of a and b can be 1 (i.e., a and b cannot be 1 at the same time)
a
b
Exactly one of A and B can be 1
a
bat least one of a, or b must be 1 (i.e., a and b cannot be 0 at the same time)
O
I
E
![Page 17: Black Box Testing](https://reader036.vdocuments.net/reader036/viewer/2022062314/56812ad4550346895d8eb6d6/html5/thumbnails/17.jpg)
Example: ATM
•For a simple ATM banking transaction system•Causes (inputs)
• C1: command is credit• C2: command is debit• C3: account number is valid• C4: transaction amount is valid
•Effects (outputs)• E1: print “invalid command”• E2: print “ invalid account number”• E3: print “debit amount not valid”• E4: debit account• E5: credit account
![Page 18: Black Box Testing](https://reader036.vdocuments.net/reader036/viewer/2022062314/56812ad4550346895d8eb6d6/html5/thumbnails/18.jpg)
C1
C4
C3
C2
and
and
and
and E3
E2
E1
E5
E4
or
![Page 19: Black Box Testing](https://reader036.vdocuments.net/reader036/viewer/2022062314/56812ad4550346895d8eb6d6/html5/thumbnails/19.jpg)
Description of processing nodes used in a Cause-effect graph
Type of processing node description
AND Effect occurs if all input are true (1)
OR Effect occurs if both or one input is true
XOC Effect occurs if one of input is true
Negation Effect occurs if input are false (0)
()
![Page 20: Black Box Testing](https://reader036.vdocuments.net/reader036/viewer/2022062314/56812ad4550346895d8eb6d6/html5/thumbnails/20.jpg)
ATM Cause-effect decision table
Cause\effect
R1 R2 R3 R4 R5
C1 0 1 x x 1
C2 0 x 1 1 x
C3 x 0 1 0 1
C4 x x 0 1 1
E1 1 0 0 0 0
E2 0 1 0 0 0
E3 0 0 1 0 0
E4 0 0 0 1 0
E5 0 0 0 0 1
Don’t Care condition
![Page 21: Black Box Testing](https://reader036.vdocuments.net/reader036/viewer/2022062314/56812ad4550346895d8eb6d6/html5/thumbnails/21.jpg)
C/E: Print Message problem
The "Print message" is a software that read two characters and, depending of their values, messages must be printed.
The first character must be an "A" or a "B". The second character must be a digit. If the first character is an "A" or "B" and the second character is a digit, the file must be updated.
If the first character is incorrect (not an "A" or "B"), the message X must be printed.
If the second character is incorrect (not a digit), the message Y must be printed.
![Page 22: Black Box Testing](https://reader036.vdocuments.net/reader036/viewer/2022062314/56812ad4550346895d8eb6d6/html5/thumbnails/22.jpg)
Step 1: Identify Causes and Effect•Causes:
•1 - first character is "A"
•2 - first character is "B"
•3 - second character is a digit
•Effects:
•70 - the file is updated
•71 - message X is print
•72 - message Y is print
![Page 23: Black Box Testing](https://reader036.vdocuments.net/reader036/viewer/2022062314/56812ad4550346895d8eb6d6/html5/thumbnails/23.jpg)
Step 2: Draw the Graph
![Page 24: Black Box Testing](https://reader036.vdocuments.net/reader036/viewer/2022062314/56812ad4550346895d8eb6d6/html5/thumbnails/24.jpg)
Step 2: Improved Graph
Exclusive Or (i.e., 1 and 2 cannot be true at same
time)
![Page 25: Black Box Testing](https://reader036.vdocuments.net/reader036/viewer/2022062314/56812ad4550346895d8eb6d6/html5/thumbnails/25.jpg)
Step 3: Convert the Graph into Decision table
T1 T2 T3 T4 T5
C1 0 0 1 0 1
C2 0 0 0 1 0
C3 0 1 1 1 0
E1:70 0 0 1 1 0
E2:71 1 1 0 0 0
E3:72 1 0 0 0 1
![Page 26: Black Box Testing](https://reader036.vdocuments.net/reader036/viewer/2022062314/56812ad4550346895d8eb6d6/html5/thumbnails/26.jpg)
Steps to create cause-effect graph Study the functional requirements and divide
the requirements into workable pieces Identify causes and effects in the specifications
Causes: distinct input conditions Effects: an output condition or a system transformations.
Assign unique number to each cause and effect Use the semantic content of the spec and
transform it into a Boolean graph Annotate the graph with constrains describing
combinations of causes and/or effects Convert the graph into a limited-entry decision
table Use each column as a test case
![Page 27: Black Box Testing](https://reader036.vdocuments.net/reader036/viewer/2022062314/56812ad4550346895d8eb6d6/html5/thumbnails/27.jpg)
Black-box testing
Ie
Input test data
OeOutput test results
System
Inputs causinganomalousbehaviour
Outputs which revealthe presence ofdefects
Ie
Input test data
OeOutput test results
System
Inputs causinganomalousbehaviour
Outputs which revealthe presence ofdefects
![Page 28: Black Box Testing](https://reader036.vdocuments.net/reader036/viewer/2022062314/56812ad4550346895d8eb6d6/html5/thumbnails/28.jpg)
Equivalence partitioning
•Black box testing that involves identifying a small set of input values that produces as many different output conditions as possible
•Input data and output results often fall into different classes where all members of a class are related
•Each of these classes is an equivalence partition where the program behaves in an equivalent way for each class member
•Test cases design involves• Identifying all partitions satisfying the input values for a product• Identifying all partitions satisfying the expected outputs for a product
•Selecting one member value from each partition to maximize complete coverage
![Page 29: Black Box Testing](https://reader036.vdocuments.net/reader036/viewer/2022062314/56812ad4550346895d8eb6d6/html5/thumbnails/29.jpg)
Equivalence partitioning
System
Outputs
Invalid inputs Valid inputs
![Page 30: Black Box Testing](https://reader036.vdocuments.net/reader036/viewer/2022062314/56812ad4550346895d8eb6d6/html5/thumbnails/30.jpg)
Guidelines for equivalence classesIf an input condition specifies range,
one valid and two invalid equivalence classes are needed
If an input condition requires a specific value,
then one valid and two invalid equivalence classes are needed
If an input condition specifies a member of a set
one valid and one invalid equivalence class are needed
If an input condition is Boolean, one valid and one invalid class are needed
![Page 31: Black Box Testing](https://reader036.vdocuments.net/reader036/viewer/2022062314/56812ad4550346895d8eb6d6/html5/thumbnails/31.jpg)
Example: ATM
•Consider data maintained for ATM•User should be able to access the bank using PC and modem (dialing)
•User needs to provide eight-digit password
•Need to follow a set of typed commands representing various transactions
![Page 32: Black Box Testing](https://reader036.vdocuments.net/reader036/viewer/2022062314/56812ad4550346895d8eb6d6/html5/thumbnails/32.jpg)
Data format for phone(xxx-xxx-xxxx)•Software accepts
•Area code: • blank or three-digit
•Prefix: • three-digit number not beginning with 0 or 1
•Suffix: • four digits number
•Password: • eight digit alphanumeric value
•Command: • {“check”, “deposit,” “ bill pay”, “transfer” etc.}
![Page 33: Black Box Testing](https://reader036.vdocuments.net/reader036/viewer/2022062314/56812ad4550346895d8eb6d6/html5/thumbnails/33.jpg)
Input conditions for ATM
Input conditions Area code:
Boolean: (the area code may or may not be present) Range: values defined between 200-999 Specific value: no value > 905
Prefix: range : specific value >200
Suffix: Value: four-digit length
Password: Boolean: password may or may not be present value : eight char-string
Command: set containing commands noted previously
![Page 34: Black Box Testing](https://reader036.vdocuments.net/reader036/viewer/2022062314/56812ad4550346895d8eb6d6/html5/thumbnails/34.jpg)
Boundary Value Analysis (BVA)• Complements equivalence partitioning• Focuses is on the boundaries of the input
• If input condition specifies a range bounded by a certain values, say, a and b, then test cases should include
• The value for “a” • The value for “b”• The values just above “a”• The values just below “a” • The values just above “b”• The values just below “b”• The nominal value
• If an input condition specifies any number of values, test cases should be exercise
• the minimum and maximum numbers, • the values just above and just below the minimum and
maximum values
![Page 35: Black Box Testing](https://reader036.vdocuments.net/reader036/viewer/2022062314/56812ad4550346895d8eb6d6/html5/thumbnails/35.jpg)
Example 2: Equivalence Partitioning (EP)
![Page 36: Black Box Testing](https://reader036.vdocuments.net/reader036/viewer/2022062314/56812ad4550346895d8eb6d6/html5/thumbnails/36.jpg)
EP: Process •Identify valid partitions•Identify invalid partitions•Identify output partitions
![Page 37: Black Box Testing](https://reader036.vdocuments.net/reader036/viewer/2022062314/56812ad4550346895d8eb6d6/html5/thumbnails/37.jpg)
Input: Valid partitions
•The valid partitions can be•0<=exam mark <=75•0<=course work <=25
![Page 38: Black Box Testing](https://reader036.vdocuments.net/reader036/viewer/2022062314/56812ad4550346895d8eb6d6/html5/thumbnails/38.jpg)
Input: Invalid partitions
•The most obvious partitions are•Exam mark > 75•Exam mark < 0•Coursework mark > 25•Coursework nark <0
![Page 39: Black Box Testing](https://reader036.vdocuments.net/reader036/viewer/2022062314/56812ad4550346895d8eb6d6/html5/thumbnails/39.jpg)
Exam mark and course work (c/w) mark
![Page 40: Black Box Testing](https://reader036.vdocuments.net/reader036/viewer/2022062314/56812ad4550346895d8eb6d6/html5/thumbnails/40.jpg)
Less obvious invalid input EP•Invalid INPUT EP should include
![Page 41: Black Box Testing](https://reader036.vdocuments.net/reader036/viewer/2022062314/56812ad4550346895d8eb6d6/html5/thumbnails/41.jpg)
Partitions for the OUTPUTS
•EP for valid OUTPUTS should include
FM
![Page 42: Black Box Testing](https://reader036.vdocuments.net/reader036/viewer/2022062314/56812ad4550346895d8eb6d6/html5/thumbnails/42.jpg)
The EP and boundaries
•The EP and boundaries for total mark
![Page 43: Black Box Testing](https://reader036.vdocuments.net/reader036/viewer/2022062314/56812ad4550346895d8eb6d6/html5/thumbnails/43.jpg)
Unspecified Outputs
•Three unspecified Outputs errors can be identified •Output = “E” (errors below zero)•Output = “A+” (errors exceeding 100)
•Output = “null” (errors corresponding to blanks)
![Page 44: Black Box Testing](https://reader036.vdocuments.net/reader036/viewer/2022062314/56812ad4550346895d8eb6d6/html5/thumbnails/44.jpg)
Total PEs
Input partitions for exams
Input partitions for course works
Output partitions for special cases
Input partitions for invalid data
Output partitions for valid data
![Page 45: Black Box Testing](https://reader036.vdocuments.net/reader036/viewer/2022062314/56812ad4550346895d8eb6d6/html5/thumbnails/45.jpg)
Test Cases corresponding to PE exam mark (INPUT)
![Page 46: Black Box Testing](https://reader036.vdocuments.net/reader036/viewer/2022062314/56812ad4550346895d8eb6d6/html5/thumbnails/46.jpg)
Test Case 4-6 (coursework)
![Page 47: Black Box Testing](https://reader036.vdocuments.net/reader036/viewer/2022062314/56812ad4550346895d8eb6d6/html5/thumbnails/47.jpg)
test case for Invalid inputs
![Page 48: Black Box Testing](https://reader036.vdocuments.net/reader036/viewer/2022062314/56812ad4550346895d8eb6d6/html5/thumbnails/48.jpg)
Test cases for outputs:1
![Page 49: Black Box Testing](https://reader036.vdocuments.net/reader036/viewer/2022062314/56812ad4550346895d8eb6d6/html5/thumbnails/49.jpg)
Test cases for outputs:2
![Page 50: Black Box Testing](https://reader036.vdocuments.net/reader036/viewer/2022062314/56812ad4550346895d8eb6d6/html5/thumbnails/50.jpg)
Test cases for Special output classes:3
![Page 51: Black Box Testing](https://reader036.vdocuments.net/reader036/viewer/2022062314/56812ad4550346895d8eb6d6/html5/thumbnails/51.jpg)
Minimal Test cases:1
![Page 52: Black Box Testing](https://reader036.vdocuments.net/reader036/viewer/2022062314/56812ad4550346895d8eb6d6/html5/thumbnails/52.jpg)
Minimal Test cases:2
![Page 53: Black Box Testing](https://reader036.vdocuments.net/reader036/viewer/2022062314/56812ad4550346895d8eb6d6/html5/thumbnails/53.jpg)
Syntax-Driven testing (SDT): 1•A test case design technique for a component or system in which test case design is based upon the syntax of the input.
•Applicable to •Specification validation written by a certain grammar (BNF)
•Input data validation
![Page 54: Black Box Testing](https://reader036.vdocuments.net/reader036/viewer/2022062314/56812ad4550346895d8eb6d6/html5/thumbnails/54.jpg)
Syntax-Driven testing (SDT): 2•Applications•GUI applications •XML/HTML files. •Command-driven software•Scripting languages•Database query languages.
•…
![Page 55: Black Box Testing](https://reader036.vdocuments.net/reader036/viewer/2022062314/56812ad4550346895d8eb6d6/html5/thumbnails/55.jpg)
Coverage criteria for SDT•Test generation method• generate test cases such that each production rule is tested at least once (production coverage)
• generate test cases such that each terminal is tested at least once (terminal coverage)
•generate test case such that each string (program sample) is tested (derivation coverage)
![Page 56: Black Box Testing](https://reader036.vdocuments.net/reader036/viewer/2022062314/56812ad4550346895d8eb6d6/html5/thumbnails/56.jpg)
Example: BNF of simple Arithmetic expressions
Example (simple expressions)<exp> ::= <exp> + <term>| <exp> -<term>|<term>
<term> ::= <term> * <factor> | <term> / <factor>|<factor>
<factor> ::= <id> | <exp><id>::= a|b|…|z
![Page 57: Black Box Testing](https://reader036.vdocuments.net/reader036/viewer/2022062314/56812ad4550346895d8eb6d6/html5/thumbnails/57.jpg)
More on SDT
•The set of test cases for SDT should include expressions that exercise each production rule, each nonterminal, and all possible input values
•Examples:•9-5+2•5 * 2 + 8/4•…
![Page 58: Black Box Testing](https://reader036.vdocuments.net/reader036/viewer/2022062314/56812ad4550346895d8eb6d6/html5/thumbnails/58.jpg)
Parse Tree (PT)
58
•Parsing?•It is a process of showing how to take a string of terminals (input) and derives it from start symbol
•Parse Tree (PT)•PT shows how the start symbol of a grammar derives a string in the language
•PT consists of : •the root (start symbol)•Interior nodes (non-terminal)•leaf nodes (terminals)
![Page 59: Black Box Testing](https://reader036.vdocuments.net/reader036/viewer/2022062314/56812ad4550346895d8eb6d6/html5/thumbnails/59.jpg)
Step 1: Use a CFG to create the syntax of the input
(example 9-5+2)
59
Exp
TermExp
Term
Exp
Term
2+5-9
![Page 60: Black Box Testing](https://reader036.vdocuments.net/reader036/viewer/2022062314/56812ad4550346895d8eb6d6/html5/thumbnails/60.jpg)
•Examples (G s n | B t n)*
![Page 61: Black Box Testing](https://reader036.vdocuments.net/reader036/viewer/2022062314/56812ad4550346895d8eb6d6/html5/thumbnails/61.jpg)
BNF Grammars
![Page 62: Black Box Testing](https://reader036.vdocuments.net/reader036/viewer/2022062314/56812ad4550346895d8eb6d6/html5/thumbnails/62.jpg)
Using Grammar
![Page 63: Black Box Testing](https://reader036.vdocuments.net/reader036/viewer/2022062314/56812ad4550346895d8eb6d6/html5/thumbnails/63.jpg)
Applications of SDT
•Here are some examples•All user interfaces•Operating command processing•All communications interfaces and protocols• Internal interface and protocol (call sequences)