white-box software testing using evolutionary algorithms
DESCRIPTION
A presentation I gave as part of our "Software Verification, Validation, and Quality Assurance" course at KFUPM.TRANSCRIPT
White-Box Software Testing using Evolutionary Algorithms
Yaser Sulaiman
1
Time for a break!
2
White-box testing
4
Test adequacy criteria
6
Statement
8
Branch
Condition Path
Weakest
Strongest
Coming up with test cases/data isn’t as easy as it sounds
9
Cover as much as you can!
11
Looks like an optimization problem.. sort of
13
So how about automating it?
14
Evolutionary algorithms
15
16 photo by ][\/][
Evolutionary Algorithms
Genetic Algorithms
(GA)
Genetic Programming
(GP)
17
GA
18
Chromosomes
19
Selection Fitness
Function
Crossover Mutation
GP
20
Specialization of GA
21
Where individuals represent computer programs
22
GP evolves programs!
23
Mathematically speaking
25
Given a set of inputs 𝑋 and outputs 𝑌, a program 𝑝 is sought which
satisfies
𝑌 = 𝑝(𝑋)
26
Fitness of a candidate program?
27
𝑌𝑒𝑥𝑝 Expected output
vs.
𝑌𝑝 = 𝑝(𝑋) Actual output
28
𝑓 𝑝 = 𝑝 𝑥𝑖 − 𝑦𝑒𝑥𝑝𝑖2
𝑋
𝑖=1
where 𝑥 ∈ 𝑋 and 𝑦𝑒𝑥𝑝 ∈ 𝑌𝑒𝑥𝑝
29
Moving on to the surveyed papers
30
Brendt* 2002
Tonella 2004
Wappler* 2006
Ahmed* 2008
31 * et al.
Brendt* 2002
Tonella 2004
Wappler* 2006
Ahmed* 2008
32 * et al.
“Breeding software test cases with genetic algorithms”
33
“Breeding software test cases with genetic algorithms”
34
Fossil record
35
Relative fitness function
36
37 Berndt et al. (2002)
3 factors affect the fitness function
38
1. Novelty
measure of uniqueness
39
2. Proximity
measure of closeness
40
3. Severity
measure of seriousness
41
“simple rules, complex behavior”
42
Explorers, prospectors, and miners
43
Brendt* 2002
Tonella 2004
Wappler* 2006
Ahmed* 2008
44 * et al.
“Evolutionary testing of classes”
45
“Evolutionary testing of classes”
46
Object-oriented testing
47
Class Under Test (CUT) &
Method Under Test (MUT)
48
Sequence of object constructions & method calls
49
Tonella (2004) focuses on representation and operators
50
A chromosome isn’t just a sequence of input values anymore
51
$a=A():$b=B():$a.m(int, $b) @ 3
52
Brendt* 2002
Tonella 2004
Wappler* 2006
Ahmed* 2008
53 * et al.
“Evolutionary unit testing of object-oriented software using strongly-
typed genetic programming”
54
“Evolutionary unit testing of object-oriented software using strongly-
typed genetic programming”
55
GP that is type-aware
56
Wappler & Wegener (2006) focus on call sequence feasibility
57
58 Wappler & Wegener (2006)
Method Call Dependence Graph (MCDG)
59
60 Wappler & Wegener (2006)
A sequence is feasible if it corresponds to an acyclic subgraph
of the MCDG
61
62 Wappler & Wegener (2006)
Brendt* 2002
Tonella 2004
Wappler* 2006
Ahmed* 2008
63 * et al.
“GA-based multiple paths test data generator”
64
“GA-based multiple paths test data generator”
65
Ahmed & Hermadi (2008) focus on the efficiency of test data generation
66
Generating multiple test data to cover multiple target paths in one
GA run
67
Requires incorporating the paths within the fitness calculation
68
// TODO: Redigest the paper.
69
Let’s compare the surveyed papers
70
71
Paper Evolutionary
Algorithm Object-
Oriented Coverage Criterion
Berndt et al. (2002)
GA Branch
Tonella (2004)
GA Branch
Wappler & Wegener (2006)
Strongly-Typed GP Branch
Ahmed & Hermadi (2008)
GA Path
A direction for future work
72
The reverse direction
73
74
Program Evolutionary
Algorithm Test Cases
75
Program Evolutionary
Algorithm Test Cases
Writing test cases first?
76
It’s called TDD
77
TDD + GP isn’t a new idea
80
Take-home messages
82
Generating test cases can be challenging
83
Testing object-oriented software provides even more challenges
84
Evolutionary algorithms can be used to solve [some of] these
problems
85
Even software developers turn to Mother Nature for inspiration
86
…</presentation> <questions>…
88