search based test data generation for server-side web...
TRANSCRIPT
![Page 1: Search Based Test Data Generation for Server-side Web ...crest.cs.ucl.ac.uk/cow/30/slides/COW30_Nadia.pdf · Bugs found by SBST (SWAT) and DSE (APOLLO) Results • Impact of seeding](https://reader034.vdocuments.net/reader034/viewer/2022050611/5fb28f48b40e0d0d890a4605/html5/thumbnails/1.jpg)
Search Based Test Data Generation for Server-side Web Application Testing
Nadia Alshahwan and Mark Harman CREST Centre University College London
![Page 2: Search Based Test Data Generation for Server-side Web ...crest.cs.ucl.ac.uk/cow/30/slides/COW30_Nadia.pdf · Bugs found by SBST (SWAT) and DSE (APOLLO) Results • Impact of seeding](https://reader034.vdocuments.net/reader034/viewer/2022050611/5fb28f48b40e0d0d890a4605/html5/thumbnails/2.jpg)
Automated web application testing using search based software Engineering (ASE 2011) • Hill Climbing (AVM) • Maximise branch coverage • Server-side code (PHP)
![Page 3: Search Based Test Data Generation for Server-side Web ...crest.cs.ucl.ac.uk/cow/30/slides/COW30_Nadia.pdf · Bugs found by SBST (SWAT) and DSE (APOLLO) Results • Impact of seeding](https://reader034.vdocuments.net/reader034/viewer/2022050611/5fb28f48b40e0d0d890a4605/html5/thumbnails/3.jpg)
Overall strategy
1
2
3 4
9
10
7 6 5 8
D
C. C. Michael, G. McGraw, and M. A. Schatz. Generating software test data by evolution. IEEE Transactions on Software Engineering,2001.
A
B C
D E F G
H I J
K
L
![Page 4: Search Based Test Data Generation for Server-side Web ...crest.cs.ucl.ac.uk/cow/30/slides/COW30_Nadia.pdf · Bugs found by SBST (SWAT) and DSE (APOLLO) Results • Impact of seeding](https://reader034.vdocuments.net/reader034/viewer/2022050611/5fb28f48b40e0d0d890a4605/html5/thumbnails/4.jpg)
Pros • Only local distance à
no need for approach level
• Accidental coverage (highest %)
![Page 5: Search Based Test Data Generation for Server-side Web ...crest.cs.ucl.ac.uk/cow/30/slides/COW30_Nadia.pdf · Bugs found by SBST (SWAT) and DSE (APOLLO) Results • Impact of seeding](https://reader034.vdocuments.net/reader034/viewer/2022050611/5fb28f48b40e0d0d890a4605/html5/thumbnails/5.jpg)
Pros • Only local distance à
no need for approach level
• Accidental coverage (highest %)
Cons • Not suitable for
specific targets • Distance calculations
affect execution time
![Page 6: Search Based Test Data Generation for Server-side Web ...crest.cs.ucl.ac.uk/cow/30/slides/COW30_Nadia.pdf · Bugs found by SBST (SWAT) and DSE (APOLLO) Results • Impact of seeding](https://reader034.vdocuments.net/reader034/viewer/2022050611/5fb28f48b40e0d0d890a4605/html5/thumbnails/6.jpg)
Pros • Only local distance à
no need for approach level
• Accidental coverage (highest %)
Cons • Not suitable for
specific targets • Distance calculations
affect execution time à Keep track of covered branches and skip
![Page 7: Search Based Test Data Generation for Server-side Web ...crest.cs.ucl.ac.uk/cow/30/slides/COW30_Nadia.pdf · Bugs found by SBST (SWAT) and DSE (APOLLO) Results • Impact of seeding](https://reader034.vdocuments.net/reader034/viewer/2022050611/5fb28f48b40e0d0d890a4605/html5/thumbnails/7.jpg)
Web Specific Issues
• Identifying inputs ($_POST[‘inputname’]) • Dynamic includes • Dynamic typing à check type at run-time • Non determinism
![Page 8: Search Based Test Data Generation for Server-side Web ...crest.cs.ucl.ac.uk/cow/30/slides/COW30_Nadia.pdf · Bugs found by SBST (SWAT) and DSE (APOLLO) Results • Impact of seeding](https://reader034.vdocuments.net/reader034/viewer/2022050611/5fb28f48b40e0d0d890a4605/html5/thumbnails/8.jpg)
Dynamic Value Seeding
if($x>=$y) { . . } if($class==$result[0]) { . . }
$x = 5 $y = 300
$class = CS $result[0] = English
![Page 9: Search Based Test Data Generation for Server-side Web ...crest.cs.ucl.ac.uk/cow/30/slides/COW30_Nadia.pdf · Bugs found by SBST (SWAT) and DSE (APOLLO) Results • Impact of seeding](https://reader034.vdocuments.net/reader034/viewer/2022050611/5fb28f48b40e0d0d890a4605/html5/thumbnails/9.jpg)
Dynamic Value Seeding
if (file_exists($lng.’.php’)) { ..}
![Page 10: Search Based Test Data Generation for Server-side Web ...crest.cs.ucl.ac.uk/cow/30/slides/COW30_Nadia.pdf · Bugs found by SBST (SWAT) and DSE (APOLLO) Results • Impact of seeding](https://reader034.vdocuments.net/reader034/viewer/2022050611/5fb28f48b40e0d0d890a4605/html5/thumbnails/10.jpg)
Evaluation - Coverage
![Page 11: Search Based Test Data Generation for Server-side Web ...crest.cs.ucl.ac.uk/cow/30/slides/COW30_Nadia.pdf · Bugs found by SBST (SWAT) and DSE (APOLLO) Results • Impact of seeding](https://reader034.vdocuments.net/reader034/viewer/2022050611/5fb28f48b40e0d0d890a4605/html5/thumbnails/11.jpg)
Evaluation - Faults
![Page 12: Search Based Test Data Generation for Server-side Web ...crest.cs.ucl.ac.uk/cow/30/slides/COW30_Nadia.pdf · Bugs found by SBST (SWAT) and DSE (APOLLO) Results • Impact of seeding](https://reader034.vdocuments.net/reader034/viewer/2022050611/5fb28f48b40e0d0d890a4605/html5/thumbnails/12.jpg)
SBST vs DSE
Bugs found by SBST (SWAT) and DSE (APOLLO)
![Page 13: Search Based Test Data Generation for Server-side Web ...crest.cs.ucl.ac.uk/cow/30/slides/COW30_Nadia.pdf · Bugs found by SBST (SWAT) and DSE (APOLLO) Results • Impact of seeding](https://reader034.vdocuments.net/reader034/viewer/2022050611/5fb28f48b40e0d0d890a4605/html5/thumbnails/13.jpg)
Results
• Impact of seeding higher with string predicates • Constant seeding might mislead the search • Test suites with the same coverage perform
differently in fault detection
![Page 14: Search Based Test Data Generation for Server-side Web ...crest.cs.ucl.ac.uk/cow/30/slides/COW30_Nadia.pdf · Bugs found by SBST (SWAT) and DSE (APOLLO) Results • Impact of seeding](https://reader034.vdocuments.net/reader034/viewer/2022050611/5fb28f48b40e0d0d890a4605/html5/thumbnails/14.jpg)
Results
• In some applications coverage is low (20%), similar results for DSE
• Branches that are not covered: – Database dependent – Environment dependent: time, OS, browser..etc – Configuration: infeasible? – Multi-user dependent
![Page 15: Search Based Test Data Generation for Server-side Web ...crest.cs.ucl.ac.uk/cow/30/slides/COW30_Nadia.pdf · Bugs found by SBST (SWAT) and DSE (APOLLO) Results • Impact of seeding](https://reader034.vdocuments.net/reader034/viewer/2022050611/5fb28f48b40e0d0d890a4605/html5/thumbnails/15.jpg)
Future Directions
• Easy to apply, better performance • Different goals not just coverage • Oracle problem: automated might not be possible
but reduce the cost