ajitha rajan school of informatics · 2016. 11. 6. · software testing widespread and applied to...
TRANSCRIPT
![Page 1: Ajitha Rajan School of Informatics · 2016. 11. 6. · Software Testing Widespread and applied to software of any size Major problems with the efficiency and effectiveness of testing](https://reader035.vdocuments.net/reader035/viewer/2022071409/610378403970c007376eb2f4/html5/thumbnails/1.jpg)
Ajitha RajanSchool of Informatics
Research Interests – Software Testing, Metrics for Quality Assurance, Optimising
Software Energy Consumption
[email protected]://homepages.inf.ed.ac.uk/arajan/
![Page 2: Ajitha Rajan School of Informatics · 2016. 11. 6. · Software Testing Widespread and applied to software of any size Major problems with the efficiency and effectiveness of testing](https://reader035.vdocuments.net/reader035/viewer/2022071409/610378403970c007376eb2f4/html5/thumbnails/2.jpg)
The ProblemSoftware testing is time consuming
Functional testing
The test suite of a non-trivial system:● could have thousands of test cases● could take hours, days or even weeks to execute
![Page 3: Ajitha Rajan School of Informatics · 2016. 11. 6. · Software Testing Widespread and applied to software of any size Major problems with the efficiency and effectiveness of testing](https://reader035.vdocuments.net/reader035/viewer/2022071409/610378403970c007376eb2f4/html5/thumbnails/3.jpg)
Software Testing● Widespread and applied to software of any size● Major problems with the efficiency and effectiveness
of testing in its current form in practice. (Annual costs in the US run up to $59.5 billion [NIST])
● Software is getting larger and more complex
● The global cost of software bugs is estimated at 312 billions of dollars annually.
12 Million Lines of Code 61 Million LOC14 Million LOC
![Page 4: Ajitha Rajan School of Informatics · 2016. 11. 6. · Software Testing Widespread and applied to software of any size Major problems with the efficiency and effectiveness of testing](https://reader035.vdocuments.net/reader035/viewer/2022071409/610378403970c007376eb2f4/html5/thumbnails/4.jpg)
Why is Software Testing Hard
Input/state spacepublic Line(int x0, int y0, int x1, int y1)
● Input per int: 232 different values:● 232 X 232 X 232 X 232 different values
Execution Sequencesfor (int i=0, i < n; i++) {
if (a.get(i) == b.get(i)) {
x[i] += 100;
} else {
x[i] /=2;
}
}
1000 lines per second: 1028 years
2n paths with n iterations
![Page 5: Ajitha Rajan School of Informatics · 2016. 11. 6. · Software Testing Widespread and applied to software of any size Major problems with the efficiency and effectiveness of testing](https://reader035.vdocuments.net/reader035/viewer/2022071409/610378403970c007376eb2f4/html5/thumbnails/5.jpg)
Systematic Partition TestingSystematic Partition Testing
Failure (valuable test case)Failures are sparse i h f but dense in someFailure (valuable test case)
No failurein the space of possible inputs ...
... but dense in some parts of the space
ues
inpu
t val
uk)
po
ssib
le i
hays
tack
pace
of p
(the
If we systematically test some cases from each part, we will
Functional testing is one way of drawing pink lines to isolate
The
sp
(c) 2007 Mauro Pezzè & Michal Young Ch 10, slide 7
p ,include the dense parts
drawing pink lines to isolate regions with likely failures
![Page 6: Ajitha Rajan School of Informatics · 2016. 11. 6. · Software Testing Widespread and applied to software of any size Major problems with the efficiency and effectiveness of testing](https://reader035.vdocuments.net/reader035/viewer/2022071409/610378403970c007376eb2f4/html5/thumbnails/6.jpg)
Research Problem
Generating software test cases using a combination of machine learning and program analysis techniques.
Test Cases
![Page 7: Ajitha Rajan School of Informatics · 2016. 11. 6. · Software Testing Widespread and applied to software of any size Major problems with the efficiency and effectiveness of testing](https://reader035.vdocuments.net/reader035/viewer/2022071409/610378403970c007376eb2f4/html5/thumbnails/7.jpg)
Ever increasing computing powerG
FLO
PS
(lo
g)
Only about 20% of the available power resources is getting used!
![Page 8: Ajitha Rajan School of Informatics · 2016. 11. 6. · Software Testing Widespread and applied to software of any size Major problems with the efficiency and effectiveness of testing](https://reader035.vdocuments.net/reader035/viewer/2022071409/610378403970c007376eb2f4/html5/thumbnails/8.jpg)
Characteristics: 1. Executions are independent 2. Executions are data parallel
The ProblemSoftware testing is time consuming
Functional testing
![Page 9: Ajitha Rajan School of Informatics · 2016. 11. 6. · Software Testing Widespread and applied to software of any size Major problems with the efficiency and effectiveness of testing](https://reader035.vdocuments.net/reader035/viewer/2022071409/610378403970c007376eb2f4/html5/thumbnails/9.jpg)
Research problem
How do you cluster tests for parallel execution on multiple cores?
Test 1
Test n
Test 4
Test 3
Test 2
Dynamic Analysis
Gather data on test executions
Create clusters of tests
Launch each cluster on a single CPU
![Page 10: Ajitha Rajan School of Informatics · 2016. 11. 6. · Software Testing Widespread and applied to software of any size Major problems with the efficiency and effectiveness of testing](https://reader035.vdocuments.net/reader035/viewer/2022071409/610378403970c007376eb2f4/html5/thumbnails/10.jpg)
Factors Affecting Energy in Software
![Page 11: Ajitha Rajan School of Informatics · 2016. 11. 6. · Software Testing Widespread and applied to software of any size Major problems with the efficiency and effectiveness of testing](https://reader035.vdocuments.net/reader035/viewer/2022071409/610378403970c007376eb2f4/html5/thumbnails/11.jpg)
Our Empirical Study
![Page 12: Ajitha Rajan School of Informatics · 2016. 11. 6. · Software Testing Widespread and applied to software of any size Major problems with the efficiency and effectiveness of testing](https://reader035.vdocuments.net/reader035/viewer/2022071409/610378403970c007376eb2f4/html5/thumbnails/12.jpg)
Approach
![Page 13: Ajitha Rajan School of Informatics · 2016. 11. 6. · Software Testing Widespread and applied to software of any size Major problems with the efficiency and effectiveness of testing](https://reader035.vdocuments.net/reader035/viewer/2022071409/610378403970c007376eb2f4/html5/thumbnails/13.jpg)
Data Collection – Metrics
![Page 14: Ajitha Rajan School of Informatics · 2016. 11. 6. · Software Testing Widespread and applied to software of any size Major problems with the efficiency and effectiveness of testing](https://reader035.vdocuments.net/reader035/viewer/2022071409/610378403970c007376eb2f4/html5/thumbnails/14.jpg)
Feature Selection
• A machine learning technique.• Selects a subset of statistically
relevant features for use in model construction.
![Page 15: Ajitha Rajan School of Informatics · 2016. 11. 6. · Software Testing Widespread and applied to software of any size Major problems with the efficiency and effectiveness of testing](https://reader035.vdocuments.net/reader035/viewer/2022071409/610378403970c007376eb2f4/html5/thumbnails/15.jpg)
Evolving SoftwareLarge software systems are usually built incrementally:
• Maintenance - fixing errors and flaws, hardware changes• Enhancements - new functionality, improved efficiency,
extension, new regulations
Programv1
Programv2
Programv3
upgrade upgrade upgrade
Research Interests1. What is the effort/cost of upgrades?2. Test adequacy criteria for regression testing3. What is the regression test effort/cost?
![Page 16: Ajitha Rajan School of Informatics · 2016. 11. 6. · Software Testing Widespread and applied to software of any size Major problems with the efficiency and effectiveness of testing](https://reader035.vdocuments.net/reader035/viewer/2022071409/610378403970c007376eb2f4/html5/thumbnails/16.jpg)
Cost Models
● Highly inaccurate and research in building these models is from a decade ago
● DATA DATA DATA – Machine learning techniques that considers program change sizes, organisational factors, developer experience, past project data.
● Combine them with precise static analysis