predicting defects for eclipse
DESCRIPTION
Thomas Zimmermann, Rahul Premraj, Andres ZellerTRANSCRIPT
Predicting Defectsfor Eclipse
Thomas Zimmermann • Rahul Premraj • Andreas ZellerSaarland University
SummaryProject Eclipse (eclipse.org)
Content Defect counts Complexity metrics
Releases 2.0, 2.1, and 3.0Level Packages and files
URL www.st.cs.uni-sb.de/softevo/
More data Eclipse source code
BugsBugs ChangesChanges
Data Source
Identifying Fixes
BugsBugs ChangesChanges
•Change history contains ordinary changes as well as bug fixes
•Use log messages to discriminate
•Search for keywords – e.g. “bug” “fix”
•Look out for bug IDs – e.g. “#33547”
Mapping Bugs
BugsBugs ChangesChanges
•Each bug report has a unique bug ID
•Bug reports contain releases(and sometimes components)
•Associate bugs with changes via bug ID
Eclipse BugsEclipse Bugs
•Program is open source
•Plenty of data available – automatically
•Program data – all sorts of code analyses
•Process data – changes, bugs, e-mail, etc.
•Data set provides bugs, metrics, tokens
Predictions at Work
<?xml version="1.0" encoding="UTF-8"?> <!-- comments --> <defects project="eclipse" release="2.0" dataversion="1.0"> <plug-in name="platform-launcher"> <counts> <count id="pre" value="0" avg="0.0" compilationunits="1" max="0"/> <count id="post" value="0" avg="0.0" compilationunits="1" max="0"/> </counts> <package name="org.eclipse"> <counts> <count id="pre" value="0" avg="0.0" compilationunits="1" max="0"/> <count id="post" value="0" avg="0.0" compilationunits="1" max="0"/> </counts> <package name="org.eclipse.core"> <counts> <count id="pre" value="0" avg="0.0" compilationunits="1" max="0"/> <count id="post" value="0" avg="0.0" compilationunits="1" max="0"/> </counts> <package name="org.eclipse.core.launcher"> <counts> <count id="pre" value="0" avg="0.0" compilationunits="1" max="0"/> <count id="post" value="0" avg="0.0" compilationunits="1" max="0"/> </counts> <compilationunit dir="/platform-launcher/library/" base="Main.java"> <counts> <count id="pre" value="0"/> <count id="post" value="0"/> </counts> </compilationunit> </package> </package> </package> </plug-in>
Eclipse Bug DataEclipse Bug Data
<?xml version="1.0" encoding="UTF-8"?> <!-- comments --> <defects project="eclipse" release="2.0" dataversion="1.0"> <plug-in name="platform-launcher"> <counts> <count id="pre" value="0" avg="0.0" compilationunits="1" max="0"/> <count id="post" value="0" avg="0.0" compilationunits="1" max="0"/> </counts> <package name="org.eclipse"> <counts> <count id="pre" value="0" avg="0.0" compilationunits="1" max="0"/> <count id="post" value="0" avg="0.0" compilationunits="1" max="0"/> </counts> <package name="org.eclipse.core"> <counts> <count id="pre" value="0" avg="0.0" compilationunits="1" max="0"/> <count id="post" value="0" avg="0.0" compilationunits="1" max="0"/> </counts> <package name="org.eclipse.core.launcher"> <counts> <count id="pre" value="0" avg="0.0" compilationunits="1" max="0"/> <count id="post" value="0" avg="0.0" compilationunits="1" max="0"/> </counts> <compilationunit dir="/platform-launcher/library/" base="Main.java"> <counts> <count id="pre" value="0"/> <count id="post" value="0"/> </counts> </compilationunit> </package> </package> </package> </plug-in>
Eclipse Bug DataEclipse Bug Data
•For three Eclipse releases:
•For all Eclipse components:
•Defect counts before and after release
•Complexity metrics
•Syntactic tokens
•Download athttp://www.st.cs.uni-sb.de/softevo/
•For three Eclipse releases:
•For all Eclipse components:
•Defect counts before and after release
•Complexity metrics
•Syntactic tokens
•Download athttp://www.st.cs.uni-sb.de/softevo/
Eclipse BugsEclipse Bugs
Where do bugs come from?
Is it the Developers?
Does Does experience experience
matter?matter?
Bug Bug density density
correlates correlates with with
experienceexperience!!
How about Testing?
Does Does code code coverage coverage
predict bug predict bug density?density?
Yes –Yes – the more the more
tests, tests, the more the more
bugs!bugs!
History?
I found lots I found lots of bugs of bugs
here. Will here. Will there be there be more?more?
Yes!Yes!
How about Metrics?
Do Do code code metrics metrics
predict bug predict bug density?density?
Yes! (but Yes! (but only with only with history)history)
Syntactic Tokens?
Which Which tokenstokens
predict bug predict bug density?density?
imports • imports • extends • extends • implementimplement
ss
Eclipse Imports
import org.eclipse.jdt.internal.compiler.lookup.*;import org.eclipse.jdt.internal.compiler.*;import org.eclipse.jdt.internal.compiler.ast.*;import org.eclipse.jdt.internal.compiler.util.*;...import org.eclipse.pde.core.*;import org.eclipse.jface.wizard.*;import org.eclipse.ui.*;
14% of all components importing 14% of all components importing uiuishow a post-release defectshow a post-release defect
71% of all components importing 71% of all components importing compilercompiler
show a post-release defectshow a post-release defect
Joint work with Adrian Schröter • Tom Zimmermann
Eclipse Imports
Correlation with failure
Correlation with success
import org.eclipse.jdt.internal.compiler.lookup.*;import org.eclipse.jdt.internal.compiler.*;import org.eclipse.jdt.internal.compiler.ast.*;import org.eclipse.jdt.internal.compiler.util.*;...import org.eclipse.pde.core.*;import org.eclipse.jface.wizard.*;import org.eclipse.ui.*;
What What makesmakes
code buggycode buggyin the first in the first
place?place?
<?xml version="1.0" encoding="UTF-8"?> <!-- comments --> <defects project="eclipse" release="2.0" dataversion="1.0"> <plug-in name="platform-launcher"> <counts> <count id="pre" value="0" avg="0.0" compilationunits="1" max="0"/> <count id="post" value="0" avg="0.0" compilationunits="1" max="0"/> </counts> <package name="org.eclipse"> <counts> <count id="pre" value="0" avg="0.0" compilationunits="1" max="0"/> <count id="post" value="0" avg="0.0" compilationunits="1" max="0"/> </counts> <package name="org.eclipse.core"> <counts> <count id="pre" value="0" avg="0.0" compilationunits="1" max="0"/> <count id="post" value="0" avg="0.0" compilationunits="1" max="0"/> </counts> <package name="org.eclipse.core.launcher"> <counts> <count id="pre" value="0" avg="0.0" compilationunits="1" max="0"/> <count id="post" value="0" avg="0.0" compilationunits="1" max="0"/> </counts> <compilationunit dir="/platform-launcher/library/" base="Main.java"> <counts> <count id="pre" value="0"/> <count id="post" value="0"/> </counts> </compilationunit> </package> </package> </package> </plug-in>
Eclipse Bug DataEclipse Bug Data
•For three Eclipse releases:
•For all Eclipse components:
•Defect counts before and after release
•Complexity metrics
•Syntactic tokens
•Download athttp://www.st.cs.uni-sb.de/softevo/
•For three Eclipse releases:
•For all Eclipse components:
•Defect counts before and after release
•Complexity metrics
•Syntactic tokens
•Download athttp://www.st.cs.uni-sb.de/softevo/