predicting defects for eclipse

Predicting Defectsfor Eclipse

Thomas Zimmermann • Rahul Premraj • Andreas ZellerSaarland University

SummaryProject  Eclipse (eclipse.org)

Content Defect counts Complexity metrics

Releases 2.0, 2.1, and 3.0Level Packages and files

URL www.st.cs.uni-sb.de/softevo/

More data Eclipse source code

BugsBugs ChangesChanges

Data Source

Identifying Fixes

BugsBugs ChangesChanges

•Change history contains ordinary changes as well as bug fixes

•Use log messages to discriminate

•Search for keywords – e.g. “bug” “fix”

•Look out for bug IDs – e.g. “#33547”

Mapping Bugs

BugsBugs ChangesChanges

•Each bug report has a unique bug ID

•Bug reports contain releases(and sometimes components)

•Associate bugs with changes via bug ID

Eclipse BugsEclipse Bugs

•Program is open source

•Plenty of data available – automatically

•Program data – all sorts of code analyses

•Process data – changes, bugs, e-mail, etc.

•Data set provides bugs, metrics, tokens

Predictions at Work

<?xml version="1.0" encoding="UTF-8"?> <!-- comments --> <defects project="eclipse" release="2.0" dataversion="1.0"> <plug-in name="platform-launcher"> <counts> <count id="pre" value="0" avg="0.0" compilationunits="1" max="0"/> <count id="post" value="0" avg="0.0" compilationunits="1" max="0"/> </counts> <package name="org.eclipse"> <counts> <count id="pre" value="0" avg="0.0" compilationunits="1" max="0"/> <count id="post" value="0" avg="0.0" compilationunits="1" max="0"/> </counts> <package name="org.eclipse.core"> <counts> <count id="pre" value="0" avg="0.0" compilationunits="1" max="0"/> <count id="post" value="0" avg="0.0" compilationunits="1" max="0"/> </counts> <package name="org.eclipse.core.launcher"> <counts> <count id="pre" value="0" avg="0.0" compilationunits="1" max="0"/> <count id="post" value="0" avg="0.0" compilationunits="1" max="0"/> </counts> <compilationunit dir="/platform-launcher/library/" base="Main.java"> <counts> <count id="pre" value="0"/> <count id="post" value="0"/> </counts> </compilationunit> </package> </package> </package> </plug-in>

Eclipse Bug DataEclipse Bug Data

<?xml version="1.0" encoding="UTF-8"?> <!-- comments --> <defects project="eclipse" release="2.0" dataversion="1.0"> <plug-in name="platform-launcher"> <counts> <count id="pre" value="0" avg="0.0" compilationunits="1" max="0"/> <count id="post" value="0" avg="0.0" compilationunits="1" max="0"/> </counts> <package name="org.eclipse"> <counts> <count id="pre" value="0" avg="0.0" compilationunits="1" max="0"/> <count id="post" value="0" avg="0.0" compilationunits="1" max="0"/> </counts> <package name="org.eclipse.core"> <counts> <count id="pre" value="0" avg="0.0" compilationunits="1" max="0"/> <count id="post" value="0" avg="0.0" compilationunits="1" max="0"/> </counts> <package name="org.eclipse.core.launcher"> <counts> <count id="pre" value="0" avg="0.0" compilationunits="1" max="0"/> <count id="post" value="0" avg="0.0" compilationunits="1" max="0"/> </counts> <compilationunit dir="/platform-launcher/library/" base="Main.java"> <counts> <count id="pre" value="0"/> <count id="post" value="0"/> </counts> </compilationunit> </package> </package> </package> </plug-in>

Eclipse Bug DataEclipse Bug Data

•For three Eclipse releases:

•For all Eclipse components:

•Defect counts before and after release

•Complexity metrics

•Syntactic tokens

•Download athttp://www.st.cs.uni-sb.de/softevo/

•For three Eclipse releases:

•For all Eclipse components:

•Defect counts before and after release

•Complexity metrics

•Syntactic tokens

•Download athttp://www.st.cs.uni-sb.de/softevo/

Eclipse BugsEclipse Bugs

Where do bugs come from?

Is it the Developers?

Does Does experience experience

matter?matter?

Bug Bug density density

correlates correlates with with

experienceexperience!!

How about Testing?

Does Does code code coverage coverage

predict bug predict bug density?density?

Yes –Yes – the more the more

tests, tests, the more the more

bugs!bugs!

History?

I found lots I found lots of bugs of bugs

here. Will here. Will there be there be more?more?

Yes!Yes!

How about Metrics?

Do Do code code metrics metrics

predict bug predict bug density?density?

Yes! (but Yes! (but only with only with history)history)

Syntactic Tokens?

Which Which tokenstokens

predict bug predict bug density?density?

imports • imports • extends • extends • implementimplement

ss

Eclipse Imports

import org.eclipse.jdt.internal.compiler.lookup.*;import org.eclipse.jdt.internal.compiler.*;import org.eclipse.jdt.internal.compiler.ast.*;import org.eclipse.jdt.internal.compiler.util.*;...import org.eclipse.pde.core.*;import org.eclipse.jface.wizard.*;import org.eclipse.ui.*;

14% of all components importing 14% of all components importing uiuishow a post-release defectshow a post-release defect

71% of all components importing 71% of all components importing compilercompiler

show a post-release defectshow a post-release defect

Joint work with Adrian Schröter • Tom Zimmermann

Eclipse Imports

Correlation with failure

Correlation with success

import org.eclipse.jdt.internal.compiler.lookup.*;import org.eclipse.jdt.internal.compiler.*;import org.eclipse.jdt.internal.compiler.ast.*;import org.eclipse.jdt.internal.compiler.util.*;...import org.eclipse.pde.core.*;import org.eclipse.jface.wizard.*;import org.eclipse.ui.*;

What What makesmakes

code buggycode buggyin the first in the first

place?place?

<?xml version="1.0" encoding="UTF-8"?> <!-- comments --> <defects project="eclipse" release="2.0" dataversion="1.0"> <plug-in name="platform-launcher"> <counts> <count id="pre" value="0" avg="0.0" compilationunits="1" max="0"/> <count id="post" value="0" avg="0.0" compilationunits="1" max="0"/> </counts> <package name="org.eclipse"> <counts> <count id="pre" value="0" avg="0.0" compilationunits="1" max="0"/> <count id="post" value="0" avg="0.0" compilationunits="1" max="0"/> </counts> <package name="org.eclipse.core"> <counts> <count id="pre" value="0" avg="0.0" compilationunits="1" max="0"/> <count id="post" value="0" avg="0.0" compilationunits="1" max="0"/> </counts> <package name="org.eclipse.core.launcher"> <counts> <count id="pre" value="0" avg="0.0" compilationunits="1" max="0"/> <count id="post" value="0" avg="0.0" compilationunits="1" max="0"/> </counts> <compilationunit dir="/platform-launcher/library/" base="Main.java"> <counts> <count id="pre" value="0"/> <count id="post" value="0"/> </counts> </compilationunit> </package> </package> </package> </plug-in>

Eclipse Bug DataEclipse Bug Data

•For three Eclipse releases:

•For all Eclipse components:

•Defect counts before and after release

•Complexity metrics

•Syntactic tokens

•Download athttp://www.st.cs.uni-sb.de/softevo/

•For three Eclipse releases:

•For all Eclipse components:

•Defect counts before and after release

•Complexity metrics

•Syntactic tokens

•Download athttp://www.st.cs.uni-sb.de/softevo/