An Entropy Evaluation Approach for Triaging Field Crashes: A Case Study of Mozilla Firefox

Foutse Khomh, Brian Chan, Ying Zou and Ahmed E. Hassan

Firefox receives 2.5 Million Crash Reports per day!

Triaging of Crash Reports

C1

C5

C3

C4

C2

C6

C1 C4

CR-A

C5 C6C2 C3

CR-B

BR-1

CR-B

Socorro ServerMozilla BugZilla

10 Beta Releases of Firefox 4.0

Fixed30%

Fixed (Du-pli-

cate)15%

Not Fixed55%

1,329 CRs

1,733 bugs

Top priority for CRs with highest # of crashes

Which Crash Reports (CR) to fix first?

Which CR are hardest?Unclear

Counts are not sufficient!15 Crashes in 2 different CRs

U1 U2 U3 U4 U5

11

1 1 1 1

U1 U2 U3 U4 U5

3 3 3 3 3

We should factor in Crash distribution across the User population

Low Entropy High Entropy

Entropy Based View of Crash Reports

Low

High

High

Entropy

# of Crashes

IsolatedRegion

HighlyDistributed

Region

ModeratelyDistributed

Region

SkewedRegion

HighLow

VeryLow Med.

Highly distributed region has the most bugs

Isolated Moderate High Skew

27%

13%

40%

20%

How do entropy priorities compare to developer assigned priorities

Priorities assigned for only 7% of bug reports

We use severity to get a priority estimate

normal critical blockermajorminortrivialnot

assigned

HighVery low MediumLow

Percentage of matching priorities

Low

High

High

Entropy

# of Users

IsolatedRegion

HighlyDistributed

Region

ModeratelyDistributed

Region

SkewedRegion

HighLow

VeryLow Med.

100%

19% 100%

100%

Only 19% matching priorities inIsolated Region

80% of bugs are ranked high by developers16% of these bugs are never fixed

Fixed bugs take 52% more time than “low” priority bugs

High priority (Iso-lated Region)

Low priority

1,680

1,104 hours

hours

(across all regions)

Isolated Region has highest % of not-fixed bugs

Isolated Moderate High Skew

16%

9.5%

6%

9%

Which CRs are hardest?

Require more time

Require more Bugzilla discussions

Which CRs are hardest?

Low Entropy High Entropy

1,296

2,279

Low Entropy High Entropy

9

20

Hours to fix # of messages

Isolated Region Bugs take the least time to fix

Isolated Moderate High Skew

1,608

4,032

4,993 5,063

Skewed Region Bugs require the most discussion

Isolated Moderate High Skew

9

20 22

67

Entropy Based View of Crash Reports

Low

High

High

Entropy

# of Crashes

Highest % of not-fixed bugs (16%)

Largest priority mismatch (80%)

Require the most discussion(3 times the median)

Most the bugs (40%)

Lowest % of not-fixed bugs (6%)

Least bugs (13%)

HighLow

VeryLow Med.