references: “pruning dynamic slices with confidence’’, by x. zhang, n. gupta and r. gupta...
TRANSCRIPT
References: “Pruning Dynamic Slices With Confidence’’, by
X. Zhang, N. Gupta and R. Gupta (PLDI 2006).
“Locating Faults Through Automated Predicate Switching’’, by
X. Zhang, N. Gupta and R. Gupta (ICSE 2006).
“Locating Faulty Code Using Failure-Inducing Chops’’, by
N. Gupta, H. He, X. Zhang, N. Gupta and R. Gupta (ASE 2005).
Automated Debugging
2
Problem Statement
Given a faulty program and an input for which the program fails, identify as small as possible a subset of executed statements that is likely to contain the faulty code.
3
Prior Approaches
• Backward Dynamic Slicing
[Korel and Laski IPL’98], [Agrawal and Horgan PLDI’90]
[Zhang, Gupta, Zhang, ICSE’03, ICSE’04]
• Delta Debugging
[Zeller and Hildebrandt TSE’02], [Zeller FSE’02]
4
Program Slice
Backward Program Slice of a variable at a point in a program is the set of statements that might affect the value of the variable at that point. (static slice)
“Program Slices: Formal, psychological, and practical investigations of an automatic program abstraction method”, Mark Wiser – [PhD Thesis, 1979].
“Programmers use Slicing when Debugging”, by Mark Wiser -- [CACM ’82].
“Program Slicing”, by M. Wiser -- [IEEE TSE, 1984].
5
Dynamic Program Slice
Dynamic Program Slice of a variable at a point in the execution trace for an input is the set of statements that actually affected the value of the variable at that point during the execution.
“Dynamic Program Slicing”, by B. Korel and J. Laski -- [IPL, 1988].
“Dynamic Program Slicing”, by H. Agrawal and J. Horgan -- [PLDI, 1990].
“Precise Dynamic Algorithms”, by X. Zhang, R.Gupta, Y. Zhang -- [ICSE 2003].
6
Backward Dynamic Slicing
Construct a dynamic dependence graph and compute a transitive closure over dynamic control and data dependences to compute the slice
• Backward dynamic slice of a faulty output is expected to contain the faulty code. However, backward dynamic slices can be much larger than the faulty code.
7
Delta Debugging
Given a faulty program and a failed run, Simplify or Isolate a minimal failure-inducing input. [Zeller and Hildebrandt TSE’02]
Isolate minimal failure-inducing differences in program states of a failed run and a successful run and build a cause effect chain in terms of relevant state differences [Zeller FSE’02]
• It is not easy to link cause effect chains to faulty source code.
8
Delta Debugging: Cause Effect Chains
Cause Effect Chain
9
Motivation
To narrow the search for faulty code by combining the potential of • Delta Debugging in identifying a minimal failure- inducing
input
• Dynamic program slicing in able to link a failed output directly to the source code causing the failure
10
Our Approach
Given a failed run:
• Identify a minimal failure- inducing input
• Compute forward dynamic slice (FS) of the input identified above
• Compute backward dynamic slice (BS) of the faulty output
• Intersect FS and BS to compute Failure Inducing Chop
“Locating Faulty Code Using Failure-Inducing Chops’’, by
N. Gupta, H. He, X. Zhang, N. Gupta and R. Gupta (ASE 2005).
11
Minimal Failure Inducing Input
Simplify Input for Failed Run ( If ) to produce minimal Input If’ such that removing any single input entity from If’ causes the failure to disappear.
Isolate minimum failure-inducing input difference between inputs for a failing and a passing run. Given inputs If and Is for a failed and a passing run respectively, compute If’ and Is’ such that any single part of If’- Is’ if removed from If’ will cause the failure to disappear if added to Is’ will cause the failure to appear.
12
Computing Minimal Failure-Inducing input
Instead of binary input division algorithms ddmin and dd from [Zeller TSE’02], a linear search algorithm by removing input from one end may also be used to identify minimal-failure inducing input.
In fact, removal of input from one end may also be done in a binary manner to speed up the search for failure inducing input.
13
Experiments: Siemens Suite
14
Experiments: Siemens Suite
15
Experiments: Siemens Suite
16
Experiments: Siemens Suite
17
Experiments: Memory Related Bugs
8164
1923
716
LOC
25854
8288
31132
-
The statement causing memory corruption was always found
Note that forward slices were much smaller than backward slices and
in the Fchop in the above experiments.
were as effective as dynamic slices in containing the failure causing code.
18
Automatic Predicate Switching
• Given an failing execution, automatically recognize an executed predicate instance, critical predicate, forcibly flipping which produces the correct output.
• The critical predicate is provided as evidence to the program failure in addition to the wrong output.
“Locating Faults Through Automated Predicate Switching’’, by
X. Zhang, N. Gupta and R. Gupta (ICSE 2006).
19
Input
Output
(a)
Bidirectional Slicing with Critical Predicates
Given a failed run:
• Identify the critical predicate
• Compute bidirectional slice (BiS) of the critical predicate
FS(CP)BiS(CP)
++CP
20
Input
Output
Coarse-grained Pruning by Intersecting Slices
failure inducing input
BS
FS
FS(CP)BiS(CP)
++CP
BS^FS
Coarse-grained pruning by intersecting multiple types (backward, forward, bidirectional) of dynamic slices -- [ASE’05, ICSE’06]
21
Forward Slice limitations
Input1 = (a=3, b=2, c=1)
failed run
Input2 = (a=1, b=2, c=1)
sucessful run
- include statements in BwdSlice that do not
depend upon any input variable into FwdSlice
22
Backward Slice limitations
Input = (a=1, b=2)
failed run
Execution trace: 1,2,3,4,6
BwdDS (y, 6, Input) = {3}
does not contain line 4.
- Use relevant backward dynamic slice
23
Further Pruning Using Confidence Analysis
Types of evidences used in coarse-grained pruning:• Wrong output• Failure inducing input• Critical predicate
Fine-grained pruning of a backward slice by exploiting partially correct output using confidence analysis
“Pruning Dynamic Slices With Confidence’’, by
X. Zhang, N. Gupta and R. Gupta (PLDI 2006).
24
Fine-grained Pruning by Exploiting Correct Outputs
……
10. A = 1 (Correct: A=3)
…...
20. B = A % 2
……
30. C = A + 2
……
40. Print (B)
41. Print (C)
Correct outputs produced in addition to wrong output.
BS(Owrong) – BS (Ocorrect) is problematic.
BS(C@41)= {10, 30, 41}
BS(B@40)= {10, 20, 40}
BS(C@41)-BS(B@40)
= {30,41}
25
Confidence Analysis
Value produced at node n can reach only wrong output nodes
nn
There is no evidence that n is correct, so it should be in the pruned slice.
Should we include n in the slice?
??
Confidence(n)=0
Confidence(n)=?; 0 ≤ ? ≤ 1
Value produced at node n can reach both the correct and wrong output nodes.
nnnn
nn
Confidence(n)=1
Value produced at n can reach only correct outputs There is no evidence of incorrectness of n.
Therefore it cannot be in the slice.
26
|)(|log1)( )|(| nAltnCf nrange−=
Confidence Analysis
nnnn
Range(n)={ a, b, c, d, e, f, g }
Alt(n)={ a }
Value(n) = a
Value(n) = bValue(n) = c
, c
• When |Alt(n)|==1, we have the highest confidence (=1) on the correctness of n;
• When |Alt(n)|==|Range(n)|, we have the lowest confidence (=0).
• |Range(n)| >= |Alt(n)|>=1
Alt(n) is a set of possible values of the variable defined by n, that when propagated through the dynamic dependence graph, produce the same values for correct outputs.
27
Confidence Analysis: Example
……
10. A = ...
…...
20. B = A % 2
……
30. C = A + 2
……
40. Print (B)
41. Print (C) 0)41( =Cf
1)40( =Cf
0)30( =Cf
1)20( =Cf
2log2
|)(|log1)10( )|(|)|(| ArangeArange
ArangeCf =−=
28
Confidence Analysis: Two Problems
How to decide the Range of values for a node n?• Based on variable type (e.g., Integer).• Static range analysis.• Our choice:
Dynamic analysis based on value profiles. Range of values for a statement is the set of values defined by all of
the execution instances of the statement during the program run.
How to compute Alt(n)?• Consider the set of correct output values as constraints.• Compute Alt(n) by backward propagation of constraints
through the dynamic dependence subgraph corresponding to the slice.
29
Computing Alt(n) Along Data Dependence
S1: T=... 9
S2: X=T+1 10 S3: Y=T%3 0
(X,T)= (6,5) (9,8)
(10,9)
(T,...)= (1,...) (3,...) (5,...) (8,...) (9,...)
(Y,T)=(0,3) (0,9) (1,1) (2,5) (2,8)
alt(T@S2)={9} alt(T@S3)={1,3,9}
alt(S1) = alt(T@S2) ∩ alt (T@S3) = {9}
alt(S2)={10} alt(S3)={0,1}
30
Computing Alt(n) Along Control Dependence
S1: if (P) … True
S2: X=T+1 10 S3: Y=T%3 0
(X,T)= (6,5) (9,8)
(10,9)
(Y,T)=(0,3) (0,9) (1,1) (2,5) (2,8)
alt(S1) = {True}
alt(S2)={10} alt(S3)={0,1}
31
Characteristics of Siemens Suite Programs
Program Description LOC Versions Tests
print_tokens Lexical analyzer 565 5 4072
print_tokens2 Lexical analyzer 510 5 4057
replace Pattern replacement 563 8 5542
schedule Priority scheduler 412 3 2627
schedule2 Priority scheduler 307 3 2683
gzip Unix utility 8009 1 1217
flex Unix utility 12418 8 525
• Each faulty version has a single manually injected error.• All the versions are not included:
No output is produced. Faulty statement is not contained in the backward slice.
• For each version three tests were selected.
32
Results of Pruning
Program DS PDSmax PDSmax / DS
PDSmin %Missed by PDSmin
print_tokens 110 35 31.8% 35 0%
print_tokens2 114 55 48.2% 55 0%
replace 131 60 45.8% 43 38.1%
schedule 117 70 59.8% 56 20%
schedule2 90 58 64.4% 50 0%
gzip 357 121 33.9% 10 100%
flex 727 27 3.7% 25 0%
On average, PDSmax = 41.1% of DS
33
Confidence Based Prioritization
DD – dep. distance
CV – confidence values
Executed statement instances examined (%)
34
The Potential of Confidence Analysis (1)
Case Study (replace v14)• 88 74 23
Dynamic SlicerWith Confidence
Pruned Slices
User Verified Statements as correct
Buggy Code
Input User
35
The Potential of Confidence Analysis (2)
Relevant slicing (gzip v3 run r1)
Potential dep.Data dep.
36
Conclusions
• Our work for the first time shows how application of forward dynamic slices in locating faulty code is enabled by the delta debugging technique.
• We have introduced the notion of switching predicate and defined a bidirectional slice to narrow the search for faulty code
• We have developed a framework for confidence analysis that exploits partially correct output
• Our approach enables combining various types of evidences such as wrong output, failure inducing input, critical predicate and partially correct output to reduce the search space for faulty code and to broaden the applicability of dynamic slicing.