lazy annotation for program testing and verification speaker: chen-hsuan adonis lin advisor:...
TRANSCRIPT
mm
Lazy Annotation for Program Testing and
Verification
Speaker: Chen-Hsuan Adonis Lin
Advisor: Jie-Hong Roland Jiang
November 26, 2010
1
mm
Paper Information
TitleLazy Annotation for Program Testing and Verification
AuthorKenneth L. McMillan
Publication (2010)Computer Aided Verification, 22nd International
Conference, Edinburgh, UK, pp. 104-118, 2010
November 26, 2010
2
mm
Outline
Introduction
Overview of algorithm
Case study
Experimental results
Conclusion
Some ideas
November 26, 2010
3
mm
Introduction
An interpolant-based approach to test generation and model checking for sequential programsSequential programs ≠ sequential circuits
The method generates Floyd/Hoare style annotations of the program on demand
Inspired from DPLL approach to Boolean satisfiability which combining search and deductionFocuses deduction where the search becomes blockedDeduce facts that guide the search away from the
failure
November 26, 2010
4
mm
Hoare Triple
A triple describes how the execution of a piece of code changes the state of the computation
where P and Q are assertions and C is a command{P} C {Q}
P is named the precondition and Q the postcondition: when the precondition is met, the command
establishes the postcondition
Example Empty statement axiom:
November 26, 2010
5
€
{P} skip {P}
Quick sort !!
€
{x +1 = 43} y := x +1 {y = 43}
mm
Outline
Introduction
Overview of algorithm
Case study
Experimental results
Conclusion
Some ideas
November 26, 2010
6
mm
Preliminary of algorithm
Designate a set of program locations as goals to be reached the goals are calls to a function error
The vertices (locations) and edges (statements) of the program’s control flow graph will be labeled with formulas
A label represents a condition under which no goal can be reachedDon’t forget goal = error
November 26, 2010
7
mm
Flow of algorithm
Initially, there are no labels (no annotation being equivalent to false)
Execute the program symbolically along some chosen pathEach input to the program is represented by
a symbolic value pi
In the symbolic state, each program variable is evaluated as a symbolic expression over these parameters
November 26, 2010
8
mm
Flow of algorithm (con’t)
A constraint is also maintained, which accumulates the conjunction of the branch guards along the chosen path
Block-actionWe say state is blocked if the current vertex
label is implied, meaning we cannot reach a goal from this state
When we are blocked, we will backtrack along the edge we just executed, annotating it with a new label that blocks that edge
November 26, 2010
9
mm
Flow of algorithm (con’t)
Decide-action (met branch)When choosing a branch to execute, we are guided
by these edge labels
A blocked edge cannot lead to a goal, so we always continue along an unblocked edge if there is one
When all outgoing edges are blocked in the current state label the current location with the conjunction of the
conditions that block the outgoing edges, thus blocking the current state and causing us to backtrack
November 26, 2010
10
mm
Outline
Introduction
Overview of algorithm
Case study
Experimental results
Conclusion
Some ideas
November 26, 2010
11
mm
Case study: simple
November 26, 2010
12
Enter this code at l1 Symbolic state is x = p0
with constraint T (true)
Branching from l1 → l2 (one choice) Constraint p0 = 0
At l2, branch to l3 or l6 ? Neither edge is labeled,
choose arbitrarily l6
At l6, only one choice, branch to l7 Obtain the UNSAT
constraint p0=0 ∧p0<0
x = p0
p0 = 0
p0 = 0
p0=0 ∧p0<0
mm
Case study: simple (con’t)
November 26, 2010
13
At l7, we are blocked, since F (false) holds in current state Current annotation is F
Backtrack, annotate edge l6 → l7 Label l6: x ≥ 0, and we
are blocked now since current state: x = 0 (x = 0 implies x ≥ 0)
Backtrack to l2, labeling edge l2 → l6 : x ≥ 0
F
x ≥ 0
x ≥ 0
mm
Case study: simple (con’t)
November 26, 2010
14
Since the edge l2 → l3 is still unblocked, and follow it (Annotation forced the search in a different direction)
Move on to l4 Have y = p1 (new input)
At l5, we have constraint p1 ≥ 0
Arrive at l6 in the state x=p0+p1 with constraints p0 = 0 and p1 ≥ 0
F
y = p1
p1 > 0
x ≥ 0x=p0+p1, (constraints
p0=0 and p1≥0)
mm
Case study: simple (con’t)
November 26, 2010
15
Since “x = p0+p1 & p0 = 0 and p1≥0” implies previous annotation l6: x≥0, we are blocked
The fact we previously learned tells us There is no path to the
goal from current state
Backtrack to l5 and take weakest precondition of x≥0, then gives us l5: x+y≥0
F
x ≥ 0
x+y ≥ 0
mm
Case study: simple (con’t)
November 26, 2010
16
Backtrack to l4, however, we observe a slight problem
The weakest precondition of x + y ≥ 0 with respect to the (assertion y ≥ 0) is y < 0∨x+y ≥ 0
The variable y is irrelevant here, and we could just as well block the state with x ≥ 0
x ≥ 0 is also a precondition, not the the weakest
F
x ≥ 0
x+y ≥ 0
?x ≥ 0
mm
Compute precondition with interpolant
It can be computed by interpolantThe precondition: l4: x ≥ 0 (not weakest)
assume y ≥ 0 l5: x + y ≥ 0
When the weakest precondition would yield l3 : ∀y. (y < 0 ∨ x+y ≥ 0)We can simplify this to x ≥ 0, but this requires
quantifier elimination, which can be very expensive
By computing preconditions with interpolants, we avoid the need for quantifier elimination
November 26, 2010
17
mm
Case study: simple (con’t)
November 26, 2010
18
Backtrack to l2, labeling the edge (l2 → l3) with x ≥ 0
At l2, both edges from l2 are blocked label it with the
conjunction of the blocking labels along these edges, yielding l2 : x ≥ 0
Finally, we label l1 : T, proving that the goal error cannot be reached from l1
F
x ≥ 0
x+y ≥ 0
x ≥ 0
x ≥ 0
T
mm
Interpolants from Proofs
Given a pair of FOL formulas (A,B), such that A ∧ B is inconsistent, an interpolant for (A,B) is a formula Ā with the following properties:A implies Ā,Ā ∧B is unsatisfiable, andĀ is in L(A) ∩ L(B)
For a given formula ϕ, L(ϕ) will denote the well-formed formulas (wff’s) over the uninterpreted vocabulary of ϕ
Interpolant always exists for inconsistent formulas in FOL (Craig’s lemma)
November 26, 2010
19
mm
Something about Interpolant
A(X,Y) Λ B(Y,Z) ≡ FALSEThere exists I(Y) such that
A(X,Y) I(Y) I(Y) Λ B(Y,Z) ≡ FALSE
The “best” interpolant:
Interpolantion is an Existential Quantification
November 26, 2010
20
€
I(Y) = (∃X )(A(X ,Y))
)),()((),( YXAXYXA
TRUEYXBYXATRUEZYBYXAX ),(),(),()),()((
mm
Outline
Introduction
Overview of algorithm
Case study
Experimental results
Conclusion
Some ideas
November 26, 2010
21
mm
Experimental Results
Comparison of test generation with and without learning Y axis: # of coverage
goals reached X axis: # of backtracks
Compare # of backtracks Saturation (all reachable
locations reached) Completion
November 26, 2010
22
The effect is more pronounced in the large program, with learning reducing backtracks to completion by factor 84
mm
Outline
Introduction
Overview of algorithm
Case study
Experimental results
Conclusion
Some ideas
November 26, 2010
23
mm
Conclusion
Deduce program annotations in response to search failure, much in the way that a DPLL SAT solver learns conflict clauses
Without learning, there are long plateaus during which many paths are explored but no new locations are reached
Learning clearly acts to push the search away from these regions, allowing the search to make steady progress
November 26, 2010
24
mm
Outline
Introduction
Overview of algorithm
Case study
Experimental results
Conclusion
Some ideas
November 26, 2010
25
mm
Some Ideas
Apply the concept of learning method of DPLL SAT solving to prune the searching space of other research topic, such as QBF, model checking, and so on
Using interpolants can avoidthe expense of quantifier eliminationPredicate image computation
November 26, 2010
26
mm
Thanks for your attention
November 26, 2010
27