specifying and checking properties of programs thomas ball sriram k. rajamani software productivity...
Post on 21-Dec-2015
215 views
TRANSCRIPT
Specifying and Checking Properties of
Programs
Thomas BallSriram K. Rajamani
Software Productivity ToolsMicrosoft Research
Software Validation
• Large scale reliable software is hard to build and test.
• Different groups of programmers write different components.
• Integration testing is a nightmare.
Property Checking
• Programmer provides redundant partial specifications
• Code is automatically checked for consistency
• Different from proving whole program correctness – Specifications are not complete
Interface Usage Rules•Rules in documentation
–Incomplete, unenforced, wordy–Order of operations & data
access–Resource management
•Disobeying rules causes bad behavior
–System crash or deadlock–Unexpected exceptions–Failed runtime checks
Example: Sockets
the "communication domain" in which communication is to takeplace; see protocols(5).
Sockets of type SOCK_STREAM are full-duplex byte streams,similar to pipes. A stream socket must be in a connectedstate before any data may be sent or received on it. A con-nection to another socket is created with a connect(2) call.Once connected, data may be transferred using read(2V) andwrite(2V) calls or some variant of the send(2) and recv(2)calls. When a session has been completed a close(2V), maybe performed. Out-of-band data may also be transmitted asdescribed in send(2) and received as described in recv(2).
The communications protocols used to implement a SOCK_STREAMinsure that data is not lost or duplicated. If a piece of
Does a given usage rule hold?
• Checking this is computationally impossible!
• Equivalent to solving Turing’s halting problem (undecidable)
• Even restricted computable versions of the problem (finite state programs) are prohibitively expensive
Scientific curiosity
Undecidability and complexity theory are the most significant contributions of theoretical computer science.
Software property checking, a very practical and pressing problem is undecidable.
Automatic property checking =
Study of tradeoffs• Soundness vs completeness
– Missing errors vs reporting false alarms
• Annotation burden on the programmer• Complexity of the analysis
– Local vs Global– Precision vs Efficiency– Space vs Time
Broad classification
• Underapproximations– Testing
• After passing testing, a program may still violate a given property
• Overapproximations– Type checking
• Even if a program satisfies a property, the type checker for the property could still reject it
Current trend
• Confluence of techniques from different fields:– Model checking– Automatic theorem proving– Program analysis
• Significant emphasis on practicality
• Several new projects in academia and industry
Outline
• Property checking of software• Undecidable• Important
• Techniques– Model checking– Automatic theorem proving– Program analysis
• Current projects
Model Checking• Algorithmic exploration of state space of the system
• Started by [Clarke-Emerson][Quille-Sifakis] in the early 80s
• Had a breakthrough with Ken McMillan’s thesis in 92 (symbolic model checking with Binary Decision Diagrams)
• Several advances: symmetry reductions, partial order reductions, compositional model checking
• Most hardware companies use a model checker in the validation cycle
enum {N, T, C} state[1..2]
int turn
init state[1] = N; state[2] = N
turn = 0
trans state[i]= N & turn = 0 -> state[i] = T; turn = i
state[i] = N & turn !=0 -> state[i] = T state[i] = T & turn = i -> state[i] = C state[i] = C & state[2-i] = N -> state[i] = N state[i] = C & state[2-i] != N -> state[i] = N; turn = 2-i
N1,N2turn=0
T1,N2turn=1
T1,T2turn=1
C1,N2turn=1
C1,T2turn=1
N1,T2turn=2
T1,T2turn=2
N1,C2turn=2
T1,C2turn=2
N = noncritical, T = trying, C = critical
Model Checking
• Strengths– Fully automatic (when it works)– Computes inductive invariants
• I such that F(I) I
– Provides error traces
• Weaknesses– Scale– Operates only on models
• How do you get from the program to the model?
Theorem proving– Early theorem provers were proof checkers
• They were built to support asssertional reasoning in the Hoare-Dijkstra style
• Cumbersome and hard to use
– Greg Nelson’s thesis in early 80s paved the way for automatic theorem provers• Theory of equality with uninterpreted functions• Theory of lists• Theory of linear arithmetic• Combination of the above !
– Automatic theorem provers based on Nelson’s work are widely used• ESC• Proof Carrying Code
Theory of Equality. • Symbols: =, , f, g, …• Axiomatically defined:
E = E
E2 = E1
E1 = E2
E1 = E2 E2 = E3
E1 = E3
E1 = E2
f(E1) = f(E2)
• Example of a satisfiability problem: g(g(g(x)) = x g(g(g(g(g(x))))) = x g(x) x
• Satisfiability problem decidable in O(n log n)
a : array [1..len] of int;
int max := -MAXINT;i := 1;{ 1 j i. a[j] max}while (i len)
if( a[i] > max) max := a[i];
i := i+1;endwhile{ 1 j len. a[j]
max}
( 1 j i. a[j] max) ( i > len)
( 1 j len. a[j]
max}
Automatic theorem proving• Strengths
– Handles unbounded domains naturally– Good implementations for
• equality with uninterpreted functions• linear inequalities• combination of theories
• Weaknesses– Hard to compute fixpoints– Requires inductive invariants
• Pre and post conditions• Loop invariants
Program analysis
– Originated in optimizing compilers• constant propagation• live variable analysis• dead code elimination• loop index optimization
– Type systems use similar analysis• Are the type annotations consistent?
Program analysis• Strengths
– Works on code – Pointer aware– Integrated into compilers– Precision efficiency tradeoffs well studied
• flow (in)sensitive• context (in)sensitive
• Weakenesses– Abstraction is hardwired and done by the
designer of the analysis– Not targeted at property checking (traditionally)
An optimizing compiler doubles performance every 18 years
-Todd Proebsting
Computing power doubles every 18 months
-Gordon Moore
Model Checking, Theorem Proving and Program
Analysis– Very related to each other
– Different histories• different emphasis• different tradeoffs
– Complementary, in some ways
Outline
• Property checking of software• Undecidable• Important
• Techniques– Model checking– Automatic theorem proving– Program Analysis
• Current projects
Industrial successes with software validation
• PREfix– Statically simulate a small number of paths
symbolically inside each function • theorem proving with under-approximations
– Scale by “summarizing” at function boundaries– Works on Windows NT (several million lines of
code)
• Other projects– ESC project at Compqq SRC– SPIN model checker and the Feaver project at
Bell Labs– Verisoft model checker at Bell labs
Current trend..
Code
Intent
Model
• Use program analysis and automated theorem proving for model extraction
• User annotations simplify the model extraction problem
• Use model checking on the model
Some current projects- Software productivity tools (MSR)
- SLAM, Vault, ESP, Behave!
- Stanford MC project- Berkeley Open Source Quality- KSU’s Bandera- IBM’s CANVAS- SRI Symbolic analysis laboratory (SAL )- Verimag IF language and Tool Set- NASA Ames’ JavaPathfinder- Compaq SRC’s ESC-Java- Bell Labs’ Verisoft and FeaVer- UMass Amherst LASER
When I use a model checker, it runs and runs for ever and never comes back… when I use a static analysis tool, it comes back immediately and says “I don’t know”
- Patrick Cousot
Predictions– The holy grail of full program
verification has been abandoned. It will probably remain abandoned
– Less ambitious tools like powerful type checkers will emerge and become more widely used
– These tools will exploit ideas from various analysis disciplines
– Tools will alleviate the “chicken-and-egg” problem of writing specifications