Differential Static Analysis: Opportunities and Challenges

Shuvendu LahiriRiSE, MSR Redmond

UC Berkeley OSQ Retreat (5/12/11)

Challenges of static assertion checking

void F(A *z, int *a){ int n = ComplexFunc(z); int j = 0; List* x = z->list; while(x != null) {

j++;if (x->d == n) { a[j] = x->g; break;}x = x->next;

}

}

assert (0 <= j && j <= ?);

Loop invariant

Postcondition

Precondition

State of static assertion checking

• Not cost-effective for an average developer (yet)– Need for specs– Need for intermediate inductive assertions– Precise analysis non-scalable

• Most functional specs are tested by regression testing (expensive/low-coverage)

Microsoft Confidential

App-Compat problem– Every developer has to worry about introducing

incompatibility rather than fixing ALL existing bugs– Library writers (releases)

Do my changes introduce a regression?

Did the “refactoring” change any observable behavior?

How does the change propagate to the public APIs

Current practice

• Run regression tests and hope breaking changes show up

• Problems– Coverage: only a small fraction of paths covered

• Especially with few inputs, loops, recursive

– Controllability: hard to exercise the paths under change

– Oracles: mostly program crashes, or a few built-in assertions

Differential static analysis

• Provide relative guarantee instead of absolute guarantee– May be easier to obtain, and possibly desirable

Precedence (equivalence checking)

• Hardware– Equivalence checking was a big success– ISA vs. RTL, timing optimizations, ….

• Software– Translation validation (intraprocedural compiler

transformations)

• However, most code changes in software change behaviors (bug fixes, features, …)– Fairly limited applicability

Potential applications: Fast feedback about changes

• Equivalence checking– Manual refactorings, translation validation, app-

compat for different versions of compilers• Conditional equivalence checking– Allow the user to express (or infer) conditions

under which two versions are compatible• Differential contract checking– Show that contracts are not violated for more

inputs after a change

Differential contract checking

void F(A *z, int *a){ int n = ComplexFunc(z); int j = 0; List* x = z->list; while(x != null) {

j++;if (x->d == n) { a[j] = x->g; break;}x = x->next;

}

}

void F(A *z, int *a){ int n = ComplexFunc(z); int j = 0; List* x = z->list; while(x != null) {

j++;if (x->d == n) { a[j] = x->h + x->g; break;}x = x->next;

}

}May be able to check that the change does not introduce new buffer overruns, with cheap abstractions

SymDiff (Symbolic Diff)

• Project underway in MSR Redmond– With Chris Hawblitzel, ..

• In a nutshell– Windiff for a program’s input/output behaviors– Source language independent (uses Boogie/Z3)– Static tool for identifying semantic differences

between two closely related program versions

SymDiff screenshot

Current directions (in progress)

• Mutual summaries – With Chris Hawblitzel– A general mechanism for comparing programs

• E.g. Can prove most compiler optimizations (translation validation)

• Application of differential contract checking – Finding precise concurrent bugs in the presence of

underspecified environment• With Akash Lal (POIROT)

• Inference of conditional equivalence– Using abstract interpretation, interpolation, …

Summary

• Problem: Static App-compat– Relative vs. absolute guarantee

• Opportunities– Impact developers, quick feedback, more coverage

than tests, fewer false alarms, cheaper abstractions

• Challenges– Understanding developer intent, communicating

differences, root cause, …

Other projects

• Linear maps– Local reasoning in classical logic [Qadeer, Walker]

• POIROT– Assertion directed symbolic search for concurrent

programs [Lal, Qadeer]

http://Rise4Fun.com