finding and preventing run-time error handling mistakes westley weimer george c. necula university...
TRANSCRIPT
Finding and Preventing Run-Time Error Handling MistakesWestley WeimerGeorge C. NeculaUniversity of California, Berkeley{weimer,necula}@cs.berkeley.edu
Handling Errors is Difficult
Writing solid error-handling code is hard1) We have a mistake-finding analysis
o And find 800 mistakes in 4M LOC (Java)o We characterize these mistakes
2) Propose a new language featureo Existing language features: insufficiento Track obligations at run-timeo Support it by case studies
Defining Terms
Run-Time Errors: Network problems, DB access errors, OS
resource exhaustion, … Typical Error-Handling:
Resend packet, show dialog box to user, …
Application-specific Error-Handling Handling Mistakes:
One example: a network error occurs and the program forgets to release a database lock with an external shared database, …
Cleanup in Practice
Most common exception handlersDo NothingPrint Stack Trace, Abort Program
High-level invariants should be restoredAside from “handling” the error, the
program should clean up after itself Often this does not happen in practice:
Considering all (error) paths is hard!
Normal Execution Path// Copy to destinationFile without doing any byte conversion.private static void _binaryCopyURLToFile(URL sourceURL, File destinationFile) throws IOException { BufferedInputStream input = new BufferedInputStream(sourceURL.openStream()); BufferedOutputStream output = new BufferedOutputStream( new FileOutputStream(destinationFile)); // The resource pointed to might be a pdf file, which // is binary, so we are careful to read it byte by // byte and not do any conversions of the bytes. int c; while (( c = input.read()) != -1)
output.write(c); output.close(); input.close();} // line 294, ptolemy2’s JNLPUtilities.java
Error Paths - Hazards// Copy to destinationFile without doing any byte conversion.private static void _binaryCopyURLToFile(URL sourceURL, File destinationFile) throws IOException { BufferedInputStream input = new BufferedInputStream(sourceURL.openStream()); BufferedOutputStream output = new BufferedOutputStream( new FileOutputStream(destinationFile)); // The resource pointed to might be a pdf file, which // is binary, so we are careful to read it byte by // byte and not do any conversions of the bytes. int c; while (( c = input.read()) != -1)
output.write(c); output.close(); input.close();} // line 294, ptolemy2’s JNLPUtilities.java
Fix It With Try-Finally// Copy to destinationFile without doing any byte conversion.private static void _binaryCopyURLToFile(…) throws … { BufferedInputStream input; BufferedOutputStream output; input = new BufferedInputStream(…); try { output = new BufferedOutputStream(…); try {
// The resource pointed to might be a pdf file, which// is binary, so we are careful to read it byte by// byte and not do any conversions of the bytes.int c;while (( c = input.read()) != -1) output.write(c);
} finally { output.close(); } } finally { input.close(); } }
Fix It With Run-Time Checks
private static void _binaryCopyURLToFile(…) throws … { BufferedInputStream input = null; BufferedOutputStream output = null; try { input = new BufferedInputStream(sourceURL.openStream()); output = new BufferedOutputStream( new FileOutputStream(destinationFile)); // The resource pointed to might be a pdf file, … int c; while (( c = input.read()) != -1) { output.write(c); } finally { if (input != null) then try { input.close(); } catch (Exception E) { } if (output != null) then try { output.close(); } catch (Exception E) { } } }
Fix It With Flags
int f = 0; // flag tracks progresstry { openX(); f = 1; work(); openY(); f = 2; work(); openZ(); f = 3; work(); } finally { switch (f) { // note fall-through! case 3: try { closeZ(); } catch (Exception e) {} case 2: try { closeY(); } catch (Exception e) {} case 1: try { closeX(); } catch (Exception e) {} }}
Fix It With Flags (Ouch!)
int f = 0; // flag tracks progresstry { openX(); f = 1; work(); if (…) { didY = true; openY(); f = 2; } work(); openZ(); f = 3; work(); } finally { switch (f) { // note fall-through! case 3: try { closeZ(); } catch (Exception e) {} case 2: if (didY) { try { closeY(); } catch … } case 1: try { closeX(); } catch (Exception e) {} }}
Outline
1) Background
2) Bug-Finding Static Analysis
3) Proposed Language Feature
Locating Handling Mistakes
We consider four generic resourcesSockets, files, streams, database locksFrom a survey of Java code
Program should release them along all paths, even those with run-time errors
Run-time errors are rare … So use a static analysis to find
mistakes
Static Dataflow Analysis
Exceptions and run-time errors correlate
Fault model:A called method can terminate normally
or raise any of its declared exceptions Build control-flow graph Symbolically execute each method Abstracting away most data values But tracking outstanding resources
Analysis Example Program
try {Socket s = new Socket();s.send(“GET index.html”);s.close();
} finally { } // bug!
Analysis Example
startnew socket
send
close
end
Analysis Example
startnew socket
send
close
end
{ }
Analysis Example
startnew socket
send
close
end
{ }
{ socket }
Analysis Example
startnew socket
send
close
end
{ }
{ socket }
{ }
Analysis Example
startnew socket
send
close
end
{ }
{ socket }
{ } { socket }
Analysis Example
startnew socket
send
close
end
{ }
{ socket }
{ } { socket }{ socket }
Analysis Example
startnew socket
send
close
end
{ }
{ socket }
{ } { socket }{ socket }
{ }
Analysis Example
startnew socket
send
close
end
{ }
{ socket }
{ } { socket }{ socket }
{ }{ }
Example Bug Report
startnew socket
send
close
end
{ }
{ socket }
{ } { socket }{ socket }
{ }{ }
False Positives
Simple analysis gives false positivesOf 100 error reports70 would be real bugs30 would be false positives
Each false positive is one of these:if (sock != null) sock.close();socket_field_we_close_later = sock;return sock;
Bug Reporting and Filtering
Filter out reports with those 3 formsIntroduces false negativesFor every 100 real bugs reported There are ~10 real bugs we miss
Removes all false positives Manually verify the rest as bugs
Analysis ResultsProgram Name Lines of
CodeMethods withError-Handling
Mistakes
Forgotten Resources
hibernate2 57k 13 DB
jaxme 58k 6 File
axion 65k 15 File
hsqldb 71k 18 DB, Strm
cayenne 86k 7 File
sablecc 99k 3 Strm
jboss 107k 40 DB, Strm
mckoi-sql 118k 37 Strm, DB
portal 162k 39 DB, File
pcgen 178k 17 File
compiere 230k 322 DB
org.aspectj 319k 27 File, Strm
ptolemy2 362k 27 File, Strm
eclipse 1.6M 126 Strm, File
… 13 others … 347k 121 File, DB
Total 3.9M 818 File, DB
Characterizing Mistakes
Model: aX must be followed by cX1) try { a1; } finally { c1; } ; a1 ; c1; 2) try { a1; a2; } finally { c2; c1; } 3) try { a1; a1; } finally { c1; }4) for (…) { a1; work; c1; } As well as tricky flag-work for
releasing resources early …
Outline
1) Background
2) Bug-Finding Static Analysis
3) Proposed Language Feature
Destructors (C++, C#, …)
Great for stack-allocated objects Error-handling contains arbitrary
codeExample adapted from code which
has 17 unique cleanups, one 34 lines long
try { StartDate = new Date(); // a1try { StartLSN = log.getLastLSN();
// a2work(1); try { DB.getWriteLock();
// a3work(2); // a3
} finally { DB.releaseWriteLock();// c3
work(3); }// c3
} finally { StartLSN = -1; } // c2} finally { StartDate = null; } // c1
Finalizers (Java, …)
Called by the garbage collector Too late!
No ordering guarantees Socket sock = new Socket(); Stream strm = new Stream(sock); sock.finalize may be called before
strm.finalize Programs do not use them
Only 13 user-defined ones in 4M LOC Not all SDKs use them for key resources
Feature Motivation
Avoid forgetting obligations No static program restrictions Optional lexical scoping Optional early or arbitrary cleanup Database / Workflow notions:
Either my actions all succeed (a1 a2 a3)
Or they rollback (a1 a2 error c2 c1)
Compensation Stacks
Store cleanup code in run-time stacksFirst-class objects, pass them around
After “action” succeeds, push “cleanup”“action” and “cleanup” are arbitrary code
(anonymous functions) Pop all cleanup code and run it (LIFO)
When the stack goes out of scopeAt an uncaught exceptionEarly, or when the stack is finalized
Compensation Concepts
Generalized destructors No made-up classes for local cleanup Can be called early, automatic
bookkeeping Can have multiple stacks
• e.g., one for each request in a webserver Annotate interfaces to require them
Cannot make a new socket without putting “this.close()” on a stack of obligations
Will be remembered along all paths Details in paper …
Cinderella Story
CompensationStack CS = new CompensationStack();try { Input in; Output out; Input in = new Input(src); Output out = new Output(dst); while (( x = in.read()) != -1) out.write(x); out.close(); in.close(); return 0;} finally { CS.runAll(); }
“Assembly Language”
CompensationStack CS = new CompensationStack();try { Input in; Output out; compensate { in = new Input(src); } with (CS) { in.close(); } compensate { out = new Output(dst); } with (CS) { out.close(); } while (( x = in.read()) != -1) out.write(x); return 0;} finally { CS.runAll(); }
With Annotated Interfaces
CompensationStack CS = new CompensationStack();try {
Input in = new Input(CS,src); Output out = new Output(CS,dst); while (( x = in.read()) != -1) out.write(x); return 0;} finally { CS.runAll(); }
Using Most Recent Stack
CompensationStack CS = new CompensationStack();try {
Input in = new Input(src); Output out = new Output(dst); while (( x = in.read()) != -1) out.write(x); return 0;} finally { CS.runAll(); }
Using “Current Scope Stack”
Input in = new Input(src); Output out = new Output(dst); while (( x = in.read()) != -1) out.write(x); return 0;
Cinderella 2 (Before)
int f = 0; // flag tracks progresstry { openX(); f = 1; work(); if (…) { didY = true; openY(); f = 2; } work(); openZ(); f = 3; work(); } finally { switch (f) { // note fall-through! case 3: try { closeZ(); } catch (Exception e) {} case 2: if (didY) { try { closeY(); } catch … } case 1: try { closeX(); } catch (Exception e) {} }}
Cinderella 2 (After)
compensate { openX(); }with { closeX(); }work(); if (…) compensate { openY(); }
with { closeY(); }work();compensate { openZ(); }with { closeZ(); }work();// using the “current scope stack” by default
Case Studies
Extend Java with compensation stacks
Annotate key interfaces (File, DB, …) Annotate existing programs to use
compensation stacksFor library resourcesAnd for unique cleanup actionsNo new error handling!
Sun’s petstore
“Amazon.com lite” plus inventory Raises 150 exceptions over 3,900
requests Avg Response: 52.06ms (std dev 100ms)
34,608 lines of Java, 123 change sites Two hours of work Three simultaneous resources (DB)
Results: 168 lines shorter (~0.5%) 0 such exceptions over 3,900 requests Avg Response: 43.44ms (std dev 77ms)
Conclusion It is difficult to write error-handling code1) We have an analysis for finding mistakes
Dataflow analysis plus fault model We found 800 mistakes in 4M LOC (Java) We characterize these mistakes
• Programmers forget some paths
2) We propose a new feature Keep stacks of obligations at run-time Existing language features are insufficient
• Timing, bookkeeping, repeated logic … And back it up with case studies
Any Questions?
I encourage difficult questions.
Aspects
We could perhaps have used AoP to do the source-to-source transform
When we started this research, tools (e.g., AspectJ) weren’t well-suited for adding, removing and editing entire lexical scoping levels
Aspect tools also seemed like the wrong framework for a path-sensitive dataflow analysis
However, using the results of this analysis you could easily roll your own “compensation stack”-like mechanism with AoP
Do Developers Care?
Beyond the scope of this work, but … ptolemy2 Developer Rating of Reported
Bugs: 11% “in tutorials or third-party code”44% 3/5 “little used or experimental
code”19% 4/5 “definitely a bug in code that is
used more often”26% 5/5 “definitely a bug in code that is
used often”