april 29, 2005 gcuu csc-299 graduate seminar 1 of 25 application reliability robert j. conti

April 29, 2005April 29, 2005 GCUU CSc-299 Graduate SeminarGCUU CSc-299 Graduate Seminar 11 of 25 of 25

Application ReliabilityApplication ReliabilityRobert J. ContiRobert J. Conti

22 of 25 of 25April 29, 2005April 29, 2005 GCUU CSc-299 Graduate SeminarGCUU CSc-299 Graduate Seminar

Prelude…Prelude…

If you thought designing and coding that program If you thought designing and coding that program was hard, you ain’t seen nothing yetwas hard, you ain’t seen nothing yet..

- GJ Myers, 1979GJ Myers, 1979

Testing of Java Web Services for RobustnessTesting of Java Web Services for Robustness. . Chen Fu, Barbara G. Ryder, Ana Milanova, & David Wonnacott. Chen Fu, Barbara G. Ryder, Ana Milanova, & David Wonnacott. ISSTA 2004.ISSTA 2004.


Today’s Agenda…Today’s Agenda…

IntroductionIntroductionA Step Back – What’s Def-Use Testing?A Step Back – What’s Def-Use Testing?Application ReliabilityApplication ReliabilityConcluding RemarksConcluding RemarksQ & AQ & A


Introduction – Drilling Down…Introduction – Drilling Down…

IOEx

Java Web

Services

Web Services

Application Reliability


Introduction - Java Exceptions…Introduction - Java Exceptions…

exceptionexception

throwthrow

trytry

catchcatch

void method1( ) {

... try {

method2(“a:\\file.txt”); }catch (IOException e) {

e.printStackTrace(System.out); }...

}

void method2(String file) throws IOException {

... InputStream in = new FileInputStream(file);int c = in.read( );...

}


A Step Back – What’s def-use testing?A Step Back – What’s def-use testing?

Black-box testingBlack-box testing

White-box testingWhite-box testing Control-flowControl-flow

e.g., statemente.g., statement Data-flowData-flow

e.g., def-usee.g., def-use



Def-useDef-use3 occurrences3 occurrences defdef c-usec-use p-usep-use



Program graphProgram graph defdef c-usec-use p-usep-use



New family of path New family of path selection criteriaselection criteria

Bridges the gapBridges the gap

Why do we care?Why do we care?


Application Reliability - Metrics…Application Reliability - Metrics…

How can you gauge reliability?How can you gauge reliability? By using e-c links and test set TBy using e-c links and test set T Determine set of faults (i.e., Determine set of faults (i.e., IOExceptionsIOExceptions)) Determine F Determine F { set of e-c links } { set of e-c links } Determine E Determine E { set of e-c links covered by { set of e-c links covered by

T }T }

Overall Exception Def-Catch Coverage = E / FOverall Exception Def-Catch Coverage = E / F


Application Reliability - Metrics…Application Reliability - Metrics…

The big pictureThe big picture

F

E

|E| / |F|

Reliability Faults

Web Service


Application Reliability - Mendosus…Application Reliability - Mendosus…

Mendosus is a fault injection engineMendosus is a fault injection engine trytry F – insert methods to request fault F – insert methods to request fault catchcatch F – insert method to record call F – insert method to record call

stack (for E) and to cancel faultstack (for E) and to cancel fault Only one fault will be tested at a timeOnly one fault will be tested at a time The facilitator behind this is the e-c link The facilitator behind this is the e-c link

analysisanalysis


Application Reliability - Mendosus…Application Reliability - Mendosus…


Part I

Part II

Application Reliability – e-c Link Analysis…Application Reliability – e-c Link Analysis…

Part I – Exception flow analysisPart I – Exception flow analysis ASTAST Call graphCall graph Compiler dataflow Compiler dataflow

equationsequations Defines e-c linksDefines e-c links Imprecise due to Imprecise due to

infeasible link infeasible link generation!generation!



Infeasible linksInfeasible links readFile( )readFile( ) readNet( )readNet( )

BufferInputStream

.fill()



Infeasible links Infeasible links (cont’d)(cont’d)

Twice the e-c Twice the e-c linkslinks

How to fix?How to fix?

readFile() readNet()

FilterInputStream.read()

BufferedInputStream.read()

BufferInputStream.fill()

FileInputStream.read() SocketInputStream.read()


Part I

Part II


Part II – Data reach analysisPart II – Data reach analysis Post pass filterPost pass filter Call graphCall graph Points-to graphPoints-to graph Removes Removes

infeasible links!infeasible links!


Application Reliability – Empirical Results…Application Reliability – Empirical Results…

FTPDFTPD

JNFSJNFS

HaboobHaboob

MuffinMuffin

Web Service Application Benchmarks



Java Web Service (Benchmark)

Configuration

E-C Link AnalysisReliability Faults

(IOEx)

The setupThe setup

E/F



Configuration Part IException-Flow

Part IIData Reach Inlining

CHA

RTA

PTA

InPTA

PTA-DR

InPTA-DR



e-c Links and Coverage

Program InPTA-DR

FTPD 85%

JNFS 84%

Haboob 83%

Muffin 83%

Overall Exception Def-Catch Coverage


Concluding Remarks…Concluding Remarks…

SummarySummary Java web services and Java web services and IOExceptionIOExceptionss White-box data-flow def-use testingWhite-box data-flow def-use testing Exception flow def-useException flow def-use Overall Exception Def-Catch CoverageOverall Exception Def-Catch Coverage MendosusMendosus e-c link analysis (E-F and DR)e-c link analysis (E-F and DR) CoverageCoverage


Concluding Remarks…Concluding Remarks…

ConclusionsConclusions Telephone SystemTelephone System Outside the boxOutside the box Compiler techniquesCompiler techniques Other Java programsOther Java programs To be continued…To be continued…


?Questions & Answers…Questions & Answers…

april 29, 2005 gcuu csc-299 graduate seminar 1 of 25 application reliability robert j. conti

Documents