Mining Test Oracles for Test Inputs Generated from Java Bytecode

Frank Xu, Ph.D.Gannon UniversityMining Decision Trees as Test Oracles for Java BytecodeXu, W., Ding, T., Wang, H., Xu. D., Mining Test Oracles for Test Inputs Generated from Java Bytecode, Proc. of the 37th Annual International Computer Software & Applications Conference, pp. 27-32, Kyoto, Japan, July 2013 Mining Decision Trees as Test Oracles for Java Bytecode (Extended version of conference paper), Accepted by Journal of Systems and Software

Bio Frank XuEducationPh.D. in Software Engineering North Dakota State UniversityM.S. in Computer Science Towson University B.S. in Computer ScienceSoutheast Missouri State University

Working ExperienceGE Transportation, 2008- present, Consultant of Locomotive Remote Diagnostics Service CenterGannon University, 2008- present, Assistant Professor of Software Engineering, Director of Keystone Software Development Institute, University VA Wise, 2007- 2008, Assistant Professor of Software EngineeringSwanson Health Products, 2005 ~ 2007, Sr. Programmer Analyst Volt Information Science Inc., 2004 ~ 2005, Software Engineer

2TeachingSource: Student Evaluation Report

ResearchSource: Google scholar: http://scholar.google.com/citations?user=9_I4ZUgAAAAJ&hl=en

Mining Decision Trees as Test Oracles Introduction Running ExampleTest Input GenerationModel Miner Empirical StudyRelated WorkConclusions

IntroductionExerciseImplementing a method to solve Triangle problem

What is Triangle Problem?

How to test Triangle?String getTriangleType (int a, int b, int c){if((a s1) = false(s0[k] ,s2) (s0, s1[l] ) Model Miner

IDabc1777211733811194117352222963043307221313813162013333345145230521531474716272822

Jimple Predicates and Attributes of Triangle ProgramJimple PredicateAttribute of UUTi0 > = $i3a > = b + ci1 > = $i4b > = a + ci2 > = $i5c > = a + bi0 != i1a != bi1 != i2b != ci0 == i1a = bi0 == i2a = ci1 == i2b = c

For a given test input generated by rule-based method, predicates produce a set of T or F values Input(a=7.b=7,v=7)ffffftttCovert Test Inputs Using AttributesTest input IDabca1a2a3a4a5a6a7a8O1777fffffttt121173ftfttfff4381119fftttfff441173tffttfff413333345fffttftf214523052fffttfft215314747ffftffff316272822tffttfff4C4.5 mining algorithm The key idea of the algorithm is to calculate the highest normalized information gain of attributes and then build a decisionnodethat splits onthe attributesToolWeka 3: http://www.cs.waikato.ac.nz/ml/weka/

IDabca1a2a3a4a5a6a7a8o1777fffffttt121173ftfttfff4381119fftttfff441173tffttfff413333345fffttftf214523052fffttfft215314747ffftffff316272822tffttfff4

Empirical StudyThree Study SubjectsLine of CodeNumber of PredicatesJavaJimpleJavaJimple (Allow duplications)Attributes (No duplication) Triangle2227388Next Date485191919Vending Machine8268214110Goal of Empirical StudiesMeasure fault detection capability # mutants killed /#mutants *100%Measure fault detection capability: ProcessStep 1: Implant mutantsStep 2: Build a decision tree modelStep 3: Find mismatchesFind possible causes Step 4: Calculate fault detectability

Mutation OperatorExamplesCategoryIDTypeOriginalReplacedArithmetic Operations1Arithmetic Operator Replacementa + ba - b2Arithmetic Operator Insertionb + c-b + cRelations3Relational Operator Replacementa != ba == bConditions4Conditional Operator Replacement(a==b) && (b==c)(a==b) || (b==c)Constants5Constant Value Modifications = as = bReturn Values6Return Value Modificationreturn sreturn s

Insert bugFaulty versionFind mismatchesTwo possible causesFound bugsassertEquals(Equilateral, new Trianlge(7,7,7).getTriType()) Model is not correctassertEquals (Isosceles, new Trianlge(7,7,7).getTriType())

Create mutant and insert mutants30ID# of Mutants # of Tests ExecutedOracle Results# Mutants Discovered# Faults in ModelsSDUSDUSDUTriangle Problem134443330002416161633311133555333000416661110005366633300063444333000Next Date Problem1313151523310026273737466200342219244340104326182732301052666222000Vending Machine23303030333000344545454440005439393944400064414141444000Total508861428Related WorkLo et al. (Lo, Cheng, Han, Khoo, & Sun, 2009), Milea et al. (Milea, Khoo, Lo, & Pop, 2012) mines a set of discriminative features capturing repetitive series of events from program execution traces. These features are then used to train a classier to detect failures. Bowring et al. (Bowring, Rehg, & Harrold, 2004) models program executions as Markov models, and a clustering method for Markov models that aggregates multiple program executions into effective behavior classifiers. (Pacheco & Ernst, 2005) Pacheco and Ernst build an operational model from observations of the software running properly. The operation model includes object invariants and properties. The object invariants are the conditions hold on entry and exit of all public methods. Our approach generates and classifies inputs based on the internal structure of the UUT. Briand (Briand, 2008) has proposed the use of machine learning techniques - including decision trees - for the test oracle problem. The decision tree model he has proposed is manually built from software requirements. ConclusionsThe first attempt to mine decision tree models from auto-generated test inputs based on static analysis of Java bytecode Our empirical study indicates that using the mined test oracles, average 94.67% mutants are killed by the generated test inputs.

ThanksFuture Research DirectionRequirements Engineering & Natural language ProcessGenerating UML diagrams, e.g., Use case, Class diagramValidating SRSDeriving test cases from SRSSoftware Design & Social Networks AnalysisUtilizing SSA for analyzing communication diagram, class diagram, and sequence diagram for improving the quality of the softwareSoftware Implementation & Big DataMining repository for software quality assurance using HadoopSoftware Testing & Mobile/Cloud ApplicationTesting mobile applications and distributed applications

Build Variable Dependency Tree (VDT)