Upload File

automatically repairing test cases for evolving method declarations

  • Home
  • Documents
  • Automatically Repairing Test Cases for Evolving Method Declarations
49
Automatically Repairing Test Cases for Evolving Method Declarations Mehdi Mirzaaghaei Fabrizio Pastore Mauro Pezzè Università della Svizzera italiana http://swiss-landmarks.ch/panos/Lugano9.jpg

Upload: icsm-2010

Post on 17-May-2015

840 views

Category:

Documents


1 download

Report

Tags:

TRANSCRIPT

Page 1: Automatically Repairing Test Cases for Evolving Method Declarations

Automatically Repairing Test Cases

for Evolving Method DeclarationsMehdi MirzaaghaeiFabrizio Pastore

Mauro Pezzè

Universitàdella Svizzeraitaliana

http://swiss-landmarks.ch/panos/Lugano9.jpg

http://swiss-landmarks.ch/panos/Lugano9.jpg
http://swiss-landmarks.ch/panos/Lugano9.jpg
Page 2: Automatically Repairing Test Cases for Evolving Method Declarations

Software evolves

Page 3: Automatically Repairing Test Cases for Evolving Method Declarations

public class BankAccount { private int balance;

public void deposit(int cents){ balance += cents; }

public int getBalance(){ return balance };}

Software evolves

Page 4: Automatically Repairing Test Cases for Evolving Method Declarations

testDeposit(){ BankAccount account = new BankAccount(); int amount = 500; account.deposit(amount); assertEquals( 500, account.getBalance());}

public class BankAccount { private int balance;

public void deposit(int cents){ balance += cents; }

public int getBalance(){ return balance };}

Software evolves

Page 5: Automatically Repairing Test Cases for Evolving Method Declarations

testDeposit(){ BankAccount account = new BankAccount(); int amount = 500; account.deposit(amount); assertEquals( 500, account.getBalance());}

public class BankAccount { private int balance;

public void deposit(int cents){ balance += cents; }

public int getBalance(){ return balance };}

Software evolves

Page 6: Automatically Repairing Test Cases for Evolving Method Declarations

public class .....

public class AccountUtil { private double dailyInterestRate = 0.00005; private int interestTerm=365; public double interest(BankAccount account return account.getBalance() *dailyInterestRate*interestTerm; } ...

public class AccountFactory {

public static BankAccount create( AccountContext ctx, boolean special ){ BankAccount account = new BankAcccount( ctx.amount ); if ( special ){ account.setInterestRate(0.0001); } ...

public class .....

testDeposit(){ BankAccount account = new BankAccount(); int amount = 500; account.deposit(amount); assertEquals( 500, account.getBalance());}

public class BankAccount { private int balance;

public void deposit(int cents){ balance += cents; }

public int getBalance(){ return balance };}

Software evolves

Page 7: Automatically Repairing Test Cases for Evolving Method Declarations

public class BankAccount { private int balance;

public void deposit(int cents){ balance += cents; }

public int getBalance(){ return balance };}

public class BankAccount { private int balance;

public void deposit(Money money){ balance += money.centsValue; }

public int getBalance(){ return balance; };}

Original classes are modified

Page 8: Automatically Repairing Test Cases for Evolving Method Declarations

public class .....

public class .....

Test cases need maintenance

testDeposit(){ BankAccount account = new BankAccount(); int amount = 500; account.deposit(amount); assertEquals( 500, account.getBalance());}

public class BankAccount { private int balance;

public void deposit(int cents){ balance += cents; }

public int getBalance(){ return balance };}

public class BankAccount { private int balance;

public void deposit(Money money){ balance += money.centsValue; }

public int getBalance(){ return balance; };}

Compilation error:Method deposit(Money) in the type BankAccount is not applicable for the arguments (int)

Page 9: Automatically Repairing Test Cases for Evolving Method Declarations

changes in method declarations: 23% changes

that impact on compilation

Page 10: Automatically Repairing Test Cases for Evolving Method Declarations

changes in method declarations: 23% changes

that impact on compilation

> 80% in maintenance releases

Page 11: Automatically Repairing Test Cases for Evolving Method Declarations

changes in method declarations: 23% changes

that impact on compilation

> 80% in maintenance releases

Automatic Repair to Save Effort

Page 12: Automatically Repairing Test Cases for Evolving Method Declarations

Automatic Test Repair

Page 13: Automatically Repairing Test Cases for Evolving Method Declarations

[Memon et. al. 2008, Grechanik 2009 ]Repair GUI tests

Well suited only for GUI tests

Automatic Test Repair

Page 14: Automatically Repairing Test Cases for Evolving Method Declarations

[Memon et. al. 2008, Grechanik 2009 ]Repair GUI tests Repair Oracles

[B. Daniel et al 2010]

testInterest(){ ... assertEquals( 50, result );} FAILURE: expected 50, found: 40

assertEquals( 40, result );

Well suited only for GUI tests Focus on oracles only

Automatic Test Repair

Page 15: Automatically Repairing Test Cases for Evolving Method Declarations

[Memon et. al. 2008, Grechanik 2009 ]Repair GUI tests

Refactoring Techniques [ReBa, Eclipse]

Repair Oracles[B. Daniel et al 2010]

testInterest(){ ... assertEquals( 50, result );} FAILURE: expected 50, found: 40

assertEquals( 40, result );

//calculate one year interestresult = account.interest();

result = account.interest( 0 );

Well suited only for GUI tests Focus on oracles only

Prevent only some compilation errors

Automatic Test Repair

Page 16: Automatically Repairing Test Cases for Evolving Method Declarations

TestCareAssistant repairs test case compilation errors

• Parameter add

• Parameter type change • Return type change

• Parameter remove

caused by method declaration changes

Page 17: Automatically Repairing Test Cases for Evolving Method Declarations

TestCareAssistant repairs test case compilation errors

testDeposit(){BankAccount account = new BankAccount();int amount = 500;account.deposit(amount); assertEquals(500, account.getBalance());

• Parameter add

• Parameter type change • Return type change

• Parameter remove

caused by method declaration changes

Page 18: Automatically Repairing Test Cases for Evolving Method Declarations

TestCareAssistant repairs test case compilation errors

testDeposit(){BankAccount account = new BankAccount();int amount = 500;account.deposit(amount); assertEquals(500, account.getBalance());

Replace: int amount = 500;with: Money amount = new Money(500); account.deposit(amount);

• Parameter add

• Parameter type change • Return type change

• Parameter remove

caused by method declaration changes

Page 19: Automatically Repairing Test Cases for Evolving Method Declarations

TestCareAssistant repairs test case compilation errors

testDeposit(){BankAccount account = new BankAccount();int amount = 500;account.deposit(amount); assertEquals(500, account.getBalance());

testDeposit(){BankAccount account = new BankAccount();Money amount = new Money(500);account.deposit(amount); assertEquals(500, account.getBalance());

Replace: int amount = 500;with: Money amount = new Money(500); account.deposit(amount);

• Parameter add

• Parameter type change • Return type change

• Parameter remove

caused by method declaration changes

Page 20: Automatically Repairing Test Cases for Evolving Method Declarations

TestCareAssistant repairs test case compilation errors

testDeposit(){BankAccount account = new BankAccount();int amount = 500;account.deposit(amount); assertEquals(500, account.getBalance());

testDeposit(){BankAccount account = new BankAccount();Money amount = new Money(500);account.deposit(amount); assertEquals(500, account.getBalance());

Replace: int amount = 500;with: Money amount = new Money(500); account.deposit(amount);

• Parameter add

• Parameter type change • Return type change

• Parameter remove

caused by method declaration changes

Page 21: Automatically Repairing Test Cases for Evolving Method Declarations

TestCareAssistant repairs test case compilation errors

testDeposit(){BankAccount account = new BankAccount();int amount = 500;account.deposit(amount); assertEquals(500, account.getBalance());

testDeposit(){BankAccount account = new BankAccount();Money amount = new Money(500);account.deposit(amount); assertEquals(500, account.getBalance());

Replace: int amount = 500;with: Money amount = new Money(500); account.deposit(amount);

• Parameter add

• Parameter type change • Return type change

• Parameter remove

caused by method declaration changes

Page 22: Automatically Repairing Test Cases for Evolving Method Declarations

TestCareAssistant repairs test case compilation errors

testDeposit(){BankAccount account = new BankAccount();int amount = 500;account.deposit(amount); assertEquals(500, account.getBalance());

testDeposit(){BankAccount account = new BankAccount();Money amount = new Money(500);account.deposit(amount); assertEquals(500, account.getBalance());

Replace: int amount = 500;with: Money amount = new Money(500); account.deposit(amount);

• Parameter add

• Parameter type change • Return type change

• Parameter remove

caused by method declaration changes

Page 23: Automatically Repairing Test Cases for Evolving Method Declarations

Repair parameter type change

public class BankAccount { private int balance;

public void deposit(int cents){ balance += cents; }

public int getBalance(){ return balance; }}

public class BankAccount { private int balance;

public void deposit(Money money){ balance += money.CentsValue; }

public int getBalance(){ return balance; }}

testDeposit(){ BankAccount account = new BankAccount(); int amount = 500; account.deposit(amount); assertEquals( 500, account.getBalance());}

Version 0 Version 1

Page 24: Automatically Repairing Test Cases for Evolving Method Declarations

Which parameter to initialize?

public class BankAccount { private int balance;

public void deposit(int cents){ balance += cents; }

public int getBalance(){ return balance; }}

public class BankAccount { private int balance;

public void deposit(Money money){ balance += money.CentsValue; }

public int getBalance(){ return balance; }}

testDeposit(){ BankAccount account = new BankAccount(); int amount = 500; account.deposit(amount); assertEquals( 500, account.getBalance());}

Version 0 Version 1

Page 25: Automatically Repairing Test Cases for Evolving Method Declarations

Which parameter to initialize?

Use code diff to identify the parameter1

public class BankAccount { private int balance;

public void deposit(int cents){ balance += cents; }

public int getBalance(){ return balance; }}

public class BankAccount { private int balance;

public void deposit(Money money){ balance += money.CentsValue; }

public int getBalance(){ return balance; }}

testDeposit(){ BankAccount account = new BankAccount(); int amount = 500; account.deposit(amount); assertEquals( 500, account.getBalance());}

Version 0 Version 1

Page 26: Automatically Repairing Test Cases for Evolving Method Declarations

How to initialize a complex object?

public class BankAccount { private int balance;

public void deposit(int cents){ balance += cents; }

public int getBalance(){ return balance; }}

public class BankAccount { private int balance;

public void deposit(Money money){ balance += money.centsValue; }

public int getBalance(){ return balance; }}

testDeposit(){ BankAccount account = new BankAccount(); int amount = 500; account.deposit(amount); assertEquals( 500, account.getBalance());}

Version 0 Version 1

Page 27: Automatically Repairing Test Cases for Evolving Method Declarations

How to initialize a complex object?

Find first uses of parameter fields in Version12

public class BankAccount { private int balance;

public void deposit(int cents){ balance += cents; }

public int getBalance(){ return balance; }}

public class BankAccount { private int balance;

public void deposit(Money money){ balance += money.centsValue; }

public int getBalance(){ return balance; }}

testDeposit(){ BankAccount account = new BankAccount(); int amount = 500; account.deposit(amount); assertEquals( 500, account.getBalance());}

Version 0 Version 1

Page 28: Automatically Repairing Test Cases for Evolving Method Declarations

How to determine fields values?

public class BankAccount { private int balance;

public void deposit(int cents){ balance += cents; }

public int getBalance(){ return balance; }}

public class BankAccount { private int balance;

public void deposit(Money money){ balance += money.centsValue; }

public int getBalance(){ return balance; }}

testDeposit(){ BankAccount account = new BankAccount(); int amount = 500; account.deposit(amount); assertEquals( 500, account.getBalance());}

Version 0 Version 1

Page 29: Automatically Repairing Test Cases for Evolving Method Declarations

How to determine fields values?

Diff to identify corresponding variable in Version 03

public class BankAccount { private int balance;

public void deposit(int cents){ balance += cents; }

public int getBalance(){ return balance; }}

public class BankAccount { private int balance;

public void deposit(Money money){ balance += money.centsValue; }

public int getBalance(){ return balance; }}

testDeposit(){ BankAccount account = new BankAccount(); int amount = 500; account.deposit(amount); assertEquals( 500, account.getBalance());}

Version 0 Version 1

Page 30: Automatically Repairing Test Cases for Evolving Method Declarations

What was the original value?

public class BankAccount { private int balance;

public void deposit(int cents){ balance += cents; }

public int getBalance(){ return balance; }}

public class BankAccount { private int balance;

public void deposit(Money money){ balance += money.CentsValue; }

public int getBalance(){ return balance; }}

testDeposit(){ BankAccount account = new BankAccount(); int amount = 500; account.deposit(amount); assertEquals( 500, account.getBalance());}

Version 0 Version 1

Page 31: Automatically Repairing Test Cases for Evolving Method Declarations

What was the original value?

Analyze the def-use chain of the variable back to the definition in the test4

public class BankAccount { private int balance;

public void deposit(int cents){ balance += cents; }

public int getBalance(){ return balance; }}

public class BankAccount { private int balance;

public void deposit(Money money){ balance += money.CentsValue; }

public int getBalance(){ return balance; }}

testDeposit(){ BankAccount account = new BankAccount(); int amount = 500; account.deposit(amount); assertEquals( 500, account.getBalance());}

Version 0 Version 1

Page 32: Automatically Repairing Test Cases for Evolving Method Declarations

What was the original value?

Analyze the def-use chain of the variable back to the definition in the test4

public class BankAccount { private int balance;

public void deposit(int cents){ balance += cents; }

public int getBalance(){ return balance; }}

public class BankAccount { private int balance;

public void deposit(Money money){ balance += money.CentsValue; }

public int getBalance(){ return balance; }}

testDeposit(){ BankAccount account = new BankAccount(); int amount = 500; account.deposit(amount); assertEquals( 500, account.getBalance());}

Version 0 Version 1

Page 33: Automatically Repairing Test Cases for Evolving Method Declarations

How can we repair the test?

public class BankAccount { private int balance;

public void deposit(int cents){ balance += cents; }

public int getBalance(){ return balance; }}

public class BankAccount { private int balance;

public void deposit(Money money){ balance += money.centsValue; }

public int getBalance(){ return balance; }}

testDeposit(){ BankAccount account = new BankAccount(); int amount = 500; account.deposit(amount); assertEquals( 500, account.getBalance());}

Version 0 Version 1

Page 34: Automatically Repairing Test Cases for Evolving Method Declarations

How can we repair the test?

public class BankAccount { private int balance;

public void deposit(int cents){ balance += cents; }

public int getBalance(){ return balance; }}

public class BankAccount { private int balance;

public void deposit(Money money){ balance += money.centsValue; }

public int getBalance(){ return balance; }}

testDeposit(){ BankAccount account = new BankAccount(); int amount = 500; account.deposit(amount); assertEquals( 500, account.getBalance());}

Version 0 Version 1

Page 35: Automatically Repairing Test Cases for Evolving Method Declarations

How can we repair the test?

Instantiate the new object, use original values to initialize fields5

public class BankAccount { private int balance;

public void deposit(int cents){ balance += cents; }

public int getBalance(){ return balance; }}

public class BankAccount { private int balance;

public void deposit(Money money){ balance += money.centsValue; }

public int getBalance(){ return balance; }}

testDeposit(){ BankAccount account = new BankAccount(); int amount = 500; account.deposit(amount); assertEquals( 500, account.getBalance());}

Version 0 Version 1

Page 36: Automatically Repairing Test Cases for Evolving Method Declarations

How can we repair the test?

Instantiate the new object, use original values to initialize fields5

Replace: int amount = 500;with: Money amount = new Money(500); account.deposit(amount);

public class BankAccount { private int balance;

public void deposit(int cents){ balance += cents; }

public int getBalance(){ return balance; }}

public class BankAccount { private int balance;

public void deposit(Money money){ balance += money.centsValue; }

public int getBalance(){ return balance; }}

testDeposit(){ BankAccount account = new BankAccount(); int amount = 500; account.deposit(amount); assertEquals( 500, account.getBalance());}

Version 0 Version 1

Page 37: Automatically Repairing Test Cases for Evolving Method Declarations

To what extent TestCareAssistant can fix test cases automatically?

Page 38: Automatically Repairing Test Cases for Evolving Method Declarations

To what extent TestCareAssistant can fix test cases automatically?

• Repair 22 test cases of 6 open source systems

• Continuum, Geronimo, xml-security, PMD, POI, Shindig

• 24 compilation errors caused by different changes

• 9 parameter types changed, 8 parameters added, 3 parameters removed, 4 return types changed

Page 39: Automatically Repairing Test Cases for Evolving Method Declarations

To what extent TestCareAssistant can fix test cases automatically?

Results Total Repaired %Test Cases 22 16 72

Compilation Errors 24 18 75

Values Initialized 36 29 80

• Repair 22 test cases of 6 open source systems

• Continuum, Geronimo, xml-security, PMD, POI, Shindig

• 24 compilation errors caused by different changes

• 9 parameter types changed, 8 parameters added, 3 parameters removed, 4 return types changed

Page 40: Automatically Repairing Test Cases for Evolving Method Declarations

To what extent TestCareAssistant can fix test cases automatically?

Results Total Repaired %Test Cases 22 16 72

Compilation Errors 24 18 75

Values Initialized 36 29 80

• Repair 22 test cases of 6 open source systems

• Continuum, Geronimo, xml-security, PMD, POI, Shindig

• 24 compilation errors caused by different changes

• 9 parameter types changed, 8 parameters added, 3 parameters removed, 4 return types changed

Page 41: Automatically Repairing Test Cases for Evolving Method Declarations

To what extent TestCareAssistant can fix test cases automatically?

• Static data flow analysis not always effective

• Use of complex data structures, e.g. hash tables

• Changes in method logic

• Changes in interfaces

Results Total Repaired %Test Cases 22 16 72

Compilation Errors 24 18 75

Page 42: Automatically Repairing Test Cases for Evolving Method Declarations

Conclusionspublic class BankAccount { public void deposit(int cents){ balance += cents; }}

public class BankAccount { public void deposit(Money money){ balance += money.CentsValue; }}

testDeposit(){BankAccount account = new BankAccount();int amount = 500;account.deposit(amount); assertEquals(500, account.getBalance());

testDeposit(){BankAccount account = new BankAccount();int amount = 500;account.deposit(amount); assertEquals(500, account.getBalance());

Page 43: Automatically Repairing Test Cases for Evolving Method Declarations

Conclusionspublic class BankAccount { public void deposit(int cents){ balance += cents; }}

public class BankAccount { public void deposit(Money money){ balance += money.CentsValue; }}

testDeposit(){BankAccount account = new BankAccount();int amount = 500;account.deposit(amount); assertEquals(500, account.getBalance());

testDeposit(){BankAccount account = new BankAccount();Money amount = new Money(500);account.deposit(amount); assertEquals(500, account.getBalance());

Page 44: Automatically Repairing Test Cases for Evolving Method Declarations

Conclusionspublic class BankAccount { public void deposit(int cents){ balance += cents; }}

public class BankAccount { public void deposit(Money money){ balance += money.CentsValue; }}

Replace: int amount = 500;with: Money amount = new Money(500); account.deposit(amount);

testDeposit(){BankAccount account = new BankAccount();int amount = 500;account.deposit(amount); assertEquals(500, account.getBalance());

testDeposit(){BankAccount account = new BankAccount();Money amount = new Money(500);account.deposit(amount); assertEquals(500, account.getBalance());

TestCareAssistant

Page 45: Automatically Repairing Test Cases for Evolving Method Declarations

Future Work

CopticChronology EthiopicChronology

Chronology + withUTC()

TestCopticUTC(){ .... assertEquals("AM",

copticChronology.withUTC() );}

Page 46: Automatically Repairing Test Cases for Evolving Method Declarations

Future Work

CopticChronology EthiopicChronology

Chronology + withUTC()

TestCopticUTC(){ .... assertEquals("AM",

copticChronology.withUTC() );}

TestEthiopicUTC(){ .... assertEquals("EE",

ethiopicChronology.withUTC() );}

CopticChronology EthiopicChronology

Chronology + withUTC()

Page 47: Automatically Repairing Test Cases for Evolving Method Declarations

Future Work

CopticChronology EthiopicChronology

Chronology + withUTC()

TestCopticUTC(){ .... assertEquals("AM",

copticChronology.withUTC() );}

TestEthiopicUTC(){ .... assertEquals("EE",

ethiopicChronology.withUTC() );}

CopticChronology EthiopicChronology

Chronology + withUTC()

Page 48: Automatically Repairing Test Cases for Evolving Method Declarations

Future Work

CopticChronology EthiopicChronology

Chronology + withUTC()

TestCopticUTC(){ .... assertEquals("AM",

copticChronology.withUTC() );}

TestEthiopicUTC(){ .... assertEquals("EE",

ethiopicChronology.withUTC() );}

CopticChronology EthiopicChronology

Chronology + withUTC()

Automate Test Generation by Identifying and Implementing Test Adaptation Patterns

Page 49: Automatically Repairing Test Cases for Evolving Method Declarations

Questions?public class BankAccount { public void deposit(int cents){ balance += cents; }}

public class BankAccount { public void deposit(Money money){ balance += money.CentsValue; }}

Replace: int amount = 500;with: Money amount = new Money(500); account.deposit(amount);

testDeposit(){BankAccount account = new BankAccount();int amount = 500;account.deposit(amount); assertEquals(500, account.getBalance());

testDeposit(){BankAccount account = new BankAccount();Money amount = new Money(500);account.deposit(amount); assertEquals(500, account.getBalance());

TestCareAssistant


Repairing Building

Alt repairing

MPORT DECLARATIONS OVERVIEW

Automatically Repairing Broken Workflows for Evolving GUI Applications

Repairing help

DECLARATIONS - Air Sailing

Scriptural Declarations TRUTH

ABAP Data Declarations

SERVICING, REPAIRING, INSTALLATION AND PREVENTATIVE ... · dcam update statement, ... repairing, installation and preventative maintenance of ... servicing, repairing, installation

Repairing Relationships

Automatically Repairing Broken Workflows for Evolving GUI Applications Sai Zhang University of Washington Joint work with: Hao Lü, Michael D. Ernst

Declarations Rasc Off

Paie Et Declarations

ML Declarations.1 Standard ML Declarations. ML Declarations.2 Declarations o Re-declaration of names in ML o The reserved words "val rec" o Pattern Matching

Declarations En

Declarations Contents - GOV.UK

Block Repairing

Speaker Declarations

Environmental product declarations

Repairing u.g.tank

Management System Declarations

31 Declarations

Armitage Declarations

automobile repairing

African Meditation Drums - MACMH · 2019. 5. 23. · RHYTHMIC NATURE: REPAIRING AND ENHANCING RELATIONSHIP “The evolving dance games focus on rhythm, timing, space and touch. These

Cracks Repairing

Automatically Repairing Broken Workflows for Evolving GUI …mernst/pubs/repair-workflow-isst… · Abroken workflow. Outline • Problem • Technique • Evaluation • Related

Declarations, Schedule, Policy Declarations CLASS LIMIT OF

DECLARATIONS - repository.unam.edu.na

Quantative Ingredients Declarations

Ship Repairing

Repairing Reptures

C++ Forward Declarations

Tribal Declarations Pilot

Mobile Repairing