combining rule-based and information retrieval techniques to assign software change requests...
TRANSCRIPT
![Page 1: Combining Rule-based and Information Retrieval Techniques to assign Software Change Requests (ASE'2014)](https://reader038.vdocuments.net/reader038/viewer/2022110310/55a6ae081a28ab48418b4608/html5/thumbnails/1.jpg)
Combining Rule-based and InformationRetrieval Techniques to assign Software
Change RequestsNew Idea
Yguarata Cerqueira CavalcantiIvan do Carmo Machado
Paulo A. da Mota S. NetoEduardo Santana de Almeida
Silvio Romero de Lemos Meira
Center for Informatics – UFPE, Brazil
ASE 2014 (Vasteras – Sweden)September 18, 2014
![Page 2: Combining Rule-based and Information Retrieval Techniques to assign Software Change Requests (ASE'2014)](https://reader038.vdocuments.net/reader038/viewer/2022110310/55a6ae081a28ab48418b4608/html5/thumbnails/2.jpg)
Agenda
1 Introduction
2 Proposal
3 Experiment
4 Concluding remarks
![Page 3: Combining Rule-based and Information Retrieval Techniques to assign Software Change Requests (ASE'2014)](https://reader038.vdocuments.net/reader038/viewer/2022110310/55a6ae081a28ab48418b4608/html5/thumbnails/3.jpg)
1 Introduction
2 Proposal
3 Experiment
4 Concluding remarks
![Page 4: Combining Rule-based and Information Retrieval Techniques to assign Software Change Requests (ASE'2014)](https://reader038.vdocuments.net/reader038/viewer/2022110310/55a6ae081a28ab48418b4608/html5/thumbnails/4.jpg)
Outline Introduction Proposal Experiment Concluding remarks References
About Change Requests (CRs)
Change Request (CR) describes a defect to be fixed, an adaptivechange, or a new functionality.
CRs are stored and managed through CR Repositories.
1/14 Yguarata Cerqueira Cavalcanti Combining Rule-based and Information Retrieval Techniques to assign Software Change Requests
![Page 5: Combining Rule-based and Information Retrieval Techniques to assign Software Change Requests (ASE'2014)](https://reader038.vdocuments.net/reader038/viewer/2022110310/55a6ae081a28ab48418b4608/html5/thumbnails/5.jpg)
Outline Introduction Proposal Experiment Concluding remarks References
CR Assignment
CR Assignment consists of matching available developers withreported CRs, considering low fixing time and highquality.
Why CR Assignment Matters?
Dozens to hundreds CRs daily [1]
Needs good knowledge on the project [2]
Despite the number of semi-automated approaches, it is stillperformed manually
Labor-intensive, time consuming, susceptible to mistakes [2]
2/14 Yguarata Cerqueira Cavalcanti Combining Rule-based and Information Retrieval Techniques to assign Software Change Requests
![Page 6: Combining Rule-based and Information Retrieval Techniques to assign Software Change Requests (ASE'2014)](https://reader038.vdocuments.net/reader038/viewer/2022110310/55a6ae081a28ab48418b4608/html5/thumbnails/6.jpg)
Outline Introduction Proposal Experiment Concluding remarks References
Motivation
Current approaches use AI. . .
Performs as a black box
Difficult adoption
Hard to deal with changing environments
3/14 Yguarata Cerqueira Cavalcanti Combining Rule-based and Information Retrieval Techniques to assign Software Change Requests
![Page 7: Combining Rule-based and Information Retrieval Techniques to assign Software Change Requests (ASE'2014)](https://reader038.vdocuments.net/reader038/viewer/2022110310/55a6ae081a28ab48418b4608/html5/thumbnails/7.jpg)
1 Introduction
2 Proposal
3 Experiment
4 Concluding remarks
![Page 8: Combining Rule-based and Information Retrieval Techniques to assign Software Change Requests (ASE'2014)](https://reader038.vdocuments.net/reader038/viewer/2022110310/55a6ae081a28ab48418b4608/html5/thumbnails/8.jpg)
Outline Introduction Proposal Experiment Concluding remarks References
New Idea
Semi-automation through
Combining Rule-based andInformation Retrieval Techniques
Aided by context information
4/14 Yguarata Cerqueira Cavalcanti Combining Rule-based and Information Retrieval Techniques to assign Software Change Requests
![Page 9: Combining Rule-based and Information Retrieval Techniques to assign Software Change Requests (ASE'2014)](https://reader038.vdocuments.net/reader038/viewer/2022110310/55a6ae081a28ab48418b4608/html5/thumbnails/9.jpg)
Outline Introduction Proposal Experiment Concluding remarks References
Strategy to Automated CR Assignment
New CR is submitted
First step:
Rule-Based ExpertSystem (RBES) runs oversimple rules (low cost)
Second step
Information Retrieval (IR)model runs and extractspotential developersRBES runs over complexrules (high cost)
5/14 Yguarata Cerqueira Cavalcanti Combining Rule-based and Information Retrieval Techniques to assign Software Change Requests
![Page 10: Combining Rule-based and Information Retrieval Techniques to assign Software Change Requests (ASE'2014)](https://reader038.vdocuments.net/reader038/viewer/2022110310/55a6ae081a28ab48418b4608/html5/thumbnails/10.jpg)
Outline Introduction Proposal Experiment Concluding remarks References
Rule-Based Expert System (RBES) – Drools
Simple rule
rule "Critical CRs, or CRs for module C"
when
$cr: CR(severity == CRITICAL || module =="C")
then
$cr.assignTo(developer("[email protected]"))
end
Complex rule
rule "Change Requests for modules A and B"
when
$cr: CR(module =="A" || module =="B")
then
$cr.assignTo(availableDeveloper(Workload.WEIGHTED ))
end
6/14 Yguarata Cerqueira Cavalcanti Combining Rule-based and Information Retrieval Techniques to assign Software Change Requests
![Page 11: Combining Rule-based and Information Retrieval Techniques to assign Software Change Requests (ASE'2014)](https://reader038.vdocuments.net/reader038/viewer/2022110310/55a6ae081a28ab48418b4608/html5/thumbnails/11.jpg)
Outline Introduction Proposal Experiment Concluding remarks References
Benefits
Fully controlled
Developers can intervene
Easy adoption
Context information promotes company dynamics
7/14 Yguarata Cerqueira Cavalcanti Combining Rule-based and Information Retrieval Techniques to assign Software Change Requests
![Page 12: Combining Rule-based and Information Retrieval Techniques to assign Software Change Requests (ASE'2014)](https://reader038.vdocuments.net/reader038/viewer/2022110310/55a6ae081a28ab48418b4608/html5/thumbnails/12.jpg)
1 Introduction
2 Proposal
3 Experiment
4 Concluding remarks
![Page 13: Combining Rule-based and Information Retrieval Techniques to assign Software Change Requests (ASE'2014)](https://reader038.vdocuments.net/reader038/viewer/2022110310/55a6ae081a28ab48418b4608/html5/thumbnails/13.jpg)
Outline Introduction Proposal Experiment Concluding remarks References
Questions
Q1: What is the accuracy of the proposed approach forautomated CR assignment?
Q2: What is the necessary effort to setup the approach ina software development project?
Q3: Does the achieved accuracy pay the necessary effortneeded in the setup?
8/14 Yguarata Cerqueira Cavalcanti Combining Rule-based and Information Retrieval Techniques to assign Software Change Requests
![Page 14: Combining Rule-based and Information Retrieval Techniques to assign Software Change Requests (ASE'2014)](https://reader038.vdocuments.net/reader038/viewer/2022110310/55a6ae081a28ab48418b4608/html5/thumbnails/14.jpg)
Outline Introduction Proposal Experiment Concluding remarks References
Experiment Design
Proposed approach versus pure Support VectorMachine (SVM)
CRs from two modules of NovoSIAFIBrazilian Federal Data Processing Service (SERPRO)
Module A = 781 CRs
Module B = 1031 CRs
9/14 Yguarata Cerqueira Cavalcanti Combining Rule-based and Information Retrieval Techniques to assign Software Change Requests
![Page 15: Combining Rule-based and Information Retrieval Techniques to assign Software Change Requests (ASE'2014)](https://reader038.vdocuments.net/reader038/viewer/2022110310/55a6ae081a28ab48418b4608/html5/thumbnails/15.jpg)
Outline Introduction Proposal Experiment Concluding remarks References
Configuration of the Proposed Approach
Rules extraction
Total of 14 rules
Context information
developers vacation
developers project allocation
developers experience
10/14 Yguarata Cerqueira Cavalcanti Combining Rule-based and Information Retrieval Techniques to assign Software Change Requests
![Page 16: Combining Rule-based and Information Retrieval Techniques to assign Software Change Requests (ASE'2014)](https://reader038.vdocuments.net/reader038/viewer/2022110310/55a6ae081a28ab48418b4608/html5/thumbnails/16.jpg)
Outline Introduction Proposal Experiment Concluding remarks References
Results I
Q1. What is the accuracy of the proposed approach forautomated CR assignment?
Our approach: Module A = 45% and Module B = 34%
SVM: Module A = 38% and Module B = 23%
An improvement of 18% on Module A and 48% on B
11/14 Yguarata Cerqueira Cavalcanti Combining Rule-based and Information Retrieval Techniques to assign Software Change Requests
![Page 17: Combining Rule-based and Information Retrieval Techniques to assign Software Change Requests (ASE'2014)](https://reader038.vdocuments.net/reader038/viewer/2022110310/55a6ae081a28ab48418b4608/html5/thumbnails/17.jpg)
Outline Introduction Proposal Experiment Concluding remarks References
Results II
Q2. What is the necessary effort to setup the approach in asoftware development project?
38 hours (rule extraction, context information, strategy)
Q3. Does the achieved accuracy pay the necessary effortneeded in the setup?
Considering 10 minutes for each CR manually assigned [2]
SVM would save 89 hours
Our approach would save 117 hours
Thus, an economy of 28 hours
However, 38 hours was needed for setup
12/14 Yguarata Cerqueira Cavalcanti Combining Rule-based and Information Retrieval Techniques to assign Software Change Requests
![Page 18: Combining Rule-based and Information Retrieval Techniques to assign Software Change Requests (ASE'2014)](https://reader038.vdocuments.net/reader038/viewer/2022110310/55a6ae081a28ab48418b4608/html5/thumbnails/18.jpg)
1 Introduction
2 Proposal
3 Experiment
4 Concluding remarks
![Page 19: Combining Rule-based and Information Retrieval Techniques to assign Software Change Requests (ASE'2014)](https://reader038.vdocuments.net/reader038/viewer/2022110310/55a6ae081a28ab48418b4608/html5/thumbnails/19.jpg)
Outline Introduction Proposal Experiment Concluding remarks References
Concluding remarks
RBES + IR: simple, yet effective !
Future work
Investigate new algorithms for workload balancing
Investigate methods and techniques for automatic extraction ofassignment rules
13/14 Yguarata Cerqueira Cavalcanti Combining Rule-based and Information Retrieval Techniques to assign Software Change Requests
![Page 20: Combining Rule-based and Information Retrieval Techniques to assign Software Change Requests (ASE'2014)](https://reader038.vdocuments.net/reader038/viewer/2022110310/55a6ae081a28ab48418b4608/html5/thumbnails/20.jpg)
Combining Rule-based and InformationRetrieval Techniques to assign Software
Change RequestsNew Idea
Yguarata Cerqueira CavalcantiIvan do Carmo Machado
Paulo A. da Mota S. NetoEduardo Santana de Almeida
Silvio Romero de Lemos Meira
Center for Informatics – UFPE, Brazil
ASE 2014 (Vasteras – Sweden)September 18, 2014
![Page 21: Combining Rule-based and Information Retrieval Techniques to assign Software Change Requests (ASE'2014)](https://reader038.vdocuments.net/reader038/viewer/2022110310/55a6ae081a28ab48418b4608/html5/thumbnails/21.jpg)
Outline Introduction Proposal Experiment Concluding remarks References
References I
[1] Cavalcanti, Y. C., da Mota Silveira Neto, P. A., Lucredio, D.,Vale, T., de Almeida, E. S., and de Lemos Meira, S. R. (2013a).The bug report duplication problem: an exploratory study. Soft.Quality J., 21, 36–66.
[2] Cavalcanti, Y. C., Neto, P. A. M. S., Machado, I. C., Almeida,E. S., and Meira, S. R. L. (2013b). Towards UnderstandingSoftware Change Request Assignment: A survey withpractitioners. In Proc. of the 17th Inter. Conf. on Evaluation andAssessment in Soft. Eng. (EASE).
[3] Jeong, G., Kim, S., and Zimmermann, T. (2009). Improving bugtriage with bug tossing graphs. In Proc. of the 7th joint meetingof the European Soft. Eng. Conf. and the ACM SIGSOFT Symp.on the Found. of Soft. Eng. (ESEC/FSE’2009), pages 111–120.
14/14 Yguarata Cerqueira Cavalcanti Combining Rule-based and Information Retrieval Techniques to assign Software Change Requests
![Page 22: Combining Rule-based and Information Retrieval Techniques to assign Software Change Requests (ASE'2014)](https://reader038.vdocuments.net/reader038/viewer/2022110310/55a6ae081a28ab48418b4608/html5/thumbnails/22.jpg)
CR Assignment
1/4 Yguarata Cerqueira Cavalcanti Combining Rule-based and Information Retrieval Techniques to assign Software Change Requests
![Page 23: Combining Rule-based and Information Retrieval Techniques to assign Software Change Requests (ASE'2014)](https://reader038.vdocuments.net/reader038/viewer/2022110310/55a6ae081a28ab48418b4608/html5/thumbnails/23.jpg)
Information Retrieval Model With Learning
SVM
Training (Black arrows)
Recommendation (Gray arrows)
2/4 Yguarata Cerqueira Cavalcanti Combining Rule-based and Information Retrieval Techniques to assign Software Change Requests
![Page 24: Combining Rule-based and Information Retrieval Techniques to assign Software Change Requests (ASE'2014)](https://reader038.vdocuments.net/reader038/viewer/2022110310/55a6ae081a28ab48418b4608/html5/thumbnails/24.jpg)
Component Diagram of the Solution
3/4 Yguarata Cerqueira Cavalcanti Combining Rule-based and Information Retrieval Techniques to assign Software Change Requests
![Page 25: Combining Rule-based and Information Retrieval Techniques to assign Software Change Requests (ASE'2014)](https://reader038.vdocuments.net/reader038/viewer/2022110310/55a6ae081a28ab48418b4608/html5/thumbnails/25.jpg)
Prototype Tool Architecture
4/4 Yguarata Cerqueira Cavalcanti Combining Rule-based and Information Retrieval Techniques to assign Software Change Requests