Download - Experiments on Design Pattern Discovery
Experiments on Design Pattern Discovery
Jing Dong and Yajing ZhaoDepartment of Computer ScienceThe University of Texas at Dallas{ jdong, yxz045100}@utdallas.edu
March 27, 2007 The University of Texas at Dallas 2
Outline
Introduction Related Work Our Approach Experiment Data Result and Discussion Precision and Recall Benchmark Conclusion and Future Work
March 27, 2007 The University of Texas at Dallas 3
Introduction
Difficult to understand large computer-based systems
No original architecture and design Patterns help on understanding systems Patterns embed future change of systems Need of design pattern discovery since
patterns are lost in source code
March 27, 2007 The University of Texas at Dallas 4
Introduction (Cont’d)
DP-Miner Experiment data: System source code
JUnit JEdit JHotDraw Java.AWT
Such experiments lead to benchmarks
March 27, 2007 The University of Texas at Dallas 5
Related Work
Au
thors
Tools
Ab
stract Factory
Ad
apter
Brid
ge
Bu
ilder
Ch
ain of
Resp
onsib
ility
Com
man
d
Com
posite
Decorator
Facad
e
Factory M
ethod
Flyw
eight
Ob
server
Prototyp
e
Proxy
Sin
gleton
Strategy
Tem
plate M
ethod
Visitor
Tsantalis 2006 X X X X X X X X X X X
Antoniol 1998 X X X X X
Heuzeroth 2003 X X X
Niere 2002 X X X
Blewitt 2001 X X X X
Balanyi 2003 X X X X X X X X X X X X X
Shi 2006 X X X X X X X X X X X X X X X
Gueheneuc 2006 X X X X X X X X X X X X X
March 27, 2007 The University of Texas at Dallas 6
Our Approach – DP-Miner
Matrix and Weight Product of Prime Numbers Structural, Behavioral, and Semantic Analysis
March 27, 2007 The University of Texas at Dallas 7
Experiment Data
JUnit: A regression test framework that helps developers to implement unit tests in Java.
JEdit: A mature and easy-to-use text editor. JHotDraw: A two dimensional graphics
frameworks for technical and structured drawing editors written in Java.
Java.AWT: A library for developing graphical user interfaces for Java programs.
March 27, 2007 The University of Texas at Dallas 8
Why Those Four Systems as Data? At the time they were developed, the idea of
design patterns is already mature and widely applied.
Other works on design pattern discovery use one or more of these systems to evaluate their approaches, which allows us to compare and evaluate our experiment results.
March 27, 2007 The University of Texas at Dallas 9
Experiment Data
Systems Version Class # File #
Java.AWT JDK1.4.2 570 345
JUnit 3.8.2 126 93
JEdit 4.2 1001 394
JHotDraw 6.0 beta 1 530 484
System information Version Number of Classes Number of Files
March 27, 2007 The University of Texas at Dallas 10
Experiment Result
Systems Adapter Bridge Strategy Composite
Java.AWT 21 65 76 3
JUnit 3 6 6 3
JEdit 17 24 24 0
JHotDraw 4 58 64 0
March 27, 2007 The University of Texas at Dallas 11
Experiment Result – Java.AWTResults for Each Analysis Phase
SystemsJava.AWT
Structural Analysis
Behavioral Analysis
Semantic Analysis
Final Result
Adapter 57 21 N/A 21
Bridge 100 76 65 65
Strategy 100 76 76 76
Composite 92 3 N/A 3
March 27, 2007 The University of Texas at Dallas 12
Experiment Result – JUnitResults for Each Analysis Phase
SystemsJUnit
Structural Analysis
Behavioral Analysis
Semantic Analysis
FinalResult
Adapter 15 3 N/A 3
Bridge 6 6 6 6
Strategy 6 6 6 6
Composite 9 3 N/A 3
March 27, 2007 The University of Texas at Dallas 13
Experiment Result – JEditResults for Each Analysis Phase
SystemsJEdit
Structural Analysis
Behavioral Analysis
Semantic Analysis
Final Result
Adapter 80 17 N/A 17
Bridge 33 24 24 24
Strategy 33 24 24 24
Composite 0 0 N/A 0
March 27, 2007 The University of Texas at Dallas 14
Experiment Result – JHotDrawResults for Each Analysis Phase
SystemsJHotDraw
Structural Analysis
Behavioral Analysis
Semantic Analysis
FinalResult
Adapter 27 4 N/A 4
Bridge 74 64 58 58
Strategy 74 64 64 64
Composite 0 0 N/A 0
March 27, 2007 The University of Texas at Dallas 15
Observations Behavioral analysis phase only deals with a small
number of classes, since it’s based on the result of structural analysis phase.
Significant reduction of the candidates between structural and behavioral analysis of Adapter and Composite pattern than those of Bridge and Strategy patterns, since the former have more behavioral characteristics than the latter do.
Result of structural and behavioral analysis of the Bridge and Strategy patterns are the same, since they have same structural and behavioral characteristics and only differ in their intents.
March 27, 2007 The University of Texas at Dallas 16
Recovery Precisions for JHotDraw
JHotDraw TP FP Precision
Adapter 4 0 100%
Bridge 53 5 91.38%
Strategy 58 6 90.63%
Composite 0 0 100%
Manually checked the results generated by our tool and see whether they are real pattern instances.
TP: True Positive FP: False Positive
March 27, 2007 The University of Texas at Dallas 17
Instances Missed Instances manually found but missed by DP-Miner Recall: 89.23%
CONTEXT STRATEGY
[1] LineConnection Connector
[2] ChangeConnectionHandle Connector
[3] ConnectionTool Connector
[4] PolygonHandle Locator
[5] LocatorHandle Locator
[6] LocatorConnector Locator
[7] SelectionTool DrawingView
March 27, 2007 The University of Texas at Dallas 18
Reasons of Discrepancy Flexibility of design pattern
Composite pattern: Collapse both the Component and Composite classes into a single class or not.
Adapter pattern: Full matching all three roles, i.e. Target, Adapter, and Adaptee, or partial matching of only Adapter and Adaptee.
Object-oriented programming languages provide special language constructs that greatly simplify the implementation of a design pattern. E.g., Java provides LinkedList, ArrayList, HashMap, and Hashtable, which make the implementation of aggregate elements in patterns easy, but the detection harder.
March 27, 2007 The University of Texas at Dallas 19
Benchmark Correct number of pattern instances and their locations
are generally not available. Lacking benchmarks is the main impediment
Hard to evaluate and compare design pattern discovery techniques.
Hard to judge whether an approach discovers all instances and whether the discovered ones are correct.
To calculate the precision and recall of a pattern matching result, it’s essential to know The number of correct pattern instances The locations of correct pattern instances
JHotDraw pattern benchmark
March 27, 2007 The University of Texas at Dallas 20
Conclusion We did a series of experiments on design pattern
discovery from open-source systems using DP-Miner Patterns concerned: Adapter, Bridge, Strategy,
Composite Experiment data: Java.AWT, JUnit, JEdit, JHotDraw We compared our experiment results with others and
found several discrepancies. We analyzed the issues and discussed possible
reasons. We argue for benchmarks for design pattern discovery.
March 27, 2007 The University of Texas at Dallas 21
Future Work
Improve our tool. Experiment on other systems Manually check results of other systems and
perfect the benchmark Research on other object-oriented languages,
such as C++.
March 27, 2007 The University of Texas at Dallas 22
Thank You!
March 27, 2007 The University of Texas at Dallas 23
Questions?
March 27, 2007 The University of Texas at Dallas 24
Overall Architecture of Our Approach
Structural Analysis Semantic Analysis Behavioral Analysis
design pattern
instances
Eliminates false positives
Extract structural information
Matrix TypeWeight
Structural constraints
Apply
Behavioral constraints
Instances from structural analysis
Apply
Instances from behavioral analysis
XMI file
Java Code
Scans Check for naming conventions for participating classes
System
Design Patternmatch
Source CodeIntermediate
Representation
March 27, 2007 The University of Texas at Dallas 25
Structural Analysis – MatrixButton
(from awt)
Canvas
base : String = "canvas"nameCounter : int = 0serialVersionUID : long
Canvas()Canvas()constructComponentName()addNotify()paint()update()postsOldMouseEvents()createBufferStrategy()createBufferStrategy()getBufferStrategy()getAccessibleContext()
(from awt)
Checkbox(from awt)
Choice(from awt)
Component(from awt)
Container(from awt)
parent
component[]
Label(from awt)
Scrollbar(from awt)
TextComponent(from awt)
March 27, 2007 The University of Texas at Dallas 26
Button
Canvas
Checkbox
Choice
Com
ponent
Container
Label
Scrollbar
TextC
omponent
Button 1 1 1 1 7 1 1 1 1
Canvas 1 1 1 1 7 1 1 1 1
Checkbox 1 1 1 1 7 1 1 1 1
Choice 1 1 1 1 7 1 1 1 1
Component 1 1 1 1 1 5 1 1 1
Container 1 1 1 1 35 1 1 1 1
Label 1 1 1 1 7 1 1 1 1
Scrollbar 1 1 1 1 7 1 1 1 1
TextComponent 1 1 1 1 7 1 1 1 1
March 27, 2007 The University of Texas at Dallas 27
Button
Canvas
Checkbox
Choice
Com
ponent
Container
Label
Scrollbar
TextC
omponent
Button 1 1 1 1 7 1 1 1 1
Canvas 1 1 1 1 7 1 1 1 1
Checkbox 1 1 1 1 7 1 1 1 1
Choice 1 1 1 1 7 1 1 1 1
Component 1 1 1 1 1 5 1 1 1
Container 1 1 1 1 35 1 1 1 1
Label 1 1 1 1 7 1 1 1 1
Scrollbar 1 1 1 1 7 1 1 1 1
TextComponent 1 1 1 1 7 1 1 1 1
Leaf Component Composite
Leaf 1 7 1
Component 1 1 1
Composite 1 35 1