criteria for the evaluation of implemented...
TRANSCRIPT
![Page 1: Criteria for the Evaluation of Implemented Architecturesericbouwers.github.io/slides/2009icsm.pdfEric Bouwers, Joost Visser, Arie van Deursen 2009/09/22. ... L. Dobrica and E. Niemel](https://reader036.vdocuments.net/reader036/viewer/2022090715/60f03f096cdcc46da129c254/html5/thumbnails/1.jpg)
Arent Janszoon Ernststraat 595-HNL-1082 LD [email protected]
Criteria for the Evaluation of Implemented ArchitecturesEric Bouwers, Joost Visser, Arie van Deursen
2009/09/22
![Page 2: Criteria for the Evaluation of Implemented Architecturesericbouwers.github.io/slides/2009icsm.pdfEric Bouwers, Joost Visser, Arie van Deursen 2009/09/22. ... L. Dobrica and E. Niemel](https://reader036.vdocuments.net/reader036/viewer/2022090715/60f03f096cdcc46da129c254/html5/thumbnails/2.jpg)
Criteria for the Evaluation of Implemented Architectures
Why evaluate software architectures?
‘Architectures allow or preclude nearly all of thesystem’s quality attributes’
-- from `Evaluating Software Architectures` by P. Clements, R. Kazman and M. Klein
![Page 3: Criteria for the Evaluation of Implemented Architecturesericbouwers.github.io/slides/2009icsm.pdfEric Bouwers, Joost Visser, Arie van Deursen 2009/09/22. ... L. Dobrica and E. Niemel](https://reader036.vdocuments.net/reader036/viewer/2022090715/60f03f096cdcc46da129c254/html5/thumbnails/3.jpg)
Criteria for the Evaluation of Implemented Architectures
Why evaluate implemented architectures?
![Page 4: Criteria for the Evaluation of Implemented Architecturesericbouwers.github.io/slides/2009icsm.pdfEric Bouwers, Joost Visser, Arie van Deursen 2009/09/22. ... L. Dobrica and E. Niemel](https://reader036.vdocuments.net/reader036/viewer/2022090715/60f03f096cdcc46da129c254/html5/thumbnails/4.jpg)
Criteria for the Evaluation of Implemented Architectures
Why evaluate implemented architectures?
![Page 5: Criteria for the Evaluation of Implemented Architecturesericbouwers.github.io/slides/2009icsm.pdfEric Bouwers, Joost Visser, Arie van Deursen 2009/09/22. ... L. Dobrica and E. Niemel](https://reader036.vdocuments.net/reader036/viewer/2022090715/60f03f096cdcc46da129c254/html5/thumbnails/5.jpg)
Criteria for the Evaluation of Implemented Architectures
Why do we want criteria for the evaluation?
Maintenance Phase
Architecture Evaluation
Result
Architecture Evaluation
Result
Architecture Evaluation
Result
Architecture Evaluation
Result
![Page 6: Criteria for the Evaluation of Implemented Architecturesericbouwers.github.io/slides/2009icsm.pdfEric Bouwers, Joost Visser, Arie van Deursen 2009/09/22. ... L. Dobrica and E. Niemel](https://reader036.vdocuments.net/reader036/viewer/2022090715/60f03f096cdcc46da129c254/html5/thumbnails/6.jpg)
Criteria for the Evaluation of Implemented Architectures
Literature reviews
A framework for classifying andcomparing software architecture
evaluation methods.
M. Babar, L. Zhu, and D. R. Jeffery
A survey on softwarearchitecture analysis
methods.
L. Dobrica and E. Niemelä.
![Page 7: Criteria for the Evaluation of Implemented Architecturesericbouwers.github.io/slides/2009icsm.pdfEric Bouwers, Joost Visser, Arie van Deursen 2009/09/22. ... L. Dobrica and E. Niemel](https://reader036.vdocuments.net/reader036/viewer/2022090715/60f03f096cdcc46da129c254/html5/thumbnails/7.jpg)
Criteria for the Evaluation of Implemented Architectures
IEEE Computer, July 2009
‘Technology transfer remainsa major challenge. There is
limited out of the box processand tool support for
organizations that want tostart reviews.’
‘Software Architecture Review: The state of Practice’
by M. Babar, I. Gorton
![Page 8: Criteria for the Evaluation of Implemented Architecturesericbouwers.github.io/slides/2009icsm.pdfEric Bouwers, Joost Visser, Arie van Deursen 2009/09/22. ... L. Dobrica and E. Niemel](https://reader036.vdocuments.net/reader036/viewer/2022090715/60f03f096cdcc46da129c254/html5/thumbnails/8.jpg)
Criteria for the Evaluation of Implemented Architectures
Software Risk Assessment (SRA)
A one-time investigation into software maintainabilityand related business risks
![Page 9: Criteria for the Evaluation of Implemented Architecturesericbouwers.github.io/slides/2009icsm.pdfEric Bouwers, Joost Visser, Arie van Deursen 2009/09/22. ... L. Dobrica and E. Niemel](https://reader036.vdocuments.net/reader036/viewer/2022090715/60f03f096cdcc46da129c254/html5/thumbnails/9.jpg)
Criteria for the Evaluation of Implemented Architectures
Architectural properties
GUI
Data Access
Business Logic
High Level Design:
Saving Paying Stocks
Modularization:
if(hasPayed() && !isAdmin()){ showThankYou();}
Separation of Concerns:
Business logic/Security
![Page 10: Criteria for the Evaluation of Implemented Architecturesericbouwers.github.io/slides/2009icsm.pdfEric Bouwers, Joost Visser, Arie van Deursen 2009/09/22. ... L. Dobrica and E. Niemel](https://reader036.vdocuments.net/reader036/viewer/2022090715/60f03f096cdcc46da129c254/html5/thumbnails/10.jpg)
Criteria for the Evaluation of Implemented Architectures
Research questions
Q1: Which system attributes do experts take into accountwhen evaluating architectural system properties?
Q2: How do these system attributes influence thearchitectural system properties?
![Page 11: Criteria for the Evaluation of Implemented Architecturesericbouwers.github.io/slides/2009icsm.pdfEric Bouwers, Joost Visser, Arie van Deursen 2009/09/22. ... L. Dobrica and E. Niemel](https://reader036.vdocuments.net/reader036/viewer/2022090715/60f03f096cdcc46da129c254/html5/thumbnails/11.jpg)
Criteria for the Evaluation of Implemented Architectures
Empirical study resources
![Page 12: Criteria for the Evaluation of Implemented Architecturesericbouwers.github.io/slides/2009icsm.pdfEric Bouwers, Joost Visser, Arie van Deursen 2009/09/22. ... L. Dobrica and E. Niemel](https://reader036.vdocuments.net/reader036/viewer/2022090715/60f03f096cdcc46da129c254/html5/thumbnails/12.jpg)
Criteria for the Evaluation of Implemented Architectures
A selection of our clients
Financials ITLogistics OtherPublic
![Page 13: Criteria for the Evaluation of Implemented Architecturesericbouwers.github.io/slides/2009icsm.pdfEric Bouwers, Joost Visser, Arie van Deursen 2009/09/22. ... L. Dobrica and E. Niemel](https://reader036.vdocuments.net/reader036/viewer/2022090715/60f03f096cdcc46da129c254/html5/thumbnails/13.jpg)
Criteria for the Evaluation of Implemented Architectures
Evaluated systems characteristics
![Page 14: Criteria for the Evaluation of Implemented Architecturesericbouwers.github.io/slides/2009icsm.pdfEric Bouwers, Joost Visser, Arie van Deursen 2009/09/22. ... L. Dobrica and E. Niemel](https://reader036.vdocuments.net/reader036/viewer/2022090715/60f03f096cdcc46da129c254/html5/thumbnails/14.jpg)
Criteria for the Evaluation of Implemented Architectures
Empirical study design overview
CreateAttribute List
Create Mapping to Properties
ValidateAttributes &Mapping
![Page 15: Criteria for the Evaluation of Implemented Architecturesericbouwers.github.io/slides/2009icsm.pdfEric Bouwers, Joost Visser, Arie van Deursen 2009/09/22. ... L. Dobrica and E. Niemel](https://reader036.vdocuments.net/reader036/viewer/2022090715/60f03f096cdcc46da129c254/html5/thumbnails/15.jpg)
Criteria for the Evaluation of Implemented Architectures
Experimental design (1/3)Creation of the attribute list
1) Check table with property ratings for argumentsif this table does not exists
use specific paragraphs
For each report:
2) For all arguments founddetermine system attributeif the attribute is not on the list
add system attribute to the list
![Page 16: Criteria for the Evaluation of Implemented Architecturesericbouwers.github.io/slides/2009icsm.pdfEric Bouwers, Joost Visser, Arie van Deursen 2009/09/22. ... L. Dobrica and E. Niemel](https://reader036.vdocuments.net/reader036/viewer/2022090715/60f03f096cdcc46da129c254/html5/thumbnails/16.jpg)
Criteria for the Evaluation of Implemented Architectures
Experimental design (2/3)Create mapping from attributes to properties
2) For all arguments founddetermine system attributeadd one to count of attribute for property
1) Extract arguments per property
For each report:
![Page 17: Criteria for the Evaluation of Implemented Architecturesericbouwers.github.io/slides/2009icsm.pdfEric Bouwers, Joost Visser, Arie van Deursen 2009/09/22. ... L. Dobrica and E. Niemel](https://reader036.vdocuments.net/reader036/viewer/2022090715/60f03f096cdcc46da129c254/html5/thumbnails/17.jpg)
Criteria for the Evaluation of Implemented Architectures
Experimental design (3/3)Validation of Attributes and Mapping
![Page 18: Criteria for the Evaluation of Implemented Architecturesericbouwers.github.io/slides/2009icsm.pdfEric Bouwers, Joost Visser, Arie van Deursen 2009/09/22. ... L. Dobrica and E. Niemel](https://reader036.vdocuments.net/reader036/viewer/2022090715/60f03f096cdcc46da129c254/html5/thumbnails/18.jpg)
Criteria for the Evaluation of Implemented Architectures
Results of Empirical Study (1/3)
CreateAttribute List
Create Mapping to Properties
ValidateAttributes &Mapping
![Page 19: Criteria for the Evaluation of Implemented Architecturesericbouwers.github.io/slides/2009icsm.pdfEric Bouwers, Joost Visser, Arie van Deursen 2009/09/22. ... L. Dobrica and E. Niemel](https://reader036.vdocuments.net/reader036/viewer/2022090715/60f03f096cdcc46da129c254/html5/thumbnails/19.jpg)
Criteria for the Evaluation of Implemented Architectures
15 System attributes
• Abstraction• Layering• Logic in Database• Module Inconsistency• Module Size• Source Grouping• Technology Combination• Textual Duplication
• Functional Duplication• Libraries / Frameworks Usage• Module Dependencies• Module Functionality • Relation Documentation and Implementation• Technology Age• Technology Usage
![Page 20: Criteria for the Evaluation of Implemented Architecturesericbouwers.github.io/slides/2009icsm.pdfEric Bouwers, Joost Visser, Arie van Deursen 2009/09/22. ... L. Dobrica and E. Niemel](https://reader036.vdocuments.net/reader036/viewer/2022090715/60f03f096cdcc46da129c254/html5/thumbnails/20.jpg)
Criteria for the Evaluation of Implemented Architectures
More details available in the paper
Inspecting the call-graph on modulelevel, matching thisagainst expecteddependencies
The staticdependencies (i.e.calls, includes)between modules
ModuleDependencies
Inspecting the listof imports andstructure of thesource- and build-files
The usage ofstandard librariesand frameworks
Libraries /Frameworks
AssessmentApproach
DescriptionName
![Page 21: Criteria for the Evaluation of Implemented Architecturesericbouwers.github.io/slides/2009icsm.pdfEric Bouwers, Joost Visser, Arie van Deursen 2009/09/22. ... L. Dobrica and E. Niemel](https://reader036.vdocuments.net/reader036/viewer/2022090715/60f03f096cdcc46da129c254/html5/thumbnails/21.jpg)
Criteria for the Evaluation of Implemented Architectures
Results of empirical study (2/3)
CreateAttribute List
Create Mapping to Properties
ValidateAttributes &Mapping
![Page 22: Criteria for the Evaluation of Implemented Architecturesericbouwers.github.io/slides/2009icsm.pdfEric Bouwers, Joost Visser, Arie van Deursen 2009/09/22. ... L. Dobrica and E. Niemel](https://reader036.vdocuments.net/reader036/viewer/2022090715/60f03f096cdcc46da129c254/html5/thumbnails/22.jpg)
Criteria for the Evaluation of Implemented Architectures
High Level Design
Modularization
Separation of Concerns
..... .....
Module Dependencies
Module Functionality
Relation Doc. / Impl.
Functional Duplication
Technology Combination
Technology Age
7 4 2 2 5 13
11 32 3 6 1 0
6 13 0 18 0 0
Attributes used for each property
![Page 23: Criteria for the Evaluation of Implemented Architecturesericbouwers.github.io/slides/2009icsm.pdfEric Bouwers, Joost Visser, Arie van Deursen 2009/09/22. ... L. Dobrica and E. Niemel](https://reader036.vdocuments.net/reader036/viewer/2022090715/60f03f096cdcc46da129c254/html5/thumbnails/23.jpg)
Criteria for the Evaluation of Implemented Architectures
High Level Design
Modularization
Separation of Concerns
..... .....
Module Dependencies
Module Functionality
Relation Doc. / Impl.
Functional Duplication
Technology Combination
Technology Age
X X
X X X
X
X
Attributes used for each property
![Page 24: Criteria for the Evaluation of Implemented Architecturesericbouwers.github.io/slides/2009icsm.pdfEric Bouwers, Joost Visser, Arie van Deursen 2009/09/22. ... L. Dobrica and E. Niemel](https://reader036.vdocuments.net/reader036/viewer/2022090715/60f03f096cdcc46da129c254/html5/thumbnails/24.jpg)
Criteria for the Evaluation of Implemented Architectures
Results of empirical study (3/3)
CreateAttribute List
Create Mapping to Properties
ValidateAttributes &Mapping
![Page 25: Criteria for the Evaluation of Implemented Architecturesericbouwers.github.io/slides/2009icsm.pdfEric Bouwers, Joost Visser, Arie van Deursen 2009/09/22. ... L. Dobrica and E. Niemel](https://reader036.vdocuments.net/reader036/viewer/2022090715/60f03f096cdcc46da129c254/html5/thumbnails/25.jpg)
Criteria for the Evaluation of Implemented Architectures
Interview results
High Level Design
Modularization
Separation of Concerns
E1
E1, E2
E1
E2
E2
..... .....
Module Dependencies
Module Functionality
Relation Doc. / Impl.
Functional Duplication
Technology Combination
Technology Age
![Page 26: Criteria for the Evaluation of Implemented Architecturesericbouwers.github.io/slides/2009icsm.pdfEric Bouwers, Joost Visser, Arie van Deursen 2009/09/22. ... L. Dobrica and E. Niemel](https://reader036.vdocuments.net/reader036/viewer/2022090715/60f03f096cdcc46da129c254/html5/thumbnails/26.jpg)
Criteria for the Evaluation of Implemented Architectures
Threats to validity
Representative data
Reliability of measurements
Generalization of the results
![Page 27: Criteria for the Evaluation of Implemented Architecturesericbouwers.github.io/slides/2009icsm.pdfEric Bouwers, Joost Visser, Arie van Deursen 2009/09/22. ... L. Dobrica and E. Niemel](https://reader036.vdocuments.net/reader036/viewer/2022090715/60f03f096cdcc46da129c254/html5/thumbnails/27.jpg)
Criteria for the Evaluation of Implemented Architectures
‘What is in it for me?’-- you
![Page 28: Criteria for the Evaluation of Implemented Architecturesericbouwers.github.io/slides/2009icsm.pdfEric Bouwers, Joost Visser, Arie van Deursen 2009/09/22. ... L. Dobrica and E. Niemel](https://reader036.vdocuments.net/reader036/viewer/2022090715/60f03f096cdcc46da129c254/html5/thumbnails/28.jpg)
Criteria for the Evaluation of Implemented Architectures
Our contributions
•A description of an empirical study using over 40 reports
•The identification of 15 system attributes that have an impact on the maintainability of an architecture
•An analysis of the projection of the found system attributes onto three architectural system properties
![Page 29: Criteria for the Evaluation of Implemented Architecturesericbouwers.github.io/slides/2009icsm.pdfEric Bouwers, Joost Visser, Arie van Deursen 2009/09/22. ... L. Dobrica and E. Niemel](https://reader036.vdocuments.net/reader036/viewer/2022090715/60f03f096cdcc46da129c254/html5/thumbnails/29.jpg)
Criteria for the Evaluation of Implemented Architectures
Application of the results
![Page 30: Criteria for the Evaluation of Implemented Architecturesericbouwers.github.io/slides/2009icsm.pdfEric Bouwers, Joost Visser, Arie van Deursen 2009/09/22. ... L. Dobrica and E. Niemel](https://reader036.vdocuments.net/reader036/viewer/2022090715/60f03f096cdcc46da129c254/html5/thumbnails/30.jpg)
Criteria for the Evaluation of Implemented Architectures
Future research opportunities
![Page 31: Criteria for the Evaluation of Implemented Architecturesericbouwers.github.io/slides/2009icsm.pdfEric Bouwers, Joost Visser, Arie van Deursen 2009/09/22. ... L. Dobrica and E. Niemel](https://reader036.vdocuments.net/reader036/viewer/2022090715/60f03f096cdcc46da129c254/html5/thumbnails/31.jpg)
Criteria for the Evaluation of Implemented Architectures
![Page 32: Criteria for the Evaluation of Implemented Architecturesericbouwers.github.io/slides/2009icsm.pdfEric Bouwers, Joost Visser, Arie van Deursen 2009/09/22. ... L. Dobrica and E. Niemel](https://reader036.vdocuments.net/reader036/viewer/2022090715/60f03f096cdcc46da129c254/html5/thumbnails/32.jpg)
Criteria for the Evaluation of Implemented Architectures
![Page 33: Criteria for the Evaluation of Implemented Architecturesericbouwers.github.io/slides/2009icsm.pdfEric Bouwers, Joost Visser, Arie van Deursen 2009/09/22. ... L. Dobrica and E. Niemel](https://reader036.vdocuments.net/reader036/viewer/2022090715/60f03f096cdcc46da129c254/html5/thumbnails/33.jpg)
Criteria for the Evaluation of Implemented Architectures
Questions or comments?
Criteria for the Evaluation of Implemented Architectures
• A description of an empirical study using over 40 SRAreports• The identification of 15 system attributes that have an
impact on the maintainability of an implementedarchitecture• An analysis of the projection of the found system
attributes onto three architectural system properties
Feel free to contact me:[email protected]