javaone’12 session 3992 - software modularity: paradoxes, principles, and architectures
DESCRIPTION
Starting from the JDK itself, a wave of migrations to module systems is bound to propagate throughout the Java landscape. In this session, expand your mental toolbox by learning what modularity is, why it is important, and how to divide your monolithic application into well-designed functional modules. First you will gain an intimate understanding of modularity by hearing about several of its mind-bending paradoxes. Then you will learn how popular design principles apply to creating modules and their APIs. Finally you will learn how common monolithic software architectures exhibit various degrees of modularization of functional features and what that means for your forthcoming modularization efforts.TRANSCRIPT
Software Modularity: Paradoxes, Principles, and Architectures
Andrzej Olszak
Jaroslav Tulach
Andrzej Olszak
• Ph.D. Research fellow @ SDU
• Creator of Featureous tool
• Fights cancer @ Dako
Jaroslav Tulach
• Founder and architect of NetBeans IDE and RCP
• API designer
• Speaker and author
http://paradoxes.apidesign.org
Agenda
1. Dividing into modules (Andrzej):
1. Architectures
2. Principles
2. Designing module APIs (Jaroslav):
1. Paradoxes
2. Principles
Motivation
Getting out of the monolithic cave
The starting point
A monolithic system
• No apparent logical parts– Everything changes
together
– Difficult to control complexity
The goal
A modular system
• Decomposed into logical modules
• Modules partial:– Comprehension
– Change
The tool
A module system
• Wraps logical modules into physical modules– a.k.a. advanced JARs
– Enforce code isolation
A possible result
• Module system ⇏Modular code– It all depends on how you design
the modules
Part I
Architectures
A running example
• Let’s assume that this is our system
– 3 layers of code that provide 3 features to the users
• This is the essence of your architecture if you use:
– MVC, MVP, Onion arch, Hexagonal arch, …
“UI”
“Logic”
“Domain Model”
Modularizing layers
• There are at lease three options:
View
Controller
Model
Modularizing layers
• There are at lease three options:
View
Controller
Model
View
Controller
Model
App
feat
ure
1
feat
ure
2
feat
ure
3
Choosing ‘best’ modularization
• Modularity is relative to change– Modularization quality depends on the future changes [Parn’72]
• Let’s consider two change scenarios:
View
Controller
Modelfe
atu
re1
feat
ure
2
feat
ure
3
Choosing ‘best’ modularization
• Modularity is relative to change– Modularization quality depends on the future changes [Parn’72]
• Let’s consider two change scenarios:
View
Controller
Modelfe
atu
re1
feat
ure
2
feat
ure
3
1. Modify user functionality
Choosing ‘best’ modularization
• Modularity is relative to change– Modularization quality depends on the future changes [Parn’72]
• Let’s consider two change scenarios:
View
Controller
Modelfe
atu
re1
feat
ure
2
feat
ure
3
1. Modify user functionality
Choosing ‘best’ modularization
• Modularity is relative to change– Modularization quality depends on the future changes [Parn’72]
• Let’s consider two change scenarios:
View
Controller
Modelfe
atu
re1
feat
ure
2
feat
ure
3
1. Modify user functionality
Choosing ‘best’ modularization
• Modularity is relative to change– Modularization quality depends on the future changes [Parn’72]
• Let’s consider two change scenarios:
View
Controller
Modelfe
atu
re1
feat
ure
2
feat
ure
3
1. Modify user functionality
Choosing ‘best’ modularization
• Modularity is relative to change– Modularization quality depends on the future changes [Parn’72]
• Let’s consider two change scenarios:
View
Controller
Modelfe
atu
re1
feat
ure
2
feat
ure
3
1. Modify user functionality
Choosing ‘best’ modularization
• Modularity is relative to change– Modularization quality depends on the future changes [Parn’72]
• Let’s consider two change scenarios:
View
Controller
Modelfe
atu
re1
feat
ure
2
feat
ure
3
2. Migrate UI to JavaFX1. Modify user functionality
Choosing ‘best’ modularization
• Modularity is relative to change– Modularization quality depends on the future changes [Parn’72]
• Let’s consider two change scenarios:
View
Controller
Modelfe
atu
re1
feat
ure
2
feat
ure
3
2. Migrate UI to JavaFX1. Modify user functionality
Choosing ‘best’ modularization
• Modularity is relative to change– Modularization quality depends on the future changes [Parn’72]
• Let’s consider two change scenarios:
View
Controller
Modelfe
atu
re1
feat
ure
2
feat
ure
3
2. Migrate UI to JavaFX1. Modify user functionality
Choosing ‘best’ modularization
• Modularity is relative to change– Modularization quality depends on the future changes [Parn’72]
• Let’s consider two change scenarios:
View
Controller
Modelfe
atu
re1
feat
ure
2
feat
ure
3
2. Migrate UI to JavaFX1. Modify user functionality
No silver-bullet modularization
Anticipating future changes
Feature-oriented evolutionary changes
75%
Other evolutionary changes25%
Anticipating future changes
Software costs in organizations [Moad’90, Nose’90, Erli’00]
Evolution &maintenance
60-90%
Initial development10-40%
Feature-oriented evolutionary changes
75%
Other evolutionary changes25%
Anticipating future changes
Software costs in organizations [Moad’90, Nose’90, Erli’00]
Modularization in practice:the case of NDVis
• Neurological analysis tool by VisiTrend
– Monolithic -> NetBeans MS
– Improve functional customizability
– Improve reusability of core algorithms
• Starting point– 10 KLOC, 27 use cases
– Unfamiliar source code
– Complex and unfamiliar domain
View
Controller
Model
feat
ure
1
feat
ure
2
feat
ure
3
Feature-oriented restructuring
Modularizing features with Featureous tool
Modularizing features with Featureous tool
1. Feature location(a.k.a. traceability)
Modularizing features with Featureous tool
1. Feature location(a.k.a. traceability)
2. Feature-orientedanalysis
Modularizing features with Featureous tool
1. Feature location(a.k.a. traceability)
2. Feature-orientedanalysis
3. Iterative Restructuring
4. Module APIs
Modularization @ 35 man-hours
• Explicit and pluggable features
• Reusable core
Part II
Principles
Separation of Concerns
• “To study an aspect of a subject matter in isolation”[Dijk’74]
• Software consists of “concerns”– Features
– Persistence
– Security
– Caching
...
• Refined by AspectJ [Kicz’96] and Hyper/J [Tarr’99]
– Multiple dimensions of concern – one dominant
– Scattering & Tangling
Reducing Scattering & Tangling
• Low Scattering
– “a concern implemented by few modules“
– reduces change scope and delocalization [Leto’86]
View
Controller
Model
feature 1
scattering
Scattering & Tangling
• Low Scattering
– “a concern implemented by few modules“
– reduces change scope and delocalization [Leto’86]
• Low Tangling
– "modules dedicated to a single concerns“
– reduces change propagation and interleaving [Ruga’95]
View
Controller
Model
feature 2
feature 1
tangling
scattering
A real-world metaphor
A real-world metaphor
http://xkcd.com/657/
Mapping SoC to other principles
SEPARATION OF CONCERNS
Mapping SoC to other principles
SEPARATION OF CONCERNS
LOW SCATTERING LOW TANGLING
Mapping SoC to other principles
SEPARATION OF CONCERNS
Low coupling[Stev’74]
Information hiding[Parn’72]
DRY[Hunt’99]
LOW SCATTERING LOW TANGLING
Mapping SoC to other principles
SEPARATION OF CONCERNS
Low coupling[Stev’74]
High cohesion[Stev’74]
Information hiding[Parn’72]
DRY[Hunt’99]
Single responsibility[Mart’02]
Common closure[Mart’02]
LOW SCATTERING LOW TANGLING
Measuring SoC
• Concern location + concern-oriented metrics
– Trialed at Motorola [Simm’06]
• Static analysis + Cohesion and Coupling
– Issues: coupling vs. DRY, “uncohesive” java.util
• Repository mining for change-sets:
http://swerl.tudelft.nl/bin/view/Main/TestHistory
clas
ses
Summary
• Module system is a tool, not the goal
• No “silver-bullet” modularization
• Restructuring layers into features is viable
• SoC – the root of all principles
References[Parn’72] Parnas, D.L. (1972). On the criteria to be used in decomposing systems into modules. Communications of the
ACM, 15(12).
[Moad’90] Moad, J. (1990). "Maintaining the competitive edge". Datamation 61-62, 64, 66.
[Erli’00] Erlikh, L. (2000). "Leveraging legacy system dollars for E-business". (IEEE) IT Pro, May/June 2000.
[Nose’90] Nosek, J. & Palvia, P. (1990). "Software maintenance management: changes in the last decade". Journal of Software Maintenance: Research and Practice 2 (3).
[Mart’11] Martin, R.C. (2011). http://blog.8thlight.com/uncle-bob/2011/09/30/Screaming-Architecture.html
[Dijk’74 ] Dijkstra, E. W. (1974). On the role of scientific thought. Selected Writings on Computing: A Personal Perspective.
[Kicz’96] Kiczales, G., Irwin, J., Lamping, J., Loingtier, J., M., Lopes, C., V., Maeda, C. and Mendhekar, A. (1996). Aspect-oriented programming. ACM Computing Surveys, vol. 28.
[Tarr’99] Tarr, P., Ossher, H., Harrison, W. and Sutton, S. M. (1999). N degrees of separation: multi-dimensional separation of concerns. In ICSE’99: Proceedings of the 21st international conference on Software engineering.
[Leto’86] Letovsky, S. and Soloway, E. (1986). Delocalized Plans and Program Comprehension. IEEE Software, 3(3).
[Ruga’95] Rugaber, S., Stirewalt, K. and Wills, L. M. (1995). The interleaving problem in program understanding. In WCRE’95: Proceedings of the 2nd Working Conference on Reverse Engineering.
[Simm’06] Simmons, S., Edwards, D., Wilde, N., Homan, J. and Groble, M. (2006). Industrial tools for the feature location problem: an exploratory study. Journal of Software Maintenance and Evolution Research and Practice, 18(6).
[Mart’02] Martin, R. C. (2002). Agile Software Development, Principles, Patterns, and Practices. Prentice Hall.
[Stev’74] Stevens, W. P., Myers, G. J. and Constantine, L. L. (1974). Structured Design. (E. Yourdon, Ed.)IBM Systems Journal, 13(2).
[Hunt’99] Hunt, A., & Thomas, D. (1999). The Pragmatic Programmer: From Journeyman to Master. Addison-Wesley Professional.
(Many of these papers are available online through Google Scholar, Citeseerx and the authors’ websites)
Feature tracer [backup]
Part III
Paradoxes of API Design
Modularity is relative to change
• API are like stars (paradox 19)
• Designing a universe
• Distributed development (paradox 2)
• Can't know all your users
• Envisioning them via use-cases
• Sustaining (paradox 3)
• One try to get API right
How to anticipate future changes?
• Client vs. Provider APIs (paradox 9)
• Open spaces
• Fixed points
• Stable API evolves
• Backward compatibility (paradox 6)
• Beware of API-less APIs (paradox 8)
• Mediawiki experience
• Loyal customer
Logical vs. Physical Design
• Design oriented on class relationship
• UML, specifications
• Packaging into JARs ignored (paradox 17)
• Influences deployment (paradox 7)
• Defines APIs
• Good common ground (paradox 11)
• Improves your design
A „weight“ of a module
• Environment for a module
• Modules don't live in vacuum
• Expressed by dependencies (paradox 17)
• Weight
• Number & scope of outgoing dependencies
• Less is more (paradox 19)
• Single responsibility principle
Use vs. Re-use
• Kirk Knoernchild
• Monolithic API is easier to use
• Modular API is easier to re-use
• Blackbox pattern (paradox 18)
• OSGi Capabilities
• Good tooling
• Wizards
SOLID APIs
• Single responsibility principle
• Meaning of modifiers (paradox 15)
• Client vs. provider APIs (paradox 9)
• Lightweight API modules
• Open/closed principle
• OK for provider APIs
• Disastrous for client APIs
• Proliferation of instanceof in user code
• Alternative behavior (paradox 16)
SOLID APIs II
• Liskov substitution principle
• AWT Frame extends Component!
• Don't expose deep hierarchies
• Use delegation rather than inheritance
• Client API should be in final classes
• 1:N factory methods
SOLID APIs III
• Interface segregation principle
• Lookup & discover
• OSGi declarative services
• Dependency inversion principle
• Code against interfaces, not implementations
• Does not imply classes are bad (paradox 9)
• Don't fear (injectable) singletons (paradox 14)
API in User Eyes
• Clueless users (paradox 1)
• Have always something else to do
• Evaluation of an API (paradox 4)
• Coolness
• Time to market
• Total cost of ownership
Collaboration
• Maintenance (paradox 10)
• Rely on patches
• Accepting unacceptable (paradox 13)
• Beauty (paradox 5)
• One writer and dozens of users
• Sacrifice the writer
Summary
• http://paradoxes.apidesign.org