journey from monolith to a modularized application - approach and key learnings - e xavier & t...
TRANSCRIPT
![Page 1: Journey from Monolith to a Modularized Application - Approach and Key Learnings - E Xavier & T Ward](https://reader031.vdocuments.net/reader031/viewer/2022030318/5a64a88b7f8b9a88568b4fe9/html5/thumbnails/1.jpg)
A Journey from a Monolith to a Modularized ApplicationApproach and Key LearningsEmanoel Xavier Senior Software Developer, Intel
Tim Ward CTO, Paremus
1
![Page 2: Journey from Monolith to a Modularized Application - Approach and Key Learnings - E Xavier & T Ward](https://reader031.vdocuments.net/reader031/viewer/2022030318/5a64a88b7f8b9a88568b4fe9/html5/thumbnails/2.jpg)
2
Notices and Disclaimers
Intel technologies’ features and benefits depend on system configuration and may require enabled hardware, software or service activation. Learn more at intel.com, or from the OEM or retailer.
No computer system can be absolutely secure.
Tests document performance of components on a particular test, in specific systems. Differences in hardware, software, or configuration will affect actual performance. Consult other sources of information to evaluate performance as you consider your purchase. For more complete information about performance and benchmark results, visit http://www.intel.com/performance.
Intel, the Intel logo and others are trademarks of Intel Corporation in the U.S. and/or other countries.
*Other names and brands may be claimed as the property of others.
© 2017 Intel Corporation.
![Page 3: Journey from Monolith to a Modularized Application - Approach and Key Learnings - E Xavier & T Ward](https://reader031.vdocuments.net/reader031/viewer/2022030318/5a64a88b7f8b9a88568b4fe9/html5/thumbnails/3.jpg)
Modularity through the ages…
3
A little history
![Page 4: Journey from Monolith to a Modularized Application - Approach and Key Learnings - E Xavier & T Ward](https://reader031.vdocuments.net/reader031/viewer/2022030318/5a64a88b7f8b9a88568b4fe9/html5/thumbnails/4.jpg)
4
Adam Smith
The Wealth of Nations (1776)
Division of Labor
“…the trade of a pin-maker: a workman not educated to this business could scarce, perhaps, with his utmost
industry, make one pin in a day, and certainly could
not make twenty.”
“But in the way in which this business is now carried on, it is divided into … about eighteen distinct operations, which, in some manufactories, are all performed by distinct hands”
“Those ten persons, therefore, could make among them upwards of forty-eight thousand pins in a day. Each person, therefore, making a tenth part of forty-eight
thousand pins, might be considered as making four thousand eight hundred pins in a day.”
SMALLER SPECIALIZED CONCURRENT
![Page 5: Journey from Monolith to a Modularized Application - Approach and Key Learnings - E Xavier & T Ward](https://reader031.vdocuments.net/reader031/viewer/2022030318/5a64a88b7f8b9a88568b4fe9/html5/thumbnails/5.jpg)
5
Henry Ford
The Moving Assembly Line (1910)
Modularity & Productivity
parts and car
“come to you”83 parts. 1 step/min
from 12h to 90 mins “continuous build”
interchangeable parts
model
![Page 6: Journey from Monolith to a Modularized Application - Approach and Key Learnings - E Xavier & T Ward](https://reader031.vdocuments.net/reader031/viewer/2022030318/5a64a88b7f8b9a88568b4fe9/html5/thumbnails/6.jpg)
“The major advancement in the area of modular programming has been the development of coding techniques and assemblers which (l) allow one module to be written with little knowledge of the code in another module, and (2) allow modules to be reassembled and replacedwithout reassembly of the whole system. This facility is extremely valuable for the production of large pieces of code, but the systems most often used as examples of problem systems are highly-modularized programs and make use of the techniques mentioned above.”
– David Parnas, On the Criteria to Be Used in Decomposing Systems into Modules (1972)
6
Modularity in Software
![Page 7: Journey from Monolith to a Modularized Application - Approach and Key Learnings - E Xavier & T Ward](https://reader031.vdocuments.net/reader031/viewer/2022030318/5a64a88b7f8b9a88568b4fe9/html5/thumbnails/7.jpg)
7
Apollo 16 Lunar Lander Module (1972)
“... the systems most often used as examples of problem systems are highly-modularized programs and make use of the techniques mentioned above.”
A telling phrase…
“Modularity is too hard to be worth it”
Mixing the problem with the solution
The problems are complex
Modularity makes the solution possible
![Page 8: Journey from Monolith to a Modularized Application - Approach and Key Learnings - E Xavier & T Ward](https://reader031.vdocuments.net/reader031/viewer/2022030318/5a64a88b7f8b9a88568b4fe9/html5/thumbnails/8.jpg)
Modularity today
8
Fast forwarding
![Page 9: Journey from Monolith to a Modularized Application - Approach and Key Learnings - E Xavier & T Ward](https://reader031.vdocuments.net/reader031/viewer/2022030318/5a64a88b7f8b9a88568b4fe9/html5/thumbnails/9.jpg)
9
1970s 2017
Modern modular software – vision
![Page 10: Journey from Monolith to a Modularized Application - Approach and Key Learnings - E Xavier & T Ward](https://reader031.vdocuments.net/reader031/viewer/2022030318/5a64a88b7f8b9a88568b4fe9/html5/thumbnails/10.jpg)
10
1970s 2017
Modern modular software – reality
![Page 11: Journey from Monolith to a Modularized Application - Approach and Key Learnings - E Xavier & T Ward](https://reader031.vdocuments.net/reader031/viewer/2022030318/5a64a88b7f8b9a88568b4fe9/html5/thumbnails/11.jpg)
11
Modularity is not magic
Forgotten past lessons
70s: Low level modularity for low level coupling
Late modularity == expensive tech debt
![Page 12: Journey from Monolith to a Modularized Application - Approach and Key Learnings - E Xavier & T Ward](https://reader031.vdocuments.net/reader031/viewer/2022030318/5a64a88b7f8b9a88568b4fe9/html5/thumbnails/12.jpg)
(Serves One)
12
The recipe for a monolith
![Page 13: Journey from Monolith to a Modularized Application - Approach and Key Learnings - E Xavier & T Ward](https://reader031.vdocuments.net/reader031/viewer/2022030318/5a64a88b7f8b9a88568b4fe9/html5/thumbnails/13.jpg)
Starting a new project
13
Nobody plans to
write bad code
Things will be different
now
Well defined, modular design
we have all been there
Sonar*, Findbugs*, 100% CC
intended modularization
![Page 14: Journey from Monolith to a Modularized Application - Approach and Key Learnings - E Xavier & T Ward](https://reader031.vdocuments.net/reader031/viewer/2022030318/5a64a88b7f8b9a88568b4fe9/html5/thumbnails/14.jpg)
Growing the project
14
Mgmt needs more code
Here, more devs
Flexible processes
more requirements.
6 months deadline
but, Karl, your best coder is
gone
That is ok, we are “Agile*”
![Page 15: Journey from Monolith to a Modularized Application - Approach and Key Learnings - E Xavier & T Ward](https://reader031.vdocuments.net/reader031/viewer/2022030318/5a64a88b7f8b9a88568b4fe9/html5/thumbnails/15.jpg)
Growing the project (6 months later)
15
almost done
processes and tools
some workarounds
needed
worked overtime. “paper” release
people catching up with Karl’s
code. code reviews
that’s ok. track and fix
later
![Page 16: Journey from Monolith to a Modularized Application - Approach and Key Learnings - E Xavier & T Ward](https://reader031.vdocuments.net/reader031/viewer/2022030318/5a64a88b7f8b9a88568b4fe9/html5/thumbnails/16.jpg)
Renewed interest(6-12 months later)
16
customer: shows
interest
sales: delivery in 6
months
no one is familiar with
the code
+4 new members, +20 years worth
work
+1 headache
no previous team member.
Karl’s code is still around
![Page 17: Journey from Monolith to a Modularized Application - Approach and Key Learnings - E Xavier & T Ward](https://reader031.vdocuments.net/reader031/viewer/2022030318/5a64a88b7f8b9a88568b4fe9/html5/thumbnails/17.jpg)
17
Do any parts of this seem familiar?
Gall’s Law:
“A complex system that works is invariably found to have evolved from a simple system that worked. A complex system designed from scratch never works and cannot be patched up to make it work. You have to start over with a working simple system.”
Nobody sets out to write bad code
Successful projects are bound to become complex
Complexity must be managed
![Page 18: Journey from Monolith to a Modularized Application - Approach and Key Learnings - E Xavier & T Ward](https://reader031.vdocuments.net/reader031/viewer/2022030318/5a64a88b7f8b9a88568b4fe9/html5/thumbnails/18.jpg)
(Serves One)
18
How OSGi helps
![Page 19: Journey from Monolith to a Modularized Application - Approach and Key Learnings - E Xavier & T Ward](https://reader031.vdocuments.net/reader031/viewer/2022030318/5a64a88b7f8b9a88568b4fe9/html5/thumbnails/19.jpg)
19
imposes
API boundaries
hidesimplementation
details
automatesdependency management
enforcesdependencies at
runtime
facilitatescomposition
![Page 20: Journey from Monolith to a Modularized Application - Approach and Key Learnings - E Xavier & T Ward](https://reader031.vdocuments.net/reader031/viewer/2022030318/5a64a88b7f8b9a88568b4fe9/html5/thumbnails/20.jpg)
20
Won’t magically turn a monolith into a modularized application
Will provide you the tools for that
Won’t automatically fix all coupling issues, “shortcuts” and technical debt
Will make them more evident
Won’t fix API leaks
Will enforce your APIs
![Page 21: Journey from Monolith to a Modularized Application - Approach and Key Learnings - E Xavier & T Ward](https://reader031.vdocuments.net/reader031/viewer/2022030318/5a64a88b7f8b9a88568b4fe9/html5/thumbnails/21.jpg)
21
Getting results
Planned and managed
API changes
Work with it,
not against it
OSGi: ongoing
commitment
![Page 22: Journey from Monolith to a Modularized Application - Approach and Key Learnings - E Xavier & T Ward](https://reader031.vdocuments.net/reader031/viewer/2022030318/5a64a88b7f8b9a88568b4fe9/html5/thumbnails/22.jpg)
How OSGi helps
22
Limits complexity to smaller pieces
– Divide and conquer
Pieces can be more easily and predictably replaced
– No runtime surprises
Easier to understand dependencies fan out
Better testability
![Page 23: Journey from Monolith to a Modularized Application - Approach and Key Learnings - E Xavier & T Ward](https://reader031.vdocuments.net/reader031/viewer/2022030318/5a64a88b7f8b9a88568b4fe9/html5/thumbnails/23.jpg)
23
OSC 101
![Page 24: Journey from Monolith to a Modularized Application - Approach and Key Learnings - E Xavier & T Ward](https://reader031.vdocuments.net/reader031/viewer/2022030318/5a64a88b7f8b9a88568b4fe9/html5/thumbnails/24.jpg)
24
How it started
Intel closed source project
Application and deployment of security virtual functions
Let’s extend it !
Let’s build a community around it!
– Spring 2016 => Summer 2017
Se
em
s fa
mil
iar?
![Page 25: Journey from Monolith to a Modularized Application - Approach and Key Learnings - E Xavier & T Ward](https://reader031.vdocuments.net/reader031/viewer/2022030318/5a64a88b7f8b9a88568b4fe9/html5/thumbnails/25.jpg)
25
OSC in SDI
LOCATIONUTILIZATION LATENCY SECURITY
InfrastructureAttributes POWER PERFORMANCE
InfrastructureAttributes
Servers
Virtualization
ContainersVirtual Router
Virtual SAN
Scale Out
Storage
Apps SDN/NFV SDS
InfrastructurePool
VMsVirtual
FirewallVirtualSwitch
Policy based automation
Self service portal
ServiceOrchestration Compute Network Storage Security
Pool of virtualized resources
Platform capability aware
Centralized and automated
![Page 26: Journey from Monolith to a Modularized Application - Approach and Key Learnings - E Xavier & T Ward](https://reader031.vdocuments.net/reader031/viewer/2022030318/5a64a88b7f8b9a88568b4fe9/html5/thumbnails/26.jpg)
register
setup
26
OSC in action
Plugins
OSC
Security Manager
SDN Controller
Data Center
Plugins
VNF Image
Virtualization Connector
Manager Connector
Security Group
Deployment
Security Policies
IPS, NGFW,
ADC
![Page 27: Journey from Monolith to a Modularized Application - Approach and Key Learnings - E Xavier & T Ward](https://reader031.vdocuments.net/reader031/viewer/2022030318/5a64a88b7f8b9a88568b4fe9/html5/thumbnails/27.jpg)
27
OSC logical components
osc-core
sec-mgr-plugins sdn-ctrlr-plugins
sdn-ctrlr-apisec-mgr-api
OpenStack4j*
RabbitMq*
Fabric8*
OpenStack
KubernetesH2 DB
VNF Catalog
Plugins
BL services Sync tasks
Security Mgr Services SDN Ctrlr Services
UI REST API
OSGi , h2db-jpa, apache.aries.jpa, vaadin, logback,
…
![Page 28: Journey from Monolith to a Modularized Application - Approach and Key Learnings - E Xavier & T Ward](https://reader031.vdocuments.net/reader031/viewer/2022030318/5a64a88b7f8b9a88568b4fe9/html5/thumbnails/28.jpg)
28
The path to open source
Community members diversity
SDN Providers and Security Managers neutral
! OSC Plugins Model
Dependency conflicts
REST Services?
Redundant endpoints
Not conducive to early adoption
Legacy plugins
![Page 29: Journey from Monolith to a Modularized Application - Approach and Key Learnings - E Xavier & T Ward](https://reader031.vdocuments.net/reader031/viewer/2022030318/5a64a88b7f8b9a88568b4fe9/html5/thumbnails/29.jpg)
29
The journey starts…
![Page 30: Journey from Monolith to a Modularized Application - Approach and Key Learnings - E Xavier & T Ward](https://reader031.vdocuments.net/reader031/viewer/2022030318/5a64a88b7f8b9a88568b4fe9/html5/thumbnails/30.jpg)
30
Modularizing the plugins
osc-server
sec-mgr-plugins
1. Build dependencies
2. Runtime dependencies
3. Startup dependencies
osc-controller
sdn-ctrlr-plugins
sec-mgr-apisec-mgr-api
pom.xml pom.xml
whiteboard pattern
whiteboard pattern
OSGi Resolver
BAR Installer
![Page 31: Journey from Monolith to a Modularized Application - Approach and Key Learnings - E Xavier & T Ward](https://reader031.vdocuments.net/reader031/viewer/2022030318/5a64a88b7f8b9a88568b4fe9/html5/thumbnails/31.jpg)
31
Modularizing the plugins
osc-server
sec-mgr-plugins
osc-controller
sdn-ctrlr-plugins
sec-mgr-apisec-mgr-api
whiteboard pattern
whiteboard pattern
OSGi Resolver
BAR Installer
Smaller footprint
Simpler service registration
Earlier plugin validation
64 /110packaged
dependenciesOSGi Http
Service Whiteboard
Plugins Lifecycle
![Page 32: Journey from Monolith to a Modularized Application - Approach and Key Learnings - E Xavier & T Ward](https://reader031.vdocuments.net/reader031/viewer/2022030318/5a64a88b7f8b9a88568b4fe9/html5/thumbnails/32.jpg)
32
Are we modular yet?
![Page 33: Journey from Monolith to a Modularized Application - Approach and Key Learnings - E Xavier & T Ward](https://reader031.vdocuments.net/reader031/viewer/2022030318/5a64a88b7f8b9a88568b4fe9/html5/thumbnails/33.jpg)
33
Perceived modularization
Implemented Lacking
"…Curved Walking in Humans”
Gregoire Courtine, Journal of Neurophysiology
![Page 34: Journey from Monolith to a Modularized Application - Approach and Key Learnings - E Xavier & T Ward](https://reader031.vdocuments.net/reader031/viewer/2022030318/5a64a88b7f8b9a88568b4fe9/html5/thumbnails/34.jpg)
34
Does it matter?
SDN/SDI
maintainability
clean dependencies
semantic versioning
isolation
extensibility
Open Source
incremental build
clean dependencies
semantic versioning
runtime enforcement
unit testing
![Page 35: Journey from Monolith to a Modularized Application - Approach and Key Learnings - E Xavier & T Ward](https://reader031.vdocuments.net/reader031/viewer/2022030318/5a64a88b7f8b9a88568b4fe9/html5/thumbnails/35.jpg)
35
The journey continues….
![Page 36: Journey from Monolith to a Modularized Application - Approach and Key Learnings - E Xavier & T Ward](https://reader031.vdocuments.net/reader031/viewer/2022030318/5a64a88b7f8b9a88568b4fe9/html5/thumbnails/36.jpg)
36
OSC uber bundle anatomy
Common monolithic to modularized migration path
Flat classpath
Transparent non-bundle dependencies
No isolation between internal packages
osc-server
uber bundle
uber bundle
core
BL services Sync tasks
DB access
UI REST API
![Page 37: Journey from Monolith to a Modularized Application - Approach and Key Learnings - E Xavier & T Ward](https://reader031.vdocuments.net/reader031/viewer/2022030318/5a64a88b7f8b9a88568b4fe9/html5/thumbnails/37.jpg)
37
Database access: osc-domain
Only @Entity classes
JPA API & Apache Aries JPA
OSGi Transaction Control Service
Unidirectional dependency
osc-server
uber bundle
core
BL services Sync tasks
DB access
UI REST API
osc-domain
hibernate-core, h2databasehibernate-jpa-api
![Page 38: Journey from Monolith to a Modularized Application - Approach and Key Learnings - E Xavier & T Ward](https://reader031.vdocuments.net/reader031/viewer/2022030318/5a64a88b7f8b9a88568b4fe9/html5/thumbnails/38.jpg)
38
User interface: osc-ui
New project: osc-service-api
UI views as DS components
OSGi managed views
osc-server
uber bundle
core
BL services Sync tasks
UI
REST API
osc-domain
osc-ui
osc-service-api
![Page 39: Journey from Monolith to a Modularized Application - Approach and Key Learnings - E Xavier & T Ward](https://reader031.vdocuments.net/reader031/viewer/2022030318/5a64a88b7f8b9a88568b4fe9/html5/thumbnails/39.jpg)
39
REST API: osc-rest-server
Uses the osc-service-api
JAX-RS service classes
No more domain entity references from the DTOs
New service APIs and DTOsosc-server
uber bundle
core
BL services Sync tasks
REST API
osc-domain
osc-ui
osc-service-api
osc-rest-server
![Page 40: Journey from Monolith to a Modularized Application - Approach and Key Learnings - E Xavier & T Ward](https://reader031.vdocuments.net/reader031/viewer/2022030318/5a64a88b7f8b9a88568b4fe9/html5/thumbnails/40.jpg)
40
Modularized OSC
5 new smaller bundles
DS, Interfaces, clear contracts
Improved dependencies scopes
Auxiliary uber bundles
– Non OSGi dependencies
Incremental build
osc-server
osc-server
core
BL services Sync tasks
osc-domain
osc-ui
osc-service-api
osc-rest-server
+
![Page 41: Journey from Monolith to a Modularized Application - Approach and Key Learnings - E Xavier & T Ward](https://reader031.vdocuments.net/reader031/viewer/2022030318/5a64a88b7f8b9a88568b4fe9/html5/thumbnails/41.jpg)
41
Are we modular yet?End of journey?
![Page 42: Journey from Monolith to a Modularized Application - Approach and Key Learnings - E Xavier & T Ward](https://reader031.vdocuments.net/reader031/viewer/2022030318/5a64a88b7f8b9a88568b4fe9/html5/thumbnails/42.jpg)
No…
42
Modularization, like good code, is an ongoing effort
There are multiple degrees of modularization.
– A gradual approach is recommended
– But don’t go back.
OSGi is not a magic wand, but makes the job possible
A monolith will only getyou so far. For a longer,pleasant (and modular)journey you will neednot only the right toolsand frameworks. Theright mindset is just asimportant.
![Page 43: Journey from Monolith to a Modularized Application - Approach and Key Learnings - E Xavier & T Ward](https://reader031.vdocuments.net/reader031/viewer/2022030318/5a64a88b7f8b9a88568b4fe9/html5/thumbnails/43.jpg)
Thank you!Emanoel Xavier Senior Software Developer, Intel
Tim Ward CTO, Paremus
43
www.opensecuritycontroller.orgCome join our community!