event-based customisation of multi- tenant saas using
TRANSCRIPT
Event-based Customisation of Multi-Tenant SaaS Using Microservices
Espen Tønnessen Nordli
Thesis submitted for the degree of Master in Informatics: Programming and System Architecture
(Software) 60 credits
Department of Informatics
Faculty of mathematics and natural science
UNIVERSITY OF OSLO
June 2020
II
III
Event-based Customisation of Multi-Tenant
SaaS Using Microservices
Espen Tønnessen Nordli
IV
© 2020 Espen Tønnessen Nordli
Event-based Customisation of Multi-Tenant SaaS Using Microservices
http://www.duo.uio.no/
Trykk: Reprosentralen, Universitetet i Oslo
V
Abstract
Context: For software vendors to exploit economies of scale, enterprise software such as
CRM and ERP are now made available on the Cloud with a multi-tenant Software as a
Service model. The increased value comes from the ability of vendors to serve multiple
tenants, with the same instance.
Objective: For tenants to widely adopt Cloud-based multi-tenant SaaS, new customisation
solutions are required. In this Master’s project, we present an event-based non-intrusive
customisation framework. The framework enables customisation for multi-tenant SaaS by
leveraging the existing events in the application.
Method: Based on a review of the literature on customisation of multi-tenant SaaS and the
requirement that the expression power should be equal to deep customisation, an approach is
created and applied to a case study.
Results: The results of the case study done on eShopOnContainers show that an event-based
customisation approach can empower an application to give the ability to customise the flow
of processing events, and integrate with tenant-specific microservices for customisation.
Conclusions: A crucial aspect for vendors of multi-tenant cloud SaaS applications is ensuring
tenant-isolation, our approach maintains tenant-isolation by altering the messaging framework
to be multi-tenant aware. Another upside to an event-based customisation approach is the
ability to reduce the number of API calls.
VI
Acknowledgements
First and foremost, I would like to thank my supervisors, Phu Nguyen, Hui Son and Franck
Chauvel for the guidance, support and feedback given throughout the process of working on
the Master’s project.
I am also thankful to my internal University of Oslo supervisor, Ketil Stølen.
I would also like to thank Sintef, for allowing me to work on a Master’s project that is so
close to the industry and making this research possible.
Additionally, I would like to thank Visma and SuperOffice for allowing me to present my
findings to them, as well as valuable feedback,
Last but not least, I am very grateful to my family and friends for their motivation and support
during the Master’s project.
VII
Table of contents
1 Introduction ........................................................................................................................ 1
1.1 Motivations .................................................................................................................. 3
1.1.1 Example ................................................................................................................ 5
1.2 Current Challenges ...................................................................................................... 9
1.3 Project Goal ............................................................................................................... 10
1.4 Research Method ....................................................................................................... 10
1.5 Approach ................................................................................................................... 10
1.6 Result and Contributions ........................................................................................... 11
1.7 The structure of this thesis ......................................................................................... 11
2 Background ...................................................................................................................... 13
2.1 Multi-tenant cloud software ....................................................................................... 13
2.2 Microservice architecture (MSA) .............................................................................. 13
2.2.1 Benefits ............................................................................................................... 15
2.2.2 Technologies ...................................................................................................... 16
2.2.3 Event-driven MSA ............................................................................................. 17
2.3 Customisation ............................................................................................................ 18
2.3.1 Controlled customisation .................................................................................... 18
2.3.2 Deep customisation ............................................................................................ 19
2.3.3 Intrusive .............................................................................................................. 19
2.3.4 Non-intrusive ...................................................................................................... 19
3 Related work .................................................................................................................... 20
3.1 Controlled Customisation .......................................................................................... 20
3.2 Deep Customisation ................................................................................................... 23
4 Approach .......................................................................................................................... 31
4.1 Requirements ............................................................................................................. 31
4.2 Customising events .................................................................................................... 32
4.3 Tenant Manager ......................................................................................................... 34
4.4 Event isolation ........................................................................................................... 35
4.5 Event capturing .......................................................................................................... 39
4.6 Architecture ............................................................................................................... 43
5 Implementation ................................................................................................................. 45
VIII
5.1 Requirements for customisation ................................................................................ 45
5.2 Reference application ................................................................................................ 46
5.3 Changes to the main product ..................................................................................... 49
5.3.1 Identity API ........................................................................................................ 50
5.3.2 Tenant Manager .................................................................................................. 51
5.3.3 Event capturing .................................................................................................. 52
5.3.4 Event isolation .................................................................................................... 54
5.4 Azure Service Bus ..................................................................................................... 57
5.5 Use cases.................................................................................................................... 61
5.5.1 Implementation of use cases .............................................................................. 61
5.6 Result of requirements ............................................................................................... 68
6 Discussion ........................................................................................................................ 71
7 Conclusion ........................................................................................................................ 75
8 Future work ...................................................................................................................... 77
References ................................................................................................................................ 79
IX
List of figures
Figure 1 Illustration of deep customisation ................................................................................ 2
Figure 2 Types of multi-tenancy [3] .......................................................................................... 4
Figure 3 eShopOnContainers architecture [4] ............................................................................ 7
Figure 4 Illustration of deep customisation with microservices ................................................ 9
Figure 5 Levels of MSA and their benefits [14] ...................................................................... 15
Figure 6 Illustration of variation points and different features[3] ............................................ 21
Figure 7 Illustration of the usage of variability modelling[1] .................................................. 22
Figure 8 Sequence diagram for deep customisation with intrusive microservices [2] ............. 24
Figure 9 An overview of the MiSC-Cloud framework[7] ........................................................ 26
Figure 10 A conceptual model of customisation for multi-tenant SaaS [22] ........................... 27
Figure 11 A reference architecture of using microservices for customisation [22] ................. 29
Figure 12 Example of generic event handler sequence diagram for the event-based approach
.................................................................................................................................................. 34
Figure 13 Generic event isolation class diagram ..................................................................... 37
Figure 14 Generic event isolation sequence diagram ............................................................... 39
Figure 15 Generic event capturing activity diagram ................................................................ 41
Figure 16 Generic event capturing sequence diagram example ............................................... 42
Figure 17 Class diagram of generic event interface for capturing events ................................ 43
Figure 18 Generic architecture adapted from[4] ...................................................................... 44
Figure 19 eShopOnContainers checkout page ......................................................................... 48
Figure 20 Implementation architecture adapted from [4] ........................................................ 49
Figure 21 IdentityDb AspNetUsers table ................................................................................. 50
Figure 22 IdentityService endpoint .......................................................................................... 51
Figure 23 Tenant Manager endpoint ........................................................................................ 52
Figure 24 AbstractEventBus template method ......................................................................... 53
Figure 25 Class diagram of event isolation and event capturing implementation ................... 54
Figure 26 RabbitMQ virtual hosts ............................................................................................ 55
Figure 27 RabbitMQ connection setup .................................................................................... 56
Figure 28 RabbitMQ Event Bus setup ..................................................................................... 57
Figure 29 Azure Service Bus connection setup ....................................................................... 58
Figure 30 Azure Service Bus Event Bus setup ........................................................................ 59
Figure 31 Azure Service Bus topics ......................................................................................... 60
Figure 32 Azure Service Bus topic with subscriptions ............................................................ 60
Figure 33 Azure Service Bus subscription with filters ............................................................ 60
Figure 34 Customisation microservice consumer main product event .................................... 62
Figure 35 Sequence diagram of shipping information use case ............................................... 63
Figure 36 Use case 1 result ...................................................................................................... 63
Figure 37 Tenant Manager registration of customisation ........................................................ 64
Figure 38 Mock endpoint to indicate that all RFID tags have been scanned ........................... 65
Figure 39 Mock Postman call ................................................................................................... 65
X
Figure 40 RFID Use case sequence diagram ........................................................................... 66
Figure 41 Result of Use case 2 before RFID tags have been scanned ..................................... 67
Figure 42 Result of Use case 2 after RFID tags have been scanned ........................................ 68
Table 1 Results of requirements ............................................................................................... 69
Figure 43 Potential event bus setups ........................................................................................ 72
Figure 44 Multi-instance event bus setup ................................................................................ 73
1
1 Introduction
To the exploit the economies of scale that gained by moving a Software as a Service (SaaS)
application from a single instance per customer solution, to a multi-tenant solution with a
single application serving multiple tenants, SaaS vendors must be able to attract a significant
number of tenants for their product [1]. Realistically, different tenants have different
requirements; as such, the software vendors of a multi-tenant SaaS application need to be able
to solve the varying requirements of a large number of potential tenants, meaning that the
SaaS application must have support for configuration and customisation on a tenant-specific
level.
Today more and more companies and organisations rely on software applications to be able to
perform their tasks and meet their goals. This software can range from applications like
Customer Relationship Management (CRM) and Enterprise Resource Planning (ERP)
applications to accounting and human resources management applications. Generally, this
type of software has to be able to fit the need of many different organisations, but this also
causes it not to be able to meet the specific needs of an organisation. To solve this, software
vendors generally provide a way to do controlled customisation. The creators of the software
predict where organisations might have different needs and provide them with application
programming interfaces (APIs), extension points and different configuration choices.
Sometimes this is not enough customisability for specific organisations, and as such, they
must perform deep customisation. Previously this used to be where the customers acquired a
specific license for the application that allowed them to customise the application to fit their
needs, and then deploy that on-premises. This type of customisation is known as deep
customisation. Figure 1 shows an illustration of the deep customisation approach. However,
as software vendors are moving their applications to the Cloud and offering them as SaaS, this
type of customisation is no longer possible if the vendor wants to run the software as a multi-
tenant application. The reason for this is that, in a multi-tenant context, the vendor must
ensure tenant isolation, security and management, which means that a feature requested by
one organisation should not affect any of the other organisation’s that use the same multi-
tenant application. Because of the need for a specific feature for one tenant not affecting the
quality of service for the other tenants, some of the mainstream multi-tenant vendors like
Salesforce and Oracle NetSuite only support controlled customisation [2]. However, this
2
means that the application might not be as flexible as required by the customer, as there are
fewer options than if they were to offer deep customisation.
Figure 1 Illustration of deep customisation
This thesis presents an approach using event-based non-intrusive microservices to offer deep
customisation capability. Customers can register their customised functionality that is to run
in addition to functionality in the main application, which modifies the result to achieve the
requirements of the customer.
The remainder is structured as follows. Section 1.1 presents the motivations for customising
enterprise software along with a motivating example, secondly, in Section 1.2, the current
challenges of achieving deep customisation for multi-tenant cloud SaaS applications. Then in
section 1.3, the project goal of the thesis. The research method used in the Master’s project is
stated in Section 1.4. Section 1.5 gives an overview of the approach that applied in the case
3
study this Master project bases itself, and highlights of the results and contributions are given
in Section 1.6. Finally, Section 1.7 gives the structure of the main content of the thesis.
1.1 Motivations
There are different layers in which software vendors can implement a multi-tenant
application, as shown in Figure 2. The three are at the application layer, middleware layer or
the virtualised infrastructure layer, and each of them has trade-offs regarding customisability,
engineering costs, operational costs and scalability. This thesis uses an approach where the
application is shared between all the tenants, i.e., on the application layer, which, arguably,
leads to lower operational costs and scalability, but makes customisability more complex as
well increasing the cost of engineering the application and increasing the complexity of the
SaaS application itself.
4
Figure 2 Types of multi-tenancy [3]
One of the significant benefits of a multi-tenant cloud application is the cost savings that the
software vendor can gain. These cost savings come from the economies of scale by serving
multiple tenants with a single instance of an application. The reason for this is that the single
instance, even if it serves multiple tenants, still requires fewer resources than if one were to
use one instance per customer of the application [3]. Additionally, it allows the software
vendors to sell their application at a lower cost, which allows them to reach more potential
tenants. Another vital aspect of multi-tenancy is that it significantly simplifies release
management. Instead of having to install multiple different instances of the same application
on different servers, it is enough to install it once in the cloud server.
The use of microservices to perform the deep customisation might also make it easier for the
customer to implement all the requirements they have for the application, either through in-
house developers or through the use of third-party developers specialising in this type of
development. As opposed to how deep customisation used to be done in on-premises
solutions; in which case it could become complicated whenever new versions of the
application are released. The new version of the software would require integration with the
customisations that the customer already has in the system. Further, the use of microservices
means that they are redeployed independently of each other, which means that one could use
DevOps and continuous integration, which allows quicker implementations and more
flexibility. It can also, in some cases, be easier to test a microservice application as it has a
5
higher cohesion and loose coupling with the other microservices compared to a monolithic
application. There can be some downsides with testing microservices as well, specifically
using mocks to replace the data from the other microservices. Additionally, the use of
microservices to perform the deep customisation allows each tenant to choose the technology
stack that fits their needs the best. The choice of technology could be either that the
requirements are simpler to implement with a specific technology or simply that they already
have extensive experience with a technology that they would like to continue using.
Another essential benefit gained by using microservices and the microservice architecture
(MSA) is the increased speed of change and safety that they can gain from it, which applies to
both the vendor of the software and the customer. In an increasingly competitive market, it
becomes more and more important for software vendors to have a short time to market.
Besides, a more modularised application allows a software vendor to sell smaller parts of an
application to a customer while still maintaining a profit due to not having wasted resources
on scaling up the other parts of the application. The use of microservice architecture also fits
very well with the DevOps mindset, and the combination of DevOps and microservice can
also lead to a shorter time to market for software vendors, which makes it easier to sell their
products to potential customers [3].
There are two directions of customisation, intrusive and non-intrusive. Intrusive means that
the customisation has access to the same context as the main product and can perform
anything the main product can. Non-intrusive means that the customisation does not send
dynamic code to the main product, which allows a software product to have a higher degree
of control over what the customisations can achieve. The non-intrusive approach might be
safer than an intrusive approach, as an intrusive approach would allow the customer full
access to the context of the main application. Which, in turn, might lead to one customer
being able to access data from another customer in an inadequate implementation.
1.1.1 Example
In this section, an open-source application, eShopOnContainers, is presented as an example to
show some of the obstacles that a software vendor faces when moving their application from
on-premises solutions to the cloud, to make it a multi-tenant application.
6
eShopOnContainers 1is an application by Microsoft. It is a shopping application that allows
the user to add different items to a basket and then purchase them; it also has login
functionality. It proposes a microservice oriented architecture implementation with multiple
autonomous microservices. The application uses HTTP as the communication protocol
between the client apps and the microservices and supports asynchronous communication for
data updates propagation across multiple services based on Integration Events and an Event
Bus [4]. Figure 3 is an illustration of the architecture of the application.
1 https://github.com/dotnet-architecture/eShopOnContainers
7
Figure 3 eShopOnContainers architecture [4]
8
A significant challenge for multi-tenancy applications is to support customisation by all the
different tenants. Say one tenant using this application wanted a feature that allowed
customers to donate a sum of money to some cause or organisation when they are checking
out their basket of item(s). This feature is a use case where first the customer has to input an
amount of money that they wish to donate. That amount has to be displayed on an overview
page along with all the items in the basket. Finally, they have to be charged the correct
amount rather than just for the items in their basket. This customisation means that two new
components for the UI-layer are needed, one input field for actually inputting the donation
amount and one component for the overview before the user pays. For the business layer, the
total price calculation has to take into account the donation, and finally, for the persistence
layer, the donation amount has to be stored as well.
However, all other tenants that are using the same application should not be affected in any
way, as illustrated in Figure 4. Here the base application consists of several microservices that
work in conjunction as a complete application as well as a Tenant Manager microservice that
keeps track of the different customisations for all the different tenants. Then, all the tenants
have the possibility of creating their tenant-specific microservices that modifies the behaviour
of the base application in some way. In this case, tenant A has decided to alter MicroserviceA
and MicroserviceB so that the application is altered for the end-users of tenant A. Tenant B,
however, has only altered MicroserviceC, which means that MicroserviceA and
MicroserviceB operate as defined by the software vendor for their end-users.
9
Figure 4 Illustration of deep customisation with microservices
1.2 Current Challenges
In the field of SaaS development, the current trend is that configuration is favoured over
customisation since the latter is considered too complicated [1]. Configuration usually
supports variations in the SaaS application through setting pre-defined parameters, which can
then alter the user interface (UI), business logic and data models. Customisation, however,
involves variations in the core of the SaaS application to address specific requirements from
tenants which cannot be solved by configuration. When comparing the two methods of
achieving variations, customisation is currently more costly for the SaaS vendors as it
introduces an increased complexity to the application as well as increasing the amount of
maintenance overhead [1].
Creating a multi-tenant SaaS application with support for deep customisation on the
application layer, as seen in Figure 2, dramatically increases the complexity of the application
as opposed to the other options. The main issue is the increased complexity that is necessary
for the SaaS application to ensure isolation between all the different tenant's data and
configurations [5]. Also, a tenant manager has to be implemented to facilitate the use of the
different tenant’s microservices to customise the main product.
10
1.3 Project Goal
This master thesis investigates how a non-intrusive microservice architecture might help with
the customisation of multi-tenancy Cloud Software. This thesis focuses on the business logic
aspect of the application, but consideration also must be given to the UI and persistence layer
to be able to create a working proof of concept.
To reach this goal, I created a proof of concept that tries to achieve deep customisation in a
tenant-specific manner in a non-intrusive way. Additionally, it must still maintain tenant
isolation, security and privacy concerns.
1.4 Research Method
To gain an understanding of how customisation is achieved in the industry, a literature review
has been performed. After the literature review is complete, a new approach is proposed
taking inspiration from the solutions that were found in the literature review. This approach
aims to give a design pattern that offers as high a customisability as possible, while also
maintaining the tenant isolation that is paramount in a multi-tenant cloud SaaS setup.
This review formed the basis of an approach. A case study has been completed by applying
this design pattern in a reference application that is based on microservice architecture. The
approach goes through multiple iterations based on feedback from sources such as industry
partners, supervisors and the feedback gained during the review process of [6]. The
implementation has been and discussed to find various upsides and possible limitations of the
approach. The review bases itself on a set of requirements and principles that are established.
1.5 Approach
The approach in the case study that this thesis bases itself on is to enable customisation of an
application for tenants by leveraging events and allowing the tenants to create microservices
that have access to the events of that application. The approach requires that the microservice
architecture application that is being customised must be based on event-based collaboration
between the different microservices that make up the application. Then the customising
microservices that are specific and owned by each tenant can leverage the pre-existing events
to achieve the desired customisations. Say, for instance, that a tenant wants an application that
11
supports automatic processing of loan applications and purchases this software from a
software vendor. This software also has support for event-based customisation. As an
additional requirement, the tenant requires that certain documents should be produced after a
loan is granted. The production of documents can then easily be solved by adding a new
microservice for that specific tenant that consumes an event that is generated whenever a loan
is granted. The new microservice can ensure that the required documents are created. The
approach for event-based customisation multi-tenant SaaS is given in Chapter 4, while the
implementation of this approach in a reference application, eShopOnContainers [4], is
explained in Chapter 5.
1.6 Result and Contributions
The case study that has been the main focus of this master’s thesis shows that it is possible to
use event-based customisation to achieve customisability for tenants. Compared to a fully
synchronous approach as in [7], the number of API calls can be reduced by using an event-
based approach, which is also asynchronous.
A short paper [6] that bases itself on the case study in this Master project has been written and
accepted by the Coordination 2020 Conference.
• E. T. Nordli, P. H. Nguyen, F. Chauvel, and H. Song, ‘Event-based Customization of
Multi-tenant SaaS Using Microservices’[6]
Additionally, the approach developed in the Master project has been presented in a workshop
with two companies in the software industry.
1.7 The structure of this thesis
The remainder is structured as follows.
Chapter 2 goes through the various background concepts and technologies required for
supporting a customisable multi-tenant SaaS application. Precisely, brief introductions to
multi-tenant cloud software, customisation, with the underlying concepts controlled and deep
customisation, and intrusive and non-intrusive customisation.
12
Chapter 3 discusses related work in the field. Chapter 0 presents the methods that this thesis
uses, as well as a high-level plan for implementation
Chapter 4 and Chapter 5 contain the approach for achieving event-based customisation and
the specific implementation in the reference application, eShopOnContainers, respectively.
Chapter 6 contains the discussion of the case study, which goes through various upsides and
downsides in this approach. Chapter 7 contains the conclusions of the Masters’ project, while
Chapter 8 gives possible directions for future work based on the discussion.
13
2 Background
This chapter goes through the main background concepts in this thesis. In Section 2.1 multi-
tenant cloud software, then in Section 2.2, the microservice architecture in general, as well as
some of the benefits gained by using it and some of the supporting technologies. Finally, in
Section 2.3 customisation with the underlying concepts; controlled customisation, deep
customisation, intrusive and non-intrusive customisation.
2.1 Multi-tenant cloud software
Multi-tenant cloud software is software that is offered by a vendor as SaaS that runs on the
cloud, with multiple clients being served by the same instance of the application. It is also a
business model that aims at serving as many tenants as possible using as little computational
resources as possible [8]. The fact that the application is shared between multiple tenants
with diverse requirements means that the software vendor has to offer some way of
customisation for each tenant so that the application can be customised to fit their needs.
Multi-tenant applications also have to guarantee tenant isolation, privacy and security, i.e.,
one tenant should not be able to access the data of another merely because they are using the
same instance of an application. The tenant isolation issue leads to the need that multi-tenant
applications have to be multi-tenant aware. Mietzner et al. defined it as follows “Multi-tenant
aware means that the software must appear to each tenant as if he was the sole tenant of the
application (e.g., keeping confidential data private), while allowing multiple tenants to use
the same instance of the application (in order to achieve economy of scale). Multi-tenant
aware applications thus allow providers to provision the hardware and software stack for the
application once and run multiple customers on that very same infrastructure.” [1]
2.2 Microservice architecture (MSA)
The microservice architecture is based on creating a collection of smaller services that
together serve as a single application. Each service is built around a specific need and should
be able to deployed independently from the other services that make up the application [9].
An example of a microservice could, for instance, be a REST API that is responsible for
14
giving information about postcodes for a larger application. Some characteristics and concepts
apply to microservices.
1. Small and focused: An important aspect is that microservices should be small and
focused, meaning that each service should focus on fulfilling one responsibility, e.g.,
an IdentityService for authentication and authorisation. Also, each microservice
should have a separate delivery pipeline and be treated as an independent application.
2. Loosely coupled: Further microservices should be loosely coupled, meaning that each
service should be able to be deployed independently of each other without
coordination between the teams working on the microservices. Additionally, the
microservices should share as little as possible with each other.
3. Language Neutral: Microservices should also be language-neutral, and choose the
technology that fits the issue and developers the best. The language neutrality means
that one could have a mix and match of different programming languages like Node.js,
Java, C#, or any other language.
4. Bounded Context: Bounded Context is a central pattern in Domain-Driven Design
[10]. It means that that a domain is divided into different Bounded Contexts as well as
defining how the different Bounded Contexts integrate [11].
As all the services run as independent processes, they need to be able to communicate with
each other, and there are multiple ways to implement this communication. A REST API might
be the easiest and most intuitive way to implement communication, but there are other
options, like an Event Bus with a dispatch/subscribe pattern. Alternatively, one could use a
remote procedure call implementation like Thrift or gRPC [12] if higher performance is
required.
The book Microservice Architecture: aligning principles, practices and culture by I
Nadareishvili et al. [13] defines three different layered characteristics that should be
considered when adopting a microservice architecture. Modularised microservice architecture
is the first, and this is the most basic level of microservice architecture, it consists of breaking
up the application into smaller parts. The second layer is cohesive microservice architecture;
this step consists of achieving service cohesion by defining the correct service boundaries
based on the existing system. Finally, there is the systematised microservice architecture in
15
which one has to examine the inter-relationships between the different services. This last step
is vital as the system availability as a whole can only be assured if the dependencies between
the services are mapped and understood. These steps, as well as benefits gained in regards to
safety and speed, are illustrated in Figure 5.
Figure 5 Levels of MSA and their benefits [14]
2.2.1 Benefits
Many benefits can be gained by using a microservice architecture, Werner Wogels from
Amazon Web Services has described the advantages that they gained as follows:
“We can scale our operation independently, maintain unparalleled system availability,
and introduce new services quickly without the need for massive reconfiguration.” [13]
As the components of the product or service that a vendor offers can be more granular than if
they were using a monolithic approach, it is also possible to manage the different services
independently and scale each of them based on their usage. The independent scaling leads to
efficiency benefits as fewer resources are wasted on scaling an entire application rather than
the individual parts that have higher resource requirements than other parts of the application.
16
It also means that one could have a higher speed of change which is something that is often
requested by the business part of an organisation [14] as well as higher safety through
independent testability and deployability.
Another of the significant benefits of a microservice architecture is that it decreases the tight
coupling that at times can be found in a monolithic application, which makes it easier to
update a single service without breaking the other services due to loose coupling.
Lastly, some monolithic applications can grow so massive that no one in the organisation
knows all of it. This lack of domain knowledge can lead to errors when a component is
updated if the developer who updated that component is unaware of every single other
component that uses the component that was updated. If that is the case, then the use of
microservices can increase the comprehensibility of a system by dividing it into smaller parts.
Moreover, when another microservice is required, the team that is developing the
microservice only has to relate to the documentation for the API with which they are
integrating. Migration from a monolithic application to microservice can be done over time as
instead of implementing new features as a part of the monolithic application; they can instead
be created and deployed as microservices.
In regards to achieving customisation of a multi-tenant SaaS application, some of the major
strengths of MSA is loose coupling in the application. The loose coupling allows
customisation on a per microservice basis. If the behaviour is still as specified by the
documentation, then no other changes should have to be made to the other microservices that
make up the application. Also, there is no need for a team that is implementing the
customisation to gain an understanding of the application as a whole, but rather, they can
relate to only the microservices that they are customising.
2.2.2 Technologies
A few key technologies that support the usage of microservice architecture is container
technology such as Docker, LXC and OpenVZ [15]. This use of container technology is much
less resource-heavy than using virtual machine images as they include an operating system in
which the application runs. Containers, however, only need to contain the application as well
as the application’s dependencies.
17
In addition to the container technology, there is Kubernetes, which is an open-source
container orchestration middleware created by Google, which offers a uniform API for
deploying and managing all the containers that contain different microservices [15].
2.2.3 Event-driven MSA
There are two different ways that communication can be implemented. Those are:
synchronous and asynchronous. In synchronous communication a call is generally made from
some sort of client to some sort of remote server, this then blocks the client until the operation
is complete. However, in asynchronous communication, the client does not wait for the
response from the remote server before continuing, and it might not even care whether the
operation in the remote server completes.
In an event-driven microservice architecture, the microservices collaborate and communicate
with each using events. Whenever something notable happens, a microservice can publish an
event to a messaging framework. This event can then be consumed by the other microservices
that make up the application if they need to react to the event in any way. In event-based
communication services are not told what to do. Instead, a client publishes an event to a
messaging framework and expects other parties to know what to do. Event-based systems by
their nature are asynchronous.
The client that publishes an event has no way of knowing what other microservices that
consume the event that is being published. This lack of tight coupling also means that new
consumers can be added to these events without the client ever having to know or take it into
account. As event-based communication is asynchronous, it allows the process to continue in
the client to complete the next task once their unit of work is complete, without having to
consider happened before or what occurs next
Some significant benefits of using an event-driven microservice architecture are:
• They are asynchronous, which is a major benefit, as explained above.
• They have inherently loose coupling. Microservices do not need knowledge of what
other services are consuming the events that they produce. The loose coupling means
that the microservices that the application as a whole is made up of can operate
independently without knowledge of the other services in the application.
18
• Further, since the microservices are loosely coupled, and by definition should be fine-
grained, i.e. the microservices should generally only perform one task, it can be easier
to track down any performance bottlenecks and then scale up that service. As the
microservices as fine-grained, it means that it can be easier to scale event-driven
microservice architecture that other types of architectures.
• Lastly, the event-driven microservice architecture allows for recovery support by
replaying events that have already happened, if, for some reason, data has been lost.
Figure 3 on page 7 shows an example of the event-driven microservices architecture in the
eShopOnContainers application. All the microservices that are responsible for the business
logic in the application can be seen to communicate mainly with the Event Bus in the system,
which is either Azure Service Bus or RabbitMQ depending on the configuration.
2.3 Customisation
Essentially there are two different types of customisation, deep customisation which involves
direct modifications of the source code and controlled customisation, which does not.
2.3.1 Controlled customisation
Configuration does not involve source code change of the SaaS application. It usually
supports variations through setting pre-defined parameters, or leveraging tools to change
application functions within a pre-defined scope, e.g., adding data fields, changing field
names, modifying drop-down lists, adding buttons, and changing business rules, etc. [16].
Configuration can support tailoring requirements within the pre-defined configurable limit.
Controlled customisation can also be done with scripting via predefined APIs that are already
exposed by the vendor. However, as the scripting language is not exposed to the same context
as the main application, the capability is limited. Some of the software platforms which
provide this are the Salesforces platform and Oracle NetSuite [2]. For the product to be as
customisable as possible, both vendors create and maintain very extensive APIs, something
that is resource-intensive and costly, and not necessarily an option for smaller software
vendors.
19
Another possible way of achieving controlled customisation is the usage of extension points.
Extension points are a type of customisation in which the software vendor makes a “socket”
of sorts, which requires an implementation within some boundaries, and then the customers of
the software can implement their extensions for that extension point.
2.3.2 Deep customisation
In an empirical study on eight companies deploying ERP systems, Rothenberger et al. [17]
observed that in most cases, ERP adopters end up implementing heavy customisations
involving code modifications. Traditionally, software vendors supported deep customisation
by allowing the customers to modify their source code directly. The customers acquire a
specific license of the application, re-developed it based on the original source code, and then
deployed it on their premises.
There are some complications with this type of customisation, as it could lead to high
maintenance costs whenever installing new releases of the application. The reason for this is
that with this type of deep customisation introduces tight coupling between the customisation
and the main software.
For this thesis, deep customisation is the type of customisation that the tenants implement in
their microservices, as opposed to doing controlled customisation via scripting, extension
points or configuration parameters.
2.3.3 Intrusive
In this context, an intrusive microservice is a way of customisation that injects code directly
into the main product. Deep customisation of an application has already been done in another
project at SINTEF using this technique [2].
2.3.4 Non-intrusive
Non-intrusive, in this context, means that the microservice that is doing the customisation
must not send dynamic code that compiles and runs in the main product as this can cause
security issues. Instead, the customisation use exposed APIs of the main product.
20
3 Related work
This chapter goes through two main categories of state-of-the-Art approaches for achieving
customisation of multi-tenant SaaS. First, in Section 3.1, solutions that offer customisation on
a controlled level; in other words, customisation approaches that are limited by the vendor's
foresight in anticipating the needs of their customers. Then in Section 3.2, solutions that offer
customisability on, or near, the same level as when one would previously redevelop the
application and deploy it on-premises.
Further, these approaches help form the baseline for developing the approach for the Master’s
project. The new approach aims to offer as high customisability as possible, while also
addressing some of the limitations of the current state of the art approaches.
3.1 Controlled Customisation
As mentioned in Section 2.3.1, controlled customisation is a type of customisation that does
not involve source code changes in the SaaS application. The idea of offering customisation
options variability management has been explored by [18] extensively. In practice, many
techniques are employed to enable customisation of SaaS applications. One source of
complexity is the fact that these types of controlled customisation options all require that the
software vendors can foresee all possible options their tenants’ might require.
Several different technical approaches can be taken to address the complexities of enabling
customisation of multi-tenant SaaS applications; such as middleware layer solutions as in [3],
software product lines, design patterns like dependency injection, or APIs made by the
software vendor.
The paper [3] looks into a middleware layer combined with dependency injection approach to
solving the customisation problem prevalent in multi-tenant applications. In their solution, the
multi-tenant middleware runs on top of a platform-as-a-service (PaaS) that allows them to
serve all tenants with the same application instance. The use of dependency injection allows
the solution to inject different variations of software into the application for each tenant.
Further, they use a feature-based approach, where all the variations are then expressed as
different features. A feature can be seen as some service, functionality, quality or
21
characteristic of the software system. Then variation points are created in the base application
in which the tenants can, for each variation point, choose between different features that the
software vendor has implemented for that variation point. These choices are then stored in a
database and retrieved during runtime using the tenant ID, which is a part of the request to the
application. This principle is illustrated in Figure 6, for the base application and the different
features I1 and I2.
Figure 6 Illustration of variation points and different features[3]
The paper focuses on the customisation of component-based multi-tier applications rather
than business processes as the latter requires a different approach where the software
variations are deployed as separate services.
The downside to this approach is that it is limited to the different features that the software
vendor is willing and able to implement, as opposed to allowing the tenant to implement their
microservices for customisation. Another point of contention is that this type of customisation
of multi-tenant SaaS is limited by what the software vendor can anticipate, at design-time.
Additionally, this causes a tight coupling between the main product and the custom code that
is performing the customisation.
There is also the paper [1] which uses variability modelling from software product line (SPL)
to customise SaaS applications. Similarly to [3], they use variation points for which the tenant
can choose between different variations as seen in Figure 7, where for the variation point
Workflow the tenants can choose between the short and the long workflow.
22
Figure 7 Illustration of the usage of variability modelling[1]
As they make use of service-oriented architecture, SOA, this also allows tenants to replace
individual service by making their implementations for that variation point. All the variation
points in the application are either optional or mandatory. The mandatory variation points
always have to be bound to a feature. Further, a variation can exclude or require other
variation points to customise the application. Tenants then have to decide which variants
should be a part of their use of the SaaS application.
Another aspect taken from SPL is the two types of variability that are used:
• External variability which is the different options that are communicated to the tenants
of the application, that they can implement their variations for.
• As well as, internal variability which is only visible to the software vendor of the SaaS
application.
Most customisation approaches to for multi-tenant SaaS applications focus on modifying the
service composition to achieve the customisation. In [18], an approach based on the automatic
transformation of a variability model to a BPEL process is presented. In this work, the
customisation is a recomposed version of the services offered by the software vendor. The
same principle is applied in [19], but the work proposes several layers of compositions. The
composite services are all customisable until the lever of atomic services, which are supplied
by the software vendors.
These approaches are all limited in expression power compared to performing deep
customisation and redevelopment of application, which is no longer an option when running
as a multi-tenant SaaS on the Cloud.
23
3.2 Deep Customisation
The paper [20] presents a novel approach of supporting multi-tenant SaaS by using the
microservice architecture. Every customisation is created as a microservice that can replace
the standard functionality of the main product. Further, the paper details several different
approaches for achieving deep customisation capability and how they address the issues of
expression power and tenant isolation. The work [2] further details this approach and it is
applied in a case study. The paper covers one possible implementation for achieving deep
customisation of an application by using an intrusive microservice to alter the behaviour of
the code based on the currently authenticated user [2]. The customisations for each tenant are
running as separate microservices. They are hosted in the same cloud as the main application.
The solution has a tenant manager that keeps track of the different microservices for the
different tenants, and further which methods should be forwarded to the custom
implementation instead of the standard one. Then the custom microservices make use of
callback code, which means that they send the code to the original service that alters the
behaviour of the method. Figure 8 shows a sequence diagram that illustrates how the solution
works.
24
Figure 8 Sequence diagram for deep customisation with intrusive microservices [2]
There are two main downsides to using this approach to achieve deep customisation firstly,
that the tenant’s microservice has full access to the context in the shared service. In a worst-
case scenario, this could allow them access to another tenant’s data, which is a severe security
issue. Secondly that there is a very tight coupling between the main application and the
tenant’s microservice, which means that updates to the main application could easily break
the tenant’s custom code, something that increases maintenance costs.
Another work that uses the same approach as [2] is [21]. Additionally, [20] suggests the need
for a non-intrusive approach. The intrusive approach offers the expression power required,
however, in practice, it might not be feasible for the industry to adopt as it allows
customisations created by third-party developers to be injected dynamically into the main
product.
The paper [7] redevelops the idea from [2]. This work describes a way of achieving deep
customisation by using non-intrusive microservices, in which tenants of the software each
25
have one or more microservices per customisation that they require. Non-intrusive differs
from the intrusive approach in the previous section in that it does not allow the customising
microservices access to the context of the main product itself, but rather it must achieve the
customisations via fine-grained APIs that are offered by the main product.
A critical aspect of the application that one is customising is that it requires a clear separation
of the frontend and business logic of the application. This separation allows for customisation
of the main product at both the user interface, business logic and database level of the
application. This type of customisation is achieved by introducing some new components to
an existing application, and these are:
• WebMVC Customiser: This is a component which is introduced at the User Interface
level of the main product, which in this case is a WebMVC type application. It
intercepts the different flows in the product and checks for customisations with the
Tenant Manager. If a customisation exists, then that is used, and a customised view is
returned rather than the standard one.
• Tenant Manager: The Tenant Manager is a service that manages and keeps track of
all the customisations for all the tenants that make use of the product. Whenever a new
customisation is activated, it is registered in the Tenant Manager with information
about which tenant the customisation should apply to, what customisation point it
makes use of and also how it extends the database of the main product.
• IAM Service: This is a new service that is used for authentication and authorization,
and it authenticates clients with OpenID Connect or OAuth 2.0 Identity Provider. The
IAM Service is a crucial aspect to maintain tenant isolation at the application level.
• API Gateway: The critical point in the non-intrusive approach is that it enables the
authorized access of the tenants’ customisation microservices to the main product
business logic (BL) via the API gateways. In this way, the tenants’ customisation
microservices can have access to the necessary execution context of the main product
BL if needed. The non-intrusive MiSC-Cloud approach can provide deep
customisation because it allows a customisation service to replace a BL component of
the main product if needed. The authorized access of the tenants’ customisation
26
microservices to the main product BL components makes the deep customisation
manageable.
Along with these new components, changes also have to be made to the Event Bus
implementation to support multi-tenancy. The changes to the Event Bus allows tenants to
subscribe and unsubscribe to events on a tenant-specific level; moreover, it allows tenants to
use events to collaborate between multiple services. All the components mentioned above can
be seen in Figure 9.
Figure 9 An overview of the MiSC-Cloud framework[7]
Since this type of deep customisation requires the use of APIs of the main product, it restricts
the types of customisations to whatever APIs that the software offers in the main product.
However, this dependency of the APIs of the main product allows the software vendors to
control what is open for customisation in the main product. This approach can, according to
the authors of the paper, be nearly as powerful as the intrusive approach presented in [2] and
[20], while being more controllable.
One downside to only using synchronous communication can be the number of calls between
the different services.
The article [22] as well as [23] is another work that is based on the work in [2], [7], and
experiences gained in the work of both and intrusive and non-intrusive approach. It sets out to
27
clarify the problem of multi-tenant SaaS customisation by outlining a conceptual architecture,
which defines the high-level concepts and problems that exist in the context of customisation
of multi-tenant SaaS. Further, a more explicit reference architecture is presented that aims to
achieve the requirements of tenant isolation, assimilation, and still gaining the economy of
scale that is the main selling point of multi-tenant SaaS. Assimilation is defined as a
requirement in [22] that customisations should not affect the performance and user experience
of the SaaS as a whole, i.e. the “look and feel” of application should not change with
customisation when compared to the original SaaS application. Further, the work identifies a
set of technical challenges inherent in implanting the reference architecture and also proposes
technical solutions to solve these challenges.
The conceptual architecture shown in Figure 10 summarizes the main concepts in customising
multi-tenant SaaS. In this model, there are three different roles. Vendors who are the ones
who provide the SaaS that is to be customised. Customers who subscribe to the product
offered by the Vendor. Finally, the Consultant, who on behalf of the Customer, perform any
customisation that they might require. Further, it introduces the concept of Product Services,
which is a version of a product that is running in a Product Environment, and one product
service services multiple tenants at the same time. Vendors often operate multiple product
services for different usages like production usage, testing and debugging. Even with
production usage, there may be multiple product services to achieve high availability. The
high availability can be used to load balance service as well as overcoming region restraints.
Figure 10 A conceptual model of customisation for multi-tenant SaaS [22]
28
The reference architecture relies on a set of principles that are introduced by the authors. First,
there are a set of principles that meet the requirement of tenant isolation.
• Every customisation is a service
• A customisation service serves one and only one tenant
• Each customisation service runs in its environment
• A customisation service has its database
• A customisation service communicates with the product service and other services
only via REST API
The principles introduced to achieve isolation can have adverse effects on economies of scale
or assimilation. The stricter isolation complicates the assimilation with the main product.
Additionally, the isolation leads to more resources being required to run additional services,
which is a drawback when considering economies of scale. The following principles
introduced by the authors aim to alleviate the adverse effects.
• A customisation environment is “close to” its product environment
• The external URL to access a customisation service is consistent with the product
URLs.
• The vendor should provide unified identity management for both product and
customisation.
• Customisation environments should have minimal footprints.
• The vendor should elastically manage all the environments.
• The vendor should facilitate the reuse-by-code for customisation applications.
By using these principles, the paper presents a reference architecture, as seen in Figure 11.
The product is a typical web-based application with one product service serving multiple
tenants at the same time. The Tenant Manager component controls the different
customisations for different flows and tenants to ensure that they are only triggered for the
29
user that belongs to the tenant that required that customisation, the identity manager controls
what users belong to the different tenants. The customisation service might need information
from the main product, and this can be achieved either by invoking an API that is offered by
the service that has the pertaining information or by using call-back code, i.e. non-intrusive vs
intrusive.
Figure 11 A reference architecture of using microservices for customisation [22]
In this architecture, the customisation services are registered in the tenant manager, which
allows it to be triggered in predefined extension points in the product service. The tenant
manager is a component that is a part of the product service, and it registers customisation
services for tenants and different extension points. When the flow reaches an extension point,
the product service checks whether there exists a customisation for the tenant that the user
belongs to and if it triggers the customisation if it exists. The triggering can be either
synchronous in which the product service waits for the customisation service to finish its
invocation, or asynchronous where it does not wait. The synchronous triggering can be used
when the customisation scenario requires the behaviour of the product service to be
influenced immediately. The asynchronous triggering can easily be implemented by event
technology. At each extension point, the product service can publish an event to a message
queue, along with context for the event that is being published, then this event can be
consumed and handled by customisation services to achieve the required customisation.
30
These approaches are closer to achieving the expression power required by software vendors
to have the ability to serve many diverse tenants. However, a fully synchronous approach is
not ideal in a MSA as it increases the coupling between the custom code and main product.
This higher level of coupling goes against the purpose of adopting MSA, and could
potentially increase the technical debt of the software.
31
4 Approach
In this chapter, we discuss the event-based approach for non-intrusive customisation and the
necessary components to enable it. We first present the prerequisite requirements for enabling
event-based customisation of multi-tenant SaaS in Section 4.1 then; Section 4.2 gives the
details of how the event-based customisation approach works. Section 4.3 details the Tenant
Manager and its role and responsibility in the event-based customisation approach. In Section
4.4, we discuss how the event-based customisation approach fulfils the requirements of tenant
isolation. Section 4.5 shows how the approach supports altering the flow of events in an
application by capturing events to allow for custom logic. Finally, Section 4.6 gives an
overview of the proposed architecture by using the eShopOnContainers application as a
reference point.
4.1 Requirements
One crucial requirement for this approach is that the application must be based on
microservice architecture, and additionally use events for collaboration between the different
services that make up the application. Additionally, the tenants’ microservices that perform
any customisations must also be allowed to consume and publish events to whatever
messaging framework the application uses. This access can either be through a direct
connection to the messaging framework, especially if the customisation is running in the same
cloud environment as the main product, or through proxy solutions like Azure Relay or other
ESB solutions.
The Event Bus is vital to enable event-based customisation of multi-tenant SaaS. Therefore, it
is a prerequisite for enabling event-based customisation that the main product already has, at
least a part, of its business logic flow orchestrated via events. If the main product already uses
an Event Bus, the Event Bus of that product has to be extended to enable event-based
customisation. If the main product does not have such an Event Bus, a new Event Bus can be
introduced as presented in [7]. Additionally, it is essential to note applications that do not use
a microservice architecture can be re-developed to allow for event-based communication at
the business logic level via an Event Bus. Several migration approaches can be used to go
from a monolithic architecture to a microservice architecture, and they show some patterns
and best practices on how to migrate from a synchronous based communication in a
32
monolithic application into an event-driven microservice architecture [24][25]. Further, the
creators of the SaaS that is being altered to enable event-based customisation can create user
or system events within their application to allow for authorized event-based integration with
third-party systems(generally of the tenants of the software). This event-based customisation
approach is similar to a more traditional approach of offering a rich REST API or a SOAP
interface for synchronous customisation [7].
Additionally, the application must be multi-tenant aware, meaning that it has to have to
support for differentiating whatever triggered each process per tenant. In the case of a flow
being initiated by an HTTP Request from a user action on a web page, the application has to
be aware of who initiated that request as well what tenant that user belongs. However, HTTP
Requests are not the only action that can trigger a process, as this has to be an event-driven
microservice application each event must also contain information of which tenant that event
belongs. The same is true for any other process that has to interact with tenant data, which
could, for example, be batch jobs or background processes.
Finally, we reuse some of the principles defined in [22] as mentioned in Section 3.2 to help
ensure that tenant isolation is achieved.
• First, every customisation must have it is own service (i.e., microservice)
• Each customisation microservice must have its database if one is required.
• A customisation service communicates primarily by using event-based
communication. However, some cases require synchronous communication as well
(REST API).
• Each customisation microservice serves one, and only one, tenant.
4.2 Customising events
If a tenant wishes to add new behaviour that extends the existing functionality of the main
product without altering the flow of the original events, it can be enough to allow the tenant to
consume an event. Say, for instance, there is a SaaS that offers automatic processing of loan
applications. A tenant of this SaaS wishes to produce certain documents after the loan is
granted and implements this in customising microservice that can consume events of the main
33
product. Then whenever the main product publishes the event that indicates that a loan is
granted, the customising microservice can consume this event and produce the required
documents in their microservice. This flow is shown in Figure 12 where the Main Product
Service on the left-hand side publishes an event to the event bus of the application, and then
this event is consumed by the tenants’ custom service, which implements the elaboration that
the tenant requires. However, some requirements might require that the flow of events is
halted; this is further explained in Section 4.5.
34
Figure 12 Example of generic event handler sequence diagram for the event-based approach
4.3 Tenant Manager
The Tenant Manager is a service that manages the registration of customisation
microservices, including the events registered for customisation for different tenants. The
service has a simple database that stores all the tenants that are using the application, all the
different events that exist in the main product and finally all the customisation microservices
that exist for tenants and specific events. Additionally, it stores an endpoint for each
customisation that is used for halting the flow of events to be discussed further in the next
section. The Tenant Manager provides an administration API that can be used to add, remove
or alter customisation registrations without having to redeploy the Tenant Manager or
accessing the database directly. We keep the tenant registration data on a small scale so that
35
even with many tenants, all data could be cached in memory in the Tenant Manager to
increase performance. Besides, the microservices could also store the same customisation
information in a hot cache or cold cache depending on performance requirements.
4.4 Event isolation
Isolation of events is an integral part of tenant isolation. Tenant isolation means that one
tenant should not be able to access the data of another tenant in any shape or form because
they are using the same instance of an application. Multiple different sources can trigger the
publishing of an event, but in the case of the reference application eShopOnContainers, there
are three different sources. The first type is whenever there is an HTTP request from one of
the clients (WebMVC, WebSPA, Mobile). The second type is when the source of one event
being created is the event handler for another event. Finally, there are the events that are
created in background tasks, which are primarily scheduled tasks that run at a set interval.
The Event Bus implementation and architecture in the main product must ensure that tenant
isolation is still preserved. Instead of having one connection to a single event bus, there must
be multiple connections, one per tenant. One example of such an event bus implementation is
based on RabbitMQ that can make use of virtual hosts. Another way would be to use Azure
Service Bus and setting up a single topic for each tenant, or one could potentially have one
instance of the chosen Event Bus per tenant. Having one instance per tenant could increase
the resource usage, which would go against the ultimate goal of multi-tenant SaaS, which is to
reduce costs by the economies of scale. However, this depends on the chosen product and its
resource overhead when creating more instances.
This way allows us to have a logical separation per tenant, and the permission can easily be
set so that each tenant is only allowed to interact with their virtual host, topic or event bus
instance. The tenant’s microservices, however, only need one single connection to their event
bus. Because there are now multiple connections to different event buses in the main product,
all the events must have the tenant information set, so that the main product knows which
event bus to use. This addition of multiple event bus instances means that the context of the
program must also contain the tenant information, i.e. which tenant owns the current process.
One approach would be to simply pass along the tenant information in the flow of the
program until a new event is created and the tenant information can be set in that event to
36
ensure that it is published to the correct Event Bus. For the events where the source is an
HTTP Request, the tenant information can be extracted from the HttpContext, as it is a
prerequisite as mentioned in Section 4.1 that the application has to be multi-tenant aware. For
the second case, the source of the event is another event handler. In this case, there is the
option of implementing a context similar to the HttpContext. This context could then be made
available to the current process. Finally, for the events that are created by a background task,
the tenant information can either be retrieved from a database or one could run one task per
tenant and pass the tenant information into the task when it is initiated.
37
Figure 13 Generic event isolation class diagram
Figure 13 shows a proposed setup of supporting multi-tenant Event Bus isolation. Wherein
there is a container class MultiEventBus that contains all the actual IEventBus instances, one
38
for each tenant that is making use of the product. The proposed logic is shown in Figure 14,
where, whenever a microservice of the main product service wishes to publish an event, it
interacts with the IMultiEventBus. Then the implementation of this interface finds the correct
Event Bus based on the tenant information in the event. This abstraction could also allow
tenants to make use of different Event Buses as the if they so desire as MultiEventBus
contains a list of interfaces and not a specific implementation. This use of multiple different
Event Buses assumes that the vendor is willing to facilitate this, as it could increase the
complexity of the application. However, it might be beneficial if there are laws that regulate
where a tenant is allowed to process data, and certain Cloud vendors might not have support
for running Event Bus frameworks in all regions.
39
Figure 14 Generic event isolation sequence diagram
4.5 Event capturing
A tenant’s customisation microservice can consume events from that are published to the
Event Bus of the main product, something whenever something of note occurs. An example is
whenever an order is initiated. Then the main product software would publish an event
OrderStartedEvent. When this event is published, it might lead to other microservices, that
make up the application, to publish more events whenever to indicate that they also updated
their business entities in some way. The event bus is designed as a multi-tenant aware
40
interface, with the necessary functionality to subscribe and unsubscribe to events, as well as
publishing them. Therefore, the flow of publishing events in the original Event Bus
implementation must be changed for customisation purposes. Instead of merely publishing to
the event, it instead checks with the Tenant Manager if there exists a customisation. This logic
is shown in Figure 15 and Figure 16. If the event is not customised, then the event is
processed in the standard fashion. Standard fashion meaning that the event is published to the
correct Event Bus instance for the current tenant. On the other hand, if the event is
customised, the event is sent to the endpoint that is part of the response from the Tenant
Manager. At this point, the tenant’s microservice is responsible for storing the event until the
required customisation has been achieved. Then, the tenant’s microservice can republish the
event to the Event Bus, along with a flag that instructs the Event Bus to not check for
customisation again, as to avoid an infinite loop. Note that in some cases when customisation
microservices would require some execution context from the main product that does not
exist in the events that they receive, they still can make authorized synchronous calls to the
APIs of the main product to obtain such context as presented in [21]. This approach is the
same, as described in [6].
41
Figure 15 Generic event capturing activity diagram
42
Figure 16 Generic event capturing sequence diagram example
This type of behaviour can be abstracted into an abstract class which is responsible for
checking whether customisations exists, before either sending the event to the tenant’s
microservice or publishing to an actual instance of an Event Bus. One way of designing this is
shown in Figure 18. This type of pattern is called Template method, wherein the skeleton
method is defined in the abstract class, and the concrete implementation is specified in the
subclass [26].
Here, abstract class AbstractEventBus has a public method Publish; this is the method that the
main product services interact with when they wish to publish an event. Instead of
immediately publishing whenever the publish method of the abstract class is used, the abstract
implementation checks with the Tenant Manager whether there exists a customisation for that
43
event and the tenant of the current user. If there is a customisation, then the event is sent to
the endpoint specified in the response from the Tenant Manager. If no customisation exists
then the event is published to the message queue and consumed by its consumers. This type of
pattern is called Template method, wherein the skeleton method is defined in the abstract
class, and the concrete implementation is specified in the subclass [26].
Figure 17 Class diagram of generic event interface for capturing events
4.6 Architecture
The result of the proposed architecture, using the reference application eShopOnContainers as
an example, can be seen in Figure 18. Wherein the only new component in the main product
itself is the Tenant Manager, the other proposed changes have to be implemented in the
EventBus implementations of the main product itself. Further, any tenant-specific
microservices to implement customisations are separate from the main product itself.
44
Figure 18 Generic architecture adapted from[4]
45
5 Implementation
In this chapter, we show a proof of concept of the approach for enabling deep customisation
of the reference application, eShopOnContainers, by leveraging the existing event-driven
microservice architecture in the application.
This chapter goes through the implementation of the case study in the reference application,
eShopOnContainers. Section 5.1 states the necessary flexibility that the proof of concept
should have to be a valid way of solving the issue of customising multi-tenant cloud SaaS.
Section 5.2 explains why the eShopOnContainers was chosen as a reference application.
Further, Section 0 shows the changes that have been made to eShopOnContainers application.
Section 5.4 details the implementation using Azure Service Bus rather than RabbitMQ, while
section 5.5 shows the implementation of the actual use cases in their microservices that makes
use of the new features to enable the customisations. Section 5.6 summarizes the results of the
case study based on the requirements stated in Section 5.1. The source code for the
implementation is available on GitHub 2.
5.1 Requirements for customisation
This work has mainly focused on customising the business logic of the application. To be able
to achieve the required level of customisation that different tenants might require engineers
should be able to customise anything in the reference application, as one could have done if
the system was deployed on-premises, and deep customisation by redeveloping the
application could be done.
The requirements boil down tenants being able to alter the existing logic in the application,
removing existing logic in the application as well as adding new logic. Also, as this is an
event-driven application, the tenants should be able to publish their events that can then also
be consumed by their microservices. An import aspect of publishing events is that tenant
isolation, privacy and security should still apply.
To sum up:
2 https://github.com/Espent1004/eShopOnContainersCustomised
46
• A tenant should be able to alter existing logic in the application.
• A tenant should be able to remove optional parts of the main product.
• A tenant should be able to extend the existing logic and add to it.
• A tenant should be able to publish their events to the Event Bus, so that the
customising microservices can communicate using a publish/subscribe pattern.
• A tenant should not be able to access and see the events of another tenant.
5.2 Reference application
The open-source application eShopOnContainers has been chosen for three main reasons.
First, eShopOnContainers has a clear separation between the user interface and the business
logic of the application. Secondly, the application follows the microservice architecture, and
as such, has loose coupling as compared to a monolithic application. Finally, the collaboration
between the microservices that the application is made up of is done using events and a
publish/subscribe system.
An overview of the architecture in eShopOnContainers can be seen in Figure 3 on page 7. It
has three different client applications; one mobile application made using Xamarin forms, a
traditional WebMVC application made using C# and a SPA Web application made with
Angular. It also uses a backend for frontend design pattern, where the client applications
interact with an API Gateway which sends the requests downstream to the correct endpoint.
The business logic in the application is contained in the different microservices (see Figure 3
on page 7).
The focus of this Master project is on enabling customisation of the business logic.
Additionally, alterations have also been made to the frontend to display the changes and
additions. For the sake of simplicity, this has only been added to one of the Client
applications, namely the WebMVC client.
These microservices uses an Event Bus for communication between the different
microservices, and the services publish events to this Event Bus whenever something notable
happens. One example of an Event that can be published is whenever a user wants to
47
checkout and place an order (see Figure 19). This cause the event
UserCheckoutAcceptedIntegrationEvent to be published to the Event Bus, this first event
starts the ordering process, which consists of multiple different events but ends in the order
achieving the status Paid.
48
Figure 19 eShopOnContainers checkout page
49
5.3 Changes to the main product
This section goes through the changes to the changes to the main product that are required to
be able to achieve customisation by using events. An overview of the new architecture of the
product can be seen in Figure 20.
Subsection 5.3.1 details the necessary changes to the Identity microservice, which is
responsible for authentication and authorization in the application. A new microservice has
been introduced to the main product, namely the Tenant Manager this is detailed in
Subsection 5.3.2. Subsection 5.3.3 shows the implementation of Event Capturing to offer the
required level of expression power for customisation purposes. Subsection 5.3.4 explains how
tenant isolation is achieved by isolating the Event Bus so that each tenant only can interact
with events that belong to them.
Figure 20 Implementation architecture adapted from [4]
50
5.3.1 Identity API
As the application has to support multi-tenancy, we need a way to identify which tenant owns
the current user. As such, the Identity API has been extended by adding a field in the service’s
database that indicates which tenant owns the user. Figure 21 shows the new TenantId field in
the AspNetUsers table. This table is found in the IdentityDb used by the Identity API.
Figure 21 IdentityDb AspNetUsers table
Besides, an endpoint was added that allows the main product to query with a username and
the Identity API returns which tenant that owns the user. This addition is a necessity as tenant
isolation needs to be ensured, and customisation of one tenant should not affect any of the
other tenants that also make use of the same product. This endpoint is added in a new
Controller in the Identity microservice (see Figure 22).
51
Figure 22 IdentityService endpoint
5.3.2 Tenant Manager
The tenant manager is a new service that manages the customisations of events for the
different tenants. The service has a simple database that stores all the tenants that are using
the application, all the different events that exist in the main product and finally all the
customisations that exist for tenants and specific events. Additionally, it stores an endpoint for
each customisation that is used for halting the flow of events which is discussed further in the
next section. This customisation information can then be queried by other microservices in the
application by using endpoint in the CustomisationsController in the Tenant Manager (see
Figure 23).
52
Figure 23 Tenant Manager endpoint
As the amount of data is quite small, even with many tenants, all this information could easily
be cached in memory in the Tenant Manager to increase performance. Besides, the
microservices could also store the same customisation information in a hot cache or cold
cache depending on performance requirements.
The Tenant Manager also has an API that can be used to add, remove or alter customisations
without having to redeploy the Tenant Manager or accessing the database directly.
5.3.3 Event capturing
As the proof of concept leverages event to achieve customisation for the different tenants that
make use of the application, adding new logic is rather easy due to the loose coupling in the
reference application. The tenant’s microservices can simply consume the events in the main
product to implement uses cases that only add new logic without any need to alter the flow of
events in the main product.
However, for some use cases, one might also want to be able to alter the existing logic in the
application or remove parts of it altogether. As such, we need a way to be able to halt the flow
of events to be able to achieve the required level of customisation. In the proof of concept,
this has been implemented in such a way that publishing events in the Event Bus
implementation have been changed. Instead of merely publishing to all the consumers of the
event, it instead checks with the Tenant Manager, whether the event is customised for the
tenant that owns the event. If the event is not customised, then the event is processed in the
standard fashion by calling the EventHandler for the different consumers of the event. In the
53
case that the event is customised, then the event is sent to the endpoint that is part of the
response from the Tenant Manager. The check for customisation is handled abstractly by
implementing this by using the template method, the abstract class AbsractEventBus checks
for customisation by querying the Tenant Manager, and if one exists, the event is sent to the
specified endpoint. In the case that there is no customisation, the actual implementation of the
publish method ActualPublish in EventBusRabbitMQ is called. At this point, the tenant’s
microservice is responsible for storing the event until the required customisation has been
achieved. At this point, the tenant’s microservice can republish the event to the Event Bus,
along with a property that instructs the EventBus to not check for customisation again, to
avoid an infinite loop. Same as for the approach, this is also detailed in [6]. The code for the
abstract implementation can be seen in Figure 24; this design pattern is called Template
method, which is a design pattern that can be used in object-oriented programming languages
[21].
Figure 24 AbstractEventBus template method
Figure 25 shows the new architecture of the Event Bus implementation in
eShopOnContainers, where the relevant classes for capturing events being the interface
IEventBus, the abstract class AbstractEventBus and the class that implements the interaction
with RabbitMQ EventBusRabbitMQ. There is also an equivalent class to EventBusRabbitMQ
for Azure Service Bus called EventBusServiceBus, but it has been left out of the figure for the
sake of simplicity, and the details of the Azure Service Bus implementation is given in
Section 5.4.
54
Figure 25 Class diagram of event isolation and event capturing implementation
5.3.4 Event isolation
As we, in this implementation, allow the tenants to publish/subscribe to the event bus, as well
as halting the main flow by use of the Tenant Manager. Isolation of events becomes vital to be
able to achieve tenant isolation.
Three different sources can create a new event in the eShopOnContainers application. First,
there is whenever there is an HTTP request from one of the clients (WebMVC, WebSPA,
Mobile). The second type is when the source of one event being created is the event handler
for another event. Finally, there are the events that are created in background tasks, which are
primarily scheduled tasks that run at a set interval.
55
To ensure that tenant isolation is still preserved, the event bus implementation and
architecture in the main product has been altered. Instead of having one connection to a single
event bus, there are multiple connections, one per tenant. As the current event bus
implementation is based on RabbitMQ, we can make use of virtual hosts [27]. The virtual
hosts allow us to have a logical separation per tenant, and the permission can easily be set so
that each tenant is only allowed to interact with their virtual host. The tenant’s microservices,
however, only need one single connection to their event bus. Figure 26 shows the virtual hosts
in the management plugin of RabbitMQ.
Figure 26 RabbitMQ virtual hosts
As there are now multiple connections to different event buses in the main product, this also
means that all the events that we want to publish have to have the tenant information set, so
that the main product knows which event bus instance to use for publishing. The current
approach taken in this case study is to pass the tenant information along with the flow in the
program until a new event is created and published to the event bus. At this point, the tenant
information can be set for that event as well to ensure that it is published to the correct
instance of the event bus. For the events where the source is an HTTP Request, the tenant
information is extracted from HttpContext, wherein the tenant information has been set by the
Identity API. For the second case, where the source of the event is another event handler, then
the tenant information is simply passed along the low of the program. Finally, for the events
that are created by a background task the user name is retrieved from the database that the
background task makes use of, after which the Identity API is queried so that the tenant
information can be set correctly before the event is published.
eShopOnContainers has a class called InMemoryEventBusSubscriptionsManager, which is
responsible for connecting events with their handlers. Originally this implementation makes
use of a standard Dictionary which uses the name of the event as the key and a list of
SubscriptionInfo as the object of that key. This list of subscriptions is then used by the
implementations of RabbitMQ and Azure Service Bus to call Handle method of the handler.
56
This implementation has to be changed to support the new multi-tenant interface, and as such
the key in the dictionary has been changed with a composite object that contains both the
event name as well as the name of the virtual host if RabbitMQ is used, and the topic name if
Azure Service Bus is enabled instead.
One improvement that could be is to add the tenant information as a scoped object that is set
per request, event handler and background task. This improvement could easily be made for
HTTP Request by using the HttpContext, and for event handlers, the same tenant information
could be set in the header of the AMQPContext. For the background task/batch jobs one could
start one per tenant, which could be run in parallel if necessary, wherein the process has
access to the necessary tenant information to be able to publish to the correct event bus
without having to query the Identity API. If one wanted to reduce the complexity of the main
product, this could be taken further by making use of aspect-oriented-programming that could
set the tenant information whenever the constructor of an event is called.
Figure 25 shows the new setup of the Event Bus with multi-tenant support. The
MultiEventBus contains all the instances of IEventBus classes in the application, and it is the
one that is interacted with directly by the main product. Additionally, the interface
IMultiRabbitMQPersistentConnections and the class MultiRabbitMQPersistentConnections
has been added as a container to hold all the connections that are used in the Event Bus
instances.
Besides, the way the microservices set up their connections to the Event Bus in the main
product also must be altered. eShopOnContainers uses either Azure Service Bus or RabbitMQ
depending on the configuration in the docker-compose.override.yml file. First, the Startup
classes set up a connection for the Event Bus based on the configuration (see Figure 27).
Figure 27 RabbitMQ connection setup
57
As there are now two tenants that make use of the product, we generate two connections, one
for each tenant. The next step sets up the Event Bus instances themselves based on the
connections that are stored in IMultiRabbitMQPersistentConnections. These connections are
then added to the MultiEventBus class, this is the class that is used by all the other classes in
the microservices, and it is injected into them with Dependency Injection. The setup of the
RabbitMQ Event Bus instances is shown in Figure 28.
Figure 28 RabbitMQ Event Bus setup
5.4 Azure Service Bus
eShopOnContainers has support for using Azure Service Bus as EventBus rather than
RabbitMQ if that is desired. As Azure Service Bus is generally a closer approximation to a
production environment than RabbitMQ, it has also been implemented in this case study to
show the feasibility of this approach in real-life situations.
The standard implementation of Azure Service Bus in eShopOnContainers makes use of a
topic in an Azure Service Bus instance, with the topic having one subscription per
58
microservice that is accessing the event bus. As in the implementation with RabbitMQ, it is
crucial to maintain event isolation to ensure tenant isolation, i.e., no tenant should be able to
access the date of another tenant. For Azure Service Bus, the case study uses topics in the
same way that Virtual Hosts were used in RabbitMQ. Each tenant has one topic that they
connect to, as well as one subscription in the topic per microservice in the main product and
customisation microservices.
This implementation makes use of the in-built configuration in eShopOnContainers to enable
Azure Service Bus rather than RabbitMQ. The connection strings for the different topics are
set in an environment variable, and then the Azure Service Bus itself is enabled in the docker-
compose file with a flag that is set per microservice.
The alteration in the Startup classes is similar to the RabbitMQ implementation. First, the two
connections are added in the IMultiServiceBusConnections, as shown in Figure 29.
Figure 29 Azure Service Bus connection setup
59
Then these connections are retrieved and used to set up the EventBusServiceBus instances.
One other necessary alteration is the addition of the topic name into the EventBusServiceBus;
this is then used for subscriptions so that the InMemoryEventBusSubscriptionsManager is set
up correctly concerning subscriptions and unsubscriptions. After the two Event Bus instances
have been created, they are added to the MultiEventBus same as for the RabbitMQ setup; this
is shown in Figure 30.
Figure 30 Azure Service Bus Event Bus setup
As we now have multiple EventBus-instances in the main product, we run into the same
problem as in the RabbitMQ implementation, and that is whenever we want to publish an
event, we need to know which EventBus should be chosen. This issue of multiple Event
Buses is solved by the same implementation as RabbitMQ, in which we find the correct
EventBus-instance based on the current context that contains information on which tenant that
owns the current process.
Compared to RabbitMQ, the use of Azure Service Bus in eShopOnContainers requires quite a
bit more work to set up. The project contains scripts to set up the necessary structure of the
Azure Service Bus for the microservices that are already in the project. The script sets up one
topic called eshop_event_bus that is used by the application. Additionally, the script generates
60
subscriptions for all the microservices in the project, correlation filters are also set up for each
subscription which matches on the name of the event.
Figure 31 shows an overview of the Azure Service Bus in the Azure Portal, where there is two
topics, eshop_event_bus_tenanta and eshop_event_bus_tenantb for Tenant A and Tenant B,
respectively. Further, Figure 32 shows all the subscriptions that are in the
eshop_event_bus_tenanta topic and Figure 33 shows the filters in the Basket subscription,
which matches the events that the Basket microservice subscribes to.
Figure 31 Azure Service Bus topics
Figure 32 Azure Service Bus topic with subscriptions
Figure 33 Azure Service Bus subscription with filters
61
5.5 Use cases
Two different uses cases have been created to test the multi-tenant customisation framework
that has been implemented in the reference application. The use cases are used to see if the
approach designed in Chapter 4 is feasible and offers the required level of expression power
to satisfy software vendors needs.
• Use case 1: Shipping information
o As a tenant, I would like to use the order information to calculate the shipping
information for every order.
• Use case 2: Radio-frequency identification
o As a tenant, I would like to mark all my goods with RFID. Before the order
status is set to confirmed, all the order lines in the order should be scanned,
and the order status should only be set to confirmed when all the items in the
order have been scanned.
Additionally, these use cases have been chosen as they satisfy different requirements for
customisation, as mentioned in Section 4.1. Use case 1 adds new logic to the main flow of the
ordering process, without altering any of the existing functionality. Use case 2 requires
alteration of the existing logic of the ordering process by halting the flow of the order.
5.5.1 Implementation of use cases
This subsection details the implementation of the use cases in their separate microservices,
how they use the new additions to the main product to facilitate the customisations and finally
show the results of the changes in the user interface.
As the external services themselves are not the focus of this work, they have been mocked.
For the shipping service, the mocked service simply uses today’s date for the shipping date
and then sets the estimated arrival date two days from now. For use case 2, an endpoint has
been added that whenever it is called with an order number, essentially sets that all order lines
have been scanned for that order.
Use case 1
The shipping information use case is relatively simple to implement. As there is no need to
alter or remove any existing logic in the application, we can simply add a new Event Handler
that consumes the OrderSubmittedIntegrationEvent as seen in Figure 34.
62
Figure 34 Customisation microservice consumer main product event
Figure 35 details the sequence of the customisation. Whenever this event is published by the
main product to the event bus of the tenant, the Event Handler consumes the event and calls
the TenantAShippingInformation Service, which is responsible for calculating the shipping
information. This information is then stored in the microservice’s database, which can then be
retrieved whenever the My Orders page is displayed.
63
Figure 35 Sequence diagram of shipping information use case
To show the changes of this customisation changes have also been made to the
OrderController in the WebMVC project. The OrderController now checks the current
tenant, and if the current user belongs to Tenant A, then the shipping information is retrieved
from TenantAShippingInformation-microservice and displayed on the My orders page. The
result of the customisation is seen on the My Orders page, as seen in Figure 36.
Figure 36 Use case 1 result
64
Use case 2
The second use case requires that the status of the order is not set to confirmed until all the
order lines in the order have been scanned. To ensure this, we need to halt the flow of the
application by capturing the OrderStatusChangedToAwaitingValidationIntegrationEvent. The
flow of events is halted by registering this event for the specific tenant in the Tenant Manager.
Additionally, an endpoint to use for the event is also added. The registration in the database of
the Tenant Manager microservice can be seen in Figure 37.
Figure 37 Tenant Manager registration of customisation
This event is then stored in the database of the microservice for this customisation until the
RFIDTagScannedIntegrationEvent is published by the TenantBRFIDService, as seen in
Figure 3. This use case also satisfies the requirement that tenants should be able to publish
events that their microservices can consume.
The use case starts when the Ordering service publishes the
OrderStatusChangedToAwaitingValidationIntegrationEvent. Next, the abstract Event Bus
implementation checks for any customisation for this event by querying the Tenant Manager.
As Tenant B has customised this event, the Event Bus sends the event to the endpoint
specified in the response from the Tenant Manager rather than publishing to the Event Bus
instance. At this point, the tenant has control of the event, and they save it to the local
database of TenantBEventService before publishing
OrderStatusChangedToAwaitingValidationEventSavedEvent to the Event Bus. This event
indicates that the event has been saved to the database of TenantBEventService. This event is
then consumed by the OrderStatusChangedToAwaitingValidationEventSavedEventHandler in
TenantBRFIDService. This event handler stores the necessary information in its database. The
next step of the use case is triggered whenever the endpoint in TenantBRFIDService is used to
65
indicate that all the order lines have been scanned. This endpoint can be seen in Figure 38.
Figure 38 Mock endpoint to indicate that all RFID tags have been scanned
As mentioned at the start of Subsection 5.5.1, this behaviour has been mocked, and it is
triggered by a call to the endpoint by using Postman. The postman call is shown in Figure 39.
Figure 39 Mock Postman call
The use of this endpoint also triggers RFIDTagScannedIntegrationEvent, which is then
consumed by the RFIDTagScanndIntegrationEventHandler in TenantB Event Service. At this
point, the original OrderStatusChangedToAwaitingValidationIntegrationEvent is re-published
to the Event Bus, and the handlers in the main product can perform their operations. A
sequence diagram that details the use case in its entirety is shown in Figure 40.
66
Figure 40 RFID Use case sequence diagram
67
The result of the customisation, before all the RFID tags are scanned, can be seen in Figure
41. The result of the customisation after the RFID tags have been scanned can be seen in
Figure 42.
Figure 41 Result of Use case 2 before RFID tags have been scanned
68
Figure 42 Result of Use case 2 after RFID tags have been scanned
5.6 Result of requirements
If we consider the requirements in Section 5.1, then the current proof of concept and use cases
satisfy the majority of them. The ability to override existing logic is shown in Use case 2
(RFID Use case), when we can halt the flow of events by capturing an event, and only
republishing that event when all the RFID tags have been scanned.
The shipping information use case allows the tenant to add new logic on top of the already
existing implementation in the application. In contrast, the ability for tenants to publish their
unique events is shown by the use of RFIDTagScannedIntegrationEvent and
OrderStatusChangedToAwaitingValidationEventSavedEvent in the RFID use case.
While no implementation has been attempted to remove optional existing logic in the
application, this seems feasible by adding the event that initiates the logic that a tenant wants
69
to remove as a customised event in the Tenant Manager. Then the tenant can discard this
event. However, this assumes that this event is optional and that the ordering flow does not
depend on this event to finish the process. On the other hand, if an application has optional
Event Handlers, an ideal solution could be to simply use configuration to turn this off or on
for tenants.
The results of the requirements for customisation can be seen in Table 1.
As all the customisation is based on leveraging the events in the application, and all the events
are also isolated so that each tenant is only able to interact with their events, this also means
that tenant isolation is still preserved.
Requirements Use case 1 Use case 2 #
Override existing logic ✓ ✓
Remove existing logic
Add new logic ✓ ✓
Publish events ✓ ✓ Table 1 Results of requirements
If we revisit the principles as defined in Section 4.1:
• First, every customisation must have it is own service (i.e. microservice)
• Each customisation microservice must have its database if one is required.
• A customisation service communicates primarily by using event-based
communication. However, some cases require synchronous communication as well
(REST API).
• Each customisation microservice serves one, and only one, tenant.
We see that the implementation aligns with these principles. A single microservice,
TenantAShippingInformation implements the shipping information use case. Additionally,
this microservice has its database which is used to store the mocked shipping information for
each order for Tenant A. Further, the only type of communication between this microservice
and the main product is via the Event Bus implementation in the product. Tenant isolation is
70
also achieved by ensuring that events are isolated for each tenant, and a tenant’s custom
microservice is only allowed to interact with events they own.
The same is true for Use case 2. The RFID implementation consists of two custom
microservices for Tenant B; namely TenantBRFIDService and TenantBEventService.
However, TenantBEventService could and should be reused if Tenant B wishes to implement
more customisations that require the flow of events to be altered. In the same way, as for Use
case 1, these microservices only communicate via events. However, the process of sending
the captured event from the main product to the customisation microservice is done via a
REST Endpoint.
71
6 Discussion
In this section, we discuss the results of the case study, as well as the pros and cons in the
implementation of the event-based customisation approach.
The event-based architecture alone might not give full expression power for a tenant to
achieve all the customisations that they might require. The reason this might occur is that the
events that are being published in the main product might not contain enough context to be
able to perform all the operations. Additionally, if the main product does not provide suitable
APIs that can provide necessary data to implement the desired behaviour, it may be
challenging to implement the requirement without customising a large part of the system. To
this end, the main product must be based on the microservice architecture, which is by
definition, fine-grained.
Another aspect is that the event-based approach requires tenants’ software developers to
integrate tightly with the system, in other words, they need to know all the available events,
what the event indicates and what information is available in that event. The leads to the need
for strict rules for the naming of events, and also proper documentation has to be available to
the tenants’ software developers. This need for proper documentation is not that different
from integrating a with a third-party API, and it should be easy to generate Swagger like
documentation for all the events in the main product. This documentation could then be made
available to all tenants through a wiki page or something similar.
Two different event buses were implemented in the case study, RabbitMQ and Azure Service
Bus. These two were chosen as they were both already implemented in the reference
application, eShopOnContainers. RabbitMQ was ideal for quick prototyping and was
implemented first in the case study. However, RabbitMQ might not be ideal in a production
environment and as such Azure Service Bus was also implemented. The RabbitMQ uses
RabbitMQ’s built-in virtual host to ensure tenant isolation and not allowing one tenant access
to events of another tenant. In Azure Service Bus there is no such feature, and as such the
implementation made use of one topic per tenant. The two different approaches are shown in
Figure 43.
72
Figure 43 Potential event bus setups
Another possible way of solving this in Azure Service Bus is to have one instance of Azure
Service Bus per tenant simply has purchased the main product. Using multiple Azure Service
Bus instances is also possible with the current implementation of the proof of concept, as the
topics generated in Azure Service have different connection strings for each topic. This setup
is shown in Figure 44.
73
Figure 44 Multi-instance event bus setup
This implementation also better shows that this type of implementation is feasible with other
types of messaging frameworks. Other examples of messaging frameworks that could be used
are Amazon EventBridge and Apache Pulsar.
One downside to using Azure Service Bus over RabbitMQ is the need for extra configuration
of the Azure Service Bus instance itself. If a tenant wants to add new events that they can
consume in their customising microservices, then first a new subscription has to be added to
the tenant’s topic for that microservices. After this has been done, filters have to be set up in
the subscription, one for each event that the tenant wishes to consume in the microservices
that matches the subscription. This setup is quite a bit more work compared to RabbitMQ,
which simply works out of the box without any need for additional configuration.
A significant upside to an event-based approach compared to a fully synchronous approach is
that it can help reduce the number of API calls that are being performed to achieve the
customisation. This reduced number of API calls is desirable as unpredictable loads caused by
tenants performing many API calls can cause performance bottlenecks that can slow the down
the entire system, which could also affect other tenants. Additionally, if the customisations
are being developed either by the tenants’ software developers or some other third-party
developers, they might not correctly use the APIs in a fully synchronous approach. This issue
of misuse could lead to the software vendor having to review and approve each customisation
to ensure it does not misuse the system. However, in an event-based approach, the context of
74
the event should have enough information to achieve the desired result in many cases and
might not require any API calls at all. This reduced number of API calls would be a
significant upside as having to review and approve each, and every customisation would be
time-consuming and costly.
An event-based approach for achieving customisation of multi-tenant cloud SaaS is only one
direction that software vendors might choose. One other direction that might be taken is a
webhook based approach. This type of approach can be especially suited for integration with
third-party systems, as the webhook allows the tenant to register the system that they want to
integrate with the webhook.
The asynchronous way of customisation requires more advanced UI implementation to
provide a better UI experience for users. This increase the cost of enabling customisation for
tenants of multi-tenant SaaS.
One key aspect of event-driven MSA and the event-based approach is that it increases the
complexity of the application. Say for instance a tenant sets a customisation for an Event in
the Tenant Manager, but registers the wrong endpoint for that customisation. This error would
lead the event being lost, and it would never be republished to the application. These types of
errors could be tricky to track down as the event is assumed by the main product to be sent to
a customisation microservice for which the tenant has responsibility. However, the tenant
would never receive the event as the endpoint is incorrect. These types of situations require
that the main product software, and also the customisation microservices, have logging that is
easy to access and read.
In the proof of concept, the main product and customisations are all assumed to run in the
same environment, and the tenant’s customising microservice have access to the Event Bus of
the main product. However, this might not be ideal or even possible in a real-world scenario.
Commercial off the shelf offerings like Azure Relay, or other frameworks with the same
capabilities, can prove to be a great boon to solve this problem. The Azure Relay service can
make the implementation of our approach even more accessible because it allows tenants to
securely expose customisation microservices that run in their corporate network to the main
product SaaS that is running on the public cloud.
75
7 Conclusion
In this Master project, we have presented an event-based customisation approach that is part
of a non-intrusive customisation framework for multi-tenant SaaS. Further, we apply that
approach in the reference application, eShopOnContainers.
As event-based communication is asynchronous, this also means that the customisation
microservices perform their customisations asynchronously when interacting with the main
product business logic for customisation purposes.
The use of event-based communication between the main product and the microservices that
perform the customisations is vital to maintain the benefits gained from adopting the MSA,
but also for enabling non-intrusive deep customisation capability.
Combining an asynchronous and synchronous approach such as [7] offers a more flexible
approach for the coordination of customisation of the microservices, which are responsible for
the business logic, of the main product with the customisation microservices of the tenants to
achieve the desired elaborations that the tenants require in a multi-tenant SaaS context.
A crucial aspect for vendors of multi-tenant cloud SaaS applications is ensuring tenant-
isolation, our approach shows how this can be maintained by altering the messaging
framework to be multi-tenant aware.
Another upside to an event-based customisation approach is the ability to reduce the number
of API calls, as opposed to a synchronous approach.
The architectural pattern, event-driven MSA, is already in use in the industry today. The
event-based approach of enabling customisation supports extensions of the existing business
logic of an application by allowing the tenant’s microservices to subscribe to the relevant
events. Allowing tenants to interact with the events in their microservices only requires that
main product SaaS is multi-tenant aware, and that tenant isolation is maintained. The
approach of only implementing event isolation could serve as a starting point for software
vendors that need to support deep customisation capability to their tenants. Tenants would
have less customisation capability as opposed to applying the full approach to their software
product, but it would serve as a more accessible starting point for the software vendor. The
event-based approach would most likely require significant changes to an existing industry
76
application and would require rigorous testing. Applying the approach in multiple stages
could reduce the risk of errors and also allow software vendors and their tenants better control
over the changes to the software product.
77
8 Future work
This section focuses on some important points of future work that can address the
shortcomings of the current implementation.
The scope of this Master project has been limited to the business logic of an application.
Rather than the application as a whole, which also includes the user interface, i.e., frontend,
and the database. As mentioned in Chapter 5, the frontend is hardcoded to retrieve the
customised information, if that specific page is customised for that tenant. One point of future
work is implementing mechanisms to automatically fetch customised information, instead of
having to hardcode per customisation. One venue for solving this can be to use the existing
tenant manager to register in what ways the tenant’s customisations extend the current date
structure that exists in the main product. Additionally, the main product would have to be
informed where it can retrieve this customised date.
This Master project bases itself on enabling customisation by giving tenant’s microservices
access to events, and then adding consumers to those events or halting them to perform other
operations before the main product should consume that event. However, this means that
tenant’s customisation microservices is bound by the context of the event that is being
published by the main product, in other words, the only information available to the
customisation is the information is contained in that event. For most customisations, this
information might be more than enough to achieve the customisation that the tenant requires,
but some more complex use cases might require more information from the APIs supplied by
the main product. Therefore, another point of future work is to combine the event-based
customisation approach with a synchronous one to give the tenants the possibility of as full
expression power as possible.
In the case study of this Master project, the customisation microservices and main product all
run in the same environment. However, in a production environment, this might not be
feasible or a wanted design pattern. One example of when this could occur is integration with
third-party systems. For example, one tenant has a pre-existing solution that they want to
extend, so it can perform some of the customisations that they require from the main product
that is being customised. The environment that this third-party system runs on might be the
tenant’s system, another cloud provider or the systems of another software vendor. However,
78
to achieve event-based customisation, the system that is performing the customisation has to
have access to the Event Bus used in the main product. The problem is that the Event Bus of
the main product is unlikely to be open to access from outside the environment that is running
in. This lack of accessibility means that another point of future work is to enable access to the
Event Bus from outside the same cloud environment that is running on. One way of solving
this is through commercial off the shelf products like Azure Relay, some sort of Enterprise
Service Bus (ESB) or a proxy solution, or it might require a custom implementation to get the
expression power required. These solutions would enable the software vendor to give access
to Azure Relay, ESB or proxy rather than giving access through the firewall for the Event Bus
directly.
Additionally, enabling event-based customisation of multi-tenant SaaS allows for offloading
of custom code Edge devices. The messaging framework in the multi-tenant SaaS application
could be open to events running on Edge devices and possibly even “things” in the IoT
context.
79
References
[1] R. Mietzner, A. Metzger, F. Leymann, and K. Pohl, ‘Variability modeling to support
customization and deployment of multi-tenant-aware software as a service applications’,
in 2009 ICSE Workshop on Principles of Engineering Service Oriented Systems, 2009,
pp. 18–25.
[2] F. Chauvel and A. Solberg, ‘Using intrusive microservices to enable deep customization
of multi-tenant SaaS’, in 2018 11th International Conference on the Quality of
Information and Communications Technology (QUATIC), 2018, pp. 30–37.
[3] S. Walraven, E. Truyen, and W. Joosen, ‘A middleware layer for flexible and cost-
efficient multi-tenant applications’, in ACM/IFIP/USENIX International Conference on
Distributed Systems Platforms and Open Distributed Processing, 2011, pp. 370–389.
[4] Microsoft, ‘eShopOnContainers’. https://github.com/dotnet-
architecture/eShopOnContainers (accessed Jun. 02, 2020).
[5] C. J. Guo, W. Sun, Y. Huang, Z. H. Wang, and B. Gao, ‘A framework for native multi-
tenancy application development and management’, in The 9th IEEE International
Conference on E-Commerce Technology and The 4th IEEE International Conference on
Enterprise Computing, E-Commerce and E-Services (CEC-EEE 2007), 2007, pp. 551–
558.
[6] E. T. Nordli, P. H. Nguyen, F. Chauvel, and H. Song, ‘Event-based Customization of
Multi-tenant SaaS Using Microservices⋆’.
[7] P. H. Nguyen, H. Song, F. Chauvel, R. Muller, S. Boyar, and E. Levin, ‘Using
microservices for non-intrusive customization of multi-tenant SaaS’, in Proceedings of
the 2019 27th ACM Joint Meeting on European Software Engineering Conference and
Symposium on the Foundations of Software Engineering, 2019, pp. 905–915.
[8] S. Walraven, D. Van Landuyt, E. Truyen, K. Handekyn, and W. Joosen, ‘Efficient
customization of multi-tenant software-as-a-service applications with service lines’, J.
Syst. Softw., vol. 91, pp. 48–62, 2014.
[9] M. Fowler, ‘Microservices a definition of this new architectural term’.
https://martinfowler.com/articles/microservices.html.
[10] M. Fowler, ‘BoundedContext’. https://martinfowler.com/bliki/BoundedContext.html
(accessed Jun. 02, 2020).
[11] D. Jaramillo, D. V. Nguyen, and R. Smart, ‘Leveraging microservices architecture by
using Docker technology’, in SoutheastCon 2016, 2016, pp. 1–5.
[12] gRPC, ‘About gRPC’. https://grpc.io/about/.
[13] I. Nadareishvili, R. Mitra, M. McLarty, and M. Amundsen, Microservice architecture:
aligning principles, practices, and culture. O’Reilly Media, Inc., 2016.
[14] N. Alshuqayran, N. Ali, and R. Evans, ‘A systematic mapping study in microservice
architecture’, in 2016 IEEE 9th International Conference on Service-Oriented
Computing and Applications (SOCA), 2016, pp. 44–51.
[15] E. Truyen, D. Van Landuyt, V. Reniers, A. Rafique, B. Lagaisse, and W. Joosen,
‘Towards a container-based architecture for multi-tenant SaaS applications’, in
Proceedings of the 15th International Workshop on Adaptive and Reflective Middleware,
2016, pp. 1–6.
[16] W. Sun, X. Zhang, C. J. Guo, P. Sun, and H. Su, ‘Software as a service: Configuration
and customization perspectives’, in 2008 ieee congress on services part ii (services-2
2008), 2008, pp. 18–25.
[17] M. A. Rothenberger and M. Srite, ‘An investigation of customization in ERP system
implementations’, IEEE Trans. Eng. Manag., vol. 56, no. 4, pp. 663–676, 2009.
80
[18] R. Mietzner and F. Leymann, ‘Generation of BPEL customization processes for SaaS
applications from variability descriptors’, in Services Computing, 2008. SCC’08. IEEE
International Conference on, 2008, vol. 2, pp. 359–366.
[19] W. Tsai and X. Sun, ‘SaaS Multi-tenant Application Customization’, in 2013 IEEE
Seventh International Symposium on Service-Oriented System Engineering, Mar. 2013,
pp. 1–12, doi: 10.1109/SOSE.2013.44.
[20] H. Song, F. Chauvel, and P. H. Nguyen, ‘Using Microservices to Customize Multi-
tenant Software-as-a-Service’, in Microservices, Springer, 2020, pp. 299–331.
[21] H. Song, F. Chauvel, and A. Solberg, ‘Deep customization of multi-tenant SaaS using
intrusive microservices’, in Proceedings of the 40th International Conference on
Software Engineering: New Ideas and Emerging Results, 2018, pp. 97–100.
[22] H. Song, P. H. Nguyen, and F. Chauvel, ‘Using Microservices to Customize Multi-
Tenant SaaS: From Intrusive to Non-Intrusive’, in Joint Post-proceedings of the First
and Second International Conference on Microservices (Microservices 2017/2019),
2020.
[23] H. Song, P. H. Nguyen, F. Chauvel, J. Glattetre, and T. Schjerpen, ‘Customizing Multi-
Tenant SaaS by Microservices: A Reference Architecture’, in 2019 IEEE International
Conference on Web Services (ICWS), 2019, pp. 446–448.
[24] D. Taibi, F. Auer, V. Lenarduzzi, and M. Felderer, ‘From Monolithic Systems to
Microservices: An Assessment Framework’, ArXiv190908933 Cs, Sep. 2019, Accessed:
Jun. 08, 2020. [Online]. Available: http://arxiv.org/abs/1909.08933.
[25] M. Mazzara, N. Dragoni, A. Bucchiarone, A. Giaretta, S. T. Larsen, and S. Dustdar,
‘Microservices: Migration of a Mission Critical System’, IEEE Trans. Serv. Comput.,
pp. 1–1, 2018, doi: 10.1109/TSC.2018.2889087.
[26] E. Gamma, R. Helm, R. Johnson, and J. Vlissides, ‘Design Patterns: Abstraction and
Reuse of Object-Oriented Design’, in ECOOP’ 93 — Object-Oriented Programming,
Berlin, Heidelberg, 1993, pp. 406–431, doi: 10.1007/3-540-47910-4_21.
[27] RabbitMQ, ‘Virtual Hosts’. https://www.rabbitmq.com/vhosts.html (accessed Jun. 09,
2020).