Whitepaper

Enterprise Service Bus in Telecommunication Domain

Author: Ritesh Arora Department: Enterprise Application Integration (EAI) TeamCompany: Wipro TechnologiesEmail: (ritesh.arora@wipro.com)

Abstract

This Whitepaper discusses the use of an Enterprise Service Bus (ESB) in Telecommunication domain and addresses the key challenges posed by traditional integration products/methodologies. However, this Whitepaper does not cover an end-to-end ESB architecture

Keywords

CSR EAI ESB OSS BSS XML SOA JMX MOM ETL SLA JCA

BackgroundIn today’s world of emerging IT technologies, business integration has become the top priority and concern for any company worth its salt. In the course of business integration, attention is focussed on achieving the integration quickly but at the same time expecting an early return on investment. In the last two decades, the IT Industry has seen the life cycle of integration starting from traditional MOMs and then graduating to vendor driven EAIs along with various other Enterprise integration technologies. The latest and one of the most promising architecture to have emerged during this period is Enterprise Service Bus. An ESB oriented solution addresses lots of key challenges in the integration space and is out to establish itself as a key trend

Introduction

The ESB approach to integration can provide the underlying integrated solution as a service based, loosely coupled, highly integrated and widely available network that extends beyond the boundaries of a traditional hub and spoke EAI broker. An ESB has the following characteristics that we would touch based upon in the later sections of the Whitepaper.

Highly Adaptable Distributed Ability to Selectively deploy Integration components Secure and Reliable Ability to orchestrate processes Monitoring

The Telecommunication Industry forms a business case for using Enterprise Service Bus as an EAI broker particularly for an OSS/BSS solution. This white paper focuses on how ESB addresses the key challenges of integration in Telecommunication domain.

ScopeThis Whitepaper discusses the use of an Enterprise Service Bus (ESB) in Telecommunication domain and addresses the key challenges posed by traditional integration products/methodologies. However, this Whitepaper does not cover an end-to-end ESB architecture

Business CaseIn a typical OSS/BSS environment, there are various disparate systems involved, from customer servicing system to the Provisioning, Billing and Mediation systems. In addition, this niche of OSS/BSS environment consists of numerous third-party systems in order to complete the communication backbone of the Enterprise. These systems are technically and functionally variant and connected using a single or multiple integration methodology/products. All these various pieces are wired together and this accomplishes the OSS/BSS architecture.

CSRs obtain the data from the customers on a regular basis and populate the database with the customer details. The CRM System communicates with various other systems like Billing, Order Management and Provisioning to facilitate the order processing. Bearing in mind that all these applications/systems are different, they communicate in a fashion that works best for them and not necessarily in a way that is best for maintaining the elasticity of the architecture framework. The following section lists the problems that arise while implementing OSS/BSS Telecommunication architecture with traditional integration methodologies.

Shortcomings of Traditional EAI

Lack of adoption of standards

In the Enterprise Industry, standards are the outcome of the Industry experience and best practices. The same criterion is applicable to the Telecommunication Industry .In the Telecommunication Industry, disparate applications like CRM, Order Management System and Provisioning are usually integrated using traditional EAI products/MOMs. The product/s have/has their inward focussed implementation of using the integration framework, i.e. proprietary data format, connectors, adapters, and communication protocols. Most of the EAI products in

Figure 1: A typical OSS/BSS EAI solution

CRM

ThirdParty

Integration communication Layer

Proprietary communication methodology

Third Party

Self Care

Third Party

OrderManagement

Third Party

Billing Mediation

Third Party

question do not fully adhere to the worldwide Industry standards such as XML ,JCA ,Web services, and JMS. Let’s take a typical example of processing a new customer order. As described in the sections above, CSRs pass in the customer details to CRM in a proprietary fashion. The proprietary approach could vary from using the FTP protocol from contact centres to executing traditional processing scripts. CRM, on having obtained the customer details, starts processing the data by sending requests or invoking API of other associated applications. The connectivity between the CRM system and other communicating applications is usually established using traditional EAI broker/MOM. This traditional product in question will have its own proprietary way of communicating with the rest of the systems/applications in terms of data, format and connectivity. Use of this type of inward focussed product/solution architecture makes the Telecommunication architecture locked, complex and deviated from standards thereby posing more unseen risks. Subsequently, once this Integration methodology gets established in the communication architecture, it becomes a costly exercise to scale and manage the entire system

Proprietary development interfaces

All the traditional EAI products have their own proprietary development interfaces and in the event of migration from one product to another, it can be a nightmare for the Enterprise. For example, if a company uses a product and for some reason the company has to migrate to another EAI product/integration methodology, then the whole migration activity is a big and complex journey. In this situation, it even becomes difficult to use standard components as the proprietary vendor methodology is used in developing interfaces. Even though a work around (plugging in standard components) is always available, it cannot resolve all the connectivity problems but instead it gives rise to mainly three types of issues:

1. The non-standard patch and play weakens the communication architecture and introduces new points of failure.

2. Basic inter component communication requirements are compromised.

3. In the whole exercise, the solution architecture moves further away from standards.

Even if the new breed of EAI brokers use SOA based in the communication architecture, they are normally inward focussed. Let us imagine a scenario in which web services are used to pick up the data from a customer-porting database. These web services would have been developed using a specific vendor product. In case the organization wants to migrate to a different application server, web services cannot be simply plugged and played and would need to be re-written up to a good extent.

Batch transfer using ETL

Integration brokers are not the only player in integration space but there are several other methodologies like ETL. Today ETL has become a non-separable component of Telecommunication industry due to its quick extraction, transformation and loading capabilities. Even though it is fast and popular, it induces certain bottlenecks in the communication flow. Due to latency in nightly batch updates between the business event and the real inventory, there is always a probability of a considerable difference between the real data and expected data. This data can be highly unsynchronised in case if the sources of data are globally distributed. This difference between the expected inventory and actual inventory can have big impact on availability and performance business services. The normal way of tackling this latency is to over bloat the inventory

In Telecommunication industry, where the services are synchronous, the traditional ETL scripts posses certain challenges like unpredictable connectivity errors, excess inventory and locking of business resources. We will see in subsequent sections, how ESB can use file services to over come the challenges posed by ETL services by reducing the errors there by providing a reliable and highly available data and re -using the resources as and when needed.

Cost equation Since huge amount of money is spent in research and development related activities for the traditional EAIs/Integration products, the licensing fee also follows a similar graph. Hence, the break-even point for projects of large enterprises that implement the traditional EAI broker/MOMs is considerably late. However, this is not the only cost factor involved. Let us imagine a scenario when the Enterprise would want to use a different integration broker/methodology. Redesigning the entire integration architecture would be a costly exercise for the company. Additionally, an extensive amount of project cost is associated in maintaining the resources that possess the appropriate skills.

Tightly coupled integration broker and Application server

In the current integration pattern between integration broker and application server, they are tightly coupled with each other. Integration broker (with or without application server stack) will require Application server at every integration end. This means that where every JCA adapter is required to connect, the architecture would need to have the deployment of application server acting as JCA container.

ESB in action in Telecommunication IndustryHaving identified a case for standard based integration required between disparate systems as depicted above, in this section we would identify why and how an ESB can be the right fit to the key challenges in the Telecommunication Industry.In the Telecommunication Industry, the Enterprise Service Bus (ESB) provides a new way to build and deploy Enterprise service-oriented architectures (SOA). The true value of the ESB is to provide suitable services within SLAs managing the service provided to operate and integrate in a heterogeneous technology environment. The diagram below depicts the components of ESB that can be used to form the communication backbone in a Telecommunication Industry

Enterprise service bus is widely distributed, highly available, service oriented communication backbone that provides the standards based integration across the enterprise.According to Gartner

An Enterprise Service Bus (ESB) is a new architecture that exploits Web services, messaging middleware, intelligent routing, and transformation. ESBs act as a lightweight, ubiquitous integration backbone through which software services and application components flow.

The Enterprise service bus (ESB) addresses the above-mentioned challenges by providing the distributed processing, standard-based integration, and Enterprise-level backbone required by the Enterprise.

ESB

Figure 2: ESB Telecommunication logical architectureServices exposed in ESB Container

Enterprise Service Bus

Data transformation Process Automation

Routing Event Transformation

JCA Connectors Web Services JMS

CSR CRM Order Management

Billing Third party

Enterprise Service Bus

Data transformation Process Automation

Routing Event Transformation

JCA Connectors Web Services JMS

CSR CRM Order Management

Billing Third party

The following sections would highlight on how the drawbacks of using a traditional EAI solution are addressed and translated into opportunity using bus architecture in the Telecommunication Industry

Basic connectivity:The core of ESB is highly scalable Enterprise messaging backbone that transports the data as messages in a secure and reliable fashion. ESB uses standard web services, JCA or standard based JMS to provide connectivity to the plugging applications. ESB uses abstract service endpoints to assemble services to form the process. In addition, it fully supports integration of services that are themselves services or that connect to other services using service endpoints. The component that makes an ESB highly distributed is ESB container*. A ESB Container is capable of hosting multiple different services in a container environment. In the Telecommunication domain, systems like self-service user interfaces, contact centres etc., can use web services to invoke various “services” like customer management, credit card authorisation etc., offered by the core ESB backbone. These services themselves invoke the appropriate services using the abstract service end points to do the desired job and come back with the reply (if desired and designed). This is important to note that services offered by ESB in can be of both synchronous and asynchronous nature. For example, credit card validation of a customer is a

Service Oriented Architecture: SOA is an architectural methodology and ESB enables the same. ESB uses registry services in which the information about the service end points is stored and configured. ESB performs routing, transformation and validation of XML inputs from these services. As defined earlier, once the Enterprise Service Bus is defined and created, integration of further applications merely involves plugging new services onto this backbone or the re-use of existing services. In short, service-oriented interactions leverage underlying messaging and event communication models. In the Telecommunication Industry, most of the requests (new customer subscription or service cancellation etc.) from contact centres can be treated as requests for “services”. Each of the services is linked to an individual set of services provided by the connecting interface (for example service for security, service for logging, service for auditing etc). Hence, the whole Enterprise backbone is predominantly built upon the service concept. Futuristically speaking, if a new business requirement evolves, the service for it can be either reused or developed as desired and plugged into the ESB

Figure 3: Services Stack for an ESB

Business Services (Process Logic)User Provisioning, Credit Card Inquiry, Problem Reporting

PresentationServicesBilling Information Portlet, Plan Information, Reports

Interaction ServicesPartner Services, Partner Profile Creation

Shared Application and Data AccessServicesCustomer Information Retrieval, Call data Update

IntegrationServicesMessage Transformation, Data Routing

InfrastructureServicesSecurity, Monitoring, Backup

Business Services (Process Logic)User Provisioning, Credit Card Inquiry, Problem Reporting

PresentationServicesBilling Information Portlet, Plan Information, Reports

Interaction ServicesPartner Services, Partner Profile Creation

Shared Application and Data AccessServicesCustomer Information Retrieval, Call data Update

IntegrationServicesMessage Transformation, Data Routing

InfrastructureServicesSecurity, Monitoring, Backup

synchronous service but the customer order activation can be an asynchronous service. For those applications/systems that do not fall into the category of communicating through web-services, standard JMS and J2EE connectors are used. A good example could be CRM systems and Billing systems. There are products in the Enterprise market that can provide the services required by

CRM and Billing systems. An ESB way for them to interact is using abstract service end points. If it is developed using the EJB server architecture, it can get on to the bus using the MDB (Message driven bean). If the application is developed in java, it can use JMS to connect to ESB. The .Net application can get connected using a .NET client. This .NET client can be plugged into the bus using the MOM internal communication protocol. Most of the ESBs today use JCA that supports both real time and batch connectivity. ESB provides ESB Container architecture that allows packaged or legacy applications to be plugged into the ESB through Service end points. Hence, by adopting the Industry wide standard of using JCA, XML and Web services and other best practices, the shortcoming of using proprietary development interfaces and protocols is nowhere in the picture any more in ESB based solution architecture. Further to this, the architecture gets inclined towards a standards based intelligent communication backbone.

backbone with no impact on the ESB backbone. The following section will list the examples of the services that can be used as synchronous/conversational services, and the services that can be used as asynchronous services.

Synchronous Services: Credit card authorisationCustomer Service Inquiries

Asynchronous ServicesOrder ManagementNumber loading from central agenciesProblem reporting and troubleshooting

Benefits of ESB

Content Based Routing

The communication architecture of ESB deviates from traditional ‘hub and spoke’ approaches to integration and enables solution architecture to be implemented in an incremental fashion and to be linearly connected across the Enterprise. The normal trend in ESB is to provide Containers to which key services such as transformation and routing are delegated. ESB provides for the routing of messages across various transports based on static routing configuration, content-based rules or load balancing. Usually, the combination of XPATH expressions and script-based routing rules to direct message delivery are applied. The details of the message itinerary are stored as XML Meta data and it travels across the bus from one ESB Container to the next. The XML Meta data contains the information like how to route the message, list of forwarding addresses etc. For content- based routing purposes, the target address is obtained, validated and routed when needed. The important point here is the CBR based routing is done at remote ESB Container as opposed to a centralised authority. This feature of CBR allows the message to reach the mirror destination even if the original destination is unavailable. For an example, In the Telecommunication Industry, during the times when the customer subscription is at its peak and it is required to provision the orders, intelligent routing can be used depending upon the configurable parameters. Using the individual ESB Container, ESB ensures that the service request is routed around the problem network. By doing so, the target is still reachable

and business process remains unaffected. For example, if one of the active Billing hubs is down, the routing logic will ensure that the secondary Billing hub can be reached. This will ensure that Business process is not stalled and later on, when the Billing hub is made available, then using the monitoring and central controlling tools, the traffic can be re-routed to the primary hub. Usage of monitoring capabilities is discussed in the section below

Manageability

Don’t build what you cannot manage.”

While the Enterprise system is up and running, it is very important to monitor the behaviour of the various applications within the ESB. It is also important that these behavioural statistics be collected and sent to the application that manages the monitoring statistics. As discussed in the above sections, ESB ESB Container is capable of using the configuration data from a directory service. At the same time, it maintains, the local copy of the configuration data. In case, the other areas of ESB including the directory service become temporarily unavailable, the ESB Container can continue to operate with no affect. Therefore, up to an extent, the ESB Containers are self-manageable. For external manageability, the usual trend in ESB implementation is to use a notification mechanism with senders. Senders broadcast the notification and listeners receive them. The normal trend within ESB is to use the JMX.

Scalability

“Get the bus if bus can’t get you”.

The ESB uses a decentralized and loosely coupled model providing complete flexibility in scaling any aspect of the integration network. A decentralized architecture enables independent scalability of individual services as well as the communications infrastructure itself. A key benefit of the decentralized model is the ability to apply fine granular control over scalability to ensure maximum effect. Taking the example of JCA in ESB scenario, The only component that would be required to install at remote location would be ESB Container* and not full-blown application server. In this case, ESB Container will act as JCA container. In some circumstances, it may be required to increase overall scalability by expanding the capacity of the Enterprise bus. Allowing brokers to communicate linearly and dynamically distributing the load on the bus can increase the scalability of the ESB backbone. In short, the distributed and scalable characteristics of ESB can be achieved by abstraction of endpoints from communication protocols and physical deployment. More brokers can be added in any topology to scale up the communication backbone. For example, in the event that an increase in the number of customer subscriptions has overloaded the capacity of their host machine(s), new machines, message brokers, ESB Containers and Service endpoints can be added to handle the load without the need to change any of the core services themselves in any topology.

Sophisticated Process management

ESB uses orchestration service, which is used to co-ordinate other services within the ESB layer. This orchestration service can manage stateful as well as stateless information about a business process that spans over a period. For the stateful processing, it the processing takes place with multiple pauses and resumes on intermittently, then orchestration service are better option. Orchestration service can work in conjunction with caching service (explained in the later sections) to keep track of process state transitions. In addition, orchestration service can manage multiple conversations by matching the requests with their respective responses arbitrarily. Unlike FIFO (First in first out) or FILO (File in last out) in which the message would need to wait until the appropriate response is matched.

Caching services Caching service use xml data of process itineraries (as they iterate through bus) to record the state information. Using XSLT and XQuery, the state information is retrieved and made available to other services. This feature of ESB allows keeping track of all the process-based itineraries. In the event of any process itinerary getting dissolved and lost if the target system goes down, itinerary can be re requested using orchestration service and caching service. This is a widely needed scenario and not confided only to Telecommunication domain.

ESB for batch transfer

ESB file service can be used to read the export file.(These export files are prepared by reading the master database). ESB file service uses the service end points to feed the data into the service bus. On the other side, another ESB file service reads the file data and writes it into the FTP directory. During the whole process, the other applications are unaware about the ESB layer induction. As ESB replaces normal ETL process effectively, it removes the unreliable layer of FTP, In addition, it ensures reliable, loosely coupled and pluggable communication architecture.

Cost equation Since ESB is based upon the best practices and industry standard of Integration architecture, the licensing cost of the product is considerable less than that of a traditional EAI broker. In addition, the resources required to maintain it will need only to be versed with current and evolving technologies rather than traditional EAI protocols and communication methodology. How ever, Licensing and training cost is not the only cost involved. In the initial section we saw how a light weight ESB container host JCA connectors to provide connectivity to external application, thus omitting out the need to install application server everywhere. The true cost is the ownership, installation, configuring and troubleshooting the on going problems of Application server.

Exported file

Enterprise Service Bus

CRM Billing/Mediation

Order Management

Provisioning

ESB file service

File export

Figure 4: Batch transfer using

Master database

ConclusionESB offers a powerful technology omitting out the proprietary development interface, without having the need to retrain the staff and providing the need based integration at much lower cost and high ROI on investments. As seen in the sections above, ESB holds lots of promise in the integration space for the Enterprise Industry. Equipped with web services and SOA under its umbrella, it could prove as one-stop solution architecture for a variety of integration needs

Keywords

Term AbbreviationCSR Contact Centres RepresentativeEAI Enterprise Application IntegrationESB Enterprise Service BusOSS Operation Support SystemBSS Business Support SystemXML Extensible Mark up languageSOA Service Oriented ArchitectureJMX Java Management ExtensionsMOM Message Oriented MiddlewareETL Extraction , Transformation and LoadingSLA Service Level AgreementJCA Java Connector Architecture

ESB Container: A ESB Container is the component where the implementation of service takes place. However, it is not limited to host a same type of services but can contain multiple and discreet services within framework.

References:Web sitesNigel Thomas & Warren Buckley (2005), Enterprise Service Bus, http://www.sys-con.com

Journal ArticlesDavid Chappell, Web services journal (2005), ESB Myth Busters: 10 Enterprise Service Bus Myths Debunked, http://www.sys-con.com

About the Author

Ritesh Arora

Ritesh is working as a Design Analyst with the EAI team of Wipro Technologies. He has experience in Designing and implementation of EAI solutions using multiple tools and technologies for Insurance and Telecom Customers

AcknowledgementI am thankful to Seema Patil, Srinivas Deshpande, Sharoakh Charna, Subhash Poojari, Santhosh Kumar and Nitesh Saini for their valuable inputs during the review period. I would also like to thank Shantanu Paknikar, Gopalakrishna Bylahalli for their valuable guidance and motivation through out when this Whitepaper was being written. Lastly, I would like to acknowledge my gratitude my entire EAI team their support.

I would like to hear from you. Please send me your suggestions and comments at ritesh.arora@wipro.com