soa pattern: data model transformation

Last Updated: Jan. 2014

Associate Technical Lead – Technical Sales

Dakshitha Ratnayake

SOA Pattern: Data Model

Transformation

2

About the Presenter

๏ Dakshitha is an Associate Technical Lead in the Solutions Architecture/Technical Sales team.

๏ She has hands on experience in Java/J2EE technologies and has worked on designing and implementing software solutions in the fields of health-care information systems and content management systems for telecommunications providers prior to her employment at WSO2. Prior to joining the Solutions Architecture/Technical Sales team she worked with the WSO2 Developer Studio team.

3

About WSO2๏ Global enterprise, founded

in 2005 by acknowledged leaders in XML, web services technologies, standards and open source

๏ Provides only open source platform-as-a-service for private, public and hybrid cloud deployments

๏ All WSO2 products are 100% open source and released under the Apache License Version 2.0.

๏ Is an Active Member of OASIS, Cloud Security Alliance, OSGi Alliance, AMQP Working Group, OpenID Foundation and W3C.

๏ Driven by Innovation

๏ Launched first open source API Management solution in 2012

๏ Launched App Factory in 2Q 2013

๏ Launched Enterprise Store and first open source Mobile solution in 4Q 2013

4

What WSO2 delivers

5

Business Model

The Integration Problem๏ As organizations move forward with their SOA

implementation, they find themselves grappling with significant problems of data integration among a wide variety of data sources.

๏ The mismatch between the data representation that one application provides and the data representation that another application expects forms the core of this long-lived problem.

๏ This data integration problem is particularly troublesome for SOA implementations because it limits the loose coupling between service providers and consumers.

Integration Needs๏ Standard Integration

๏ Entity Aggregation / Canonicalization

๏ Standards Compliance

A Narrowed Down Problem๏ How can services interact with programs that

communicate with different data formats?

๏ Services may use incompatible schemas to represent the same data, hindering service interaction and composition.

๏ Even when organizations use metadata standards such as XML to format messages, the schema of one Service might not match the schema of another, and fields with the same meaning might use differing tags.

The Straight-Forward Approach๏ When an application or service must transform data

from one format to another in order to achieve some desired data integration goal, the most common and straightforward approach is to embed data transformation logic directly within the application.

Disadvantages of Using Embedded Transformation๏ Other applications or Services have no practical way of

reusing the embedded transformations.

๏ It's impossible to offload the CPU overhead that the embedded data transformations cause.

๏ The transformation logic is tightly coupled to the business logic within the application, reducing the flexibility and reusability of the applications.

๏ The transformation logic is handled programmatically vs. declaratively through metadata, further impeding the flexibility of the application.

A Better Approach๏ An alternative to embedding transformation logic within an

application is to use an external transformation service to handle the task.

๏ A transformation service, loosely-coupled in a SOA implementation, accepts incoming data in one format, and outputs data with the same semantic meaning in another format.

๏ The use of data transformation services improves loose coupling by separating the concerns of service providers and consumers.

Solution for ‘same format different schema’ problem ๏ A data transformation technology can be incorporated

to convert data between disparate schema structures.

๏ Mapping logic needs to be developed and deployed so that data compliant to one data model can be dynamically converted to comply to a different data model.

Solution Pattern: Data Model Transformation

๏ The requirements statement for Data Model Transformation is:

How can services interoperate when using different data models for the same type of data?

Reference Architecture: Standard Integration with XSLT transformation

Service 2Service 2

Service 1Service 1

Data Model Transformation

Data Model Transformation

Schema1.xsdSchema1.xsd

Schema2.xsdSchema2.xsd

Schema1.xsdSchema1.xsd

Schema2.xsdSchema2.xsd

Transform.xslTransform.xsl

WSO2 Solution Architecture: Standard Integration

Service 2Service 2

Service 1Service 1 Schema1.xsdSchema1.xsd

Schema2.xsdSchema2.xsd

Schema1.xsdSchema1.xsd

Schema2.xsdSchema2.xsd

Transform.xslTransform.xsl

WSO2 Offerings

The WSO2 ESB supports transformations using the following transformation mediators:

WSO2 Data Services Server supports transformation using XSLT Transformation, where the user can define the transformation in XSLT.

Entity Aggregation

๏ There are essentially three reasons for the aggregation of entities in different systems:

๏ Single View of Entity—Having a single view of all common entities spread across enterprise systems.

๏ Horizontal Partitions—There are situations where entity information is distributed across many different services, based on factors such as geographical boundaries.

๏ Cross-Join scenarios—These situations require cross-join among entities present in different services.

Reference Architecture: Entity Aggregation

ERP Service ERP Service

Data Model Transformation

(Entity Aggregation)

Data Model Transformation

(Entity Aggregation)

CustomerERPCustID

LNameFnameOrgID

CustomerERPCustID

LNameFnameOrgID

Transform.xslTransform.xsl

CustomerAPICustomerAPI

CRM Service CRM Service

CustomerCRMCustID

LNameFname

AddressTel

OrgID

CustomerCRMCustID

LNameFname

AddressTel

OrgID

CustomerERPCustID

LNameFnameOrgID

AddressTel

CustomerERPCustID

LNameFnameOrgID

AddressTel

WSO2 Solution Architecture: Entity Aggregation

ERP Service ERP ServiceCustomerERPCustID

LNameFnameOrgID

CustomerERPCustID

LNameFnameOrgID

Transform.xslTransform.xsl

CustomerAPICustomerAPI

CRM Service CRM Service

CustomerCRMCustID

LNameFname

AddressTel

OrgID

CustomerCRMCustID

LNameFname

AddressTel

OrgID

CustomerERPCustID

LNameFnameOrgID

AddressTel

CustomerERPCustID

LNameFnameOrgID

AddressTel

Standards Compliance

๏ Adherence to standards insures, to some degree, the loose coupling and composition of services and data exchange mechanisms with a guarantee that the data exchanged with an individual service will be universally comprehensible.

๏ E.g. – HL7, other domain-specific formats

Demo

ERP Service ERP ServiceCustomerERPCustID

LNameFnameOrgID

CustomerERPCustID

LNameFnameOrgID

CustomerAPICustomerAPI

CRM Service CRM Service

CustomerCRMCustID

LNameFname

AddressTel

OrgID

CustomerCRMCustID

LNameFname

AddressTel

OrgID

CustomerERPCustID

LNameFnameOrgID

AddressTel

CustomerERPCustID

LNameFnameOrgID

AddressTel

Resources

๏ http://searchsoa.techtarget.com/tip/The-role-of-transformation-services-in-SOA

๏ http://soa.dzone.com/articles/soa-pattern-week-6-canonical

๏ http://www.asprom.com/note/wp20.pdf

๏ http://www.ibm.com/developerworks/data/library/techarticle/dm-0803sauter/

๏ http://msdn.microsoft.com/en-us/library/ms954596.aspx

Contact us !