soa integration blueprint with oracle soa suite

2011 © Trivadis

BASEL BERN LAUSANNE ZÜRICH DÜSSELDORF FRANKFURT A.M. FREIBURG I.BR. HAMBURG MÜNCHEN STUTTGART WIEN

Welcome Trivadis SOA Integration Blueprint @Work

Matthias Furrer

Principal Consultant Technologies

April 2012

April 2012TVD SOA Integration Blueprint @Work

1

2011 © Trivadis

Introduction

TVD SOA Integration Blueprint @Work2

• Matthias Furrer� Long-standing experience in SOA and ERP Area

� Architect, Consultant, Lead-Developer and Project Manager

� SOA Certified Professional

� Oracle SOA Blackbelt Consultant

� Trainer with Trivadis Oracle Middleware und SOA

� Reviewer of Technical Publications

� More than 20 years of software development experience

� Contact: [email protected]

April 2012

2011 © Trivadis

TVD SOA Integration Blueprint @WorkApril 2012

3

Initial Situation

Customer• 3 different companies – developing and running various applications operated in 3 different service centers

• Purchasing Oracle SOA Suite – including Oracle Service Bus

• Oracle Service Bus intended to become strategic enterprise-wide backbone for all integration tasks – externally and internally

• Trivadis engaged for PM in the ESB Platform building project and creating a customer-specific Integration Blueprint

Basis for the customer specific Integration Bluepri nt• Trivadis Book “Service Oriented Architecture: An Integration Blueprint” by Guido Schmutz,

Daniel Liebhart, Peter Welkenbach

Implementation Projects• First solutions developed and currently in testing phase

• Go-live date for first solutions: June 2012

• Trivadis designed and implemented Oracle SOA platform, CI Concept and first implementations (Java, OSB and SOA-Suite)

2011 © Trivadis


4

Trivadis SOA Integration BlueprintElements – Views, Domains and Layers

The customer-specific Integration Blueprint is built upon the Integration Architecture Blueprint from Trivadisand is focused on the integration-specific components in this specific view of the system architecture. The main focus lies on the technical domain “Integration” and describes the information flow between the systems while following the layered architecture principle.

2011 © Trivadis


5

Trivadis SOA Integration BlueprintNotation

The customer-specific Integration uses the same notation as introduced in the TrivadisIntegration Architecture Blueprint to visualize the scenarios.

2011 © Trivadis


6

Goals of the Integration Blueprint• Unified Implementations of Integration Requirements (Design Principles)

• Best -Practices for specific Tasks (Design Principles)

• Ensuring Interoperability of Services (Design Principles and Standards)

• Detailed Integration Scenarios as Design Patterns (Integration Scenarios)

• Templates for specific Tasks (Reference Implementations)

Answers in the Integration Blueprint• Which Design Pattern is best suited for a specific use case

• Which Integration scenario can be used for specific requirements

• Which tool should be used for a specific requirement

• How to ensure transaction reliability and consistency

• How to avoid and handle faults

• How to implement validation and transformation

• How to secure services

• How to integrate basic services (e.g. .NET services)

SOA Integration BlueprintWhy Considerung an Integration Blueprint

2011 © Trivadis


7

Infrastructure• Availability and Performance Requirements

• System Topology and Platform **

Architecture• Domain Model

• Service Models

• Service Categorization *

• Integration Scenarios *

Development• Design Guidelines and Standards **

• Tools Choice *

• Transaction Reliability *

• Error Handling *

Administration• Manage, Build, Test and Deploy (CI) **

• Versioning

Common Challenges and Considerations in a SOA ProjectWhat was covered in the Customer Integration Blueprint

• Security

• Service Granularity

• Message Exchange Patterns *

• Decoupling and Events *

• Validation and Transformation *

• Canonical Schema *

• Shared Artefacts and Metadata *

• Service Access Security *

• Transport/Message Protocols *

• Repository and Registry

*) covered in the customer-specific Integration Blueprint

**) covered in other, dedicated concepts or project documents by Trivadis

2011 © Trivadis


8

SOA Integration BlueprintGoals and Rules for Usage in the specific Customer Project

Content and Goals

The Integration Blueprint introduces concepts and usage scenarios as well as components for implementation in an abstract form, which will be replaced by application-specific modules in implementation projects.

Design Rules will provide best practice approaches and ensure a consistent implementation of integration projects. Executable Sample Implementations will illustrate the described scenarios and can be used for reference purposes in implementation projects.

Rules for usage in the project

• Variations in the solution design from solution scenarios or design principles as described in the Integration Blueprint are possible, however these must be reasonable, documented and approved by the Quality Gate.

• The notation for integration scenarios as used in the Integration Blueprint must also be used for documentation of integration projects.

• Described Design Rules apply to integration projects only and are not valid for application or service development.

2011 © Trivadis


9

Customer Integration BlueprintContent and Chapters

Chapter Title Description

1 Fundamentals Description of basic elements, terminology and notations in the Integration Blueprint

2 Implementation with Oracle SOA Suite and OSB

Implementation of the Blueprint with Oracle SOA Suite and Service Bus. Type Mapping to Frameworks and development components.

3 Composite Services Guidelines for development of Composite Services – particularly regarding development platform to be used (BPEL vs. OSB)

4 Error Handling and Preventing Guidelines and sample implementations for error handling and strategies to avoid errors for all service types

5 Validation and Canonical Schema Principles of Validation and Transformation Services and Canonical Schema in message exchanging

6 Security Implementation guidelines for access security(Draft Version)

7 Basic Services Integration Guidelines and scenarios for the development of .NET /Ruby/GWT Services and integration with Oracle Service Bus(Draft Version)

8 Integration Blueprint (Interface Type) Reference Implementation for specific Interface Type.(Open)

9 Design Principles (Summary) Overview and Summary of all Design Principles and Reference Implementations

2011 © Trivadis


10

Customer SOA PlatformOracle SOA Suite / OSB 11gPS4

• 3 Tier DMZ-Architecture

• Load-Balancing in front of HTTP Service

• Currently no Clustering on Integration Tier

• SSL Offload in Web Tier

• Dedicated Servers for Production Environment

• Dedicated WL Domains for OSB and SOA Suite

2011 © Trivadis


11

Customer SOA BlueprintAbstract SOA Blueprint and Service Interaction

Abstract SOA Blueprint

Multi-Layer Model for a holistic view of the IT landscape and partitioning of the entire system into logical units for Identification, borderline and definition of components and interactions within the system as well as establishing methods, standards and tools for development.

2011 © Trivadis


12

Customer SOA Service TypesCategorization into Service Types for Common Understanding

� Process Services: Implementation of the Business Process in an enterprise with application-comprehensive transactions – often including Human Workflow functionality as e.g. approval processes.

� Composite Services: Multiple Transactions or Queries – typically application-comprehensive, which are composed from multiple Basic Services – or other Composite Services.

� Basic Services: Logic- or data-centric Services, as the foundation of a SOA represent the basic functionality in an application domain..

� Logic Services: Implementation of business logic in non agnostic" context: Example: CalculatePrice

� Entity Services: Data operations on a specific entity. Example: GetCustomerAddress, AddCustomer

� Utility Services: Domain independent operations: Example: ValidateBirthDate, FormatSSCNo

� Decision Services: Decision Rules for controlling processes and workflows

� Virtualization Services: Service abstraction from logical and physical endpoints

Service Types used for this SOA Integration Blueprint

Introducing service types allows to divide the system into components with common characteristics in order to apply different rules to different types of services. A well defined terminology is crucial to establish a common understanding in the project.

2011 © Trivadis


13

Customer SOA Integration BlueprintMapping Integration Blueprint to Oracle FMW Products

Level Layer BuildingBlock

Developer Framework Developer Component Technology

Transport Level Communication Layer

Transporter - - HTTP, JMS, FTP, SMTP, IIOP, RMI, MQ etc.

Integration Domain Level

Collection/Distributïon Layer

Adapter - Oracle SOA Suite - Tech Adapter (File, FTP, DB, AQ, MQ etc.)- JMS Adapter- WS Adapter- HTTP/Socket Adapter- EJB Adapter

- JCA- JMS- SOAP- TCP- EJB

Mapper - Oracle SOA Suite (implicit) - Implicit Functionality in Adapter Framwork

-

Mediation Layer Translator - Oracle Service Bus- Oracle SOA Suite

Transformation (Action)Mediator (Service) *)

XSLT, XqueryXSLT

Router - Oracle Service Bus - Proxy Services proprietary

Application Level

Process Layer Orchestrator - Oracle SOA Suite BPEL PM WS-BPEL

The following components will be used in the first phase of the implementation project:

• Service Bus• Mediator• BPEL

Candidates for usage in the mid-term planning are:

• Human Workflow• Business Rules

2011 © Trivadis


14

Customer SOA Integration BlueprintMapping Service Types to Oracle FMW Products

Service Type Subtype BusinessLogic

CrossAppl

Reuse-abilityPotential

Sync/Async

WS SOAPInboundrequest

WS SOAPOutboundreply/callback

InstanceTracking / stateful

Development Framework

Process Service yes yes Low async yes yes yes BPEL

Composite Service yes yes Medium async *) yes *) yes yes *) - BPEL- (OSB)

Basic Service Logic yes no Medium sync yes **) yes **) no - Native (Java, .NET etc.)- (BPEL, Mediator)

Entity yes no Medium sync yes **) yes **) yes *) - BPEL- Native (Java, .NET etc.)

Utility no yes High sync yes yes no - Native (Java, .NET etc.)- (BPEL, Mediator)

Decision yes no Medium sync yes yes no Business Rules Engine

Virtualization no yes High sync *) possible possible no - OSB- (Mediator) ***

*) Typical characteristics, can vary

**) Existing Functions of Legacy Systems (e.g. PL/SQL) can be used and exposed via Adapters or Wrappers

***) Only for XML Validation and Data Transformation

2011 © Trivadis


15

Customer SOA Integration BlueprintDesign Principles - Usage of Oracle Service Bus (Extract)

• Grundsätze für den Einsatz von OSB als Mediationsla yer

� Sämtliche anwendungsübergreifenden Aufrufe von Webservices müssen immer über den Oracle Service Bus erfolgen. Das heisst wenn beispielsweise das Customer Center (Ruby Anwendung) einen Web-Service aus dem Samba Integration Layer konsumiert der mit .NET implementiert wurde, muss dieser über den OSB (Virtualisierungsservice) erfolgen [DP-IA-2-001].

� Kommunikation zwischen Webservices innerhalb der gl eichen Anwendung können direkt (point-to-point) implementiert werden und müssen nicht über einen Virtualisierungsservice erfolgen. Wenn beispielsweise ein Websservice aus dem Samba Integrationslayer als Consumer für einen anderen Webservice aus dem Samba Integrationslayer agiert, und beide sind mit WCF implementiert, ist kein Mediationslayer erforderlich [DP-IA-2-002].

� EJB Komponenten die aus einer Anwendung oder einem Service konsumiert werden , können direkt angesprochen werden und müssen nicht über einen Virtualisierungsservice bezogen werden [DP-IA-2-003].

� Processservices (BPEL) die Webservices konsumieren , müssen diese immer über den Mediationslayer beziehen. Als Protokoll wird dabei nicht SOAP, sondern das RMI-basierende «SOA direct» Protokoll eingesetzt [DP-IA-2-004].

� Anwendungen oder Services, die als SCA Komponenten implementierte Services konsumieren , müssen diese immer über den Mediatonslayer im OSB beziehen. Das bedeutet dass sämtliche Aufrufe von BPEL Processservices, Business Rules oder Transformationsservices die mit Mediator implementiert wurden, über einen Virtualisierungsservice erfolgen müssen. Ausnahmen sind EJB’sdie innerhalb eines SCA’s implementiert wurden oder SCA Komponenten die von anderen SCA Komponenten konsumiert werden[DP-IA-2-005].

2011 © Trivadis


16

Customer SOA Integration BlueprintImplementation of Orchestration Services – BPEL vs. OSB

SLA Monitoringrequired

- Instance Tracking required OR- Compensation required OR- Human Workflow included- Restart Capability required

ImplementationOSB

ImplementationBPEL

Implementationcombined

YES NO X - -

YES YES - - X

NO NO X - -

NO YES - X

OSB and BPEL developer environment both provide capabilities to implement Orchestration and Composite Services. However, based on exclusive features and restrictions in the respective frameworks the following rules can be defined for the usage of BPEL on the integration layer for those service types :

• Design Principle: [DP-IA-3-001]

2011 © Trivadis


17

Customer SOA Integration BlueprintIntegration Scenario and Sample Implementation OSB and BPEL Combined

Referenzimplementierung im Blueprint

C3-1 - AddCustomerAsyncWithBPEL

http://esb-test.ltvintra.ltv.ch:7011/Blueprint/AddCustomerAsyncWithBPEL?WSDL

2011 © Trivadis


18

Customer SOA Integration BlueprintDesign Principles - Fault Handling (Extract)

� Design Rules – Rückgabe von Fehlern der Service Prov ider� Technical Faults, die nicht in der entsprechenden Instanz selber behandelt werden, müssen mit HTTP Response Status «500» antworten[DP-

IA-4-001].

� Business Faults sind gültige SOAP Response - die entsprechende Serviceinstanz muss jedoch immer mit HTTP Response Status «500» antworten[DP-IA-4-002]. *)

� Business Faults, Validation Errors und Processing Errors sollten als SOAP Fehlercode <faultcode> immer den Wert «soapenv:Client» (SOAP 1.1) oder «soapenv:Sender (SOAP1.2) zurückgeben. Dies gilt generell als ein Indikator, dass kein Retry versucht werden sollte[DP-IA-4-003].

� System Errors und Security Errors sollten als SOAP Fehlercode <faultcode> immer den Wert «soapenv:Server» (SOAP 1.1) oder «soapenv:Receiver (SOAP1.2) zurückgeben. Dies gilt generell als ein Indikator, dass ein Retry ausgeführt werden kann [DP-IA-4-028].

� Business Faults und Validation Errors müssen die entsprechende Fehlerinformation immer über eine im WSDL des entsprechenden Services definierte Fault Message anzeigen. Eine ausschliessliche Anzeige über Elemente im Payload ist nicht ausreichend. Damit wird unter anderem auch die Fehlerbehandlung via Fault Policy in den SCA Komponenten ermöglicht[DP-IA-4-004].

� Bei reinen Virtualisierungsservices zur Serviceabstraktion im OSB muss die Fehlermeldung des Serviceproviders unverändert an den Client zurückgegeben werden. Als Virtualisierungsservices in diesem Kontext gelten [DP-IA-4-008]:• nur ein Service Provider im Flow• keine Veränderung (z.B. Transformation) der Request oder Response Message

• Bei unbehandelten Fehlern (mit Ausnahme von Business Faults), die unverändert an den Client zurückgegeben werden, muss die ursprüngliche Fehlermeldung des Service Providers in das Error-Log des OSB geschrieben werden. Bei kritischen Transaktionen kann dazu auch ein Pipeline-Alert verwendet werden[DP-IA-4-009].

Service Provider URI 2



Proxy Service

Business

Service

2011 © Trivadis


19

Customer SOA Integration BlueprintDesign Principles - CompensationBei Virtualisierungsservices ist der Service Provider für die Transaktionssicherheit zuständig. Auf der Integrationsebene wird lediglich der entsprechende Fehler an den Consumer zurückgegeben [DP-IA-4-024].

Bei Composite Services ist dieser selber für die Transaktionssicherheit zuständig. Dabei wird die Steuerung der Compensation von dieser Komponente übernommen. Für Process Services gelten die gleichen Regeln. [DP-IA-4-025].

2011 © Trivadis


20

Customer SOA Integration BlueprintDesign Principles – Error Prevention

Fehlerszenario Strategien

Nicht-Erreichbarkeit von Service Providern oder Ressourcen aufgrund geplanter oder ungeplantem Ausfall von Infrastrukturkomponenten

- Implementierung von Retry Verfahren- Service Pooling im OSB- Implementierung des Messaging über Queues anstelle synchronem Request/Reply

Messaging

Ausfall der ESB Service-Infrastruktur - Implementierung des Polling Consumer Patterns über Queues oder DB Persistierung

Überlastung der ESB-Infrastruktur - Implementierung von Split-Join Patterns in OSB- Work Manager in OSB

Überlastung von Service-Endpoints - Caching im OSB- Service Throttling in OSB zur Limitierung von gleichzeitigen Anfragen

Interoperability Issues für externe Consumer in Enterprise Services

- WS-I Compliance Validierung der Services- Implementierung von WS-I Checks im OSB

The Customer SOA Integration Blueprint contains Explanations, Design Principles, Usage Scenarios and a reference Implementation for the below listed error preventing strategies.

Proxy Service

Business Service

Legacy Service

Message Buffer

2011 © Trivadis


21

Customer SOA Integration BlueprintError Prevention – Polling Consumer

� Design Regeln

• Initiierung von geschäftskritischen Transaktionen ü ber Polling Consumer Pattern . Geschäftskritische Transaktionen müssen immer über einen Persistenzlayer mit passiven Polling Mechanismen implementiert werden. Die verhindert den Verlust von Transaktionen bei nicht verfügbarer Integrationsplattform. Falls möglich soll dazu eine JMS Queue verwendet werden. [DP-IA-4-010].

• Falls eine JMS Queue oder Topic für die Persisitierung von Nachrichten für Polling Consumer verwendet wird, darf der dazugehörige JMS Server nicht auf der gleichen physischen Plattform wie der OSB ausgeführt werden. Andernfalls könnten Nachrichten bei einem Ausfall der OSB Plattform ebenso wenig wie im synchronen Request/Reply Pattern zugestellt werden [DP-IA-4-027].

2011 © Trivadis


22

Customer SOA Integration BlueprintDesign Principles - Validation and Transformation

Action Type Action Subtype Standards Exposedas Service

Codingrequired

ConsumedBy

WS SOAPInterface

Sync/Async

Development Framework

Data Validation Technical Validation XML Schema no no - OSB- BPEL- Mediator

- - -

Functional Validation - Native- Schematron *)

yes yes **) - OSB- BPEL- Mediator- Legacy App

yes sync - Java, .NET, PL/SQL etc.- (BPEL, OSB)

Data Transformation ***) Data Transformation - XSLT should yes - OSB- BPEL

yes sync Oracle Mediator

Data Mapping - XSLT- XQuery

possible yes - OSB- BPEL- Mediator

(yes) (sync) - JDeveloper- Eclipse

Data Formatting - no yes - XSLT- XQuery

- - - JDeveloper- Eclipse

Data Value Mapping - possible no - XSLT (yes) (sync) - Oracle SOA Suite Domain Value Mapping

Validierung und Datenkonversion [DP-IS-5-006]

Untenstehende Abbildung zeigt eine Übersicht der einzelnen Aktionen, Services mit deren Ausprägungen, Eigenschaften und jeweilig bevorzugt einzusetzenden Development Framework.

*) Einsatz von Schematron offen**) Vorhandene Services aus Legacy Applikationen können über Wrapper angesprochen werden (wo möglich)***) Sollte immer in einer Operation implementiert werden

2011 © Trivadis


24

Summary

Summary and lessons learned• The customer specific SOA Integration Blueprint currently contains approx. 60 Design Principles for

the implementation of integration projects with Oracle SOA Suite and Service Bus

• The customer specific SOA Integration Blueprint currently contains approx. 20 Reference Implementations implementing related patterns, design principles and integration scenarios. More integration scenarios are described but not implemented as executable implementations.

• Design Principles and patterns are based on common best-practices, implementation experiences and general architecture principles. Nevertheless, these rules and patterns can prove not to be completely applicable or suitable in practical usage. Reasons can be limitations or restrictions of the platform, too complex to implement while not providing expected benefit in some cases or changing requirements and needs. Therefore the establishment of such a Blue Print should be considered as an on-going process which has to be continuously reviewed, monitored and adjusted as needed.

• Each customer or customer project has individual requirements , constraints and environments. A generic Integration Blueprint can be used as a starting point to define architectural and conceptual rules and patterns, but should always be adjusted specifically.

2011 © Trivadis

BASEL BERN LAUSANNE ZÜRICH DÜSSELDORF FRANKFURT A.M. FREIBURG I.BR. HAMBURG MÜNCHEN STUTTGART WIEN

Thank you.Trivadis AG

Matthias Furrer

Europa-Strasse 5CH-8152 Glattbrugg

[email protected]

April 2012TVD SOA Integration Blueprint @Work

25

soa integration blueprint with oracle soa suite

Presentations & Public Speaking