fusesource camel-persistence-part2-webinar-charles-moulliard

Copyright © 2010 Progress So3ware Corpora6on and/or its subsidiaries or affiliates. All rights reserved. 1 A Progress So3ware Company

A Progress So3ware Company

Transac6on Management with Apache Camel, ServiceMix -‐ EIPs Part II

Charles Moulliard – Raul Kripalani, November 22nd, 2011


When you joined today’s session …

Audio is broadcast from your computer

Submit your questions via the Chat Window

Contact today’s Host via the Chat Window


Learn More at hLp://fusesource.com


Speaker – Charles Moulliard

Engineer in Agronomy & Master in Zoology

17 years of experience in IT world development (J2EE, .NET) • Specialised in new technologies

web/Internet & integra6on

• Project manager in bank, financial, telco, insurance and transport world

Solu6on Architect at FuseSource CommiLer on Apache projects :

• Apache ServiceMix, Apache Karaf (PMC) , Apache Camel, Fuse Fabric, Scalate

Follow me on :

hLp://twiLer.com/cmoulliard

hLp://www.linkedin.com/in/charlesmoulliard

hLp://cmoulliard.blogspot.com

hLp://www.slideshare.net/cmoulliard


Agenda

Goal : Transac6on Management with Apache Camel, ServiceMix -‐ EIPs • Transac6ons with Camel • JMS and JDBC alone • Fuse ESB Tx Manager

• Using Global Transac6on Manager (XA) • Persistence with EIPs

• Idempotent processor

• Aggregator • Summary


Table of contents

Transac6ons with Camel JMS and JDBC alone

Fuse ESB Tx Manager

Using Global Transac6on Manager (XA)

Persistence with EIPs • Idempotent processor • Aggregator

Summary


Transac6ons with Camel

Camel does not re-‐invent the wheel and use Spring or Blueprint API (on Apache ServiceMix)

Can work with local or provided Tx Manager • Local = Spring (JMS, JPA, JDBC, Hibernate), Atomikos, Bitronix, jOTM

• Provided = J2EE (WebSphere, JBoss, Glassfish, BEA, Geronimo), Apache ServiceMix


Transac6ons with Camel

Factors affec6ng Architecture depends on : • Component (Transac6onal or not), • Type of Transac6on Manager and DataSource (XA or not),

• Route (simple, complex), • Transacrion Propaga6on Type (REQUIRED, NEW, MANDATORY, ….)

How to configure a Camel route to be transac6onal 2 approaches exist and they can be combined


Transac6ons with Camel (1)

A) Using a Transac6onal component for producer or consumer

How To : Check which component are Tx

aware jpa, sql, jms, file, hibernate, iba6s.

Configure a Transac6on Manager for the endpoint



Simple Transac6onal JMS client configura6on example

Spring JMS Tx manager is defined

Here we provide the parameters



B) Configure a Camel route as Transac6onal

Advantage : allow to include processors (beans, jpa, hibernate) with a Transac6onal route

How to configure it add <transacted> to the route How does it work

• Camel will automa6c lookup to find a Spring transac6on manager (org.springframework.transac6on.PlasormTransac6onManager)

• Transac6on Manager creates a Transac6on (according to PROPAGATION Policy)

• Transac6onError handler will take care of excep6ons occuring during camel exchange and propagated it to the caller



Example of Camel Transac6on route A

B

Camel will lookup for “PROPAGATION_REQUIRED”



Reference to the bean

Reference of the SpringTransac6onPolicy can be provided



Another example using <Policy> DSL and SpringTransac6onPolicy is provided to the TxErrorHandler

Transacted is replaced by Policy



Transac6on policy can be defined using class org.apache.camel.spi.TransactedPolicy

Policies can be mixed in Camel routes to use different transac6on propaga6ons (NEW, REQUIRED, …)



Example of 2 Camel routes using different Policy Required policy : will use an existing Transaction or request to create a new

Requires-New policy will request a new Transaction



Transac6on rollback occurs when a run6me excep6on is raised but you can also manage it using <rollback> DSL

We force a Rollback by generating a org.apache.camel.RollbackExchangeException



Using markRollBackOnly generates a Rollback without excep6on in the Camel route

Rollback occurs in BookService (DB), exception is not send to direct (which is not Tx aware) and exception is send to mock endpoint



When Apache Camel producer/consumer is not Transac6onal, a compensa6on mechanism can be used.

Uses a synchronisa6on mechanism = callback (mimic Tx) to group tasks as a UnitOfWork

Will react with onComplete (=commit) or onFailure (=rollback)



onCompletion + link to the bean implementing the rollback logic



Rollback strategy



Be carefull about threads, threadPool • Only one should be used, • Don't use seda://, vm://, NMR:// which are async to link Camel routes

Tx async are not supported in Camel 2.x (planned for Camel 3.x)

More info : hLp://fusesource.com/docs/router/2.8/transac6ons/TxnDemarca6on-‐ErrorHandling.html


Table of contents


Fuse ESB Tx Manager

Using a Global Transac6on Manager (XA)


Summary


JMS and JDBC

Integra6on project, most probably, will use not only 1 Tx Manager but a combina6on using JMS + JDBC, JDBC + JDBC, JMS + JMS, …

Such a project can use Spring or Fuse ESB Tx Managers 2 Approaches are available on Fuse ESB

• Spring / Spring DM (Dynamic Modules)

• Aries Blueprint Apache Aries project = implementa6on of Enterprise features of OSGI EE specifica6ons (Transac6on, JPA, JNDI, Web)


JMS and JDBC

Use Case descrip6on


JMS and JDBC (1) -‐ Spring

Example of a project containing a JPA, Datasource and a camel route with Ac6veMQ JMS queues

DataSource is created using Apache Commons DBCP

And is exported as OSGI Service wit h2 filters



JPA Persistence file – transac6on-‐type = "RESOURCE-‐LOCAL"



Spring will inject persistence context

Method will be added into an existing Transaction


JMS and JDBC (4) -‐ Spring DAO will be scanned by spring to find @Annotation

Our Spring JPA Tx Manager is exported and will be used by Camel route

DAO is exposed as a service

Spring will add Tx demarcation

Based on @Transactional annotations


JMS and JDBC (5) -‐ Spring Spring JMS Tx Manager

Message rollbacked will be moved directly into the DLQ

Transacted = true, ref to Tx manager, CACHE_NONE to avoid that Spring uses another session (thread))


Camel Spring configura6on

JMS and JDBC (6) -‐ Spring We search for JPA Tx Manager

We configure 2 Transaction Policy (one for JMS, other for JPA)



JMS + JDBC

Apache Camel routes

JMS + JMS


Table of contents


Fuse ESB Tx Manager



Summary


Fuse ESB – Transac6on Manager

Apache ServiceMix 4.x plasorm (Fuse ESB) provides a Tx Manager

Based on Apache Geronimo Tx Manager (JTA 1.1)

Deployed using feature transac6on Contains the following bundle


Fuse ESB – Transac6on Manager (1)

Can be used as a Global JTA Tx Manager with XA Resource

Expose through OSGI service registry interfaces

Proxified objects available


Fuse ESB – Transac6on Manager (2)

Timeout and Tx log file can be customised


Table of contents


Fuse ESB Tx Manager



Summary


Global Tx Manager -‐ XA

Using a Global Transac6on Manager on Fuse ESB just requires that we use JTA and configure XA resources

Verify careful regarding to the choice of the "drivers" or "datasource" that you will use (XA aware)

You do not need to use Spring JDBC, JPA, JMS, Hibernate Transac6on Manager Aries Tx Manager


Global Tx Manager -‐ XA

Use Case


Global Tx Manager -‐ XA (1)

What will change : Datasource (JDBC) Add a XA Datasource

Export XA and non XA Datasource as OSGI services (will be used by JPA container)



JPA -‐ persistence.xml Use JTA instead of RESOURCE-LOCAL

Configure JTA and non-JTA datasource



JPA -‐ spring We call OSGI Service to retrieve the PlatformTransactionManager object managed by Aries TX Manager



JMS – XA Connec6on Pool / Resource Manager

ActiveMQ requires JTA Manager and XAPooledConnectionFactory



JMS – Ac6veMQ component

Transacted is false in this configuration as Aries Tx xManager will take care !



Apache Camel Route We use the Global Tx Manager exposed as OSGI Service through Spring TransactionManager to get it for JPA Service



Apache Camel Route Nothing has changed except that you could use XA and non XA connection pool for JMS


Demo Transac6ons in Ac6on on Fuse ESB !


Table of contents


Fuse ESB Tx Manager



Summary


Idempotent processor

Filter out duplicate messages Uses an Expression to calculate a unique message ID, which is looked up in the repository

By default, memory based Supports several persistent mode

• FileIdempotentRepository

• HazelcastIdempotentRepository (Available as of Camel 2.8) • JdbcMessageIdRepository (Available as of Camel 2.7)

• JpaMessageIdRepository


Idempotent processor (1)

To use JPA Repository, simply configure a persistence file

We use as Model MessageProcessed



Model contains info about propre6es that we will store into the DB

MessageID = key used to filter out messages



Spring Tx and JPA Template must be configured

Spring Template



Setup JpaMessageIdRepository

Reference to the JPA template

Ref for the Spring bean jpaStore


Demo Idempotent Consumer with JPA repo


Table of contents


Fuse ESB Tx Manager



Summary


Aggregator

Allows to combine messages together into a single message

Support : • JdbcAggregatorRepository (Available as of Camel 2.7)

• HawtDBAggregatorRepository (in-‐memory DB)

We keep in DB the aggregated object as blob

Allows to restart aggrega6on process from where it stops !


Aggregator (1)

How agregator works


Aggregator (2)

Configura6on of the Persistent Repository

And DB

Configure Repo with DataSource and Tx Manager


Aggregator (3)

MyAggrega6onstrategy simply concatenates the Body using old and new exchange


Aggregator (4)

Class contains Users and set the Key id = FUSE that the aggregator will use on the Exchange

Id = key used


Aggregator (5)

Apache Camel route refers to the Persistent Repository

Persistent repository


Demo Aggrega6on with JDBC repository


Table of contents

Transac6ons with Camel Fuse ESB Tx Manager

JMS and JDBC alone



Summary


Summary

Transac6ons in more detail • Transac6onal client <-‐> Transacted / Policy mode • Propaga6onPolicy • Rollback strategies • UnitOfWork to synchronise

Present a configura6on with JDBC, JMS Tx managers Introduce Fuse ESB Tx Manager Global Transac6on Manager

EIPs persistent • Idempotent processor/Consumer

• Aggregator


For More Informa6on

fusesource.com • Free downloads • Documenta6on • Geyng Star6ng demo, tools and advice

• Webinars on beginner and advanced topics

• White papers

• Customer case studies

Get Help • Forums

• Virtual training • Consul6ng • Enterprise subscrip6ons


Useful references

Transac6ons with camel / Fuse ESB

• hLp://fusesource.com/docs/router/2.8/transac6ons/index.html

EIPs paLerns • hLp://fusesource.com/docs/router/2.8/eip/MsgEnd-‐Idempotent.html

• hLp://fusesource.com/docs/router/2.8/eip/MsgRout-‐Aggregator.html

Use Case – Devoxx Hands on Lab • hLps://github.com/cmoulliard/Devoxx-‐2011-‐HandsOnLab

fusesource camel-persistence-part2-webinar-charles-moulliard

Technology