wtui11 - effective application development in ibm integration bus
DESCRIPTION
What are the best practices for IBM Integration Bus development? How do you design integrations that conform to external and internal standards? Learn how to use the products's built-in capabilities to apply common usage patterns and develop, refine and share your own patterns that solve your company's integration requirements. This session will also introduce the application development enhancements in the latest release of IBM Integration Bus that allow you to implement solutions more quickly than ever.TRANSCRIPT
© 2014 IBM Corpora/on
Please Note IBM’s statements regarding its plans, direc/ons, and intent are subject to change or withdrawal without no/ce at IBM’s sole discre/on. Informa/on regarding poten/al future products is intended to outline our general product direc/on and it should not be relied on in making a purchasing decision.
The informa/on men/oned regarding poten/al future products is not a commitment, promise, or legal obliga/on to deliver any material, code or func/onality. Informa/on about poten/al future products may not be incorporated into any contract. The development, release, and /ming of any future features or func/onality described for our products remains at our sole discre/on.
Performance is based on measurements and projec/ons using standard IBM benchmarks in a controlled environment. The actual throughput or performance that any user will experience will vary depending upon many factors, including considera/ons such as the amount of mul/programming in the user’s job stream, the I/O configura/on, the storage configura/on, and the workload processed. Therefore, no assurance can be given that an individual user will achieve results similar to those stated here.
1
2
Effec.ve Applica.on Development
▪ What should you consider when building connec/vity solu/ons? ▪ What technology is provided to help you? ▪ What types of integra/on asset are appropriate in each scenario?
▪ My top /ps will be highlighted as we move along!
▪ We will consider development from several design perspec/ves: o User-‐defined nodes o Subflows o Deployable subflows o Libraries o Applica/ons o PaWerns o PaWern authoring o Integra/on API
▪ I’ll focus on WMB V8, IIB V9 but will point out some V10 Open Beta improvements too!
Effec.ve Applica.on Development Tools of the Trade
Tools of the Trade!
4
• Service Discovery aims to facilitate access to and cataloguing of key systems – Integra/on Bus tools discover and represent these key data – Integra/on services extends access to exis/ng end-‐user applica/ons over new
channels
• New integra/on tools discover key database assets – Connect to external system (e.g. DBMS, MQ) – Discover source artefacts (tables, queues, etc.) – Map CRUD opera/ons to service interface – Save in canonical WSDL document – Re-‐use WSDL in mul/ple scenarios
• Many uses for service defini/on – Drag and drop WSDL to automa/cally create access methods – Create new integra/on service to exploit customized database access – Provide pre-‐populated drop-‐downs for built-‐in nodes
• Use of built-‐in registry to store discovered services – Currently MQ service defini/ons (and workload management policies) only – Designed for future sharing between integra/on nodes and other services
Service Discovery
5
Products Partners Customers Accounts
Effec.ve Applica.on Development Your integra.on logic
7
Applica.ons, Libraries and Services ▪ Applica/ons package end-‐to-‐end connec/vity solu%ons
▪ The concept of an applica/on is shared between the toolkit and run/me ▪ Applica/ons are deployed and managed as a single unit of isola/on
▪ Libraries package resources for reuse (flows, scripts, models) ▪ Resources in an applica/on are not visible to anything else
▪ Use applica/ons to manage your solu/ons inside an execu/on group ▪ A Service is an Applica/on with a well defined interface (WSDL)
Flows and Nodes
• Built-‐in nodes encapsulate transports, technologies and applica/ons – Our intent is always to make the common tasks easy, and the rest possible! – Use the built-‐in nodes to reduce the amount of custom code required – This makes best use of the built-‐in facili/es like ac/vity trace and resource sta/s/cs
8
Transforma.on Op.ons ▪ WMB and Integra/on Bus have several transforma/on op/ons:
▪ Mapping ▪ XSLT ▪ ESQL ▪ Java ▪ PHP ▪ .NET
▪ Reflects the importance of transforma/on in connec/vity solu/ons ▪ User-‐defined nodes supported for Java and C/C++
▪ Every transforma/on op/on has strengths and weaknesses! – Performance and scalability – Backend integra/on – Skill sets and learning curve – Developer usability – Portability and maintenance
• Use a transforma/on technology appropriate to the problem at hand!
9
▪ Subflows are simply message flows that are invoked from another flow ▪ Input and output nodes in the subflow become terminals in the main flow ▪ Use subflows to break up large problems into smaller more manageable chunks
▪ Subflows are directly deployable to the run/me ▪ Shared subflows deployed just once per execu/on group (or applica/on) ▪ No need to redeploy message flows aier changes to shared rou/nes are made ▪ Redeployment of a subflow is automa/cally picked up by any consumers
11
Subflows
Message Modelling • Models are needed for parsing, valida/on and transforma/on
– Models avoid the need to write custom code to parse messages!
12
§ Graphical mapper requires models to display the message structure – ESQL editor provides in line valida/on of code that navigates message trees
§ XML documents and messages are modelled using XML schemas – XSDs and WSDLs are deployed directly to the run/me – no import required!
§ Non XML data (both text and binary) is defined using DFDL schemas – Data Format Descrip/on Language standardised through the Open Grid Forum – DFDL schemas replace message set projects and the MRM message domain – Input nodes support DFDL alongside XMLNSC, JSON, MRM etc
§ New model based tooling to design and test DFDL schema models – Test parsing, test data genera/on and parse-‐by-‐example all built into editor! – As with XSDs, DFDL schemas are also deployed directly to the run/me
Message Modelling DFDL and XML Schemas
13
Design for Performance • There are several areas that incur a processing cost
– Parsing and serializa/on of messages as they enter and exit the flow – Business logic such as transforma/ons executed during message processing – Transforma/on cost includes general complexity, tree naviga/on and tree copying – External resources such as interac/ons with databases and queue managers
• Message flow design is cri/cal to achieve high performance and scalability – For example using global units of work to achieve transac/onality across a flow
• Performance is a large (and important) topic in its own right – There is a wealth of informa/on and guidance available online!
14
Effec.ve Applica.on Development PaJerns
16
The Challenge
From: MQ
File
WSDL
To:
17
PaJerns for Simplified Development § Creates top-‐down, parameterized connec/vity solu/ons
§ Reduces common problems in flow development
§ Establishes best prac/ces for healthcare integra/on § Reduces /me-‐to-‐value for solu/on development
§ PaWerns are easily extended with regular IIB func/onality
PaJern Genera.on • PaWern genera/on creates the produc/on ready projects
– Generated projects reflect the configura/on choices of the paWern user – Configura/on is saved so that the paWern can be re-‐generated if required
18
Built-‐In PaJerns • Integra/on Bus provides a core set of built-‐in paWerns • These implement a variety of common scenarios
– Web service front end to a MQ based applica/on – Processing data stored in a file and rou/ng to one or more queues – Adding a proxy in front of a web service provider – Processing data from an SAP system and rou/ng to MQ – Shredding messages and rou/ng to one or more queues
• PaWerns are selected based on client feedback and field experience • This core set of paWerns con/nues to grow with each release
19
Effec.ve Applica.on Development PaJern Authoring
21
PaJern Authoring • PaWerns becomes even more compelling when you can create your own!
– Every organiza/on has their own repea/ng connec/vity paWerns! – PaWern authoring is the name we give to this technology in WMB and Integra/on Bus
• We recommend you start with a working solu/on – One or more projects
• PaWern authoring is a design ac/vity – It may be long lived – It is oien not sequen/al
• Using paWerns is a top-‐down ac/vity driven by a requirement, but: – Authoring a working solu/on is (typically) a boWom-‐up ac/vity – So paWern authoring bridges these two different approaches
• PaWerns have their own development cycle – PaWern Authoring editor supports this design ac/vity
Create Your Working Solu.on
• No change at all -‐ design your solu/ons as you do today – PaWern authoring does not change the tools you use to create solu/ons – The key to a good paWern is to create a good working solu/on!
22
Design Your PaJern • Straighporward to create paWerns using the PaWern Authoring editor!
– Design the user interface which is presented to your paWern users – Easy to add branding -‐ style sheets, images and other files as required
23
PaJern Refinement • PaWern authoring in supports property changes
– Node, user-‐defined proper/es (UDPs) and promoted node proper/es
• Property variability is the most common type of variability that a paWern might need to express -‐ there are many others:
– Generate applica/on text files such as ESQL scripts – Make structural changes to Message Flows – Create administra/on files such as MQSC scripts
• It is impossible to try and predict all the possible extensions that a paWern author might wish to implement
• We provide two ways to extend paWern authoring – Java code that is invoked when paWern instances are generated – PHP templates that generate text files in paWern instance projects
24
Packaging a PaJern
25
• The Integra/on API is our Java applica/on development API – Packaged along with the administra/on API in ConfigManagerProxy.jar – Provides a complete Java API to manipulate message flows (and subflows)
• The Integra/on API can be used in standalone applica/ons or in paWerns – Wide variety of applica/ons are now possible such as business data mashups – Easy to implement mass crea/on of message flows during migra/on projects
26
Integra.on API
PaJern Communi.es • PaWern authoring creates paWerns whose value mul/plies as they are shared and used by a community of developers
• Public and private communi/es are equally interes/ng! • A community needs to offer more than just a repository of assets!
– Space for content such as paWerns and subflow nodes – Forums to host discussions and threaded conversa/ons – Categorisa/on is important as the community grows! (taxonomies) – Essen/al administra/ve func/ons such as user management – News ar/cles and broadcasts keep the site fresh
• There are literally dozens of content management systems available • On the commercial side we have tried Lotus Connec/ons and Ra/onal Asset Manager
• Open source projects such as Drupal and Joomla are compelling!
27
Crea.ng a Community Space
28
29
Effec.ve Applica.on Development From small to large!
User-‐defined nodes (UDNs)
Opaque node implementa/on hides intellectual property Extend the built-‐in nodes with your own business logic Excellent poten/al re-‐use across many applica/ons
Subflows A piece of reusable logic implemented as a subflow Copy-‐and-‐pasted into the main flow by the BAR compiler
Deployable subflows Separately deployed from the main flows that invoke them
Libraries Packaging mechanism to facilitate reuse of many resources Shared understanding between the toolkit and run/me
Applica/ons Connec/vity solu/ons which are managed as a whole Typically combines one or more libraries together
PaWerns Re-‐use working solu/ons quickly and easily Creates produc/on ready connec/vity solu/ons Builds on all of the above technologies!
• Use the features of WMB & IIB to their full extent! – Message modelling, transforma/on op/ons, applica/ons and libraries – Capabili/es link in with ac/vity trace, record edit replay, and more
• Libraries and subflows make for simpler designs – Deployable subflows and libraries extend the benefits to the run/me
• Consider produc/on and performance early on in your design – In par/cular, applica/ons and configurable services – Your system administrator will thank you for this!
• PaWerns address /me-‐to-‐value for your solu/ons – It takes just a few minutes to create a paWern! – Why not create a paWerns community in your organisa/on?
• Tell us what you need – we really do listen • Developer Edi/on
– Free edi/on of IB for use in evalua/on, dev, test and produc/on usage – Fully func/onal: all nodes available and no /me limita/ons – Throughput rate limited to 1TPS per integra/on flow
30
Summary