InvenireAude.com Copyright (C) 2015 Invenire Aude Ltd.

Practical Guide to

Invenire Aude System

over IBM WebSphere MQ

Introduction

Versatile framework

● Data Processors

● Transport and Database Connectors

● Fault tolerant deployments

No Java, small memory footprint and high performance.

Working with IBM WebSphere MQ

From

Easy to use ad-hoc command line message manipulation tool,

through

Quick to use build-in processing patterns (proxies, gateways),

to

Complete, the fault-tolerant deployment of hundreds of efficient script processing nodes for advanced SOA and CEP scenarios

Features HighlightsTransport protocols:

● HTTP(s) and native TCP (SSL), ● IBM Websphere MQ – client and server ● Native in-memory Queues● Files, Directories and more● Formats

● JSON (with inheritance pattern),● XML – reasonable subset

● Database options (XA support if applicable)● IBM DB2 – commercial module● Oracle – commercial module● SQLite and more ...

● PASCAL-like syntax, easy to digest, processing language– no explicit compilation required, eclipse highlights plugin- small final code footprint

● Built-in modules (event counters, log analyzers and more).

● Extendable with custom modules (logic, parser, databases).

One to Process Them All

For simplicity of this graphics we skip some facts:one can define and use many inputs and outputs, including

requesters, responders, publishers, subscriberswithin or without the transaction control.

Disclaimer :)

Although, one can build a full

CEP or SOA solution using this framework.

we will start small scale ...

The examples that follow are not the typical use cases of the Data Processor, they are meant to give you some hints how this tool can accompany you in your everyday tasks.

The next part will present some advanced scenarios and examples how and what sort of the applications can be built upon this framework.

For more information on the Data Processors, visit:

http://www.invenireaude.com

Examples Part One

Basic Command Line

Accessing Queues

Displaying message content:

ias_qs_processor ­i mqm://MQTEST/Q1 ­o file:stdout ­l fwdias_qs_processor ­i mqm://MQTEST/Q1 ­o file:stdout_fmt ­l fwd

Moving message to the file:

ias_qs_processor ­i mqm://MQTEST/Q1 ­o file:output.xml ­l fwd

­ l fwd – specifies the built-in forwarder logic.

Parsing

export IAS_LANG_XSD=${your_project}/data/customer.xsd

Note the exclamation mark right after the logic name:

­l fwd vs ­l fwd! Turns the parsing off.

Data Format Change

 ias_qs_processor ­i mqm://MQTEST/Q1?mode=input \   ­o mqm://MQTEST/Q2?format=JSON ­l fwd

 

Working with RFH

 ias_qs_processor ­i mqm://MQTEST/Q1?mode=input \   ­o 'mqm://MQTEST/Q2?*Folder.ATTR1=VALUE' ­l fwd

The '*' attributes provided in URL are considered as message meta attributes and will be seen in RFH2 or message properties according to the application or IBM WebSphere MQ version.

 

Examples Part Two

Protocol Gateway

Example Queue Setup

DEF QL(GATEWAY.IN)DEF QL(SERVICE.IN)DEF QA(GATEWAY.OUT) TARGQ(SERVICE.IN)

Echo Service

If you do not have a service for this example just start the following command to setup the echo program.

ias_qs_processor \ ­i 'mqm://MQTEST/SERVICE.IN?mode=responder&timeout=­1' \  ­o dummy: ­l fwd!

 

Gateway Synchronous Proxy

ias_qs_processor \  ­i 'srvhttp://localhost:50000/mode=input&responderName=input' \  ­o 'mqm://MQTEST/GATEWAY.OUT&mode=requester&inputDestination=GATEWAY.IN&timeout=20000' \  ­l proxy! ­m server

Note the srvhttp service and the proxy logic. 

Gateway Asynchronous Proxy

ias_qs_processor \  ­i 'asrvhttp://localhost:50000/mode=input&responderName=input' \  ­o 'mqm://MQTEST/GATEWAY.OUT&mode=requester&inputDestination=GATEWAY.IN&timeout=­1' \  ­l aproxy!

Note the asrvhttp service and the aproxy logic.

Testing

curl ­X POST ­d "{ 'key': 'value1234' }" \     ­H "Content­Type: application/json" \        http://localhost:50000/

You can use the HTTP client of your choice.

Examples Part Three

Basics of Scripting

Environment

Data model:IAS_LANG_XSD=/home/ias/data/xsd/model.xsd

Script Sources:IAS_LANG_SRC_DIRS=/home/ias/lang

First Script

 vi /home/ias/lang/update_msgs.y

The Eclipse source highlighting plugin is available.

See the Script Language for more details (for example on writing procedures accessing databases and files and more).

Processing Data

Testing, browsing and displaying result on the screen:

ias_qs_processor ­i mqm://TEST/Q1 \ ­o file:stdout_fmt ­l exe update

Processing, moving and saving changed content in the output queue:

ias_qs_processor ­i mqm://TEST/Q1?mode=input \ ­o mqm://TEST/Q2 ­l exe update

Performance Notes

Options:

 ­T <no. of threads>

 ­C <commit count>

 ­N <no. of messages to process> ­O <no. of messages to skip>

Examples Part Four

Advanced Configurations

Advanced Configuration

You do not have to specify everything as command line options !!!... and you should NOT.

When developing an application, you:

● Organize your source as usual (namespaces, version control).● Prepare configuration files (XML or JSON) for your processors.● Use the Service Manager as processors execution monitor.● Easily switch transport layers for testing (eg. files vs. messages)

As no compilation is required the deployment is usually an VCS export/checkout.

Configuration Files

  ias_qs_processor ­f <configuration_file>

Configuration (XML or JSON)

- input and output specification (Queues, HTTP(S), Files)

- data sources (data bases)

- logic(s) configuration

Environment Specific Settings

IBM WebSphere MQ Configuration

  <inputs xsi:type="io:SubscriberInput"           inputName="input" >    <connection>      <protocol>mqm</protocol>      <host>MQTEST</host>    </connection>      <destination>T.EVENT.ACCTXN</destination>      <txnMode>TXN</txnMode>      <timeout>­1</timeout>        </inputs>

IBM WebSphere MQ Driver Features

  Connection types: ­ Server (mqm) ­ Client (mqic)

  Transaction support:­ NONTXN, TXN, XA (Requires the standard qm.ini setup)

  Access to the message properties (RFH2).

  Typical processing patterns:­ datagrams, requests, responses, publish­subscribe

Some MQ Specific MQMD fields are accessible via Meta attributes (feedback, report, expiration, etc.)

Learn More

Visit:

http://www.invenireaude.com/content/articles/index.html

Build your installation (see Ubuntu Quick Start) in less than 15 minutes.

Download and learn from these tutorials:Customer Information File:

- Database access with Queues and HTTP/REST- Event Driven ESB- Gateways Examples

Complex Event Processing- account transaction processing rules- system log screening