move-3: openedge®, sonic esb®, and soa: qad's service interface layer fred yeadon, qad inc

MOVE-3: OpenEdge®, Sonic ESB®, and SOA:

QAD's Service Interface Layer

Fred Yeadon, QAD Inc.

MOVE-3: OpenEdge, Sonic ESB, and SOA: QAD's Service Interface Layer

2

Agenda

IntroductionUse Cases and Architecture Issues and Challenges


3

QAD GLOBAL FOOTPRINT

NASDAQ: QADI

Founded: 1979

IPO: 1997

FY05 Est. Revenue: $232-237 million

Licensed Sites: 5,200 worldwide

Customer Renewals: 90%+

Offices: 26 countries

Employees: 1,275 worldwide

Headquarters: Santa Barbara, CA

Fiscal Year End: January 31

ABOUT QADQAD COMPANY INFORMATION

A leader in providing ERP and supply chain execution software applications and services to global manufacturers in target vertical markets.

QAD Snapshot


4

Our Core Issues

Very large legacy base

Growing set of acquired/partner products

Tight UI – business logic coupling

Over 8,000 installed sites in over 26 countries.

Over 12,000 source files in MFG/PRO.

Customers resist upgrades!

Complementary add-ons. Progress and non-

Progress. Interoperability

challenges!

Difficult to place new skin on product.

Most ‘APIs’ are UI-dependent!


5

QAD Reference Architecture

Data Access

Business Services

BusinessEntities

BusinessTasks

Structured Unstructured

Workflow

ProcessOrchestrations

Service Interfaces

BusinessRules

Integration

BusinessReports

BusinessIntelligence

Data Warehouse

OLAP

ESB

Develo

pm

en

t & D

ep

loym

en

t Tools

Presentation

Web Local

Controller

Model

Web Service

EDI IV&I

Event Pub


6

Service Interface: Responsibilities

Expose business services to be called by various types of clients

Dispatch incoming requests to correct local service program method

Provide local copy of a public APIs for a remote service

Must be agnostic to remote service physical location

Send outgoing requests to correct remote service program method

Insulate business logic from the interfaces of all other services and their callers

Business Services

BusinessEntities

BusinessTasks

Service Interfaces

BusinessRules

BusinessReports

ESB


7

Foundation Technologies

Web services– Any request may be sent and processed as XML SOAP

message– OpenEdge Web Services Out (WSO) used on client side– ESB Adapter, ProxyGen used on server side

ProDataSets– Used to package all application data, method signatures– XML message syntax is based on dataset structure

Sonic ESB– Transport for web-service messages– Message transformation and content-based routing– Hosts API methods as services

AppServers– Any request-reply may be executed as direct AppServer™

call


8

Why Web Services?

Rapidly becoming foundation/prereq technology for:– Business messaging– System-level services– SOA, in general

Strategic for major QAD technology suppliers– Progress and Sonic Software– Microsoft– IBM– Emerging SysAdmin tools (Actional, …)

Basis for new generation of interoperability standards– WS-Security, WS-Addressing, …

BUT…– It is ‘slow’ – don’t use for everything!– It is still maturing– It complicates the technology stack (for now)

SOAPHTTP

WS-*WSDL


9

OpenEdge ProDataSets and XML

Dataset

SODetail

Comments

.

.

.

.

.

.

SODetail

Ship-ToAddr

SalesOrder

XML<callLoader xml:ns0=“urn:schemas-qad-com:xml-services> <dsSessionContext>… <dsSalesOrder> <ttSalesOrder> <soNbr>SO001</soNbr> <shipToID>DEF</shipToID> <ttSalesOrderDet> <domain>WHQ</domain>

<itemNumber>10-10000</itemNumber><qtyOrder>45</qtyOrder>…<ttSOL_Comments> <comments>Package req… </comments></ttSOL_Comments>

</ttSalesOrderDet> </ttSalesOrder> </dsSalesOrder></callLoader>


10

Why Sonic ESB?

J2EE-based– Cross platform– Custom services built using Java™, JavaScript

Standards-compliant– Java Messaging Service (JMS)– Web services– XML, XSLT, XQuery– Java Management Extensions (JMX) APIs used for

administration Well regarded in ESB-MOM-EAI market space

– ESB ‘pioneers’ and thought leaders– Used successfully in financial and other industries

Owned by Progress Software– OpenEdge ‘Sonic ESB Adapter’ provides Sonic interoperability– Less mature than OpenEdge product line


11

ESB

<SOAP-ENV:Envelope …><SOAP-ENV:Header>..<SOAP-ENV:Body><callLoader xml:ns0=“urn:schemas-qad-com:xml-services> <dsSessionContext>… <dsSalesOrder> <ttSalesOrder> <soNbr>SO001</soNbr> <shipToID>DEF</shipToID> <ttSalesOrderDet> <domain>WHQ</domain>

<sodPart>10-10000</sodPart> <sodQtyOrd>45</sodQtyOrd> … <ttSOL_Comments> <comments>Package req… </comments> </ttSOL_Comments>

</ttSalesOrderDet> </ttSalesOrder> </dsSalesOrder></callLoader></SOAP-ENV:Body></SOAP-ENV:Envelope>

DistributedOrder

Management

MFG/PROeB2.1

Database

Router

Inter-Product Messaging

…

……


12

Key Features … Request-reply interactions One-way (async) request processing and

acknowledgment UI communications Transport transparency Location transparency Remote session transparency State-free/stateless operation

… And Gaps! ESB error and queue administration Publish-subscribe Security API versioning Entity state management XML syntax inconsistent with QDoc XML


13

Agenda



14

Distributed Order Management

Take an order from anywhere

Source it from anywhere

Maintain centralized control


15

Category # Entities Implemented

Order – Customer – Supplier - Item

7

Common Codes 13

Infrastructure 5

User Interface** 21

Total 46

* As of initial early-adopter release

Business and Service Entities in DistOM*

** Data structures and methods built to service the UI


16

The Global Sales Order

Customer requestGlobal credit check for enterprise riskGlobal order line independence for

ship-to address and sourcingSplit and routed to appropriate

business unit (‘domain’)Aggregate representation of executable

legal ordersOrders confirmed on a business unit

level


17

Distributed Order Management Architecture

WebSpeedAgents

AppServer

SI

SI

SI

WebBrowser

DOM MFG/PRO eB2.1/eB3(multi-domain)

domain 1

domain 2

domain 4

domain 3

AppServer

…

… OE Clients

DB

MFG/PROChUI orDesktop

DB

Sonic ESB


18

Sales Order Processing

DOM

Serv

ice Inte

rface

Sonic ESB

MFG/PRO

ED

I EC

om

merc

eServ

ice Inte

rface

Cache MFG/PRO data

Price, total sales order

Sales order (850/860)

Functional acknowledgment(997 equivalent)

Process acknowledgment(855/865)

Pro

pagate

SO

Ack

to P

BU

Yes

NoMulti-level

demand?

SO ack for PBU (855/865)

FileExternal SO

(850/860)

Error?


19

Key Service Scenarios

Request-Reply Via AppService– Used for remote data caching, sales order

pricing, credit checking.

Request-Reply Via WebService– Not currently used.

One-way Request With Acknowledgment– Used for all business document processing:

orders, changes, shipment notices.


20

Request-Reply Scenario: AppServices

AppServiceCaller

AsyncReply

Handler

SessionManager

Service/BusinessEntities

RPCRequestService

ServiceProxies/Adapters

OE Client-Server

Sync

Async

OE AppServerOE AppServer

ConfigFile

Cache MFG/PRO data

Price, total sales order

DistOM MFG/PRO


21

Alternate Request-Reply: ESB Web Services

WS

OWebServiceCaller

AsyncReply

Handler

OE AppServer OE AppServer

Sonic ESB Container

Sonic ESB Container

SessionManager

Pre-ProcessRequest

XSLTCBR

……

HTTP-SOAP


XMLRequestService

OESvc

ES

B A

dap

ter


Post-ProcessResponse

XSLT

QADServiceProxyGen.pWSDLProxyGenWSDLWSDL

CBRReply-To?

ConfigFile

DistOM MFG/PRO


22

Request-Reply Web Services: Long Term

WS

OWebServiceCaller

AsyncReply

Handler


Container

Container

SessionManager

CBR

……

HTTP-SOAP OE

Svc

ES

B A

dap

ter

Service Proxies/Adapters

WSDLWSDL ProxyGen


•Reply-•To?

CBR

ConfigFile

DistOM MFG/PRO


23

One-Way Request Scenario: ESB Web Services

WS

O

WebServiceCaller

XMLAck

Service


Sonic ESB Container

Container

SessionManagerCBR …

…HTTP-SOAP1-Way


XMLRequestService


Post-Proc Ack

XSLT

Process Doc

OESvc

ES

B A

dap

ter

Pre-Proc Ack

XSLT

Post User Credentials

XSLTCust

MaintainUsername-Password

CBR

GenerateAck

DistOM MFG/PRO


24

SOAP Message Headers: Sample<SOAP-ENV:Header>

<ns0:Action xmlns:ns0=“http://www.w3.org/2005/08/addressing”/>

<ns0:To xmlns:ns0="http://www.w3.org/2005/08/addressing">urn:services-qad-com:AckProc:myServiceCaller</ns0:To>

<ns0:MessageID xmlns:ns0="http://www.w3.org/2005/08/addressing">urn:messages-qad-com:myCompany:2005-11-09T100:00:00.000-05:00:AckProc:myServiceCaller:ProcessAcknowledgmentTest.p:processAcknowledgment:myDocument001</ns0:MessageID>

<ns0:ReplyTo xmlns:ns0="http://www.w3.org/2005/08/addressing"/>

<ns0:RelatesTo xmlns:ns0="http://www.w3.org/2005/08/addressing">urn:messages-qad-com:myServiceCaller:2005-10-25T15:30:20.261-05:00:TestService:myCompany:SvcIntTestService.p:pingMe:myDocument001</ns0:RelatesTo>

<ns0:ReferenceParameters xmlns:ns0=“http://www.w3.org/2005/08/addressing”/>

</SOAP-ENV:Header>


25

Enterprise Service Bus

Web Service Caller

Service Proxies

XML Response XML Request

Adapters

DatasetTo XML

Datasets

Calling Out to Other Services

XML To Dataset

AppService Caller

<ExternalApplicationAppServer>

Datasets

Response

Request

Business Logic


26

Sample DistOM Proxy Code

/* Dataset passed to-from remote service */define dataset dsCustomerCredit for ttCustomerCredit.

/* Defines standard method to retrieve API dataset */{BusinessService.i &DATASET1=dsCustomerCredit }

/* Proxy method (defined in application-specific wrapper) */

{dom/DOMServiceProxy.i &PROGRAMNAME="socustcr.p" &METHODNAME="getCreditCheckParameters" &PRIMARYBUFFER="ttCustomerCredit" &SERVICEINSTANCEFIELD="domain" &DATASETINOUT="dsCustomerCredit" }


27

Sample Proxy Call from DistOM Adapter

/* Start the proxy service */run dom/CreditVisibilityProxy.p persistent set hProxy ( input "WEBSERVICE", input "mfgpro", input "dom/DOMWSOConnectionTest.xml“ ).

/* Get the dataset used by the service */run getServiceDataset in hProxy ( input "dsCustomerCredit", output hdsCustomerCredit ).

/* Formulate domain query */ .../* Iterate across all MFG/PRO target domains */do while not hqDomain:query-off-end: /* Map-copy the portion of the local dataset for each

domain present in the data */ ...


28

Sample Proxy Call from DistOM Adapter (cont)

if domainFound then do: /* Set domain to run method in correct session context */ run setQADContextProperty ( input “domain”, input domainID ). /* Call service method */ do on error undo, leave: run getCreditCheckParameters in hProxy ( input-output dataset-handle hdsCustomerCredit by-

reference ). end. ... end. /* if domainFound */ hqDomain:get-next().end. /* do while not hqDomain:query-off-end */


29

Sample Web Service Request

<SOAP-ENV:Envelope …><SOAP-ENV:Header>…</SOAP-ENV:Header><SOAP-ENV:Body><callLoader xml:ns0=“urn:schemas-qad-com:xml-services> <dsSessionContext>… <dsSalesOrder> <ttSalesOrder> <soNbr>SO001</soNbr> <shipToID>DEF</shipToID> <ttSalesOrderDet> <domain>WHQ</domain>

<sodPart>10-10000</sodPart> <sodQtyOrd>45</sodQtyOrd> … <ttSOL_Comments> <comments>Package req… </comments> </ttSOL_Comments>

</ttSalesOrderDet> </ttSalesOrder> </dsSalesOrder></callLoader></SOAP-ENV:Body></SOAP-ENV:Envelope>

MessageBody

(Payload)

SOAP Headers(used for

CBR)

ApplicationDatasetMessage

Envelope

Method Name Session

ContextDataset


30

Service Method Signatures

Dataset dsSessionContext (I-O)– Session context properties.– Target method/program name.

Dataset dsExceptions (O)– Unhandled exceptions thrown from service.– Contains temp-table temp_err_msg.

<Application dataset> (I)<Application dataset> (I-O)<Application dataset> (O)

• Up to 1 per I-O mode => 8 possible

signatures

Not passed to application programs


31

OpenEdge AppServer

Enterprise Service Bus

ESB OpenEdge Service

XMLRequestService

Service Programs/Entities

Business Logic

API Methods

XML Request XML Response

Sonic ESB AdapterLongchars and temp-

tablesNon-persistentgateways

SessionManager

Coarse-grained

Fine-grained

Calling In to Business Logic

RPCRequestServiceDatasets

<ExternalApplicationAppServer>

Datasets

Response

Request

ExceptionManager

State-free/

StatelessApp

Server


32

Sample Service Program Code

/* Service dataset with temp-table */{sottcrdt.i}define dataset dsCustomerCredit for ttCustomerCredit.

/* Standard methods implemented by all service programs */{bussvc.i &DATASET1 = dsCustomerCredit}

procedure getCreditCheckParameters: define input-output parameter dataset for

dsCustomerCredit. /* Process API request */ ...

end procedure. /* getCreditCheckParameters */


33

Sample Session Context Properties…<callLoader xml:ns0=“urn:schemas-qad-com:xml-services> <dsSessionContext> <ttContext> <propertyQualifier>QAD</propertyQualifier> <propertyName>sessionID</propertyName> <propertyValue>mfg12345</propertyValue> </ttContext> <ttContext> <propertyQualifier>QAD</propertyQualifier> <propertyName>methodName</propertyName> <propertyValue>getSalesOrderTotals</propertyValue> </ttContext> </dsSessionContext> <ttContext> <propertyQualifier>QAD</propertyQualifier> <propertyName>programName</propertyName> <propertyValue>sototapi.p</propertyValue> </ttContext> … </dsSessionContext> <dsSalesOrder>…</dsSalesOrder></callLoader>…

Sessioncontext

properties


34

Agenda



35

Issues and Challenges

Retrofitting To Legacy ApplicationHeadless Authentication and Session

ManagementMore Complex Technology StackOE 10.0 Bugs in Web Services OutPerformance


36

Future Directions

Upgrade to OpenEdge 10.1Enhanced security, sign-onBullet-proof error managementSupport data caching using Sonic XML

ServerDeploy with .NET User Interface


37

Questions?


38

Thank You!

move-3: openedge®, sonic esb®, and soa: qad's service interface layer fred yeadon, qad inc

Documents

remote servicemust

foundationprereq technology

technology stack

global manufacturers

new skin

interoperability challenges

public apis

installed sites