positioning of process orchestration and data services

Positioning of Process

Orchestration and Data Services

Florian Koeller, SAP Consulting

Matthias Allgaier, Product Management, Integration & Orchestration

Sue Waite, Solution Management, Enterprise Information Management

01/2013

© 2013 SAP AG. All rights reserved. 2

The information in this presentation is confidential and proprietary to SAP and may not be disclosed without

the permission of SAP. This presentation is not subject to your license agreement or any other service or

subscription agreement with SAP. SAP has no obligation to pursue any course of business outlined in this

document or any related presentation, or to develop or release any functionality mentioned therein. This

document, or any related presentation and SAP's strategy and possible future developments, products and

or platforms directions and functionality are all subject to change and may be changed by SAP at any time

for any reason without notice. The information in this document is not a commitment, promise or legal

obligation to deliver any material, code or functionality. This document is provided without a warranty of any

kind, either express or implied, including but not limited to, the implied warranties of merchantability, fitness

for a particular purpose, or non-infringement. This document is for informational purposes and may not be

incorporated into a contract. SAP assumes no responsibility for errors or omissions in this document, except

if such damages were caused by SAP´s willful misconduct or gross negligence.

All forward-looking statements are subject to various risks and uncertainties that could cause actual results

to differ materially from expectations. Readers are cautioned not to place undue reliance on these forward-

looking statements, which speak only as of their dates, and they should not be relied upon in making

purchasing decisions.

Legal disclaimer


Scope of this document

Focus

On-premise variants of SAP NetWeaver Process

Orchestration and SAP Data Services

Goals

Explain the motivation behind those two platforms

Enable customers to elaborate their usage patterns for

both platforms

Not in scope of this document

Delimitation with other related products such as SAP

NetWeaver Gateway

Alignment to related methodologies such as Enterprise

Architecture, Business Process Management,

Information Modeling and Data Governance


Table of Contents

Process and Data Integration with SAP – Overview

The Delineation

Outlook – SAP HANA Cloud Integration

APPENDIX: Use Cases in SAP Integration

Process and Data Integration

with SAP – Overview


There are different levels of integrating business applications / parties.

In this document we focus on:

Process Integration*

Data Integration

Both architectural styles differ in their embedding into the application context.

Architectural Styles of Application Integration

Application 1

Process Integration

Data Integration

User

interaction

Data

Business Logic

Connectivity

Application 2

User

interaction

Data

Business Logic

Connectivity

* Remark: this relates to the approach, not the product

SAP NetWeaver Process Integration


Process Integration is now part of SAP NetWeaver

Process Orchestration

SAP NetWeaver Process Orchestration helps IT and line-of-

business professionals improve business performance by

orchestrating the work of people and software systems into

automated business processes.

SAP NetWeaver Process Orchestration includes:

Business Process Management (BPM) Model and execute business processes that augment best

practices of packaged applications, such as SAP ERP

Business Rules Management (BRM) Centrally manage business policies that can be maintained by

business and IT

Process Integration (PI) Efficiently integrate processes across heterogeneous IT

landscapes as well as between business partners including a

wide range of supported connectivity options (such as the PI

B2B Add-On or partner adapters)


Business Process Management and Integration with SAP

NetWeaver Process Orchestration


SAP Data Services Integrate a variety of data from any data source, enabling high quality results

Big Data

– Volume

– Variety

– Velocity

Information Governance

– Quality Data

– Across Processes

– For Analytics

Simplified

– Unified

– Intuitive

– Flexible

One Runtime Architecture &

Services

Business UI (Information Steward)

Unified Metadata

Technical UI (Data Services)

SAP Data Services 4.1

ETL

Data Quality

Profiling

Text Analytics

One Administration Environment

(Scheduling, Security, User Management)

One Set of Source/Target Connectors


Process Integration - Characteristics

A business process step triggers the integration from inside of the business

process

The business process steps involved have agreed on the data structure

The exchanged data is produced with specific business process knowledge

Typically the data is PUSHed from the sending business application1,2

What are the typical units of data exchange?

Business Objects (orders, invoices, …)

Methodology related to Business Process Management (BPM) and Enterprise

Architecture Management (EAM) approaches

1 Frequently, batch jobs running within the sending business application are used to generate messages to be processed by the middleware. 2 In addition to the typical PUSH paradigm, process integration also supports PULL scenarios through synchronous interfaces.


Data Integration - Characteristics

The trigger of the integration is from outside of the business process

The access to the source data pool and the target data pool is also from outside of

the business process

Data transformation provides a generic (i. e. not a business process specific)

processing of the exchanged data

This data transformation requires (static) application knowledge in general,

but no respective (dynamic) business process knowledge.

Typically the data is PULLed out of the business application/database, such

as realized by a QUERY/READ pattern

What are the typical units of data exchange?

Databases tables, records

Methodology related to Data and Information Modeling approaches


Further approaches

There are further related and interrelated integration approaches such as

Enterprise Application Integration (EAI)

Service-oriented architecture (SOA)

Extract Transform Load (ETL)

B2B / EDI

We will not discuss all these different approaches in detail in this document

The Delineation


In an on-premise SAP environment…

…process integration, EAI, SOA, B2B and EDI are in scope of SAP NetWeaver

Process Orchestration

More information: http://scn.sap.com/community/process-orchestration

More information on B2B: https://scn.sap.com/docs/DOC-31685

…data integration, data quality management, and text data processing are in

scope of Enterprise Information Management, in specific SAP Data Services

More information: http://scn.sap.com/community/data-services

As mentioned above: there is a certain grey area between all these

approaches

Consequently: a grey area between SAP NetWeaver Process Orchestration and

SAP Data Services

This means: you can solve certain requirements with both

The delineation between these two solutions is the focus of this document

http://scn.sap.com/community/process-orchestration



https://scn.sap.com/docs/DOC-31685




There are different levels of integrating business applications / parties.

Both architectural styles differ in their embedding into the application context.

For a given integration requirement, which of the two platforms should be

used?

Deciding for the Architectural Styles of Application

Integration

Application 1

Process Integration

Data Integration

User

interaction

Data

Business Logic

Connectivity

Application 2

User

interaction

Data

Business Logic

Connectivity

?


How to decide?

As mentioned above: there is a grey area, an overlap

The delineation depends on various parameters, such as

…the technical and functional capabilities of SAP NetWeaver Process Orchestration

and SAP Data Services

…the organizational environment

…the type of integration to be done

…the technical environment (e. g. hardware sizing)

Technical and functional capabilities of SAP NetWeaver Process

Orchestration and SAP Data Services are discussed in the following

All other parameters are subject to your environment


Decision Tree Extract (Example)

Recommendation: elaborate a decision tree that considers all these

parameters

It can look like this

(this is just an extract): …


Process-centricWorkflows

Integration-centricWorkflows

Basic to intermediateData Transformation

Advanced DataTransformation

Unstructured TextData Processing

Big Data

Data QualityManagement

Dataset LevelDatabase

Connectivity

Batch Processing

Real-TimeProcessing

B2B/EDI

Message Level

SOA Capabilities

Reliable Messaging

ProcessOrchestration

DataServices

Radar Map


Capability Capability Detail Process

Orch.

Data

Services

Notes

Data Volume Small to medium

Data Services moves big data (such as

billions of records) in a non-transactional

mode. PO processes a large number of

messages containing small or medium

amounts of data efficiently. Big Data

Processing Mode Scheduled / Batch based

Data latency with DS tends to be larger

(minutes rather than seconds). PI offers

mechanisms for real-time synchronous

transactions. Real-Time to near Real-

Time (both synchronous

and asynchronous)

Event- / Trigger-based

Pattern for Direction

of Request

Data is pushed out of

sender application

Data Services processing is typically

scheduled (at a certain point in time, it

pulls the data from the source), however, it

has capability to receive events via web

services or other means and process

these events. PO is typically triggered by

incoming data.

Data is pulled from sender

application

Reliable Messaging Guaranteed Delivery DS does not support guaranteed delivery

in a robust fashion.

Capabilities – Details



Orch.

Data

Services

Notes

Data Transformation Basic (Data type

conversions, string

manipulation, calculations,

etc)

Both Data Services and PO apply

transformations (Mapping, split,

aggregation, …), with Data Services

having a special focus on advanced

transformations needed for Dat

Warehouse or data migration.

In addition, Data Services has the ability to

cleanse data.

Intermediate

(Lookup/replace,

aggregations,

summarizations, etc)

Advanced (Slowly-

changing dimension

management, matching,

etc)

Data Quality Management

Business Process

Management

(Workflows)

Integration-centric Both PO and Data Services offer support

for integration-centric workflows. Data

Services workflows tend to be simple

having to do with process sequencing and

error handling.

PO provides full workflow capabilities for

both integration- and process-centric

workflows in an SAP environment (through

SAP NetWeaver BPM).

Process-centric

Capabilities – Details (2)



Orch.

Data

Services

Notes

Connectivity File Handling

Database Handling is the core functionality

of BusinessObjects Data Services. It is

able to extract tables in a complex

manner.

PI is able to access databases via JDBC.

* EDI is available as integration content (e.

g. SAP B2B Add-on) or as an on-demand

solution (SAP Information Interchange)

SAP Applications (Proxy,

RFC, IDoc,…)

Legacy Protocols (JMS,

…)

Databases

Industry Standards (e. g.

EDI, …)

SOA (Service-

oriented Architecture)

Web Service / Enterprise

Service consumption and

provision

SAP NetWeaver PI as part of PO is

positioned as the SOA middleware. Data

Services is able to act as a service

provider or consumer, but does not offer

further SOA functionality (e. g. with

regards to governance).

SOA governance and

operations

Data Processing

Level

Message Level PO exchanges messages between

systems while Data Services rather

exchanges data sets. Dataset Level


*



Orch.

Data

Services

Notes

Handling of

Unstructured Data

Handling of unstructured

and semi-structured data

such as Excel, PDF, etc

Handling of Unstructured Data is one of

the core functionalities of SAP Data

Services.

For PI, the Conversion Agent is available;

however, it is not released for PI 7.3 and

PI 7.31 anymore. For details, please see

SAP note 894815.


Outlook – SAP HANA Cloud

Integration


Outlook - SAP HANA Cloud Integration Process and data integration capabilities in one offering

Notes: SAP NetWeaver Process Integration and SAP Data Services will also support integration with SAP cloud solutions.

Cloud Integration

Technology

Process and data integration

capabilities to enable integration

of SAP cloud solutions to:

SAP on-premise solutions

SAP cloud solutions

Third-party cloud solutions

Third-party on-premise solutions

Cloud Integration Content

Prepackaged integration content

for SAP to SAP, for example,

via rapid-deployment solutions

Customers, ecosystem, and

community can extend and

create new content

On-Premise

SAP Cloud

ERP CRM ….

On-Premise

Agent

Siebel … PeopleSoft Legacy

SAP On-Premise Solutions Third-Party On-Premise

Solutions

Non-SAP

Clouds

SAP integration content

Partner integration content

Application enablement

SAP HANA Cloud Integration

Mapping and

Transformation

Data

Security

Monitoring and

Administration

Development &

Lifecycle Mgmt

Governance

& SLAs

Extensibility /

Adapter SDK

…...

Connectivity

Service

Data

Quality

Data

Consolidation

SAP Cloud Solutions

People Customers Suppliers Money

APPENDIX: Use Cases in SAP Integration


SAP NetWeaver Process Orchestration for various

business scenarios

For certain integration scenarios SAP and partners provide Process

Orchestration content

A&

D

Au

to

Ch

em

CP

EC

&O

HiT

ec

h

Ind

MC

Lif

eS

c

Mil

lPro

Oil

&G

Air

lin

e

Lo

gS

P

Me

dia

Po

sta

l

Pro

fSr

Rail

Wy

Reta

il

Te

lco

Uti

lity

Wh

Sa

l

Ban

kin

Hlh

Cr

HiE

d

Ins

ur

Pu

bS

cr

PS

ec

ur

Enterprise

Service

Integration

Scenarios

Service

Interfaces

Message

Mapping

Industry

Adapters

Manufacturing Industries Services Industries Financial & Public Ser

Partner Solutions Industry-specific Non industry-specific Adapters SAP Solutions


SAP NetWeaver Process Orchestration for B2B/EDI (on-

premise)

SAP NetWeaver Process

Orchestration

B2B Add-On

B2B Infrastructure Services

B2B Technical Adapters

Converter Modules

Security

EDI splitting

Numbering

Acknowledge-

ments

Archiving

Mapping

functions

Mapping

templates

AS2

OFTP

ANSI X.12

EDIFACT

TRADACOM

Odette

VDA

Plain Text

Secure Connectivity Add-

On

Technical Adapter

Adapter Module

SFTP

PGP

SAP offers a

comprehensive

B2B/EDI add-on for

SAP NetWeaver

Process

Orchestration

Business

Partners

X.400

http://imagedirectory.wdf.sap.corp:6085/ImageLibrary/ImageDetailsView?imageId=180f12d0-7fed-4d31-a1a9-8b69e794b201


SAP NetWeaver Process Orchestration for B2B/EDI (within

on-demand SAP Information Interchange offering)

SAP Information Interchange (formerly Crossgate)

Turnkey B2B Content Engine that allows companies to exchange documents

electronically

Prebuilt business partner profile and out-of-the box process integration

Plant

Supplier

3PL

Customers

SAP Business Suite

PLM

SCM

SRM

CRM

ERP

SAP Information Interchange

Orders In- / Outbound

Other Docs, etc…

Order Acknowledgement In- / Outbound

Invoice In- / Outbound

ASN In- / Outbound

Delivery Forecast In- / Outbound

Remittance Advice In- / Outbound

Delivery JIT In- / Outbound

Inventory Report In- / Outbound

Self Billing Invoice In- / Outbound

Shipment In- / Outbound

PLUS TM & SNC Enterprise Services Wareho

use

SAP

NetWeaver

Process

Orchestration


SAP Data Services for all kinds of database access

Direct, native access to all kinds of databases is a typical use case of SAP

Data Services

SAP Data Services

Oracle

DB2

HANA

Sybase IQ & ASE

SQL Server

Hadoop

Informix

Teradata

ODBC

MySQL

Netezza

HP NeoView

…


SAP Data Services for batch loading into databases (SAP

HANA, Sybase IQ, …)

Load into HANA, Sybase IQ or any 3rd party target database

Support for latest versions/revisions of these databases

Optimized for “push-down” to HANA and Sybase IQ, leveraging the power of the

target system for E-LT scenarios (*).

Any

Source

Data Load

Metadata

Open

Hub

SAP Data Services**

In-Memory

Computing

Engine

Modeler

SAP HANA

BW

Repository

Server

(*) E-LT = First Extract from the source and Load raw data into the target, finally Transform inside the target. As opposed to traditional ETL.

(**) Data Integration license for ETL/ELT is bundled with HANA to enable data loads into HANA.


SAP Data Services for Business Warehouse Integration

Non-BW customers

Now get native access to the Business Suite without having to go through BW, including delta

queue support

Data is streamed, no staging files (no ftp, no shared directory, ...)

BW customers

Can now easily apply data quality transformations when loading SAP data into BW

Can now use one single tool to define all extraction, validation, and cleansing rules to load all

data (SAP and non-SAP) into BW

SAP

Business Suite

All 3rd-Party

Data Sources SAP Data Services

SAP Extractors

DWH,

data marts,

applications, ...

SAP NetWeaver

BW

Hadoop / Hive


SAP Data Services for Data Migration (within SAP Rapid

Data Migration Solution)

Software SAP Data Services for ETL and Data

Quality

SAP BI Platform for reporting

Migration Services for value mapping

Pre-configuration Best Practices

Mapping-Templates

Pre-configured mapping and validation

jobs

WebI Reports

Enablement content Quick Guide

Extension Guides

Predefined service

(via SAP or partners) Starter Service

Fixed scope, fixed cost

SAP

DATA MIGRATION

1

ANALYSIS

2

EXTRACT CLEAN

5

LOAD

6

RECONCILE VALIDATE

Governance and Visualization

4 3

Thank You! Florian Koeller SAP Consulting [email protected]

Sue Waite SAP Enterprise Information Management [email protected]

Matthias Allgaier SAP Integration & Orchestration [email protected]


© 2013 SAP AG. All rights reserved.

No part of this publication may be reproduced or transmitted in any form or for any purpose

without the express permission of SAP AG. The information contained herein may be

changed without prior notice.

Some software products marketed by SAP AG and its distributors contain proprietary

software components of other software vendors.

Microsoft, Windows, Excel, Outlook, PowerPoint, Silverlight, and Visual Studio are

registered trademarks of Microsoft Corporation.

IBM, DB2, DB2 Universal Database, System i, System i5, System p, System p5, System x,

System z, System z10, z10, z/VM, z/OS, OS/390, zEnterprise, PowerVM, Power

Architecture, Power Systems, POWER7, POWER6+, POWER6, POWER, PowerHA,

pureScale, PowerPC, BladeCenter, System Storage, Storwize, XIV, GPFS, HACMP,

RETAIN, DB2 Connect, RACF, Redbooks, OS/2, AIX, Intelligent Miner, WebSphere, Tivoli,

Informix, and Smarter Planet are trademarks or registered trademarks of IBM Corporation.

Linux is the registered trademark of Linus Torvalds in the United States and other countries.

Adobe, the Adobe logo, Acrobat, PostScript, and Reader are trademarks or registered

trademarks of Adobe Systems Incorporated in the United States and other countries.

Oracle and Java are registered trademarks of Oracle and its affiliates.

UNIX, X/Open, OSF/1, and Motif are registered trademarks of the Open Group.

Citrix, ICA, Program Neighborhood, MetaFrame, WinFrame, VideoFrame, and MultiWin

are trademarks or registered trademarks of Citrix Systems Inc.

HTML, XML, XHTML, and W3C are trademarks or registered trademarks of W3C®,

World Wide Web Consortium, Massachusetts Institute of Technology.

Apple, App Store, iBooks, iPad, iPhone, iPhoto, iPod, iTunes, Multi-Touch, Objective-C,

Retina, Safari, Siri, and Xcode are trademarks or registered trademarks of Apple Inc.

IOS is a registered trademark of Cisco Systems Inc.

RIM, BlackBerry, BBM, BlackBerry Curve, BlackBerry Bold, BlackBerry Pearl, BlackBerry

Torch, BlackBerry Storm, BlackBerry Storm2, BlackBerry PlayBook, and BlackBerry App

World are trademarks or registered trademarks of Research in Motion Limited.

Google App Engine, Google Apps, Google Checkout, Google Data API, Google Maps,

Google Mobile Ads, Google Mobile Updater, Google Mobile, Google Store, Google Sync,

Google Updater, Google Voice, Google Mail, Gmail, YouTube, Dalvik and Android are

trademarks or registered trademarks of Google Inc.

INTERMEC is a registered trademark of Intermec Technologies Corporation.

Wi-Fi is a registered trademark of Wi-Fi Alliance.

Bluetooth is a registered trademark of Bluetooth SIG Inc.

Motorola is a registered trademark of Motorola Trademark Holdings LLC.

Computop is a registered trademark of Computop Wirtschaftsinformatik GmbH.

SAP, R/3, SAP NetWeaver, Duet, PartnerEdge, ByDesign, SAP BusinessObjects Explorer,

StreamWork, SAP HANA, and other SAP products and services mentioned herein as well

as their respective logos are trademarks or registered trademarks of SAP AG in Germany

and other countries.

Business Objects and the Business Objects logo, BusinessObjects, Crystal Reports, Crystal

Decisions, Web Intelligence, Xcelsius, and other Business Objects products and services

mentioned herein as well as their respective logos are trademarks or registered trademarks

of Business Objects Software Ltd. Business Objects is an SAP company.

Sybase and Adaptive Server, iAnywhere, Sybase 365, SQL Anywhere, and other Sybase

products and services mentioned herein as well as their respective logos are trademarks or

registered trademarks of Sybase Inc. Sybase is an SAP company.

Crossgate, m@gic EDDY, B2B 360°, and B2B 360° Services are registered trademarks

of Crossgate AG in Germany and other countries. Crossgate is an SAP company.

All other product and service names mentioned are the trademarks of their respective

companies. Data contained in this document serves informational purposes only. National

product specifications may vary.

The information in this document is proprietary to SAP. No part of this document may be

reproduced, copied, or transmitted in any form or for any purpose without the express prior

written permission of SAP AG.

positioning of process orchestration and data services

Documents

architectural styles differ

data services notes

business process management

sap cloud solutions

sap data services

sap information interchange

data integration capabilities

architectural styles