soa summit 2014

Better Business Results with Open Source Integration

Kenneth PeeplesJBoss Technology Evangelist

2 RED HAT JBOSS MIDDLEWARE

Agenda for SOA Summit

● Overview of Open Source Integration with Data Virtualization

● Preview of JBoss Middleware Integration Offerings● Integration examples that will help you get started


http://www.devnation.org

http://www.devnation.org/


Red Hat is…

“By running tests and executing numerous examples for specific teams, we were able to prove […] not only would the solution work, but it will perform better & at a fraction of the costs.”

MICHAEL BLAKE, Director, Systems & Architecture

“By running tests and executing numerous examples for specific teams, we were able to prove […] not only would the solution work, but it will perform better & at a fraction of the costs.”

MICHAEL BLAKE, Director, Systems & Architecture

Overview of Open Source Integrationwith Data Virtualization

Better Business Results through Open Source Integration


SOAs and abstraction

Decoupling is the key concept in SOAs achieved through abstraction based on service interfaces

Business processes and services make use of business information

Business information is exposed to business services as data services


Bridging the Gap

The need to bridge data gaps in a more straightforward, streamlined and scalable way is due to:

● More and more data

● Growing need for consolidated views of information

● Drive toward interoperability and standards


Data Services through Data Virtualization

Key requirements of data services

● Expose integrated information in one or more desired formats, even if the original data are in different formats

● Expose information through one or more consistent, standard interfaces, even if the original data are accessed through different interfaces

The requirements are achieved by

● Modeling to define the required format of data, integrated from the underlying sources

● A query engine for processing these abstract definitions efficiently, exposing the integrated information through one or more interfaces


Data Virtualization essential to a SOA

Data Virtualization provide the necessary interface to data for all business services. Data Virtualization:

● Expose all data through a single uniform interface● Provide a single point of access to all business services in the

system

● Expose data using the same paradigm as business services - as "data services"

● Expose legacy data sources as data services

● Provide a uniform means of exposing/accessing meta-data● Provide a searchable interface to data and meta-data● Expose data relationships and semantics● Provide uniform access controls to information


Data Virtualization and SOA Goals

● Create an enterprise data foundation for streamlined development, better integration and more efficient data governance

● Improve delivery of consistent, complete information from any data source to the application and business processes

● Externalize business knowledge buried in applications and databases

● Expand automation, moving beyond reports to active analysis and decision-making

● Build business processes in code that are actually followed by the organization


What is Data Virtualization?

Data Virtualization provides a service that:

● Encapsulates data-oriented logic● Enable loose coupling between the application and the data stores● Manages real-time connectivity to data stores● Offers a higher level of data abstraction compared to interacting

directly with data stores

● Provides appropriate views of data

● Allows views of data to be cached based on performance requirements


Translators and Resource Adaptors

A translator

● provides an abstraction layer between the run time engine and physical datasource

● Knows how to issue query commands into source specific commands

● Execute them using the resource adapter

A Translator also defines the capabilities of a particular source, like whether it can natively support query joins (inner joins, cross joins etc) or support criteria.

A Translator along with its Resource Adapter must be configured on a Source Model


Translators and Resource Adaptors (Continued)

A Resource Adapter provides the connectivity to the physical data source

This also provides way to natively issue commands to the source and gather results

Translator capabilities define what processing each translator/source combination can perform.

For example, most relational sources can process joins and unions, whereas when processing delimited text files these operations cannot be performed by the resource adapter or the "source" (in this case, the file system).


Modeling Environment for Data Virtualization

Virtual Databases contain two primary varieties of model types - source and view.

Source models represent the structure and characteristics of physical data sources, whereas view models represent the structure and characteristics of abstract structures you want to expose to your applications.

Models used for data integration are packaged into a virtual database (VDB).

The models must be in a complete and consistent state when used for data integration.

That is, the VDB must contain all the models and all resources they depend upon.

Source models must be configured with a Translator and a Resource Adapter with them before a VDB is tested in Designer or deployed for data access


Runtime Environment for Data Virtualization

● Connect to multiple, heterogeneous data sources

● Optimize performance● Manage data security● Take advantage of run-time system management features● Audit data service usage

Preview of JBoss MiddlewareIntegration Offerings



Innovate faster, in a smarter way

A family of a lightweight, enterprise-grade products that are ideal for open hybrid cloud environments.


There is a better way – Red Hat JBoss Integration Product LineRed Hat JBoss A-MQ

● A small-footprint, high-performance messaging platform● Product Page – http://www.redhat.com/products/jbossenterprisemiddleware/amq/● Developers Download the Code - https://www.jboss.org/products/amq.html

Red Hat JBoss Fuse● A small-footprint, flexible enterprise service bus (ESB)● Product Page – http://www.redhat.com/products/jbossenterprisemiddleware/fuse/● Developers Download the Code - https://www.jboss.org/products/fuse.html

Red Hat JBoss Fuse Service Works● A service design, development and integration platform that enables

organizations to transition to an open hybrid cloud● Product Page – http://www.redhat.com/products/jbossenterprisemiddleware/fuse-service-works/● Developers Download the Code - https://www.jboss.org/products/fsw.html

Red Hat JBoss Data Virtualization (Only Open Source Data Virtualization product on the market)

● A lean, virtual data integration that unlocks trapped data and delivers it as easily consumable, unified, actionable information

● Product Page – http://www.redhat.com/products/jbossenterprisemiddleware/data-virtualization/● Developers Download the Code - https://www.jboss.org/products/datavirt.html


Integration Everywhere


JBoss Data Virtualization v6


What does Data Virtualization software do?Turn Fragmented Data into Actionable Information

Data Virtualization software virtually unifies data spread across various disparate sources; and makes it available to applications as a single consolidated data source.

The data virtualization software implements 3 steps process to bridge data sources and data consumers:

● Connect: Fast access to data from diverse data sources

● Compose: Easily create unified virtual data models and views by combining and transforming data from multiple sources.

● Consume: Expose consistent information to data consumers in the right form thru standard data access methods.

Virtual Consolidated Data SourceVirtual Consolidated Data Source

BI Reports

Data Virtualization Software• Consume• Compose• Connect

SAP Salesforce.comOracle DW Hadoop

Siloed & Complex

VirtualizeAbstractFederate

Easy,Real-time

InformationAccess

SOA Applications

DATA CONSUMERS

DATA SOURCES


Turn Fragmented Data into Actionable Information

ConnectConnect

ComposeCompose

ConsumeConsume

Unified Customer View

Unified Customer View

Unified Product View

Unified Product View

Unified Supplier View

Unified Supplier View

BI Reports & AnalyticsMobile Applications

SOA Applications & Portals

Unified Virtual Database / Common Data Model

ESB, ETL

Native Data ConnectivityNative Data Connectivity

Standard based Data ProvisioningJDBC, ODBC, SOAP, REST, OData

Standard based Data ProvisioningJDBC, ODBC, SOAP, REST, OData

JBOSSDV

DATA CONSUMERS

DATASOURCES

Design ToolsDesign Tools

DashboardDashboard

OptimizationOptimization

CachingCaching

SecuritySecurity

MetadataMetadata

Hadoop NoSQL Cloud AppsData Warehouse

& Databases MainframeXML, CSV

& Excel Files Enterprise Apps

Siloed & Complex

VirtualizeAbstractFederate

Easy,Real-time

InformationAccess


JBoss Data Virtualization:Supported Data SourcesEnterprise RDBMS:• Oracle • IBM DB2 • Microsoft SQL Server• Sybase ASE• MySQL• PostgreSQL• Ingres

Enterprise EDW:• Teradata • Netezza • Greenplum

Hadoop:• Apache• HortonWorks• Cloudera• More coming…

Office Productivity:• Microsoft Excel • Microsoft Access• Google Spreadsheets

Specialty Data Sources:• ModeShape

Repository• Mondrian• MetaMatrix• LDAP

NoSQL:• JBoss Data Grid• MongoDB • More coming…

Enterprise & Cloud Applications:• Salesforce.com• SAP

Technology Connectors:• Flat Files, XML Files,

XML over HTTP• SOAP Web Services• REST Web Services• OData Services


Key New Features and Capabilities for v6● Data connectivity enhancements

– Hadoop Integration (Hive – Big Data),

– NoSQL (MongoDB – Tech Preview) and JBoss Data Grid

– Odata support (SAP integration)

● Developer Productivity improvements – New VDB Designer 8 and integration with JBoss Developer Studio v7

– Enhanced column level security,

– VDB import/reuse, and native queries

● Simplify deployment and packaging – Requires JBoss EAP only; included with subscription

– Remove dependency with SOA Platform

● Business Dashboard– New rapid data reporting/visualization capability


Self-Service Business Intelligence

The virtual, reusable data model provides business-friendly representation of data, allowing the user to interact with their data without having to know the complexities of their database or where the data is stored and allowing multiple BI tools to acquire data from centralized data layer. Gain better insights from Big Data using JBoss Data Virtualization to integrate with existing information sources.

360◦ Unified View

Deliver a complete view of master & transactional data in real-time. The virtual data layer serves as a unified, enterprise-wide view of business information that improves users’ ability to understand and leverage enterprise data.

Agile SOA Data Services

A data virtualization layer deliver the missing data services layer to SOA applications. JBoss Data Virtualization increases agility and loose coupling with virtual data stores without the need to touch underlying sources and creation of data services that encapsulate the data access logic and allowing multiple business service to acquire data from centralized data layer.

Regulatory Compliance

Data Virtualization layer deliver the data firewall functionality. JBoss Data Virtualization improves data quality via centralized access control, robust security infrastructure and reduction in physical copies of data thus reducing risk. Furthermore, the metadata repository catalogs enterprise data locations and the relationships between the data in various data stores, enabling transparency and visibility.

● JBoss Data Virtualization – Use Cases


There is a better way – Red Hat JBoss Integration Product Line

JBoss A-MQ

JBoss Fuse(Includes JBoss A-

MQ)

JBoss Fuse Service Works

(Includes JBoss Fuse)

A small-footprint messaging platform, eliminating batch delivery

“JBoss A-MQ for real-time messaging”

A Integration/ESB platform for integrations that can extend outside the data center

“JBoss Fuse for integration/ESB requirements”

A Service design, development and integration platform that enables responsive and innovative enterprise

“JBoss Fuse Service Works for integration/ESB+SOA+Orchestration+Governance requirements”

Additional Capabilities

Additional Capabilities


JBoss A-MQ v6.1

New in 6.1:

● Full support of AMQP 1.0 – provides wire-level compatibility across connections● Improved high availability (HA) – embedded message store for shared-nothing

HA


JBoss Fuse v6.1

New in 6.1

• Full support of AMQP 1.0 – provides wire-level compatibility across connections• Vast library of connectors – over 200 out-of-the-box-connector via Apache Camel• Managed integration routes - start, stop, measure and trace Camel routes on-premise or in

the Cloud• Improved high availability (HA) – embedded message store for shared-nothing HA


JBoss Fuse Service Works v6

Enables organizations to Simplify integrations and focus on business innovation


SwitchYard Application Overview

Composite

ComponentComponent-Service

Service-Binding

Composite Service

Component-Reference

Component-Binding

Reference-Binding


Design Time and Run Time Governance

Integration examples to help you get started



Sample 1: Big Data with Data Virtualization

● Objective:– Determine if sentiment data from the

first week of the Iron Man 3 movie is a predictor of sales

● Problem:– Cannot utilize social data and

sentiment analysis with sales management system

● Solution:– Leverage JBoss Data Virtualization to

mashup Sentiment analysis data with ticket and merchandise sales data on MySQL into a single view of the data.

ConsumeComposeConnect

Consumer 1: Excel Powerview to analyze the aggregated data

JBoss Data Virtualization

Hive

SOURCE 1: Hive/Hadoop contains twitter data including sentiment

SOURCE 2: MySQL data that includes ticket and

merchandise sales

Consumer 2 : DV Dashboard to analyze the aggregated data

https://github.com/DataVirtualizationByExample/Mashup-with-Hive-and-MySQL

https://github.com/DataVirtualizationByExample/Mashup-with-Hive-and-MySQL


Sample 2 : Travel Triage

http://www.ossmentor.com/2013/10/sap-nw-gateway-with-fuse-and-dv.html

http://www.ossmentor.com/2013/10/sap-nw-gateway-with-fuse-and-dv.html


Sample 3: Home Loan v1

https://github.com/FuseServiceWorksByExample/Home-Loan-Reference-Architecture

https://github.com/FuseServiceWorksByExample/Home-Loan-Reference-Architecture


Sample 3: Home Loan v1 (Continued)

Closing


http://www.devnation.org

http://www.devnation.org/


Question and Answer Session


References

Data Virtualization Datasheet - http://www.redhat.com/resourcelibrary/datasheets/red-hat-jboss-data-virtualization-datasheet

Fuse Service Works Datasheethttp://www.redhat.com/resourcelibrary/datasheets/red-hat-jboss-fuse-service-works-datasheet

Red Hat JBoss Data Virtualization for financial services Whitepaperhttp://www.redhat.com/resourcelibrary/whitepapers/data-virtualization-fsi

Red Hat JBoss Fuse Service Works technology overview Whitepaperhttp://www.redhat.com/resourcelibrary/whitepapers/red-hat-jboss-fuse-service-works-technology-overview

Turn data into actionable information with Red Hat JBoss Middleware Whitepaperhttp://www.redhat.com/resourcelibrary/whitepapers/turn-data-into-actionable-information-with-red-hat-jboss-middleware

http://www.redhat.com/resourcelibrary/datasheets/red-hat-jboss-data-virtualization-datasheet

http://www.redhat.com/resourcelibrary/datasheets/red-hat-jboss-data-virtualization-datasheet

http://www.redhat.com/resourcelibrary/datasheets/red-hat-jboss-fuse-service-works-datasheet

http://www.redhat.com/resourcelibrary/datasheets/red-hat-jboss-fuse-service-works-datasheet

http://www.redhat.com/resourcelibrary/whitepapers/data-virtualization-fsi

http://www.redhat.com/resourcelibrary/whitepapers/red-hat-jboss-fuse-service-works-technology-overview

http://www.redhat.com/resourcelibrary/whitepapers/red-hat-jboss-fuse-service-works-technology-overview

http://www.redhat.com/resourcelibrary/whitepapers/turn-data-into-actionable-information-with-red-hat-jboss-middleware

http://www.redhat.com/resourcelibrary/whitepapers/turn-data-into-actionable-information-with-red-hat-jboss-middleware