Creating flexible data services for

enterprise S OA with

WS O2 Data S ervices

S umedha Rubas inghe(sumedha@ wso2.com)

~ WS O2 Data S ervices Team ~

WS O2 Background● Founded in August 2005 by leaders in XML and

Web services technologies & standards, and open source

● Building complete SOA platform, all 100% open source

● Founders / leading contributors to all key Apache Web services / SOA projects

● Selling support, training and services around the software & solutions

● Global corporation with R&D center in Sri Lanka and offices in US & UK, totaling 60+ employees

Problem● Data locked away in monolithic application silos● No unified away of accessing data● Need to access most recent data● Mergers/acquisitions/partnerships

Challenges● SOA demands access to data in heterogeneous

data sources● Business users does not differentiate between

application functionality or data access● Financial & budgeting constraints

Alternative approaches to data access in a S OA● Direct access to databases (JDBC,ODBC,..etc)● Use of O/R mapping frameworks (Hibernate,

Ibatis,...)● Enterprise Java Beans (EJBs)● CORBA● Problem

– Each approach demands different way of access

– Developer time spent on data access, rather than implementing business functionality

● Thus comes the need for Data services

How Can WS O2 Data S ervices Help?● Ability to access most up-to-date data using a

service call (brings you to the front of the wave)● Single layer for accessing all the data needs of your

SOA● Optimization efforts will be available to all the

service consumers● Developer productivity – save time on developing

similar data access mechanism for different applications

● QoS – battle tested runtime

WS O2 Data S ervices in a nutshellA mechanism to take relational and other data lockedaway in databases and make them available as WS-*style Web services or as a set of REST style webresources.

S imple example● Expose Employee contact data in MySQL database

as a service● Table Structure

● How to do this?

S imple example – GUI wizard s tep (1)

S imple example – GUI wizard s tep (2)

S imple example – GUI wizard s tep (3)

S imple example : configuration file<data name="ContactInfoS ervice"> <config> <property name="org.wso2.ws.dataservice.driver">com.mysql.jdbc.Driver</property> <property name="org.wso2.ws.dataservice.protocol">jdbc:mysql://localhost:3306/dataservice_sample</property> <property name="org.wso2.ws.dataservice.user">dsuser</property> <property name="org.wso2.ws.dataservice.password">user123</property> <property name="org.wso2.ws.dataservice.minpoolsize">1</property> <property name="org.wso2.ws.dataservice.maxpoolsize">5</property> </config>

<query id="contactInfoQuery"> <sql>select firs tName,lastName,extens ion,email from employees where lastName = ? </sql> <param name="name" sqlType="S TRING" /> <result element="Employees" rowName="ContactInfo"> <attribute name="LastName" column="las tName" /> <element name="FirstName" column="firs tName" /> <element name="Extens ion" column="extens ion" /> <element name="Email" column="email" /> </result> </query>

<operation name="getContactInformation"> <call-query href="contactInfoQuery" > <with-param name="name" query-param="name" /> </call-query> </operation></data>

WS DL for the S ervicehttp://<IP>:<PORT>/services/ContactInfoService?wsdl2

Invoking the S ervice – HTTP bindinghttp://<IP>:<PORT>/services/ContactInfoService/getContactInformation?name=Patterson

Invoking the S ervice – us ing TryItServices > ContactInfoService > TryIt

Data S ervices Description Language● In-house developed language for writing data

services● Maps service requests to your SOA to queries

operating on your database objects (tables, views, procedures & functions)

● Maps query results to XML responses● Available online @

http://wso2.org/wiki/display/wsf/Data+Services+and+Resources

Data S ervices Architecture● Data Services are SOA equivalent of the Data

Access Object(DAO) Pattern● DS deal with different data sources similar to what

DAO does● But, DS operate on a different layer

WS O2 Data S ervices (DS ),WS AS & Axis2● First appeared on WSAS 2.0● WSO2 DS runs on top of Axis2 SOAP Engine

– Equinox OSGI container– First of new breed of products

● Also available on WSO2's Web Services Application Server (WSAS)

● Growing feature list with every new release● Open source (Apache licensed)

Currently S upported Data S ources● Relational Databases

– Any database accessible via JDBC– Tables, views, stored procedures, functions

● URL accessible Comma Separated Value (CSV) files● Microsoft Excel (97 – 2003) spread sheets

– operation per work book● Coming up

– Support for LDAP Trees

Zero code approach● Simple XML configuration file● Easy configuration for

– Database Administrators– System Administrators

● Easily portable

CRUD support● Create/Read/Update/Delete most up to date data

Web console / Eclipse plugin wizard● Simple Web based UI wizard● Eclipse plugin wizard

Customizable Response● Service consumer – different input formats● Response format can be customizable● XSLT transformation capabilities (future feature)● Advance Transformations through ESB (eg. WSO2

ESB)

Data as Resources(RES T)● Unique URL for each data item

– http://myserver.com/services/products/{product-code}– http://myserver.com/services/products/AB342– http://mybooks.com/services/book/ISBN2823

● Support HTTP GET,POST,PUT,DELETE methods● Simple configuration

S ingle service - multiple data sources● 'Customer data' in MSSQL DB & 'Order data' in

'Derby DB'● The need – Orders along with Customers placed

them● Link heterogeneous data sources (MySQL & CSV

file)● Consolidated logical view – distribute data

Derby

M S SQL

Expose S tored Procedures & Functions● Business logic in Stored procedures/functions ● Expose them as web services within minutes● Power of pre-compiled procedures + Data Service

in SOA

Built-in caching● Infrequent updates to database data● Cache response & use it● Configurable options

– Timeout– Cache size– memory/file system

Access Throttling● Control access frequency● Two options,

– Rate based throttling– Frequency based throttling

High Availability● HA via clustering● Using Apache Tribes – Tomcat Clustering

Communication module● Graceful shutdown

S ecurity● Supports WS Security using Apache Rampart● 15 Security Scenarios

– Basic (4)● Confidentiality● UsernameToken● Integrity● Non-repudiation

– Advance (11)

Built-in connection pooling● Apache DBCP as the built-in pooling manager● Greatly increases the performance

● Ability to connect to Data sources bound to JNDI name

● Use a pooling manager of your choice

'Try It' for testing services● Test your data services without leaving console● Ability to dynamically change the configuration &

see the results

Management Console● Monitor system usage● Statistics gathered include:

– Service usage – Endpoint load– Overall system performance

● Enable tracing of a running system and capture service level message traces

● Configure system level logging and monitor logs directly from the console

● Monitoring via JMX

Up coming features● LDAP Directories as data sources● SQL support for CSV & Excel● Response Compression● Reading database credentials from runtime● Batch insert/update● APP support● support for SQL in (...where country_id in

('SL','US','UK');)● Content filtering based on logged in User● Improved Eclipse Plugin● More user friendly Web UI

Getting s tarted ...● Many online resources on WSO2 Oxygen Tank

(wso2.org)● How to use MySQL 5 INOUT variables in Data Services? (http://wso2.org/blog/sumedha/3082)● How to call MySQL 5 Stored Function from Data Service? (http://wso2.org/blog/sumedha/3029)● How to call MySQL 5 stored procedure from Data Services?(http://wso2.org/blog/sumedha/2623)● How to call MySQL 5 view from Data Services? (coming up)● Data Service in 2 minutes - Movie (coming up)● Getting started with Data Services using Oracle (http://wso2.org/blog/sumedha/3051)● Getting started with Data Services using WSO2 WSAS-2.0 (http://wso2.org/blog/sumedha/2573)

- using Derby as database● Getting started with Data Service Samples - Movie (http://wso2.org/blog/sumedha/2590) - using

Derby as database● How to page (http://wso2.org/library/3183)

● Forum (http://wso2.org/forum/404)● Mailing list (ds-java-user@wso2.org)

Profess ional S ervices & support● WSO2 is the leading provider of Open Source

support for Apache WS projects● With over 60 employees in three geographies we

can offer first class support● Range of support options up to 24x7 production● Support Backed by key Apache and OSS

committers● More information - http://wso2.com/support/

S ummary

S ummary● Service enable data locked away in relational

databases● Support for tables,views, functions & stored

procedures● Data Service Description Language● Not limited to RDBMS (CSV,EXCEL,...)● Improved QoS● Professional support

Trainings● Introduction to Data services

– http://wso2.com/training/courses/owas001/● Advance Data Services

– http://wso2.com/training/courses/owas002/● Visit http://wso2.com/training/ for more information

Thank you