download file
DESCRIPTION
TRANSCRIPT
![Page 1: Download File](https://reader037.vdocuments.net/reader037/viewer/2022102518/548e4b27b47959891d8b466e/html5/thumbnails/1.jpg)
INT420 - Federated Database Solutions with Sybase Middleware
Lynn Weatherwax
Director of Engineering – Data IntegrationInfrastructure Platform Group, [email protected]
![Page 2: Download File](https://reader037.vdocuments.net/reader037/viewer/2022102518/548e4b27b47959891d8b466e/html5/thumbnails/2.jpg)
2
Abstract
Many companies today seek optimized ways to access structured and unstructured data in their enterprises, without extensive programming or costly database consolidation.
Virtual databases with links to distributed heterogeneous sources enable applications to consistently access and combine data in new ways.
Sybase solves this problem while supporting the optimized query capabilities and scalability of an enterprise class relational database, via ASE, Enterprise Connect Data Access, Enhanced Full Text Search, and ASE Web Services.
Leverage SQL and XML standards to make your data work for you in your enterprise.
![Page 3: Download File](https://reader037.vdocuments.net/reader037/viewer/2022102518/548e4b27b47959891d8b466e/html5/thumbnails/3.jpg)
3
Agenda
Enterprise Information Integration (EII)
Strategies for Integrating Federated Data
Federated Database Access
Document Access - Full Text Search
XML
Web Services
Summary and Resources
![Page 4: Download File](https://reader037.vdocuments.net/reader037/viewer/2022102518/548e4b27b47959891d8b466e/html5/thumbnails/4.jpg)
4
Enterprise Information Integration (EII)
“Enterprise information integration (EII) is seeing a new surge of interest in 2003, largely because its virtual and/or federated approach to data integration has potential benefits.
Compared to other data integration technologies, solutions based on a vendor’s EII tool tend to be quick to deploy, inexpensive in terms of licensing and very flexible since they’re based on a metadata-based view of enterprise data.
In addition, EII relies on distributed queries to effect data integration in near real-time, thus delivering fresher data than traditional batch methods.
Many companies are evaluating vendors’ EII products in 2003, so 2004 should see an upsurge in EII implementations.”
Philip Russom, IdeaByte
![Page 5: Download File](https://reader037.vdocuments.net/reader037/viewer/2022102518/548e4b27b47959891d8b466e/html5/thumbnails/5.jpg)
5
Factors Driving EII Today
Changes in company structure Increased competition, requiring better intelligence Government regulations
Patriot Act Sarbanes-Oxley Act HIPPA
![Page 6: Download File](https://reader037.vdocuments.net/reader037/viewer/2022102518/548e4b27b47959891d8b466e/html5/thumbnails/6.jpg)
6
Every enterprise has federated data, due to
• Mergers and acquisitions
• Legacy mainframe applications
• Applications required to work with specific databases
EII Challenge
XML
DB2 UDB
SybaseMicrosoft
InformixVSAM
Oracle
Adabas
Documents
![Page 7: Download File](https://reader037.vdocuments.net/reader037/viewer/2022102518/548e4b27b47959891d8b466e/html5/thumbnails/7.jpg)
7
Option: Migrate All Data/Apps to One Database…
Sometimes this is feasible, but often there are time and $$ restrictions…
And what about the next acquisition, or the next undiscovered information island?
XML
DB2 UDB
My CentralizedDatabase
Microsoft
Informix
VSAM
Oracle
Adabas
Documents
![Page 8: Download File](https://reader037.vdocuments.net/reader037/viewer/2022102518/548e4b27b47959891d8b466e/html5/thumbnails/8.jpg)
8
Enterprise Information Integration Requirements
Leverage existing hardware and software investment Ensure secure access Integrate legacy data and applications while they
remain in place – short-term or long-term Metadata transparently aggregates data, no
programming required Components that work together, forming a complete,
end-to-end solution Supported in standard client/server environments, yet
fully e-business ready (supports XML, Java, etc.)
![Page 9: Download File](https://reader037.vdocuments.net/reader037/viewer/2022102518/548e4b27b47959891d8b466e/html5/thumbnails/9.jpg)
9
Agenda
Enterprise Information Integration (EII)
Strategies for Integrating Federated Data
Federated Database Access
Document Access - Full Text Search
XML
Web Services
Summary and Resources
![Page 10: Download File](https://reader037.vdocuments.net/reader037/viewer/2022102518/548e4b27b47959891d8b466e/html5/thumbnails/10.jpg)
10
Strategies for Integrating Federated Data
Data Access Document Access - Full Text Search Messaging/Data Replication XML Web Services
![Page 11: Download File](https://reader037.vdocuments.net/reader037/viewer/2022102518/548e4b27b47959891d8b466e/html5/thumbnails/11.jpg)
11
Enterprise Connect Data AccessEnterprise Connect Data Access
Make all your enterprise data look like one database
Any Any mainframe mainframe datadata
OracleOracleMS SQL ServerMS SQL ServerSybaseSybaseDB2 UDB for OS/390, NT/UNIXDB2 UDB for OS/390, NT/UNIXAS/400AS/400InformixInformixODBC-enabled databasesODBC-enabled databasesDocuments/FilesDocuments/Files
CustomCustomSpecialty Data StoresSpecialty Data Stores
ASE/CISMainframe Connect
Distributed
Proxy Tables
Specialty Data Store
Toolkit
Any ClientOr BrowserAny Client
Or Browser
![Page 12: Download File](https://reader037.vdocuments.net/reader037/viewer/2022102518/548e4b27b47959891d8b466e/html5/thumbnails/12.jpg)
12
Mainframe Connect - Extending the Mainframe
EnterpriseConnect VSAM
IMSDatacom IDMSAdabas
Mainframe
SybaseASE
ReplicationServer
EnterprisePortals,
App Servers
Direct Connect
for OS/390
DB2
Mainframe Applications and Transactions
Any Client
Any Client
Mainframe as a Client(Client Options)
Access to any MFData/Applications (Server Options and XJS/390)
Dynamic Accessto DB2(DB2 UDB for OS/390 Option)
“Gatewayless”configuration
![Page 13: Download File](https://reader037.vdocuments.net/reader037/viewer/2022102518/548e4b27b47959891d8b466e/html5/thumbnails/13.jpg)
13
XJS/390 Enterprise Integrator
ASE/CIS
VSAM IMSDatacom IDMS Sequential Adabas
MainframeSoap, XML, HTML, and Scripts
ASE
ReplicationServer
EnterprisePortals, App
Servers
Direct Connect
forOS/390
DB2
Mainframe Applications, and Transactions
Any ClientOr BrowserAny Client
Or Browser
MQSeries
MQSeriesAccess to MF Data and Applications (Server Options and XJS/390)
CICS Programs, Data HTTP, MQ, Web Services
IMS Programs, Data, HTTP, MQ
Navigate CICS Transactions
Integrate with MainframeCONNECT
No data is too complex to web- or message- enableNo data is too complex to web- or message- enable
“Gatewayless”configuration
![Page 14: Download File](https://reader037.vdocuments.net/reader037/viewer/2022102518/548e4b27b47959891d8b466e/html5/thumbnails/14.jpg)
14
Scenario – Customer Service
Problem: Phone Service Two-Step
• Taking 20+ minutes/call to help customer buy new options
• Customer data from IMS mainframe “green screens”
• Regional service data from distributed Informix databases
Goal: Faster Service
• Present all data at once in easy-to-use GUI application so customer service operators can provide faster service
• Must leverage existing application logic and data
InformixRegional service data
Mainframe IMS customer data
![Page 15: Download File](https://reader037.vdocuments.net/reader037/viewer/2022102518/548e4b27b47959891d8b466e/html5/thumbnails/15.jpg)
15
Solution – Customer Service
• OS/390 IMS customer data
• Informix local telephone service data
Enterprise ConnectMainframe Connect
GUI Customer Service application.
Solution:
<2 minutes/call with GUI customer service application transparently accessing mainframe and distributed data
Proxy Tables
![Page 16: Download File](https://reader037.vdocuments.net/reader037/viewer/2022102518/548e4b27b47959891d8b466e/html5/thumbnails/16.jpg)
16
Data Access – How It Works
Set up connectivity to remote servers Define remote servers and security in ASE-CIS Automatically set up proxy tables mapped to remote
heterogeneous tables (federated data) Set up security as desired Metadata retained within ASE-CIS Now queries work with the proxy tables as if they were local Create stored procedures, views for proxy tables as needed Federate data is accessible transparently via SQL
Enterprise ConnectMainframe Connect
![Page 17: Download File](https://reader037.vdocuments.net/reader037/viewer/2022102518/548e4b27b47959891d8b466e/html5/thumbnails/17.jpg)
17
Agenda
Enterprise Information Integration (EII)
Technologies for Integrating Federated Data
Federated Database Access
Document Access - Full Text Search
XML
Web Services
Resources
![Page 18: Download File](https://reader037.vdocuments.net/reader037/viewer/2022102518/548e4b27b47959891d8b466e/html5/thumbnails/18.jpg)
18
Document Access – Full Text Search
Store and index documents for quick search PDF, Microsoft Word, many other formats supported Ability to join database data with results of full text search Automated indexing, backup, restore
![Page 19: Download File](https://reader037.vdocuments.net/reader037/viewer/2022102518/548e4b27b47959891d8b466e/html5/thumbnails/19.jpg)
19
ASE – Enhanced Full Text Search Option (EFTS)ASE – Enhanced Full Text Search Option (EFTS)
Full Text Search of Wide Variety of DocumentsCombined Search Results with Data from Other Tables
ASE/CIS Full Text Search Engine
• Documents stored as binary in user-created source tables with identity column for unique id
• CIS index tables (proxy tables) mapped to text search engine collections
Any ClientOr BrowserAny Client
Or Browser
Collections (Directories with text indexes and unique identifiers)
Supports over 200 formats including Microsoft Word, XML, SGML, PDF
![Page 20: Download File](https://reader037.vdocuments.net/reader037/viewer/2022102518/548e4b27b47959891d8b466e/html5/thumbnails/20.jpg)
20
Scenario – Resume Search
Problem: Find Resumes with Certain Skills, Match with other Data on Potential Employees
> Scan individual resume documents in various formats for candidate with appropriate skills
> Once candidate is located, query ASE database containing other information on the candidate
> Assemble information by hand in report for interviewers
DB2 database with other candidate info
Resume Documents
![Page 21: Download File](https://reader037.vdocuments.net/reader037/viewer/2022102518/548e4b27b47959891d8b466e/html5/thumbnails/21.jpg)
21
Solution – Resume Search
ASE-CIS
Resume Search Application
Solution: • Single point of access
for both documents and other data
• One query locates matching documents based on relevance, joins this data w/ related federated data on candidates, presents complete info to end user
EFTS Option for ASE
ASE database with documents and other data on candidates
Text indexes
Collection
![Page 22: Download File](https://reader037.vdocuments.net/reader037/viewer/2022102518/548e4b27b47959891d8b466e/html5/thumbnails/22.jpg)
22
Enhanced File Text Search – How It Works
Setup Store documents in ASE as binary data Set up EFTS remote server defined as server class “sds” Run stored procedures that extract and index the text of each
document into “collections” Proxy tables w/in ASE-CIS mapped to indexes called
“collections” Each document has unique id in database As documents are changed/updated, triggers track changes
so indexes can be kept in sync
![Page 23: Download File](https://reader037.vdocuments.net/reader037/viewer/2022102518/548e4b27b47959891d8b466e/html5/thumbnails/23.jpg)
23
Enhanced File Text Search – How It Works
Run-time Query for particular text string is sent through proxy
mapping to EFTS for query against collection Document ids returned, based on relevance (degree of match
with query) If desired, join with other data Return data to client application Client application may then display lists of docs or actual doc
content as desired
![Page 24: Download File](https://reader037.vdocuments.net/reader037/viewer/2022102518/548e4b27b47959891d8b466e/html5/thumbnails/24.jpg)
24
EFTS – Advanced Capabilities
Topics Groupings of information that represents a concept or subject
area. Precompiled query made available for run-time searches by
any user Clustering
Group similar documents Query by Example
Search for documents that have similar phrases or are similar to docs already indexes
Custom Thesaurus Build a custom thesaurus with synonyms you define
![Page 25: Download File](https://reader037.vdocuments.net/reader037/viewer/2022102518/548e4b27b47959891d8b466e/html5/thumbnails/25.jpg)
25
Enhanced Full Text Search – More Features
Sample Text Load program Loads data into text or image columns into a source table Delivered as source code
Stored procedures to help automate EFTS configuration Alteration of cluster capabilities Backup and recover text indexes Display information about a text index Optimize indexes
![Page 26: Download File](https://reader037.vdocuments.net/reader037/viewer/2022102518/548e4b27b47959891d8b466e/html5/thumbnails/26.jpg)
26
ASE 12.5 Content Management Option(External File System Support – EFS)
File System Directory Access SQL access to file system directories and their underlying files is
enabled via proxy tables Each directory entry is visible as a row in a proxy table Regular file content is available as an image column in the proxy
table
Combine with Full Text Search to index and search on documents external to ASE via ASE queries
![Page 27: Download File](https://reader037.vdocuments.net/reader037/viewer/2022102518/548e4b27b47959891d8b466e/html5/thumbnails/27.jpg)
27
File System Directory Access – How it Works
Create a proxy table for the directory The directory pathname must reference directory visible to/searchable by ASE Proxy table is created which maps column names to file attributes and content Option to include subdirectories
Attributes of directory contents converted to SQL data types Content of regular files returned via content column Always treated as an image datatype
SQL Commands supported Insert causes new file to be created in directory Update allows filename change Delete causes file to be deleted Select returns columns as required Readtext will retrieve contents of the file Writetext will modify contents of the file
File content may be queried with the select statement
![Page 28: Download File](https://reader037.vdocuments.net/reader037/viewer/2022102518/548e4b27b47959891d8b466e/html5/thumbnails/28.jpg)
28
File Access
SQL access to individual files within a file system enabled via proxy tables
Contents of file must be readable characters, 255 char max records delimited with newline
Useful for exporting ASE data to other applications that cannot otherwise be accessed via CIS
SQL alternative to bcp out or bcp in Create proxy_table command creates a proxy table with one
column called “record”, 255 varchar To specify column names and datatypes use the create
[existing] table command
![Page 29: Download File](https://reader037.vdocuments.net/reader037/viewer/2022102518/548e4b27b47959891d8b466e/html5/thumbnails/29.jpg)
29
File Access – How It Works
Example, to read individual records within the ASE error log file:
create proxy_table errorlog external file at "/usr/sybase/ase12_5/install/errorlog"
select record from errorlog where record like "%server%"
This query will return all rows from the file that match the like pattern. Rows longer than 255 bytes will be truncated. To specify longer rows:
create existing table errorlog(
record varchar(512) null )external file at "/usr/sybase/ase12_5/install/errorlog"
In this case records up to 512 bytes in length will be returned
![Page 30: Download File](https://reader037.vdocuments.net/reader037/viewer/2022102518/548e4b27b47959891d8b466e/html5/thumbnails/30.jpg)
30
Agenda
Enterprise Information Integration (EII)
Technologies for Integrating Federated Data
Federated Database Access
Document Access - Full Text Search
XML
Web Services
Resources
![Page 31: Download File](https://reader037.vdocuments.net/reader037/viewer/2022102518/548e4b27b47959891d8b466e/html5/thumbnails/31.jpg)
31
Support for Storing, Searching and Querying XML documents
Key functionalities:
XQL to query XML documents Store parsed XML documents in ASE Query parsed XML documents Enhanced Full-Text Search is XML-aware
Benefits Simple: Java classes to extract contents General: extraction can be generalized Optimized: store once, query many times Tight integration with SQL language
ASE 12.5 as an XML Server
![Page 32: Download File](https://reader037.vdocuments.net/reader037/viewer/2022102518/548e4b27b47959891d8b466e/html5/thumbnails/32.jpg)
32
New XML Features with ASE 12.5.1
Native XML Engine Lets you more efficiently store, query, manage and exchange
XML data in its native format Based on W3C, XML and other open Internet standards Integrates SQL and XML Data Management functionalities
natively in the same server reduces cost of ownership reduces number of moving parts in a complex IT infrastructure provides a solid data management platform to exploit the
power of XML while leveraging existing IT investments.
![Page 33: Download File](https://reader037.vdocuments.net/reader037/viewer/2022102518/548e4b27b47959891d8b466e/html5/thumbnails/33.jpg)
33
ASE 12.5.1 Native XML Engine
Native XML Processor Running in ASE Implements XPath – a significant subset of XQuery language
standard for querying XML data Upwardly compatible w/ XQL provided w/ ASE 12.5
Supports for xml extension of select statements Map SQL statements to standard SQLX-XML Java-based forsqlj functions allow mapping of SQLX-XML back to
SQL script that recreates the data
![Page 34: Download File](https://reader037.vdocuments.net/reader037/viewer/2022102518/548e4b27b47959891d8b466e/html5/thumbnails/34.jpg)
34
More on 12.5.1 XML Features
Native XML Support Store and Retrieve well formed XML data Schema independent
SQL and XML Duality Complete interoperability between SQL and XML at language and storage
level
XML Indexing Renders high query performance Self defined indexes – No user input necessary
XPATH & XQUERY Support Granular results – Returns documents, document fragments, elements Wild card support for complex queries
![Page 35: Download File](https://reader037.vdocuments.net/reader037/viewer/2022102518/548e4b27b47959891d8b466e/html5/thumbnails/35.jpg)
35
How Does this Relate to Federated Data?
Combine ASE and XML Management option with ECDA to Query XML documents located in heterogeneous databases Use SQL to extract and join data from heterogeneous databases, and convert
it automatically to XML format
ASE/CIS + ECDA Distributed Heterogeneous
Data
Proxy Tables
Any ClientOr BrowserAny Client
Or BrowserSQL
XML
XML
SQL
![Page 36: Download File](https://reader037.vdocuments.net/reader037/viewer/2022102518/548e4b27b47959891d8b466e/html5/thumbnails/36.jpg)
36
Agenda
Enterprise Information Integration (EII)
Strategies for Integrating Federated Data
Federated Database Access
Document Access - Full Text Search
XML
Web Services
Summary and Resources
![Page 37: Download File](https://reader037.vdocuments.net/reader037/viewer/2022102518/548e4b27b47959891d8b466e/html5/thumbnails/37.jpg)
37
What are Web Services?
Self-contained, modular applications that can be described, published, located, and invoked over a network, generally, the Web.
Generally associated with SOAP, WSDL and UDDI protocol specifications.
Enable loose coupling of independent applications
![Page 38: Download File](https://reader037.vdocuments.net/reader037/viewer/2022102518/548e4b27b47959891d8b466e/html5/thumbnails/38.jpg)
38
What can web services do?
Allow applications to share data. Discrete units of code; each handles a limited set of tasks. Based on XML, the universal language of Internet data exchange
Can be called across platforms and operating systems, regardless of programming language.
![Page 39: Download File](https://reader037.vdocuments.net/reader037/viewer/2022102518/548e4b27b47959891d8b466e/html5/thumbnails/39.jpg)
39
What About Data Available from Web Services?
ASE Web Services Option for ASE 12.5.1 Expose ASE functionality (Stored procedures within
ASE) as Web Services – ASE as Web Services Producer
Extend ASE to access external Web Services and use the results in the SQL constructs seamlessly – ASE as a Web Services Consumer
Application data (e.g. Peoplesoft) exposed as Web Services can participate in the Federated Data world
![Page 40: Download File](https://reader037.vdocuments.net/reader037/viewer/2022102518/548e4b27b47959891d8b466e/html5/thumbnails/40.jpg)
40
ASE Web Services Producer
ASE
ASE Web Services
Producer
Any SOAP Client
Input: Soap
Output: Soap
Input: JDBC
Output: JDBC
![Page 41: Download File](https://reader037.vdocuments.net/reader037/viewer/2022102518/548e4b27b47959891d8b466e/html5/thumbnails/41.jpg)
41
ASE Web Services Consumer
ASE ASE Web ServicesConsumer
SOAP Server
2. RPCEvent
3. SOAPrequest
4. SOAPresponse5. Results
1. select
6. Results
![Page 42: Download File](https://reader037.vdocuments.net/reader037/viewer/2022102518/548e4b27b47959891d8b466e/html5/thumbnails/42.jpg)
42
Agenda
Enterprise Information Integration (EII)
Technologies for Integrating Federated Data
Federated Database Access
Document Access - Full Text Search
XML
Web Services
Summary and Resources
![Page 43: Download File](https://reader037.vdocuments.net/reader037/viewer/2022102518/548e4b27b47959891d8b466e/html5/thumbnails/43.jpg)
43
Sybase EII MiddlewareComplete Access to Federated Data
Sybase EII Middleware Enterprise Connect Data Access and Mainframe Connect ASE Options for EFTS, Content Management, XML
Management, Web Services
Benefits Leverage existing hardware/software investment Leverage existing IT skills Leverage Sybase support for and experience with
heterogeneous environments
![Page 44: Download File](https://reader037.vdocuments.net/reader037/viewer/2022102518/548e4b27b47959891d8b466e/html5/thumbnails/44.jpg)
44
The Big Picture
How Today’s Sybase Solutions Enable Information Liquidity
![Page 45: Download File](https://reader037.vdocuments.net/reader037/viewer/2022102518/548e4b27b47959891d8b466e/html5/thumbnails/45.jpg)
45
For More Information
Product Documentation - see Enterprise Connect Data Access, Mainframe Connect, ASE Component Integration Services, XML Management Option, Web Connect
http://www.sybase.com/support/manuals/
Product Information and White Papers
http://www.sybase.com/products/middleware
Newsgroups
sybase.public.gateways
sybase.public.omni
![Page 46: Download File](https://reader037.vdocuments.net/reader037/viewer/2022102518/548e4b27b47959891d8b466e/html5/thumbnails/46.jpg)
46
Sybase Developer Network (SDN)
Developer software, services, up-to-date technical information:
White papers and documentation Collaboration with other developers and Sybase engineers Code samples and beta programs Technical recordings Free software Join today: www.sybase.com/developer or visit SDN at
TechWave’s Technology Boardwalk
![Page 47: Download File](https://reader037.vdocuments.net/reader037/viewer/2022102518/548e4b27b47959891d8b466e/html5/thumbnails/47.jpg)
47
SDN Presents CodeXchange
Forum for exchanging samples, tools, scripts, etc. New features enable community collaboration
Download samples created by Sybase or external users Leverage contributions of others to extend Sybase tools Contribute code or start your own collaborative / open source project with input
from other product experts
Any SDN member can participate Log in using your MySybase account via SDN
Join the collaboration already underway http://www.codexchange.sybase.com or via SDN at
www.sybase.com/developer
A New Benefit for Sybase Developers
![Page 48: Download File](https://reader037.vdocuments.net/reader037/viewer/2022102518/548e4b27b47959891d8b466e/html5/thumbnails/48.jpg)
48
Check Out These Related TechWave Sessions!
ASE111 - Integration, XML, and Web Services--Why are They Important to a DBA?
ASE114 - Querying and Transforming XML with XQuery, XPath, and SQLX
INT404 - Integrating Adaptive Server Enterprise with Web Services
Boardwalk demos
![Page 49: Download File](https://reader037.vdocuments.net/reader037/viewer/2022102518/548e4b27b47959891d8b466e/html5/thumbnails/49.jpg)
INT420Federated Database Solutions with Sybase Middleware
Lynn Weatherwax
Director of Engineering – Data IntegrationInfrastructure Platform Group, [email protected]
Q & A