A Winning Pair

OpenInsight andU2 Databases

What is OI for U2?Rev Guy rockin’with the band

What is OI for U2?Bob Carten doing

Web 2.0

What is OI for U2?• A special version of OpenInsight with a built-in U2

Connector toolset that communicates directly with U2 database servers.

• The U2 Connector works through a Base Filing System (BFS), sometimes called a Bond.

• While OI for U2 is marketed to the general MultiValue community, the U2 Connector ships with all versions of OpenInsight from 7.0 and up (at least to 8.1).

• In the future, it might be separated into two distinct technical products.

What is U2?

U2 is a family of databases, Universe and Unidata, which use extended relational, or multivalued fields.

They are designed for high performance and extreme scalability.

Each one is ideal as an embedded vertical solution for small to medium businesses and they offer rapid, cost-effective application development.

Paraphrase from http://www-306.ibm.com/software/data/u2/

What is OpenInsight?

Goals for Today• To encourage those in the Revelation community to

seriously consider U2 databases as an alternative to linear hash as a relatively easy way to increase performance and extend applications into new markets.

• To convince you that working with U2 using OpenInsight as a front-end is not rocket science. I truly believe that working with U2 is a natural fit for anybody in the Revelation community.

YOU CAN DO IT!

RESOURCES

International Spectrum Magazine• A great resource for all

things “MultiValue”

• Since Mar/Apr 2003 issue,Revelation has had aregular article

• http://www.intl-spectrum.com

• Jan/Feb 2004 issue hasthe first OI for U2 article byMike Ruane

• Mike basically tells theMultiValue community thattheir “green-screens” areancient, dated, andyesterday’s news

International Spectrum Magazine

• Jan/Feb 2008 issue hasBob Carten’s article,“Distributed ComputingUsing OpenInsight andIBM U2”

• A nice article giving aprimer on MVC (model-viewcontroller) using OpenInsight

International Spectrum Magazine

• Mar/Apr 2008 issue hasMike Ruane’s article,“Swapping your OISystem’s Database to U2”

• Mike gives an overview ofwhy U2 databases mightbe considered and how tomake the migration

International Spectrum Magazine

Revelation Tutorials• http://www.revelation.com and click on the

Tutorials link

• U2 Connector:

• Quick Start Guide

– A nice walk throughtutorial for setting up a U2connection and creatingapplication components

– Enter OpenInsight for U2 in the Revelation website Search then click on theProduct Documentation and Articles link

Documentation

• Quick Start Guide

– A nice walk throughtutorial for setting up a U2connection and creatingapplication components

– http://www.revsoft.co.uk/devwhitepapers.htm

Documentation

Marketinghttp://www-306.ibm.com/software/

data/u2/and click on the U2 Business Value

Video link

INITIAL COMMENTS

Give Credit Where Credit is DueA lot of gratitude goes to the following people…

• Bob Fernandes• Bob Carten• Bob Catalano• Bryan Shumsky

• David Sigafoos• Stacy Mecklenberg• Pat McCann

• Mary Jean Blink

• Bob Orsini

Disclaimers• I am not getting paid by Revelation or IBM for this

presentation.

• I am not creating waves by suggesting that people consider U2 as an alternative to the native OpenInsight linear hash database system.

• I am not a U2 developer.

– Therefore…I reserve the right to defer all U2 specific questions to others in the room.

..but I play one on TV.

Who Should be Interested in OI for U2?• Existing U2 developers who are interested in an alternative

environment to extend their applications into the GUI desktop or other front-ends that OpenInsight can provide.

• OI developers who need to an easy to way increase performance and scale up their applications.

• OI developers who need brand-name recognition to increase their marketability and credibility.

GETTING STARTED

Tools of the Trade• Current version of OpenInsight

– Ideally 8.0.1, 8.0.2, or 8.0.6– Eventually 8.1

• Access to a U2 database

• At a minimum…a copy of the InterCall library files in stored in your WinDir\SYSTEM32 folder:

File Name Sizeunirpc32.dll 64 KB

UVCLNT32.dll 52 KB

UVIC32.dll 36 KB

Installing the U2 Database• Go to the U2 home page

– http://www-306.ibm.com/software/data/u2/– Make sure you sign in or register

• Click on the Trials and demos link

• Select U2 Personal Editions (Personal Edition Extended Relational Data Server)

File Description File Name SizeIBM UniVerse Personal Edition

UVPE_WINDOWS_10.2.8.zip

22 MB

IBM UniVerse Clients UV_10.2Acl.zip 400 MB

Installing the U2 Database• Run the INSTALL.EXE program from UVPE_WINDOWS_10.2.8.zip file

– Select UniVerse RDBMS from the initial screen– After going through the screens Universe will be installed on your

system– Note: Universe requires three services to be running– Use the Windows Services utility or the Universe Control utility to

start or stop them accordingly

• Run the INSTALL.EXE program from UV_10.2Acl.zip file – There are several options– Recommended: UniAdmin and UniDK– Optional: Dynamic Connect

Installing the InterCall Library Files• If you already downloaded the IBM UniVerse Clients file

then this contains the UniDK package which contains the library files.

• The UniDK package can also be downloaded seperately from Revelation Software in the same page where the Quick Start Guide is located.– Smaller footprint (36MB) thus quicker download– Missing UniAdmin. Need IBM UniVerse Clients for this

• Ask somebody who already has the three InterCall library files to send them to you (152 KB total)

WORKING WITH U2

Making a U2 Connection• Use the Database Manager

– Utilities -> Other MV Tables -> U2 Connection Setup

• Provide the connection parameters

• Click on the Attach volume button and see what happens

• Confirm that tables are attached by clicking on the Table Properties tab.

Interacting with the U2 Data• The U2 BFS makes the U2 data available seamlessly

• U2 data is multivalued just like OpenInsight data

• U2 indexes, however, are different from OpenInsight. Therefore, logic that specifically works with OpenInsight indexes might have mixed results

• We can even use an MFS, but this has to be hacked into the .DBT file

• Copy_Table_FS is supposed to allow you to copy OpenInsight data tables, but I haven’t been able to get it to work in my version of the U2 bond.

Basic +

Basic Interpreter (OpenEngine)

MFS

BFS (RTP57, U2_BFS)

Physical Data

Programming with U2• Current OpenInsight programs will work as they always

have

• U2 programs are data records stored in U2 database tables (e.g. BP)

• These can be opened by any editor that works within OpenInsight

• Use the System Editor+ Keyword Configuration window to list "source" tables as "Tables Requiring Coloring". In the SRP Editor, this can be done in the Options window

Programming with U2• Compiling U2 programs is automatic with the SRP Editor

• Compiling can also be done through a console that is connected to the U2 database server

• Or…you can roll out your own compiler using the U2_ExecProc Basic+ command:

FileName = "BP" Program  = "TEST_PROG" TableVolume = Xlate("SYSTABLES", FileName, 1, "X") Connection  = Xlate("SYSVOLUMES", TableVolume, 2, "X") rv = U2_ExecProc(Connection, "BASIC ":FileName:" ":Program, Results) rv = U2_ExecProc(Connection, "CATALOG ":FileName:" ":Program, Results)

Programming with U2• There are many Basic+ commands that are designed to

work with the U2 database system but there are two that are the most important

• U2_ExecProc is analogous type entering a command in the TCL prompt. This will execute a VOC command. However, UV procedures with parameters cannot be executed this way

• U2_SubCall is analogous to the Call @Sub statement. This will execute a U2 stored procedure and is the only way to execute code with parameters

• Both are documented in the Programmer’s Reference Guide

BENCHMARK RESULTS

Benchmark Process• My goal was to compare the difference between U2 and

OpenInsight file I/O operations

• All tests were done on my machine in order to provide an equal playing field

• Tests were done in three different environments– Pure U2 environment…running U2 basic logic against U2 tables– Pure OI environment…running OI basic logic against OI tables

• Indexes used for comparison when appropriate– Hybrid environment…running OI basic logic against U2 tables

Write Test Methodology• 1 million rows of data written to an empty table

• Sequential Key ID (1 through 1000000)

• A randomizer was used to create a “contacts” database based on hardcoded lists– Ten first names– Ten last names– Ten cities– Ten states

• Average record size is about 60 bytes

• Tables in each environment were created with optimized capacity– U2 (CREATE.FILE CONTACTS 30 1 4 SEQ.KEY RECORD.SIZE 60)– OI (Database Manager GUI, Presized to 1000000 records, Average Size = 60,

Frame Size = 2048, Sizelock = 2)

Write Test Results (Average of 10 tests)

Read Test Methodology• Used table from the Write Test (sequential Key IDs from 1

through 1000000)

• 1000000 read requests

• Key ID randomized with each read request

• Used Read statement:    For RecCnt = 1 to NumRecords         ID = Rnd(1000000) + 1         Read Rec from hFile, ID else debug     Next RecCnt

Read Test Results (Average of 10 tests)

Select Test Methodology• Used table from the Write Test (sequential Key IDs from 1

through 1000000)

• Three subsequent SELECT statements performed– FIRST_NAME = 'MARY’– FIRST_NAME = 'MARY' AND WITH LAST_NAME = 'SMITH'– FIRST_NAME = 'MARY' AND WITH LAST_NAME = 'SMITH' 

AND WITH STATE = 'CA'

• CLEARSELECT performed between each select

• Times for all three selects are combined

Select Test Results (Average of 10 tests)

Sort Test Methodology• Used table from the Write Test (sequential Key IDs from 1

through 1000000)

• Three subsequent SORT statements performed– BY FIRST_NAME– BY FIRST_NAME BY LAST_NAME– BY FIRST_NAME BY LAST_NAME BY STATE

• CLEARSELECT performed between each select

• Times for all three selects are combined

Sort Test Results (Average of 10 tests)

Benchmark Observations• Some operations yield 80x performance benefit in a pure U2

environment versus a pure OI environment (cf. Select Test Results)

• Hybrid environments (i.e. same OI code but using U2 tables) yields immediate benefits

• Read testing yielded minimal benefits in hybrid environments whereas sort testing suggests that a pure OI environment is better

• Hybrid environments will perform better as the U2 Bond technology improves and more operations are passed to the U2 server rather than being performed internally

FINAL COMMENTS

Good to Know…• Use the U2_KeepAlive command or a TIMER event to keep the

connection alive

• Use the U2_BUFFERSLC insert to increase the size of the buffer

• DICT.SYSPTRS is missing. This prevents the U2_SYSPTRS Volume Name popup from displaying

• There is a list of "do not detach" tables. One of these is VOC (for AREV32). As such, U2 tables will be renamed as U2_tablename (e.g. U2_VOC)

• U2 file names can support dots, but OI will represent them as underscores. However, when referring to these tables in code you sometimes have to use the "." name

Maximizing OI for U2• A straight conversion won’t necessarily improve

performance in all areas (cf. Benchmark Observations)

• OpenInsight will still perform database operations on the client side

• Optimized applications should push all I/O operations onto the server

• Good application design will use Client-Server, MVC, or n-Tier architectures

Thank You

Any Questions?