how iam significantly reduces vsam processing time · as to how much vsam dataset processing is...

1

How IAM Significantly Reduces VSAM Processing Time A White Paper

The Mainframe Software Partner for the Next 50 Years

2

https://support.compuware.com/


3

Contents

How IAM Significantly Reduces VSAM Processing Time ............................................................. 1

Introduction ................................................................................................................................ 5

What IAM Is ............................................................................................................................. 5

What IAM Does ....................................................................................................................... 6

How Much VSAM Processing .................................................................................................. 6

3rd Party Vendor Software ..................................................................................................... 7

Benefits ....................................................................................................................................... 8

Why Should I Consider IAM? .................................................................................................. 8

User Experiences ..................................................................................................................... 8

Summary of User Experience with IAM ................................................................................ 10

Benchmarks ............................................................................................................................... 10

Relevance of Benchmark Results .......................................................................................... 10

Benchmark Tests ................................................................................................................... 10

File Load Performance .......................................................................................................... 11

Access is Dynamic Performance ........................................................................................... 12

Sequential Read Performance .............................................................................................. 14

IAM/AIX Processing Performance ......................................................................................... 15

Random Insert Performance ................................................................................................. 17

Summary of Benefits ................................................................................................................. 18

Summary of IAM Benefits ..................................................................................................... 18

How IAM Achieves Results ........................................................................................................ 19

Overview ............................................................................................................................... 19

I/O REDUCTION ..................................................................................................................... 19

CPU TIME SAVINGS ............................................................................................................... 22

DASD SPACE SAVINGS ........................................................................................................... 22

Converting from VSAM to IAM ................................................................................................. 24

VSAM INTERFACE .................................................................................................................. 24

IAM FILE STRUCTURE ............................................................................................................ 27


4

Support for Record Level Sharing ............................................................................................. 29

IAM RECORD LEVEL SHARING (RLS) ...................................................................................... 29

Journaling & Additional Features.............................................................................................. 30

JOURNALING AND RECOVERY ............................................................................................... 30

Analysis Tools ............................................................................................................................ 32

HOW MUCH VSAM DATA DO YOU HAVE? ............................................................................ 32

HOW MUCH VSAM I/O PROCESSING DO YOU HAVE? .......................................................... 33

Conclusion ................................................................................................................................. 36

Summary ............................................................................................................................... 36


5

Introduction This white paper presents how IAM significantly reduces the processing time of VSAM applications. VSAM has been in existence for some 30+ years. In spite of the great success of DB2, there are still large amounts of data stored and processed on mainframes in VSAM format. In fact VSAM use can still be quite extensive even when DB2 use is predominate. For example a large DB2 user recently emailed us the following comment:

“I recently recommended converting a VSAM KSDS that was being used by an application here as a work file to manipulate a large amount of data unloaded from DB2 to IAM. This was very successful and dramatically reduced the I/O to

the file and has cut the elapsed time by more than half.”

The performance of accessing VSAM data remains of high importance for many z/OS users for their critical applications. The IAM product can help improve the performance and reduce the processing costs for VSAM applications. For many applications the benefits have been quite substantial.

What IAM Is

IAM is a reliable high performance record oriented disk file access method that can be used by applications that use VSAM KSDS, ESDS, RRDS or Alternate Index data sets for batch, TSO and CICS transaction processing. IAM offers a level of performance and reduction in the use of computer system resources that provides substantial savings for most applications that utilize VSAM data sets. VSAM requires a significant and costly manual tuning effort, and still for most applications will not process data as quickly as IAM does. IAM drastically reduces DASD space requirements through maximizing the storage capabilities of each DASD device and a Data Compression feature that does not consume excessive CPU time. IAM utilizes a sophisticated self-tuning buffering mechanism, called Real Time Tuning that responds quickly to the I/O demands of application programs, resulting in substantial decreases in physical I/O.

Today, IAM is being used in hundreds of information processing centers to provide high performance combined with lowering processing costs when compared to the use of VSAM alone. IAM is being used for a diverse array of mission critical applications including:

Accounts Payable & Accounts Receivable

ATM Processing

Payroll

Credit Card Transaction Clearing/Processing

Order Entry


6

Insurance (Including Medical) Claims

Retail Transaction Processing

Patient Care Processing in Hospitals

Data Engine for CICS-WEB Applications

Report Management Systems

The IAM product is continuing to be enhanced, offering additional functionality and performance enhancements.

What IAM Does

The savings that IAM offers over standard VSAM processing are going to vary, but the figures below are an indicator of the kind of savings that are typically achieved. For some applications the savings may be less. For many applications the savings are even higher.

20% - 70% Reduction in CPU usage

30% - 70% Reduction in DASD space usage

40% - 80% Reduction in I/O (EXCPs)

20% - 80% Reduction in Batch Elapsed Times

20% - 80% Reduction in CICS Transaction Response Times

In the following section on the Benefits of IAM you will be able to read about the benefits realized by various users in utilizing the IAM product, along with an overview of results from IAM benchmark tests.

How Much VSAM Processing

You may be wondering if you really have enough VSAM processing to really benefit from IAM. Compuware has available two no cost and no obligation software tools to help determine just how much VSAM data and processing is occurring on your systems. The first, FDRQUERY quickly runs through all of the online DASD volumes and produce a report with all of the VSAM datasets found, and the amount of DASD space used. FDRQUERY quickly and easily answers the question about how much VSAM data you have. The second tool, IAMSMFVS, reads through SMF data and reports on all of the VSAM datasets that were processed for the time period covered by the amount of data provided, and present reports that include the top 100 VSAM datasets for I/O activity, and the largest 100 VSAM datasets. IAMSMFVS answers the question as to how much VSAM dataset processing is actually occurring. From these reports, Compuware can provide assistance to determine whether or not IAM will be beneficial on your systems. For many users, the savings from using IAM for just 25 or 30 VSAM datasets will more than justify the cost of the IAM product. Additional information and examples of the reports produced by these tools are in the following section on Analysis Tools.


7

3rd Party Vendor Software

The following table identifies some of the common application software products from other software vendors that customers are using IAM with.

VENDOR PRODUCT

ACI OCM24, Action 2000

American Software FCT and DRP

Atos Origin CARDLINK

BMC CMF Monitor

CCI (AMS) AMS Advantage

CSC Hogan

CSI-BankTrade Trade Finance

Fidelity National Systematics Banking Applications BASE2000

First Data VisionPLUS

Hewitt (CYBORG) Payroll

IBM RMDS

Infor (GEAC) Millenium

JDA Software (Western Data Systems)

Compass Contract

LSI Popims

Mackinney Software JES Queue Print

VENDOR PRODUCT

Metavante Customer Statement Formatter (CSF)

Napersoft Automated Correspondence Tools

OMI ABS, Biceps, Prompt

Pitney Bowes (Group 1)

Finalist, Code 1, Mailstream Plus

SEA TRMS

Siemens Medical Systems

Invision

Sigma SAM (Student Aid Management)

SunGuard (SGT) SIS+, HRS, FRS

Tessseract Payroll, HR, Benefits

VIPS Medicare Part B, SuperOP

Walker Accounts Pay Purchase Order (APPO)


8

Benefits Why Should I Consider IAM?

The goal of this section is to answer just that question. Before looking at how IAM works and all of the features and functions IAM provides, we’ll take a look at the results of using IAM. First consider what your needs are. For example:

Are you using all of your CPU processing capacity? If so, are you deferring a processor upgrade at this time, and need something to gain back some of your processing capacity?

Is your batch work taking too long to complete?

Is your online transaction processing response time too long?

Are you running out of DASD space with no immediate plans to acquire more?

Are your batch applications frequently ending after their deadline?

Are your users aggravated because it is taking too long to process their work?

Is it costing too much to run your applications because of the bills for CPU usage?

If you answered YES to any of these questions, then IAM may be a part of the solution. Below is a quick look at what some users are saying about IAM.

User Experiences

Here are some quotes and other comments from some of our users. Certain portions of quotes have been removed to protect their identity.

“Our … application have had problems for a number of years with a large … VSAM file (…..) that had grown to over 35 gb and required frequent maintenance. This file was identified as an ideal candidate for conversion to IAM, with the aim of reducing its size, dependence on guaranteed space and batch processing time. The introduction of ‘dynamic dictionaries’ for hardware compression made the conversion simpler and more attractive to the application support team as they had experience in using IAM. The conversion was performed in early August and reduced the size of the dataset to 6 gb but more importantly reduced the run time of the main batch job that processed the file by between 3.5 to 5 hours, depending on the day of the week. Following the success of this conversion the support team decided to convert a number of other datasets.”

“We have 50% reduction in elapsed time and 20% reduction in CPU time after using IAM on our BASE2000 credit card application.”


9

“Said they’ve also converted a number of CARDLINK files as well. With the implementation of IAM with WALKER & CARDLINK … Said they’ve dropped 5-8hrs from their batch window and reduced their DASD by 30-80% over a number of files.”

“We implemented IAM for a key group of about 12 files about 4 years ago rather than do a costly hardware upgrade and it has worked out great. We were regularly (75%) over-running our batch window on weekdays (by hours!) so our interest in IAM was processing speed, not the compression. Since implementation, we cut about 3 hours from the 2 biggest job streams and have only missed the batch window when there was an application abend requiring programmers to fix code at night.”

“The advantages that IAM offers is far more than what can be achieved with VSAM compressed files or Tuning can do!! IAM has reduced a job from 7.5 hours to 4 mins 2 seconds!!! IAM saves 70 days processing per year with one application alone!!!!!!”

Below are the stats for a bank testing IAM and getting good results with an in-house application:

Elapsed time: From 44 minutes to 13 minutes under IAM.

Dataset size: From 150,000 TRK to 60,000 (using software compression)

CPU time: From 2 minutes to 1 minute under IAM.

They were very impressed with the results and IAM transparency. “We’ve completed our trial of IAM with the TechFIM VSAM files. We had the following findings:

43% reduction in I/Os

47% reduction in run time (Currently TechFIM nightly runs 4 hours; TechFIM monthly runs 22 hours)

20% reduction in CPU utilization (Currently TechFIM nightly uses 30+ CPU minutes)

34% reduction in DASD space utilization (Currently TechFIM (test & prod total) uses (9) 3390-3 volumes)

We feel these improvements justify the purchase of the product.”

“$AVRS daily accumulation is about 25,000 jobs that are kept online for 10 days.

Problem before IAM: Afternoon backlog waiting to be accumulated by $AVRS was 5,000 to 10,000 jobs. Accumulation of a 1000 line job with five files was about 1.5 seconds.

After IAM: Accumulation time of the 1,000 line job was almost cut in half and the accumulation backlog disappeared.”


10

Summary of User Experience with IAM

From the user experience shared above we see that while actual specific results vary, what is consistent is that IAM is providing savings for applications using VSAM. These savings are for many applications above what tuning efforts are able to accomplish, whether it be manual tuning efforts or automated tuning from the various VSAM performance software enhancement products that are available, including System Manage Buffering. IAM’s ability to reduce I/O, CPU utilization, and DASD space leads to savings in elapsed time in batch jobs and reduced response times for online transactions. As you will learn from the subsequent sections, IAM provides more than just a sophisticated buffer management enhancement for VSAM. IAM provides a different way for managing data records with compatibility for VSAM application programs.

Benchmarks Relevance of Benchmark Results

Benchmark tests provide for a known and repeatable set of tests from which performance comparisons can be done. By changing one variable in the test and keeping everything else as constant as possible, one can draw conclusions about the positive or negative influence of that variable on the processing being performed. While benchmark tests may not be representative of real world application processing, they are useful for identifying types of processing and the expected range of performance that can be expected. With that in mind, we will take a look at the results of some benchmark tests that were run comparing IAM and VSAM.

Benchmark Tests

A number of different benchmark tests were performed to compare IAM performance with VSAM performance. A variety of different types of processing were done along with a variety of different record lengths. Tests were run without any data compression and with hardware data compression. The VSAM tests were done using System Managed Buffering with a record access bias set to SYSTEM using DFSMS Extended Format datasets. IAM tests were done using the TURBO mode of IAM’s Real Time Tuning buffer management on Enhanced IAM files in standard sequential datasets. A complete description of the benchmark tests, benchmark environment, and the results are published in the IAM Version 8.1 User Manual.

The benchmark tests are being presented here in a bar chart type of graph. The bar chart shows the external throughput rate (ETR) in terms of records processed per elapsed second for the indicated function, i.e. file load, random reads, etc. When considering overall performance, this is the best single indicator, as it incorporates into it the key elements of CPU time (TCB time and SRB time), uncaptured CPU time, EXCP’s and I/O service time into a single numeric value. External throughput is reliable providing that the benchmark tests are performed in an environment where no other processing is being performed, which was created in the


11

laboratory environment for these tests. Included with the graphs are the ranges for CPU time savings, EXCP savings, and elapsed time savings.

File Load Performance

First impressions do count, and the first impression users have of IAM is with file load processing. It is also an important aspect of VSAM performance. Many VSAM files are loaded or reorganized (which performs a file load) on a regular basis such as daily, weekly, or monthly. When the VSAM files are being loaded or reorganized they are not available for other processing. Clearly file load processing is an important aspect of VSAM performance.

The chart below shows the throughput performance of file load processing of 2,000,000 records for IAM and VSAM, with and without hardware data compression.

125 250 500 750 1000

IAM 152,788 131,062 76,394 49,285 33,996

VSAM 69,037 49,603 31,581 22,282 18,471

IAM Comp 71,429 51,374 31,990 23,375 18,267

VSAM Comp 43,131 33,162 20,760 16,472 13,394

In comparing IAM and VSAM file load without hardware compression:

IAM used 39% to 41% less CPU time

IAM ran in 46% to 62% less elapsed time

IAM used 85% less EXCP’s

IAM VSAM IAM Comp. VSAM Comp.

0

20,000

40,000

60,000

80,000

100,000

120,000

140,000

160,000

125

250

500

750

1000

Rec

ord

s p

er S

eco

nd

Record Length

File Load Performance


12

IAM used 8% to 17% less DASD space

The results with hardware compression are subject to larger variations than without compression because the amount IAM and VSAM will be able to actually compress varies on a file by file basis based on the data patterns. Each product uses a different technique to build the compression dictionary, which results in different results for different datasets. This can result in different results for CPU time, EXCP’s, elapsed time and DASD space utilized. For these particular test files IAM achieved greater compression. However for some files VSAM may end up with greater compression than IAM when using the IAM automatic dictionary build process. IAM does allow for users to build a customized dictionary, as documented in the IAM manual, which generally achieves even greater compression than IAM or VSAM automated dictionary build process.

Access is Dynamic Performance

A type of processing called ACCESS IS DYNAMIC is frequently used by COBOL batch application programs. The reason is that ACCESS IS DYNAMIC allows the application to perform random and/or sequential processing. This causes a problem for System Managed Buffering and other VSAM buffer enhancing software that must select a buffering method at OPEN time. Such software does not know whether the dataset will be primarily randomly accessed or sequentially accessed. VSAM SMB selects NSR buffering for files opened in this manner, unless otherwise manually specified. NSR buffering works well for sequential and predominately sequential processing, but in most cases LSR buffering is much more beneficial with random processing.

This is an area where the IAM Real Time Tuning feature really shines because IAM adjusts the buffering to meet the current mode of processing being performed by the application.


13

Results are shown below for IAM and VSAM without compression. The results labeled VSAM are for the VSAM dataset with the default of ACCBIAS=SYSTEM from the Data Class. The results labeled VSAM DO are for the VSAM dataset with ACCBIAS=DO on the DD card for the VSAM dataset that indicates predominately random processing resulting in use of an LSR buffer pool.

125 250 500 750 1000

IAM 13,870 12,407 12,563 12,300 11,274

VSAM 162 150 145 145 144

VSAM DO 10,823 10,373 10,081 9,425 7,502

The results of IAM performance compared to VSAM SMB using the ACCBIAS=SYSTEM are:

IAM used 93% less CPU time

IAM used 99% less EXCP’s

IAM used 99% less elapsed time

Even with the manual specification in JCL of ACCBIAS=DO for VSAM the results are still in IAM’s favor:


IAM used 40% to 53% less EXCP’s


IAM VSAM VSAM DO

0

2,000

4,000

6,000

8,000

10,000

12,000

14,000

125

250

500

750

1000

Rec

ord

s p

er S

eco

nd

Record Length

Access is Dynamic Performance


14

The point here is that even with System Managed Buffering or any other automated VSAM buffering software, one cannot be sure without analysis of the results whether or not an appropriate buffering mechanism along with the appropriate number of buffers are being used to obtain optimal performance. With IAM, that is not a concern. IAM manages buffers in response to the actual I/O requests being performed currently by the application without any manual intervention. IAM handles an all sequential, an all random, or a mix of random and sequential I/O requests efficiently!

Sequential Read Performance

Sequential processing is another important aspect of VSAM performance. Many VSAM files are sequentially processed daily to create sequential backups and for reorganizations. Results are shown for reading datasets that were not data compressed, for the case with no record inserts and for a case after 38,000 random inserts were done. As anticipated, the sequential processing performance for both access methods decreases as a result of a less than optimal data organization due to the inserted records.

125 250 500 750 1000

IAM 128,287 117,440 95,102 80,873 64,851

VSAM 95,511 78,771 56,883 40,040 33,025

IAM w/Inserts 119,048 107,933 85,580 69,276 57,110

VSAM w/Inserts 89,807 71,633 47,996 33,904 27,747

IAM VSAM IAM Comp. VSAM Comp.

0

20,000

40,000

60,000

80,000

100,000

120,000

140,000

160,000

125250

500

750

1000

Rec

ord

s p

er S

eco

nd

Record Length

Sequential Read Performance


15

In comparing the results of IAM sequentially reading records versus VSAM we see that:




IAM/AIX Processing Performance

Processing with alternate indexes is another area where IAM offers very significant performance benefits. A VSAM base cluster with alternate indexes actually consist of multiple VSAM datasets, however when opened in a sphere they must all use the same buffering technique. So if an application were reading a file sequentially through an alternate index, then the alternate index would preferably be buffered with NSR buffering, and the base cluster is read randomly so it should preferably be buffered with LSR buffering. VSAM does not provide that capability. IAM with Real Time Tuning always uses the appropriate buffering technique resulting in consistent optimal buffering for datasets with alternate indexes.

Shown below are the results of performing a variety of I/O requests through an alternate index. The base cluster actually had two alternate indexes in this run, and some of the updates performed did require updates the second alternate index.

The following types and amounts of I/O requests were processed:

256,000, Random Read

160,000 Updates, 1/2 of which required an update to the other alternate index

80,000 Inserts

16,000 Deletes


16

16,001 Points (Start browses) with 80,000 sequential reads. (5 records read per start browse)

125 250 500 750 1000

IAM-Comp. 7,380 6,750 5,445 4,514 3,759

VSAM-Comp. 273 268 272 260 261

IAM 6,630 4,867 3,018 2,160 1,620

VSAM 266 256 257 245 261

In comparing the results of IAM/AIX processing versus VSAM we see that:




IAM’s Real Time Tuning buffering results in a very significant performance advantage over VSAM SMB. SMB does not use LSR buffering for VSAM clusters with alternate indexes. In this circumstance, VSAM’s performance could have been improved somewhat with the use of LSR, although IAM still would have outperformed VSAM. Use of LSR would require JCL changes to use the Batch LSR facility, or changes to the application program to build and specify the LSR buffer pool.

IAM-Comp. VSAM-Comp. IAM VSAM

0

1,000

2,000

3,000

4,000

5,000

6,000

7,000

8,000

125

500

1000

I/O Requests per Second

Record Length

IAM/AIX Processing Performance


17

Random Insert Performance

As discussed later in this paper, one of the significant advantages of IAM is with handling record inserts.

125 250 500 750 1000

IAM-Comp. 5,390 3,572 2,276 1,660 1,249

VSAM-Comp. 802 659 486 390 325

IAM 3,251 1,897 1,092 882 783

VSAM 709 506 363 255 225

This was put to test in this benchmark test. The chart below shows the throughput results for randomly inserting 38,003 records within a range of 400,000 records, which amounts to 9.5% inserts within that portion of the dataset. Both IAM and VSAM datasets were defined with 10% CI free space to help accommodate the inserted records. Because of the random pattern, VSAM did require CI and CA splits to handle some of the inserts, and IAM did require the use of its extended overflow area. The results indicate just how much faster IAM is at performing inserts with its file structure as compared to VSAM with its file structure.

IAM-Comp. VSAM-Comp. IAM VSAM

0

1,000

2,000

3,000

4,000

5,000

6,000

125

250

500

750

1000

Rec

ord

s p

er S

eco

nd

Record Length

Performance of Inserting Records


18

In comparing the results of IAM inserting records versus VSAM we see that:




For uncompressed data, IAM used 26% to 32% less space.

The amount of space savings for hardware compressed files varies on a file by file basis, due to the expected different amount of compression that will be achieved. These results show that not only does IAM greatly outperform VSAM on file inserts, the IAM file structure uses a lot less space than VSAM with its CI and CA split technique.

Summary of Benefits Summary of IAM Benefits

As can be seen from the user experiences and from the benchmark results presented above, IAM clearly provides performance benefits:

CPU time savings

Reduction in physical I/O

Reduction in DASD space requirements

Reduction in elapsed/response times.

The evidence indicates that the amount of savings with IAM is going to vary depending on a number of variables including the data record size and the type of processing being performed. This expected variance leads to the IAM savings being presented as a range of values rather than an absolute value.

Another interesting conclusion is that even with a VSAM performance enhancing facility, such as in the benchmarks above the use of System Managed Buffering, IAM still provides considerable value and savings.

Many of the users cited above already had some VSAM performance enhancement software installed, and still saw considerable benefit from IAM.


19

How IAM Achieves Results Overview

There are a few key features of IAM that are responsible for providing the bulk of the savings that users experience with IAM. These include:

Elimination of Index I/O

Elimination of CI and CA Split Processing

Dynamic Buffer Management through IAM’s Real Time Tuning

Optimized File Load Buffering

Record Cache (Dynamic Tabling)

Performance Oriented Design

Data Compression

Device Efficient Block Sizes

Each of these features are explained and described below.

I/O REDUCTION

Eliminates Index I/O

IAM uses virtual storage to reduce physical I/O’s. The first thing that IAM does whenever an open is performed for an IAM dataset is to read the index of the dataset into virtual storage. To minimize storage use, the index is generally in a compressed format. This virtual storage is either part of the job’s region, in which case it is all in 31-bit addressable storage, or it can be contained in a data space. Depending on the size of the file, this may take a little bit of time, but for most files the overhead is not noticeable. With the index residing in virtual storage, IAM never has to perform I/O operations to read the index in response to application I/O requests. This means that IAM can locate and read in any record within the dataset with at most one physical I/O. IAM does not have any index component buffers to be tuned. This function alone can result in significant I/O savings when using IAM instead of VSAM.

Eliminates CI and CA Splits

IAM’s record insert strategy significantly reduces the CPU time and I/O to add records into an IAM dataset. IAM eliminates the high cost of the VSAM CI and CA splits. IAM utilizes a record based overflow area to accommodate expanded records that no longer fit within the block they are in, and for inserted records that do not fit in the block in which they would be placed. When a record has to be placed in an overflow block, or perhaps has to be moved to an overflow block with more room, IAM finds an overflow block with sufficient space, and writes the record in that overflow block. IAM maintains in virtual storage a list of overflow blocks that have free space. If there is inadequate space available within the existing extended overflow blocks, then a new overflow block is obtained from the extended area of the IAM dataset. If there is no


20

room left for additional blocks, IAM attempts to obtain an additional extent to acquire more DASD space for the extended area. Once added to an overflow block, an index entry is added in virtual storage for the overflow record. Because there is no relation by key of the records within any particular overflow block, any record can be placed in any overflow block regardless of its key value. Additionally, when a record is deleted from an overflow block, the space is immediately available for reuse for any record that needs overflow space, regardless of the key.

This technique has significant performance advantages for many VSAM applications. The advantages of the IAM technique are:

Reduced physical I/O to perform the overflow processing than CI/CA split processing

Reduced CPU time to add records to the file

More efficient use of DASD space

No more than one physical I/O to read any requested record.

The potential disadvantages of this technique are the increase in virtual storage required, potential for long open times when a file has very large number of records in the overflow area, and the possibility of excessive I/O when sequentially reading the dataset. Most VSAM applications realize noticeable performance benefits from the IAM overflow technique. The disadvantages may be minimized by appropriately scheduled file reorganizations.

Dynamic Buffer Management

IAM’s Real Time Tuning feature is one of the major contributors to improving VSAM application performance. IAM dynamically monitors the I/O activity to an IAM dataset, and adjusts various parameters dynamically, including buffer management technique and the number of buffers being used for the dataset. This feature simplifies and reduces the effort involved in tuning VSAM applications using IAM files. Let’s take a look at the functions of IAM’s Real Time Tuning, and how it improves VSAM application performance.

The first function of Real Time Tuning is dynamic buffer management. IAM utilizes a buffer pool for each opened IAM dataset. Those buffers are managed based on how the application program is accessing the data within those buffers. IAM utilizes a least recently used (LRU) buffer management for randomly accessed data blocks. IAM also automatically defers writes for batch updates on datasets with share options of 1 or 2. For sequential processing, IAM reads ahead multiple blocks, read multiple blocks per physical I/O, and write multiple updated blocks per physical I/O. As the sequential process completes use of a buffer, it is made available for immediate reuse. Additionally, when performing a sequential multi-block read, IAM attempts to reuse the same buffers it used for the prior sequential I/O. For applications performing a mix of random and sequential I/O, IAM dynamically balances the number of buffers used for each type of buffer management based upon the actual application use to best meet the application’s current requirements. IAM also recognizes whether or not it is running under a batch job or CICS online system, and for example is more aggressive with sequential


21

buffer management for batch jobs than for CICS online systems, because CICS online systems typically have a lot more concurrently active I/O requests than a batch application.

IAM does not lock buffers to strings (place holders). This eliminates the CI lockout / exclusive control problems that may be experienced when using VSAM. Instead, IAM uses an internal record level locking technique when a record is read for update. Elimination of this delay helps to provide improved response times to CICS online system users.

A key function of IAM’s Real Time Tuning is that it dynamically adjusts the number of buffers that it is using, on a dataset by dataset basis. IAM works within a default or user specified range of number of buffers, referred to as MINBUFNO and MAXBUFNO. IAM does this by monitoring the applications use of buffers, with internal statistics being kept as if IAM actually had more buffers than it presently has. On a periodic basis based on actual file I/O, IAM reviews the activity statistics and increase or decrease the number of buffers one at a time. IAM examines increasing buffers more frequently than decreasing buffers to avoid being in a state of constantly releasing and acquiring of buffers. The intent is to achieve as stable of a state as possible with buffers. IAM considers the quantity and frequency of physical I/O requests along with whether the I/O activity is random, sequential or a mix to make a decision on revising the number of buffers.

IAM Version 8.1 includes a TURBO mode for the Real Time Tuning buffer adjustment. When running in TURBO mode, the default, IAM is much more aggressive in increasing the number of buffer being used. This enables IAM to respond quickly to applications that have high I/O rates to improve response times for online transactions and reduce elapsed times of batch jobs.

A frequent question is how to tune IAM files. One of the key tuning factors in improving IAM’s performance is to increase the maximum number of buffers. This can easily be done at a job step level through the use of IAM overrides, the JCL parameter BUFND, or at a global level with the IAM Global Options. Then the question is for what file(s) will a buffering increase be beneficial. It is easy to locate such files because IAM displays a message (IAM368) in the IAMINFO report when the file is closed. If the IAM SMF records are being recorded then the IAMINFO reports can be specifically reported on by IAMSMF with a control card input indicating to produce reports only for files that have the more buffer message by specifying ATTRIBUTE=MOREBUFFER.

IAM sets MAXBUFNO for batch jobs to the number of buffers that fit in 64 megabytes of virtual storage on a file by file basis. For IAM datasets opened under CICS, a lower value of 1 megabyte is used because there are frequently many more IAM files opened under a CICS region than any particular batch application. The default values can be changed at an installation level through the IAM Global Options Table.

With IAM’s Real Time Tuning feature, tuning is simplified and performance of VSAM applications is improved. While results vary, typical results are in the range of 30% to 80% reduction in elapsed time.


22

File Load Buffering

File load buffering is different than file access buffering, because the file load is a sequential output process. IAM defaults to acquiring buffers for two cylinders worth of blocks. During processing, when half of the buffers are filled, IAM issues an EXCP to write out that set of buffers. Concurrent with that I/O, IAM allows the application to continue processing sending records to the other half of the buffers. With the default number of buffers, IAM is writing out one cylinder per I/O during the file load process. Typically a data space is used to hold the index however a temporary dataset can be used as an alternative.

For DFSMS Extended Format datasets, IAM uses BSAM to load the file. IAM sets up the I/O so that multiple blocks can be written per physical I/O, however IAM has no direct control over the number of blocks written per physical I/O, as this is determined dynamically by BSAM. For such datasets, the EXCP count for the file load is equal to the number of blocks written out to the file, as opposed to the actual physical I/O count.

Dynamic Tabling

IAM has a unique feature referred to as Dynamic Tabling. This function essentially provides a record based cache in a z/OS Data Space for frequently retrieved randomly accessed records from an IAM file. This function has proven to be very beneficial for I/O reduction particularly for certain transaction processing applications and helps reduce storage usage by reducing the number of data buffers required to achieve the same level of performance. This feature is activated for selected files.

CPU TIME SAVINGS

Designed for Speed

IAM is designed and intended to be a performance oriented software product. The identified main line execution paths are carefully designed and coded to reduce path length and utilize instruction combinations and coding techniques that are proven to minimize the CPU time required to perform the intended process. Algorithms are carefully selected for their performance attributes. The reduction of physical I/O requests also eliminates a lot of processing required to perform a physical I/O.

DASD SPACE SAVINGS

Data Compression

IAM provides data compression functions that can be used to reduce the amount of DASD space for VSAM data that is managed by IAM. Additional data compression benefits include reducing physical I/O operations to read/write/update VSAM data that can result in reducing response times for online transactions, and reducing elapsed time for batch jobs. With the data residing in fewer data blocks, IAM has a smaller index that reduces the amount of virtual storage required for the index.


23

Data compression can be used on any type of IAM file, KSDS, ESDS, RRDS, Variable RRDS, or AIX. The downside of data compression is that it uses processor cycles to compress and decompress the data. For many datasets, even with data compression, IAM still uses less CPU time than what was used for an uncompressed VSAM dataset. However, as the record sizes increase, so does the data compression overhead. Because of the benefits of data compression, we generally recommend its use. However, if saving CPU time is a priority, then turning off data compression either selectively or across the board will provide you with CPU time savings.

IAM has two different methods of compression. The first is a proprietary software compression algorithm that compresses data records by eliminating repeating values. For example, a string of blanks can be reduced to one byte. There is no compression dictionary used by this algorithm, and it was designed to provide some data compression with a low CPU time cost. Typical results are 20% to 50% compression, and with some files even better compression may be achievable. IAM’s software compression typically uses less CPU time to compress the data than the zSeries hardware compression, but may use slightly more CPU time to decompress the data.

IAM can alternatively compress data with the zSeries hardware compression instruction. Use of the zSeries hardware compression requires a data dictionary to compress and decompress the data. IAM automatically creates a data compression dictionary as the dataset is being loaded using the first few megabytes of data. Alternatively, a user can create their own data compression dictionary following the instructions in the IAM manual. The most compression is typically achieved by the manual dictionary creation process, which for some datasets can achieve very substantial space savings. CPU times can be rather high when loading datasets with hardware compression that can result in elongated elapsed times.

Block Size

IAM selects a block size for a dataset based on a variety of criteria that includes the user specified CI size if any, the maximum record size, the type of device the dataset resides on and by the IAM Global Options. IAM chooses a block size that maximizes the amount of data stored on a track once it has determined how many blocks per track it will use. In general IAM selects a block size that results in four blocks per track. Depending on the record size and user CI size, IAM may choose a larger block size of three or two blocks per track. For the vast majority of datasets converted to IAM, the IAM selected block size provides excellent performance and rarely needs to be modified. What this means is that users seldom have to be concerned about having to revise the IAM determined block size, either for DASD capacity or performance.

One of the reasons that IAM can utilize a larger, more efficient block size than VSAM is that IAM only locks out access at the record level, whereas VSAM locks out at the CI level. For some files users have had to go to very small CI sizes to limit the number of records in each CI. These smaller CI sizes are much less space efficient than the larger sizes. While CICS has developed a process with VSAM that may reduce CI lock out issues, that process does cause more overhead


24

due to additional VSAM I/O requests being required and does not completely eliminate the problem.

The most commonly selected IAM block size allows for up to 53.4KB per track on a 3390. The most commonly used CI sizes for VSAM allows for up to 48KB per track on a 3390, which provides IAM with the potential of having up to 11.25% more data per track even before compressing the data records. The actual space savings for any particular file is going to vary based on the actual record lengths and the defined free space values.

Converting from VSAM to IAM While the key performance oriented features are described in the previous section, this section focuses on the rest of the story. We will present information on IAM’s VSAM interface, a brief description on using IAM datasets, a description of the IAM file structure, and a summary of some additional features of the IAM product.

VSAM INTERFACE

Transparent VSAM Interface

IAM’s system level VSAM Interface (VIF) provides application program transparency. IAM allows any program (e.g. COBOL, assembler, PL/1, C) access to IAM files in place of VSAM KSDS, ESDS, RRDS, or VSAM Alternate Index files. The IAM VSAM Interface (VIF) is normally started as part of the IPL procedure to dynamically install hooks into various system services that are used to process VSAM datasets, such as OPEN, CLOSE, and a few others. Once activated, the IAM VSAM interface remains active in the system until either IAM is deactivated or an IPL is performed. The IAM VSAM Interface essentially acts as a router, sending requests for real VSAM datasets to the appropriate VSAM routine, and routing requests for IAM files to IAM. This enables IAM files to appear as VSAM files to application programs, whether running under batch, TSO, or CICS.

IAM does not interfere with or replace any VSAM processing. For example, during OPEN if the file being opened is an actual VSAM dataset then IAM routes control to VSAM. From that point forward, all processing is handled by normal VSAM routines and is never again seen by IAM. IAM does not get control or interfere with any VSAM I/O processing. Conversely, if the file is an IAM file, IAM handles all of the processing so that VSAM code never interferes with IAM.

IAM has full support for the SMS environment. This support includes recognizing and using the SMS classes for allocation, honoring file attributes specified in the Data Class, and support for JCL allocation of IAM files; including temporary data set support. When using the SMS JCL allocation feature specifying $IAM in the data set name, in the Storage Class name or in the Data Class name on the DD Statement results in an IAM file being allocated.


25

Using IAM Datasets

IAM files are processed in a manner that is identical to VSAM. First, IAM files must be defined. IAM files can be defined with the IDCAMS DEFINE utility, through MVS JCL VSAM allocation, or by the IAM ISPF panels. Once defined, IAM files must be initialized with user data. IAM files can be loaded with application programs, IDCAMS REPRO, or as system SORT output (SORTOUT). After a successful file load, IAM files can then be processed and updated. Alternate indexes and paths can be defined and built using IDCAMS. After significant update activity, an IAM file may need to be reorganized, just as VSAM KSDS files. The frequency of file reorganization of IAM files may be either less or more than what was required with VSAM. Compuware offers a file reorganization product, called FDRREORG™, which can automate the file reorganization process for IAM files, as well as VSAM and PDS type of files.

Defining an IAM File

Before using an IAM dataset, it must be defined. This define process is identical to what is required for VSAM datasets. During the define process IAM allocates the DASD space for the dataset, catalogs the dataset, and stores the file attributes within the dataset itself. IAM datasets can be defined by using IDCAMS, through JCL DD cards, or using a variety of methods under TSO, including through the IAM ISPF panels. Many other software products that are used to define VSAM datasets will generally also be able to define IAM datasets. There usually are no changes required to the DEFINE parameters other than an indication that the file is to be an IAM file.


26

How to IAM a Dataset

For a dataset to become IAM instead of VSAM, an indication must be provided on the file definition indicating that the file is to be an IAM file. The ways to indicate this are:

1. Add the parameter OWNER($IAM) to the IDCAMS DEFINE command, or 2. Change the dataset name to include the literal $IAM somewhere within the name, or 3. For SMS managed IAM datasets, use an SMS Data Class or Storage Class with the literal

$IAM as part of the class name.

Below is an example of the changes required to the IDCAMS control cards to convert it to define an IAM file:

Original VSAM Define

//DEFINE EXEC PGM=IDCAMS //SYSPRINT DD SYSOUT=* //SYSIN DD * DEFINE CLUSTER - (NAME(EXAMPLE1.DATASET) - VOLUMES(VOL001) - CYL(10 1) - SPEED REUSE ) - DATA( - NAME(EXAMPLE1.DATASET.DATA) - RECORDSIZE(200 256) - KEYS(16 0) - CISZ(4096) - FREESPACE(10 10) - INDEX( - NAME(EXAMPLE1.DATASET.INDEX) - CISZ(1024) IMBED) ) LISTCAT ENT(EXAMPLE1.DATASET) ALL

IAM Define

//DEFINE EXEC PGM=IDCAMS //SYSPRINT DD SYSOUT=* //SYSIN DD * DEFINE CLUSTER - (NAME(EXAMPLE1.DATASET) - VOLUMES(VOL001) - CYL(10 1) - OWNER($IAM) ADD - SPEED REUSE ) - DATA( - NAME(EXAMPLE1.DATASET.DATA) - RECORDSIZE(200 256) - KEYS(16 0) - CISZ(4096) - FREESPACE(10 10) - INDEX( - NAME(EXAMPLE1.DATASET.INDEX) - CISZ(1024) IMBED) ) LISTCAT ENT(EXAMPLE1.DATASET) ALL

IAM File Load Process

After successfully defining an IAM Dataset, the next step is to load data into the dataset. Generally, the file load process is where the dataset is populated with records, and the index structure is built. For IAM datasets, this portion of the index is called the prime index. Once the load is completed, the prime index for the dataset is established, and does not change. The load process is restrictive, as is VSAM, in that the only I/O operation allowed is to write new records into the dataset. For KSDS type of files, the records must be written in ascending key sequence. The file load must write one or more records to the dataset, and subsequently successfully close the dataset for it to be considered a loaded dataset. Once loaded, the dataset can be processed with the full range of I/O operations.


27

IAM datasets can be loaded by any program designed to load VSAM clusters. This includes system utility programs, such as IDCAMS REPRO and the SORT, as well as application programs.

IAM File Processing

After the IAM dataset has been defined and subsequently loaded with data, the IAM dataset is ready to be used for typical online and batch processing. Processing an IAM dataset instead of a VSAM cluster generally significantly reduces CPU times, physical I/O’s, and elapsed times for most programs. The IAM VSAM interface was developed with the intent of eliminating, as much as possible, the need to change any program, JCL, or CICS tables. On rare occasion, there may be a need for a change, which can frequently be accomplished with an IAM override. IAM supports almost all of the typical VSAM processing options, including specification of LSR pools, user exits specified via the VSAM EXLST macro, asynchronous I/O, and so forth.

IAM FILE STRUCTURE

Block Structure

To achieve the high performance and space savings IAM uses a different data structure and dataset organization than VSAM. We will start with the basic unit of data transfer for an IAM file, which is a fixed length block. One might wonder isn’t that similar to VSAM’s fixed length control intervals? It is, but only in the concept of using fixed length blocks. The organization of data is completely different, as are many of the other attributes of an IAM block. First, the size of an IAM block is not restricted to an arbitrary set of sizes as VSAM is, which means that IAM can use block sizes that allows full track utilization based on the device geometry on which the data set resides. Therefore depending on data record lengths and CI sizes, in many circumstances IAM can store more data on a track then VSAM.

Data Set Organization

IAM files reside on DASD as non-VSAM datasets, typically with a DSORG of PS (physical sequential). To exceed the 64K track per volume limitation, IAM files can also be stored as Large Format Sequential datasets in z/OS 1.7 or higher, or optionally as DFSMS Extended Format sequential datasets. This means that the cluster name specified when defining an IAM dataset, is the name and only name of the dataset that is in the catalog structure as a non-VSAM dataset and on the DASD VTOC. All of the descriptive information about the IAM dataset, the user data records, and the index is kept within the single IAM dataset itself. IAM datasets do not have control areas or components. Rather, it is a collection of fixed length blocks containing internal file description information and user data.

Structure of IAM Data Set

Within the IAM dataset itself are basically four different areas. The first area and first blocks within an IAM dataset contain the file description information, things like record length, key size, key length, and information on the file structure and index, such as where within the file the other areas are and their respective sizes. Following this descriptive information is what


28

IAM calls the prime data area. These are blocks containing application data records. The size of this area is based on the amount of data that is initially loaded into the IAM dataset, plus any embedded free space. At the end of the prime data area is the index to the prime data area. This index is read into virtual storage when the IAM dataset is opened, so there is no index I/O to retrieve records once the IAM dataset has been opened. This prime index structure is never updated, until it is rewritten when the IAM file is reorganized.

Control Information

Prime Data Area

Index

Extended Area

Extended Area

After the index area there may be an extended area. The extended area is used to contain records that have been added or records that expanded in size for which there was no room within the free space in the Prime Data Area. The extended area has a dynamic structure, which includes an area for the some blocks describing the usage of the subsequent blocks, and then the extended prime blocks and the extended overflow blocks that can be intermixed. Extended prime blocks are used to contain records that are added to the logical end of the dataset that is with keys higher than the high key initially loaded into the dataset. This is similar to the prime data blocks, and is indexed by the high key in each block. The extended overflow blocks contain data records that have been inserted into the file or expanded, and did not fit within the targeted prime block. The extended overflow blocks are indexed by record.

When a record has to be placed in an overflow block, or perhaps has to be moved to an overflow block with more room, IAM finds an overflow block with sufficient space, and write the record in that overflow block. IAM maintains in virtual storage a list of overflow blocks that have free space. If there is inadequate space available within the existing extended overflow blocks, then a new overflow block is obtained from the extended area of the IAM dataset. If there is no room left for additional blocks, IAM attempts to obtain an additional extent to acquire more DASD space for the extended area. Once added to an overflow block, an index entry is added in virtual storage for the overflow record. Because there is no relation by key of the records within any particular overflow block, any record can be placed in any overflow block regardless of its key value. Additionally, when a record is deleted from an overflow block, the space is immediately available for reuse for any record that needs overflow space, regardless of the key.

Advantages

This technique has its advantages over the VSAM CI/CA split technique. The advantages of the IAM overflow processing include:


29

Reduced physical I/O to perform the overflow processing than CI/CA split processing

Reduced CPU time to add records to the file

More efficient use of DASD space

Retaining no more than one physical I/O to read any requested record

Most VSAM applications do realize noticeable performance benefits of the IAM overflow technique. Some files may benefit by increasing the CI free space value, or more frequent reorganization.

Support for Record Level Sharing IAM RECORD LEVEL SHARING (RLS)

Why RLS

In today’s 24 x 7 environments it is no longer viable to restrict a file to the rigid protocol of “CICS-only access” during the day, and “Batch-only access” (i.e. one-job-at-a-time) during the evening. Now, to fit increasing amounts of work into decreasing windows of time, data centers need to be able to safely access and update their files, whenever they want and from wherever it is necessary.

Clearly, the issues of sharing CICS/VSAM files with Batch are becoming increasingly important to many companies. What is the situation in your company?

Are you currently closing down your CICS/VSAM applications during the day so that you can run urgent batch work? If so, what impact is this having on the productivity of your online users?

Or…are you compromising by running daytime batch work against a copy of the online files? If so, what affect does this have on the users who are receiving out-of-date (and possibly inaccurate) information?

Or…are you just delaying all of your batch work until the evening shift, after CICS has shut down. If so, does this delay cause problems for the users who are waiting for the batch output? Are you perhaps having trouble fitting all of the batch work into the available overnight window? And does the long-running batch work occasionally have a knock-on effect on the following day’s startup of CICS?

IAM RLS

Whichever way you are currently handling the file sharing of your CICS/VSAM files, you will be pleased to know that IAM’s Record-Level Sharing (RLS) feature allows you to have multiple applications (CICS Regions, Batch Jobs, TSO Users, etc.) concurrently sharing an IAM file within a single system/LPAR image.


30

This is accomplished by routing all I/O for the selected files to an IAMRLS address space, which handles the I/O requests. IAMRLS provides buffering, record level locking with deadlock protection, journaling and recovery capabilities (see below). This version of IAMRLS does not require the use of a Coupling Facility or SYSPLEX services.

You can have multiple IAMRLS address spaces on a single LPAR or z/OS system image. This can be an aid for customers requiring isolation of different applications, for testing, security or accounting purposes. Each IAMRLS address space is assigned a four-character identifier (referred to as the RLSID), and you can select which RLS address space is used for particular jobs or CICS regions.

While no changes are required to batch or CICS programs to implement IAMRLS, customers with long-running batch jobs are encouraged to implement and use a provided IAM Batch Syncpoint capability. This provides for a recovery point-in-time, and allows the record locks to be released so that other batch jobs or CICS transactions are not locked out of too many records for an excessively long time.

RLS support is available on KSDS files, Alternate Indexes (AIX files), RRDS and ESDS files. Not all IAM files used within a CICS system, or referenced by a batch job, or accessed by a TSO user have to be RLS capable. It is possible to mix-and-match to suit requirements. No extra JCL or programming is required to implement basic RLS support.

Journaling & Additional Features JOURNALING AND RECOVERY

Journaling and Recovery

IAM provides the facility to write BEFORE and/or AFTER journal records, which can be employed in either a “forward” or “backout” recovery of a file:

A “backout recovery” is performed in the event of a failure/abend of a single (updating) job. At the point of failure, the “Before” images of the updated data records are written back to the IAM file, working backwards in time, thus removing all the updates.

A “forward recovery” is used if an older copy of an IAM file has to be restored, perhaps after encountering a media failure, or data corruption, or maybe even in the event of Disaster Recovery. Once the older copy of the file is back on disk, the “After” images can then be reapplied to the file, working forwards in time, thus recreating the updates that had taken place.

These procedures, which have been available for some time in IAM, have been enhanced to accommodate the changes introduced by the RLS support described above.


31

IAM also features a “Dynamic Job Backout” (DJB) facility, which provides a controlled, automated and fully dynamic “backout recovery” of IAM RLS files. If an updating job step abends all uncommitted updates for that job are automatically removed from all RLS files being updated by that job. This negates the need to run separate, manual, batch-driven recoveries for each affected file.

Additional Features

There are a number of other functions and features of the IAM product that enhance IAM’s usefulness and performance for many users. These include the following:

Automatic Space Release on completion of the file load. This feature eliminates unused DASD space that come about from the space savings offered by IAM and the frequent over allocation of space for VSAM datasets.

Secondary Space Adjustment increases the secondary DASD space quantity when it is determined that the space allocation when the file was defined was insufficient. This can occur on file loads or as the file expands in size.

With IAM’s data structure, IAM has NEVER had a 4 gigabyte limit on dataset size.

Potential for up to three terabytes of compressed data within a single IAM dataset when IAM datasets are defined as Large Sequential Format (DSNTYPE=LARGE) under z/OS 1.7 or higher, or as DFSMS Extended Format dataset.

IAM can perform backups and reorganizations without decompressing and recompressing the data for software compressed files under IDCAMS or FDRREORG. Using this feature can reduce backup and reload time of IAM software compressed datasets.

IAM produces a one page statistics report referred to as the IAMINFO report whenever an IAM dataset is closed. This report includes file description information, the number of the various I/O requests issued, and data on storage utilization and EXCP’s. The data to produce those reports can also be written as an SMF record.

IAM unique functions and parameters can optionally be specified through the use of the IAM Override facility. This provides a mechanism through JCL to alter IAM parameters.

Compuware offers two no cost and no obligation tools that determine just how much VSAM data you have, and quantify the actual VSAM I/O activity that is occurring. The FDRQUERY program quickly determines the amount of DASD space used and the amount of data that is stored in non-DB2 VSAM datasets. The IAMSMFVS program reads through SMF data to report on the amount of VSAM I/O activity and produce three reports, including the 100 VSAM clusters with the highest I/O activity, and the 100 largest VSAM clusters. The reports produced from these tools provide the information needed to determine whether there is VSAM data and activity that will benefit from IAM. If you send these reports to Compuware, then an analysis report with an estimate as to the minimum I/O reduction that can be expected from IAM will be provided. From that information, you can make a decision as to the potential for IAM to make a beneficial difference in reducing the cost and time of processing the VSAM data.


32

Analysis Tools HOW MUCH VSAM DATA DO YOU HAVE?

FDRQUERY

As mentioned above, the FDRQUERY program can quickly and easily determine the amount of VSAM data that you presently have. If you already have the FDR V5.4/62 or higher from Compuware, then you have the FDRQUERY program needed to produce this report. If FDR at V5.4/62 or higher is not available, then Compuware will provide you a free, no-cost, and no-obligation version of FDRQUERY to produce this report for you.

The VSAMSCAN function of FDRQUERY scans your online DASD volumes for VSAM datasets. Unless otherwise requested, DB2 datasets, catalogs, and VVDS datasets are automatically excluded. The VSAMSCAN produces a report that includes the amount of tracks used for each VSAM component, along with an estimate on the size of the dataset in bytes. These values are totaled per volume and for the entire run. To get a more accurate figure for the number of bytes, you can run the VSAMSCAN with the keyword DISABLE=ESTIMATE, in which case FDRQUERY requests the actual size in bytes from catalog services, however that runs longer than just doing an estimate. An example of a portion of the report produced by FDRQUERY is shown below.

FDR400 FDR SMF/VTOC DATA EXTRACT/QUERY-FDRQUERY VOL D/S OP DATA SET NAME VOLSER SEQ CRDATE LRDATE ORG RECFM CD BKSIZ NEX ALLOC BYTES ------------------------------------- ------ --- -------- -------- --- ----- -- ----- --- ----- -------- SYSPXCF.CICSIDP2.DFHJ01.A0000000.DATA SYTS17 001 2003.104 2006.242 EF U 80 18432 1 8340 461.169M IAMCICSV.CICSB22F.DFHBRNSF.DATA SYTS17 001 2006.104 2006.241 EF U 80 18432 1 1 0.055M IAMCICSV.CICSB22F.DFHBRNSF.INDEX SYTS17 001 2006.104 2006.241 EF U 80 18432 1 1 0.055M IAMCICSV.CNTL.CICSB32I.DFHINTRA.DATA SYTS17 001 2007.104 2007.212 EF U 80 18432 1 9 0.498M IAMCICSV.CICSB23F.DFHEJOS.DATA SYTS17 001 2006.104 2006.212 EF U 80 18432 1 180 9.953M IAMCICSV.CICSB23F.DFHEJOS.INDEX SYTS17 001 2006.104 2006.212 EF U 80 18432 1 1 0.055M IAMCICSV.VSAM.RLS.KSDS1.DATA SYTS17 001 2007.104 2007.312 EF U 80 18432 1 285 15.759M IAMCICSV.CICSB32I.DFHEJDIR.DATA SYTS17 001 2007.104 2007.137 EF U 80 18432 1 30 1.659M IAMCICSV.CICSB32I.DFHEJDIR.INDEX SYTS17 001 2007.104 2007.137 EF U 80 18432 1 1 0.055M IAMCICSV.CICSFORT.DFHLRQ.DATA SYTS17 001 2007.104 2007.345 EF U 80 18432 1 30 1.659M IAMCICSV.VSAM.RLS.KSDS1.INDEX SYTS17 001 2007.104 2007.312 EF U 80 18432 1 1 0.055M IAMCICSV.CICSFORT.DFHLRQ.INDEX SYTS17 001 2007.104 2007.340 EF U 80 18432 1 1 0.055M IAMCICSV.CICSFORT.DFHADEM.DATA SYTS17 001 2007.104 2007.340 EF U 80 18432 1 60 3.318M IAMCICSV.CICSFORT.DFHADEM.INDEX SYTS17 001 2007.104 2007.340 EF U 80 18432 1 1 0.055M TOTALS — VOLUME=SYTS17 — 14 COMPONENTS OCCUPYING 8941 TRACKS — 494.402M BYTES … TOTALS — FINAL — 61336 COMPONENTS OCCUPYING 1332527 TRACKS — 73472.404M BYTES


33

How to Run FDRQUERY

The JCL to run FDRQUERY is very simple. The only change to the JCL shown below is to provide the load library name of the library containing FDRQUERY. If you have the FDR products and are using that version, and have FDR in your link list, then you can completely remove the STEPLIB. Submit the JCL and let it run. Make sure that you have the proper level of FDRQUERY (which is V5.4 Level 62 or higher). The JCL and control card is shown below:

HOW MUCH VSAM I/O PROCESSING DO YOU HAVE?

IAMSMFVS

IAMSMFVS identifies your most active and largest VSAM clusters by scanning SMF data and totaling the usage for each VSAM cluster. IAMSMFVS lists your ‘top 100’ most active VSAM clusters (by EXCPs) and the ‘top 100’ largest VSAM clusters, along with other file characteristics.

The four (4) reports produced by IAMSMFVS are:

1. A Summary Report that lists the total VSAM activity including total EXCP’s and DASD space utilization.

2. An EXCP Report listing the top 100 VSAM clusters by the number of I/Os (EXCPs) issued against the clusters. This report shows the total EXCP’s against the file, the percentage of each components portion of the EXCP’s for the cluster, and the number of records in the cluster. File activity is displayed (Reads, Inserts, Updates, and Deletes) as is the number of CI and CA splits, along with the size of each component in tracks.

3. A Data Set Summary Report listing all of the VSAM clusters that were processed during the time interval of the provided SMF data in dataset name sequence. The file characteristics of each VSAM cluster (Record Length, Key Size, Key Offset, CI Size, etc.) are included in this report.

4. A Size Report listing the top 100 VSAM clusters by size. The total size for each VSAM cluster is given in tracks along with each of its components and the number of extents they occupy.

//QUE1 EXEC PGM=FDRQUERY,REGION=0K //STEPLIB DD DISP=SHR,DSN=fdrquery.loadlib //SYSPRINT DD SYSOUT=* //ABRMAP DD SYSOUT=* VSAMSCAN /*


34

Examples of two of these reports are provided below. The full details on running IAMSMFVS will be provided with the IAMSMFVS program. Please take note of the SMF records that MUST be available for the IAMSMFVS program to produce these reports, and make sure that they are being collected at your site.

SMF type 30 subtype 4 records, or type 4 records that contain Step Termination statistics.

SMF type 64 records containing VSAM data set activity statistics.

VSAM EXCP Report

The VSAM EXCP Report provides a listing of the 100 VSAM clusters with the highest physical I/O activity. This report is of particular value for finding VSAM datasets for conversion to IAM when reducing CPU time and/or reducing I/O along with elapsed time or response times is desired.

Using the VSAM EXCP Report

The VSAM EXCP Report is used to identify the VSAM datasets that are likely to receive the largest performance benefit from conversion to IAM. The primary benefits are I/O reduction and CPU time reduction that result in shorter elapsed/response times for the jobs processing those datasets. For most users converting the top 20 or 30 datasets leads to a significant savings. Many users convert hundreds of datasets to IAM because of the great performance IAM provides. Other users convert VSAM datasets related to selected particular applications that need a performance boost. The point is that you can obtain significant savings by converting a few datasets to IAM then over time gain additional savings by converting additional VSAM files to IAM.

VSAM EXCP REPORT CLUSTER TOTAL SPLITS ALLOC DATA SET NAME EXCPS EXCPS RECORDS READS INSERTS UPDATES DELETES CI CA TRKS -------------------------------- ------- --------- -------- --------- ------- -------- ------- ----- --- ------ BA.VMR.VPMOBE.VMOBE 100 668613807 BA.VMR.VPMOBE.VMOBE.DATA 29 191811021 5453127 279413926 0 0 0 0 0 14790 BA.VMR.VPMOBE.VMOBE.AIX1.DATA 28 187232598 4649193 198669957 0 0 0 0 0 7815 BA.VMR.VPMOBE.VMOBE.INDEX 25 165400609 986 0 0 0 0 0 0 56 BA.VMR.VPMOBE.VMOBE.AIX1.INDEX 19 124169579 4 31874 0 0 0 0 0 25 BA.V01.VPMRCF.VMRCF 100 26361787 BA.V01.VPMRCF.VMRCF.DATA 59 15643582 43897700 370338231 458791 584818 0 13769 22 89925 BA.V01.VPMRCF.VMRCF.INDEX 41 10718205 5953 0 0 13876 0 22 16 297 BA.VMR.VPMDNB.VMDNB 100 25267534 BA.VMR.VPMDNB.VMDNB.DATA 85 21473526 10509731 207289431 0 0 0 0 0 8595 BA.VMR.VPMDNB.VMDNB.INDEX 15 3794008 578 0 0 0 0 0 0 28 BA.V01.VPBRRS.VBRRS 100 23188107 BA.V01.VPBRRS.VBRRS.DATA 98 22666884 52138196 556218577 0 0 0 0 0 113745 BA.V01.VPBRRS.VBRRS.INDEX 2 521223 6171 0 0 0 0 0 0 570 BA.V01.UACXXX.VACRH 100 19365800 BA. V01.UACXXX.VACRH.DATA 100 19365800 901440 32744440 0 10889502 0 0 0 18780


35

One user has reported the following:

“I used IAMSMFVS…to analyze our file usage. It revealed a few small VSAM files had very heavy usage. I found that a very long running job that produces ad-hoc reports pounded those files mercilessly. After converting those files to

IAM, we cut about 1/3 from the clock time. After using IAM’s dynamic tabling, we cut another 1/3 and I am not finished tuning it yet! We have now moved

that batch run out of the daytime window just by converting a few key files to IAM in only 1 job! This one job used to run 4-7 hours and now takes less than 2

hours.

I am now using the IAMSMFVS output to analyze other jobs.”

VSAM Size Report

The VSAM Size Report provides a listing of the 100 VSAM clusters using the most DASD space. This report is of particular value for finding VSAM datasets for conversion to IAM when reducing DASD space usage is desired. Even if space savings is not a priority, you can still potentially find some VSAM datasets that end up saving a lot of DASD space when converted to IAM.

VSAM SIZE REPORT ALLOC TOTAL USE AVG MAX KEY DATA SET NAME TRKS EXCPS COUNT EXTENTS LRECL LRECL LEN RKP CISIZE ----------------------------------..------ ------- ----- ------- ----- ----- -- --- ----- AOPC.VIEWDRT.TPC.T000002.D040101 512850 1132899 2590 AOPC.VIEWDRT.TPC.X000002.D040101 510000 1031785 2590 19 100 100 60 0 18432 AOPC.VIEWDRT.TPC.I000002.D040101 2850 101114 2590 24 408 9 60 0 4096 AOPC.VIEWDRT.TPC.T000001.D040101 361950 990225 3704 AOPC.VIEWDRT.TPC.X000001.D040101 360000 822316 3704 17 100 100 60 0 18432 AOPC.VIEWDRT.TPC.I000001.D040101 1950 167909 3704 12 408 9 60 0 4096 AOPC.VIEWDRT.TPC.T000002.D030101 336960 8041 143 AOPC.VIEWDRT.TPC.X000002.D030101 335310 1781 143 7 100 100 60 0 18432 AOPC.VIEWDRT.TPC.I000002.D030101 1650 6260 143 1 408 9 60 0 4096 AOPC.VIEWDRT.TPC.T000002.D020101 248475 2827 55 AOPC.VIEWDRT.TPC.X000002.D020101 246975 1892 55 6 100 100 60 0 18432 AOPC.VIEWDRT.TPC.I000002.D020101 1500 935 55 1 408 60 0 4096 AOPC.VIEWDRT.TPC.T000001.D030101 239415 5465 308 AOPC.VIEWDRT.TPC.X000001.D030101 238290 3027 308 5 100 100 60 0 18432 AOPC.VIEWDRT.TPC.I000001.D030101 1125 2438 308 1 408 9 60 0 4096 AOPC.VIEWDRT.TPC.T000001.D020101 198885 2503 93 AOPC.VIEWDRT.TPC.X000001.D020101 197760 1636 93 5 100 100 60 0 18432 AOPC.VIEWDRT.TPC.I000001.D020101 1125 867 93 1 408 9 60 0 4096


36

Using the VSAM Size Report

The VSAM Size Report is used to identify the VSAM datasets that can benefit by IAM’s space reduction abilities. Even if your primary objective is to reduce CPU time, you may still be able to take advantage of data compression on some of the larger VSAM datasets. By using the information from the VSAM Size Report you can selectively use IAM data compression on a limited number of datasets that realize the most benefit from data compression. For some of the largest datasets it may be very worthwhile to build a customized hardware compression dictionary, which will likely achieve the most compression possible. While results will vary, we have seen some files use up to 90% less space with a customized dictionary. In the example above, based on the similar dataset names, it is possible that one customized hardware dictionary may work well for all of these files if the data is similar enough.

Summary of IAMSMFVS Reports

The VSAM EXCP report and the VSAM SIZE report are very helpful in identifying VSAM datasets that can be converted to IAM to achieve significant benefits. The benefits include CPU time reductions, I/O reductions, and DASD space savings, along with reductions in elapsed times. To be of most use, it is highly recommended that the reports be run with at least one week’s worth of data, and preferably up to a month’s worth of data. Compuware will be happy to review your IAMSMFVS reports, and provide an analysis report with recommendations for files to convert to IAM, and a spreadsheet of the datasets from the VSAM EXCP report.

Conclusion Summary

IAM is a high performance transparent alternative to VSAM that can easily be used by VSAM application programs. No programming changes and generally no JCL changes are required to use the IAM product for application programs. Typically the only change required is to specify the OWNER($IAM) parameter when defining the dataset. IAM’s high performance comes by reducing the processing resources to utilize VSAM to read, write, and update data records. IAM reduces the quantity of physical I/O’s being performed, reduce CPU time being used and reduce DASD space used that yields reductions in elapsed times for batch jobs and reductions in response times for online transactions. In addition to being better able to meet processing time requirements, IAM can provide real dollar savings by delaying the need for additional hardware or by reducing costs of outsourcing the processing.

The benchmark data has shown the potential for savings in several aspects of application use of VSAM, and customer experience has confirmed the savings and value of the IAM product. Here are a couple of additional user comments:


37

“I wanted to let you know that our applications people are thrilled with your product. We have converted about 40 files from our financial products and have seen wonderful reductions in batch run times. For example, 1 job that

ran for 1 1/2 hours before IAM is now completing in 10 minutes!!!”

Typical savings from using IAM fall into the following ranges:

20% - 70% Reduction in CPU usage

40% - 80% Reduction in I/O (EXCPs)

30% - 70% Reduction in DASD space usage

20% - 80% Reduction in Batch Elapsed Times

20% - 80% Reduction in CICS Transaction Response Times

Find out today if IAM can help you reduce your z/OS processing costs by running the free FDRQUERY and IAMSMFVS tools from Compuware.

how iam significantly reduces vsam processing time · as to how much vsam dataset processing is...

Documents