ukcmg 2011 mxg bof may 16, 2011 thame, oxfordshire
TRANSCRIPT
UKCMG 2011
MXG BOFMay 16, 2011
Thame, Oxfordshire
MXG Version 29.04 dated May 17, 2011.
UPDATES TO MXGZ/OS 1.12 SupportCICS/TS 4.2 SupportZ196 SupportDB2 V10 Support EXITCICS COMPRESSED CICS AND DB2TYPE113 SupportVMXGSRCHVMXGFINDSAS 9.2 WINDOWS SEVEN AND MSEANALDUPEJCLSIMPL/JCLSPSMx/BLDSIMPL/BLDSPSMx
1. New stuff in MXG 28.01, Mar 9, 2010:
VMXGFIND 28.012 Kewl tool, find all obs in all datasets meeting,
a test (all obs with JOB=‘CICS’ in all PDB.
TYPESTC 28.005 Support for Sun StorageTek VSM Version 6.2/7.0
TYPE89 28.015 Support for z/TPM SMF 89 record, subtype wrong.
TYPENTSM 28.042 New Sentry VM 3.1.4.3 adds VMWARE objects/metrics.
TYPE30 28.031 z/OS 1.11 GA added variables to SMF 30 and SMF 71.
TYPEZTPF 28.043* zTPF has major revisions in Performance Data
TYPETMS 28.040* CA-1 Retention and VMRECORD extensions.
Changes:
TYPE74 28.039 R7451RID now one byte, R7451FLG/TYPE74CA overlays.
BUILDPDB 28.037 PDB.SMFINTRV have EXCP/IOTM counts for FLUSHED.
TYPE103 28.036 TYPE1032 deaccum needed PORTNR, label changed.
VGETOBS 28.034 %TRIM() references here removed, still in VMXGSUM.
IMACICMR 28.032 Protect 200-byte CMRDATA on CICS/TS 3.2 (s/b 256).
VGETDDS 28.014 Colon in DDNAMES= worked only with DDNAMES=PDB:)
TYPEDB2 28.010 Variable SHIFT (from QWHSSTCK, END) kept.
2. Major enhancements added in MXG 28.02, dated Apr 14, 2010
VMXGINIT 28.081 VARLENCHK=NOWARN eliminates SAS V9.2 WARNINGS
VMXGINIT 28.057 ERROR MACRO %ABORT IS NOT IN SAS 8.2, ONLY.
TYPE84 28.077 Support for all JES3 JMF TYPE84 subtypes.
ASMIMSL6 28.066 Support for IMS Version 11 (INCOMPATIBLE).
TYPEIMS7 28.066 Support for IMS Version 11 (INCOMPATIBLE).
TYPEMVCI 28.065 Support for BMC CICS CMRDETL C660 for CICS/TS 4.1
TYPEDB2 28.051 Support for DB2 APAR PK62161 new SQL Counters.
TYPETMNT 28.079 TOTAL LENGTH OF VARIABLES MUST BE LT 32760.
TYPEDB2 28.073 DB2STATS had missing values for QW0225 variables.
TYPE42 28.072 TYPE42X4 Above the BAR LRU variables wrong.
BUILDPDB 28.071 PDB.STEPS/PDB.JOBS duplicates if FLUSHED steps.
MXGSAS92 28.070 SAS 9.2 TS2M2 DSNAMES may have changed
TYPERMVV 28.048 RMFV CPUG3 was misaligned in z/OS 1.1122.
3.
BLDSMPDB 28.125 Support for Week/Month if 1st-day-of-week NOT MON.
CONFIGVx 28.128 ERROR APPARENT MACRO TRIM NOT RESOVED: MORE DOC.
VMXGSRCH 28.147 Kewl Tool. Find all instances of VARIABLE='VALUE'.
BUILDPDB 28.139 Recently added SMF30xxx vars kept in PDB.STEPS.
Major enhancements added in MXG 28.03, dated May 25, 2010
TYPEWPMO 28.086 Support for Windows Performance Monitor PERFMON.
TYPECTCP 28.108 Support for CleverView for TCP/IP TN3270 SMF.
TYPE80A 28.107 TOKDANAM BINDDN,BINDPW,APPLNAM,UTYPE,JPNUM.
VMXGSUM 28.105 Optional KEEPWEEK/MNTH/YEAR/DAYS/ keeps TRENDs.
TYPE7072 28.099 Variable CPULHKTM, CPU TIME Lock Promoted, 72GO.
VMXGSET 28.098 DSETOPT= optional argument for data set options.
SMFRECNT 28.089 BUILDPDBs PDB.SMFRECNT now has bytes and counts.
TYPE110 28.087 Internal Decompression Algorithm use now ERROR:'d.
TYPECIMS 28.084 BMC IMF INPUTCLS and LASTCLAS variables restored.
4.
Many 28.175 Support for z/OS 1.12 (REQUIRES MXG 28.05).
DO NOT USE MXG 28.04 WITH z/OS 1.12 DATA INCOMPAT.
TYPE70 28.175 Support z196: REQUIRED ONLY WITH GT 64 ENGINES.
(Lots of new data added compatibly.)
TYPE113 28.166 Major revision - TYPE113 - John Burg's SHARE 2010.
Calculation of RNI, new z196 fields, new metrics.
TYPE119 28.175 Support for SMF 119 new subtypes 32-37 and 48-52.
TYPEITRF 28.162 Support for ITRF V420 IF2 (COMPATIBLE).
TYPECTCP 28.160 Support for CleverView GMT offset, CTCPIPAD fixed.
TYPE42 28.158 Support for APAR OA31648 TYPE42D1/D3 buff gets
TYPEVM 28.157 Support for VM ACCOUNT ID='09' ISFC record.
TYPE102 28.156 Support for IFCID=27 specific variables.
TYPENMON 28.176 Support for SARMON - Solaris SAR in NMON format.
IMACCADI 28.172 Support for CA-Dispatch V11 SMF 6 INCOMPATIBLE.
TYPETPFC 28.152 Support for zTPFC TPF Continuous Monitoring .
TYPEZCOS 28.151 Support for zCost AutoSoftcapping V 1.5.00 SMF.
UTILPDSL 28.179 Utility to read PDS/PDSE directories of a concat.
IMACZDAT 28.174 Example to set ZDATE when you rebuild a past PDB.
5. Major enhancements added in MXG 28.05, dated Aug 18, 2010
The z196 processor with more than 64 engines REQUIRES MXG 28.05.A z196 with LESS THAN 64 engines DOES NOT require MXG 28.05, aslong as the operating system is z/OS 1.11 or earlier.IBM Maintenance APARs OA30563,OA33976 REQUIRES MXG 28.05.
Major enhancements added in MXG 28.06, dated Oct 7, 2010
TYPEWSMQ 28.233 Support for WebSphere MQ Version 7 Accounting
TYPEDB2 28.222 ITRM only, DB2STAT4 NOT SORTED ERROR.
ASUMDB2- 28.220 DB2 Summary ASUMDBxx and Trending TRNDDBxx.
EXITCICS 28.223 Support for DB2 V10 Compressed SMF records.
DFH$MOLS 28.223 JCL example to use IBM CICS decomp utility.
TYPEITRF 28.227 INPUT STATEMENT EXCEEDED RECORD LENGTH type=17x.
TYPEIMFS 28.193 Full support for IMF records in SMF format.
TYPE113 28.226 Variable LPBUSY,LPARBUSY replaced LPARCPU. m
TYPE74 28.212 TYPE74ID (small) created, saves pass TYPE74CA.
6. MXG Version 28.07 dated Nov 5, 2010.
Major enhancements added in MXG 28.07, dated Nov 5, 2010
TYPEDB2 28.264 Support for DB2 Version 10. COMPLETELY INCOMPAT:
MXG 28.06 was required to process the V10 data,
now, MXG 28.07 has full support.
TYPEOMMQ 28.263 Support for IBM/OMEGAMON XW MQ file (INCOMPAT)
TYPEMIM 28.262 Support for CA MIM RESOURCE SHARE R11.7 (COMPAT)
GRAFCEC 28.261 SAS/GRAPH example charts CEC Util by engine
TYPETPX 28.260 IP address and Port Number now decoded in TPX
UTILEXCL 28.259 Spurious "WRONG LENGTH OF 200 FOR CMRDATA"
TYPE120 28.258 Support for WebSphere ID=120 SUBTYPE=20 records
ANALID 28.257 ERROR: VARIABLE IDANDSUM ... with PDB,DISP=OLD
READDB2 28.250 COPYONLY logic now works.
VMXGSUM 28.249 VMXGSUM enhanced with MODE and MEDIAN statistics
TYPE110 28.247 Example using _Kdddddd to create new datasets
TYPE30 28.246 New CPITCxTM/CPISRxTM wrong in MXG 28.06.
TYPESTC 28.244 STC/STK/Oracle VSM user SMF records support
7. MXG Version 28.08 dated Dec 13, 2010, thru Change 28.288.
TYPE113 28.279 "Near duplicate" ASUM113 intervals corrected.
TYPE89 28,282 Support for APAR OA31615, zIIP/zAAP CPU times.
TYPENMON 28.275 Support for NMON FCREAD/FCWRITE/XFERIN/XFEROUT
TYPETNG 28.273 Support for more than 9999 instances in CA NSM.
TYPETMVT 28.287 Support for ASG-TMON for VTAM subtype 'SX'.
TYPE110 28.285 CICS Statistics Subtype 2 STID=143 corrected.
ASUMUOWT 28.284 ASUMUOWT (for ASG-TMON MRO) uses VMXGUOW.
ASUMCICR 28.281 Count/avg response time by DATE for each APPLID.
DB2ACCT 28.277 NETSNAME/UOWTIME only created QWHCATYP=4 (CICS).
TYPE89 28.272 SMF89HOF/SMF89DTO SCRT don't use last 3 nybbles.
WEEKBLD 28.269 TYPE72DL NOT SORTED after Clock Set Back.
UTILNPRT 28.268 Identify non-print chars, SAS Enterprise Guide
8. MXG Version 28.28 dated Jan 18, 2011, thru Change 28.331.
TYPE89 28.331 INVALID DATA FOR SMF89CZT if APAR OA31615 on.
TYPE111 28.329 CTG records had zero obs in TY111CXI "IPIC".
JCLINSTT 28.328 JCL example to ftp/unterse/install on z/OS.
TYPENDM 28.327 Connect Direct/NDM 'RT' record INCOMPAT change.
TYPE102 28.325 DB2 SQL-text vars only 100 bytes COMPRESS=NO.
TYPEIMSA 28.311 Support for IMS/DBCTL transactions in IMSTRAN.
TYPEIMS7 28.310 Support for IMS/DBCTL transactions in IMS0708.
TYPE0 28.313 Variable CVTTZ in TYPE0 could be one sec wrong.
BUILDPDB 28.305 PDB.NJEPURGE did not contain all NJE-variables.
ANALDUPE 28.308 Removal of Duplicate SMF (or any) records.
TYPEVMXA 28.315 PFXCPUAD in VXSYTCUM is the LCPUADDR, no CPUID.
TYPEVMXA 28.307 Short LINUXKRNL MONWRITE record caused errors.
UTILGETM 28.312 No Reporting Class data in SMFSMALL NRECORD=10.
TYPE89 28.304 SMF 89 with no usage segment INPUT EXCEEDED.
TYPE30 28.302 TYPE30MU duplicate records exist, non-dupes.
9. MXG Version 29.01 dated Feb 4, 2011, thru Change 29.022.
These two impacted MONTHLY build:
MONTHBLD 29.017 SERIOUS ERROR CORRECTED: last day's PDB skipped.
BLDSMPDB 29.017 LIBNAME WEEK1 not found corrected.
These two eliminate possibility of NOTSORTED errors:
BLDSMPDB 29.008 SORTEDBY=NO default to eliminate NOTSORTED.
WEEKBLD 29.008 MXGNOBY default to eliminate NOTSORTED exposure.
MONTHBLD 29.008 MXGNOBY default to eliminate NOTSORTED exposure.
TYPEENDV 29.012 Support for Endeavor Version 14 (INCOMPATIBLE).
TYPE111 29.001 Support for IPIC creates obs in TY111CXI.
TYPE115 29.015 Support for MQ Version 7 compression statistics.
TYPE89 29.002 Support for APAR OA31615, zIIP/zAAP times added,
and false error messages are eliminated..
10. MXG Version 29.02 dated Mar 1, 2011, thru Change 29.050.
VSETMNTH 29.041 POSSIBLE LOSS OF MON DATA IN FEB MONTHLY PDB. (Unfortunately, EVEN with the newest MXG 29.01). ONLY IF USING NEW VSETMTH LOGIC IN MONTHBLD. Future MONTHly's can also be missing one or more day's PDBs in your MONTH PDB. Complete details are in Change 28.041, below.
TYPENDM 29.042 Support for NDM-CDI Version 5 records (COMPAT).VMACDB2H 29.037 DB2 V9.1 false "INVALID DISTRIBUTED HEADER" msg.TYPE30 29.034 Invalid data for SMF30RGT is true, circumvented.TYPECIMS 29.033 Support for IMF Version 4.5 is in place.TYPE0 29.032 PDB.IPLS, now, DOES always report a SYSTEM IPL.TYPEDB2 29.031 DB2 V9.2 only, QBGxxx variables DB2GBPST wrong.TYPEVMXA 29.026 Support for zVM APAR VM64794 (COMPATIBLE).TYPE30 29.025 Small negative CPUUNITS now set to zero.TYPE26J2 29.024 Cosmetic: INREASON NOT DECODED msgs corrected.
11. MXG Version 29.03 dated Apr 19, 2011, thru Change 29.094.
TYPE110 29.094 1st MXG 29.03 ONLY. CICSTRAN CPUTM fields WRONG.TYPE116 29.057 Support for Websphere for z/OS MQ Version 7.0.1.TYPE115 29.057 Support for Websphere for z/OS MQ Version 7.0.1.TYPEBBMQ 29.056 Support for MainView MQ (MVMQ) Version 4.4.TYPEQACS 29.078 Support for OS/400, AS/400 Version 7.1 (INCOMPAT)TYPE110 29.076 CICS CPUTM exceeds ELAPSTM, zAAP/zIIP EquivalentTYPE120 29.081 Support for User Field in SMF 120 Subtype 9.TYPETPMX 29.071 Support for Throughput Manager subtype 10 and 16.TYPENTSM 29.075 Support for 62 new objects and 1425 new metrics.UTILVREF 29.075 MXG creates DATASET names up to 32 characters.BUILDPDB 29.068 28.28-29.02. ABEND=JCL obs missing in PDB.JOBS.TYPERACF 29.067 RACF UNLOAD dataset RACF0270 UID limit variables.TYPEBETA 29.059 Support for Beta 93 Version 4.2 subtypes 25/50.TYPE30 29.058 Variable CPUCEPTM always a missing value.MONTHxxx 29.052 SAS 9.1.3 Only. %QCMPRES needed versus %CMPRES.TYPE85 29.093 INPUT STATEMENT EXCEEDED st 79, z/OS 1.12.ASUM70PR 29.092 ZIPCPUS/IFACPUS included parked time.TYPEVMXA 29.092 z/VM new PDB.VXINTUSR sums all engines each VM.
12. MXG Version 29.04 dated May 17, 2011, thru Change 29.115.
TYPE105 29.100 Support for GDPS Global Mirror V3R8 SMF 105.
DB2ACCT 29.111 DB2 CICS TRAN name wrong, now from QWHCCV.
TYPEIMSA 29.110 the exit _IMSTRN was accidentally removed.
BUIL3005 29.106 JES3 PDB.JOBS variable JOBCLAS8 after change.
VMXGSRCH 29.103 RESULTS=FINDVAR finds all datasets with a var.
TYPE70PR 29.098 Counts ICFCPUS/IFLCPUS/IFACPUS/ZIPCPUS too high.
TYPE110 29.097 INPUT EXCEEDED 110-2 MNSEGLC=5 with DPL segment
13. VMXGFIND
EXAMPLES: 1. FIND ALL OBSERVATIONS FOR A JOB NAME: %VMXGFIND(PDB=PDB SPIN, PDBOUT=WORK, KEEPIN=JOB, FIND= IF JOB=:'CICS'; , PRINT=99); 2. FIND ALL OBSERVATIONS FROM/FOR A SPECIFIC USER FROM PDB1-PDBN AND SPIN LIBRARIES: %VMXGFIND(PDB=PDB: SPIN, PDBOUT=WORK, KEEPIN=FSRUID JOB RACFUSER QWHCAID, FIND=IF RACFUSER='XYZ12345' OR QWHCAID ='XYZ12345' OR FSRUID ='XYZ12345' OR JOB ='XYZ12345';, PRINT=YES);
14. VMXGSRCH EXAMPLES: TO FIND ALL OBSERVATIONS IN ALL SAS DATASETS IN THE PDB LIBRARY THAT HAVE ANY VARIABLE THAT CONTAINS "SYSD", AND PRINT THE FIRST TEN OBSERVATIONS OF EVERY FOUND DATASET:
%VMXGSRCH(LIBNAME=PDB,RESULTS=PRINT,VALUE=SYSD,NOBS=10);
TO FIND ALL VARIABLES IN ALL ALLOCATED DATA LIBRARIES WITH THE TEXT "CPU" IN THE LABEL
%VMXGSRCH(VALUE=CPU,RESULTS=LABEL);
WILL REPORT EVERY SAS DATASET AND VARIABLE IN THOSE "PDBS" THAT HAVE CPU IN THEIR LABEL. TO FIND ALL VARIABLES WITH TIME IN THE FORMAT (WILL ALSO FIND DATETIME)
%VMXGSRCH(VALUE=TIME,RESULTS=FORMAT);
WILL REPORT EVERY SAS DATASETS/VARIABLES WITH TIME IN THE FORMAT
15. CICS Record Size GrowthCICS DATA-LENGTH
NUMBER-FIELDS MXG
TS RELEASE MCTSSDRL MCTSSDCN VERSION1.6.1 176 541.7 312 602.1 320 613.1 380 703.2.1 388 71 9.93.3.0 444 83 10.014.1.0 572 107 13.094.1.1 852 165 13.09
1.1 5.1.0 664 124 14.071.2 5.2.0 732 134 15.061.3 5.3.0 1288 203 16.092.1 6.1.0 1564 230 18.112.2 6.2.0 1620 239 19.082.3 6.3.0 1692 252 21.043.1 7.1.0 1848 286 22.123.2 7.2.0.5 2352 307 25.034.1 7.3.0.4 2512 317 26.14.1 7.3.0.5 2640 330 27.01
16. Assemble and Install the EXITCICS
THIS ASM PROGRAM IS ASSEMBLED AND LINK-EDITED INTO A LOAD MODULE
THAT IS NAMED CICSIFUE THAT IS STORED IN A LOAD LIBRARY DATASET
THAT IS CONCATENATED TO THE //STEPLIB DD STATEMENT IN YOUR MXGSASVN
JCL PROCEDURE. THE LOAD MODULE IS A SAS INFILE EXIT.
1) %LET SMFEXIT=CICS;
2) FILENAME SMF 'dsname of dataset' ENGINE=CICS;
DATA X; INFILE SMF; INPUT ...
3) DATA X; INFILE SMF CICS; INPUT ....
THAT FIRST %LET STATMENT CAN BE PLACED IN THE "IMACKEEP" MEMBER
IN YOUR USERID.SOURCLIB TAILORING LIBRARY, OR IT CAN BE PLACED IN
THE //SYSIN DD STREAM AS THE FIRST STATEMENT..
EXITCICS decompresses CICS 110, 112, and DB2 100, 101, and 102 records.
17. The INTERNAL SAS algorithm on z/OS is VERY CPU intensive
(and it takes a long time, too!). DFH$MOLS and read UNCOMP
only slightly slower than reading COMPRESSED+EXIT, BUT
uncompressed file needs nearly 5 times the disk space.
I/O activity with DFH$MOLS (read comp, write uncomp, read
took six times the EXCPs and four times the IOTM (Connect Time)
Reading of the compressed file with the EXITCICS exit is best.
On Windows/ascii platforms, SAS Infile Exits do not exist.
Only two ways to process compressed CICS data on ASCII:
a. Use DFH$MOLS first to uncomp file and read UNCOMP.
c. Use MXG's internal algorithm to read COMP NO EXIT.
Elapsed User SYS Size
a. DFH$MOLS .4 .07 .00 212/970
ftp download 2.0 .04 .00 970 MB
UNCOMPRESSED .4 .23 .05 970 MB
total 2.8 .34 .05
c. ftp download 0.5 .01 .00 212 MB
INTERNAL SAS 3.8 2.71 .05 212 MB
total 4.3 2.72 .05
18. SAS Version Requirements information:
MXG 29.04 executes best with SAS V9.2, or with SAS V9.1.3 with
Service Pack 4, on any supported SAS platform.
And, only for z/OS 1.10 with SAS V9.1.3 with ANY version of MXG,
the SAS Hot Fix for SN-35332 is REQUIRED (to be completely safe).
Old MXG code may continue to execute with SAS V8.2, but V8 is now
"Level B" support from SAS Institute, and there are known errors
in V8.2 that are only fixed in SAS V9. I no longer QA with V8.2;
While many MXG programs (accidentally) will still execute under
V8.2, I can not guarantee that all of MXG executes error free.
19. Install of SAS V9.2 for z/OS is rumored to be difficult,
BUT BY following the SAS Install Instructions at:
http://preview.tinyurl.com/443mlpd
upload and install of the (7.5 GB) z/OS SAS/BASE took 2 hours
BUT ONLY if everything needed is already in place!).
Size of the depot between 5GB and 17GB depending on products
SAS Depot will be uploaded to a ZFS filesystem
z/OS restriction of 4GB for size of normal zFS mount point
Need authority to define a larger zFS mount point
Either must be created with the SMS Data Class EXTENDED attribute.
Need two zFS directories of 5GB to 17GB because 2nd SASHOME
But, one last note, do NOT use that EXTENDED attribute for the Data
Class of your SAS Data Libraries on z/OS; it is not supported due
to SAS's use of the EXCP access method.
20. Using MicroSoft Security Essential, MSE, causes various errors the MXG QA job is run, never at the same place in the job:
ERROR: File Deletion Failed For MXGSUM1 (after 293 succeeded)
ERROR: An I/O error occurred on file WORK._tf6737.UTILITY
ERROR: Rename of temporary member WORK.OPTVAR.DATA failed.
BUT BY disabling MSE Settings to: -exclude the SAS.EXE process, and -exclude files *.sas7bdat and
-exclude the c:\qa directory, where all output is written, & -exclude the c:\sastemp "WORK" directory.
were both errors were eliminated, with WINDOWS 7 64 bit.
“SAS NOTE: Do not run your antivirus software in real-time mode.
OR be sure to exclude the following file types from your
virus scan: SAS* LCK* UTL* “
21. Issue open with SAS Institute and Microsoft since Feb, 2010; In October, a new MicroSoft "Senior Escalation Engineer" attempted resolution, providing instructions to install several MicroSoft diagnostics tools that either failed to initialize or failed to capture the event data, including runs with TTTracer that generated over 85 GigaBytes of trace (how do you send a file that big??)
that still did not capture anything of use to MS.
Nov 2010: 14 MicroSoft updates auto installed, error went away.
MS engineer was unable to identify why the error disappeared.
Moral: Disable MSE for SAS.
MAY 2011: WINDOWS 7 ULTIMATE AUTOINSTALL SP1 REINSTATED ERRORS.
DISABLING MSE REAL TIME SCAN WAS REQUIRED TO RUN QA.
USING MSE SETTINGS (process/dir/file) DID NOT CIRCUMVENT.
ADDITIONALLY: RUN SAS AS ASMINISTRATOR MAY BE REQUIRED WITH SP1.
ERROR: User does not have appropriate authorization level for library C:\PROGRAM FILES\SAS\SASFOUNDATION\9.2
(to write qasas.log and qasas.lst files).
22. Removal of duplicate (SMF) records on z/OS - new ANALDUPE.
500,000 SMF Records Processed
Several techniques for removal of duplicate SMF records on z/OS
are provided in the new ANALDUPE member. Two approaches are both
sort-based but are limited by requirements for MASSIVE amounts of
disk space or tape drives and require more CPU time than the two
elegant solutions created by MP Welch, who imagined a solution and
discovered that the SAS V9 MD5 (digital signature) function could
be used to create a unique Hash Value for each record, and the Hash
Values are then sorted (instead of the full record), to MASSIVELY
reduce the disk and CPU requirements. A one-pass solution using
a hash table works fine, but could rapidly exhaust virtual storage,
so the recommended solution creates the MD5 Hash Value, but then
uses a second step (freeing temp space of the first step) and a SAS
Format for the look up table to remove duplicates.
23.
SORT ONE SORT TWO SORT THREE SORT FOUR DISK BASED TAPE BASED MD5 HASH FUNCTION MD5 HASH 29 SORTWRK 7 TAPE DRIVES HASH TABLE SAS FORMAT 1000 CYL REQUIRED ONE PASS TWO STEP
CPU 41.40 SEC 49.80 SEC 16.2 SEC 16.2 SECSRB 3.60 SEC 6.60 SEC 0.6 SEC 1.2 SECEXT 31,780 K 31,776 K 43,848 K 50,284 KSYS 11,860 K 11,864 K 11,884 K 12,060 KEXCP 484,000 463,000 84,000 126,000CONN 27.052 SEC 20.40 SEC 15.00 SEC 18.00 SECCLOCK 648.00 SEC 1380.00 SEC 18.00 SEC 42.00 SEC
Clearly it's much more efficient to hash a record and operate on ashorter value than operating on the full record itself. In thiscase, it works particularly well because there is no expectationnor requirement to reorder the records. The Hash table filled 2GBof memory at 3.5 million unique records. But the two pass hash willhandle hundreds of millions of records in most shops.
24.
JCLSIMPL and JCLSPxxx examples use UTILBLDP/BLDSMPDB and
are THE now-recommended z/OS jobs for a "SIMPLE" BUILDPDB
or the "SPLIT SMF" family of "BUILDPDB" jobs.
JCLSIMPL creates a "simple", PDB library, with one job
that reads the SMF file, showing how to add an SMF record
and invoking all of the default ASUMxxxx members to build
a "single", default PDB data library from raw SMF data.
You could do the same with BUILDPDB and the EXPDBxxx exit
members, but these more recent utility macros are now the
recommended way to build/tailor a simple BUILDPDB:
UTILBLDP - defines what data is to be created in a PDB
you can add, subtract, or change what's kept
by each of these jobs using UTILBLDP to create
a specific suit of MXG datasets in a PDB
built from SMF data records.
BLDSMPDB - flexible job manager creates day/week/etc
PDBs using the UTILBLDP execution preceding
its invocation to define the PDB contents.
Will process non-SMF data records.
25.
JCLSPxxx is a family of jobs to read "split" subsets of
SMF and other data records to parallelize the BUILDPDB,
using the above+ UTILBLDP and BLDSMPDB members:
JCLSPGDG - run once to create GDGs, and then never again
unless there is a need to alter a GDG base or
to change dataset names..
JCLSPLIT - first job in daily stream - standalone -
splits the daily SMF into pieces for
subsequent processing
SMF.ALL - All SMF for archive
SMF.CICS - SMF 110.1
SMF.DB2 - SMF 101/102
SMF.IO - SMF 14/15/42/61/65/66/74/240/241
SMF.MQ - SMF 115/116
SMF.SPLITPDB - All other SMF records
JCLSPSMA/JCLSPSMB/JCLSPSMC/JCLSPSMD/JCLSPSME can be run
concurrently to process the split SMF files:
26. Family of jobs to read the SPLIT SMF files:
JCLSPSMA - Read only CICS SMF 110.1, create:
JCLSPSMB - Read only DB2 SMF 101/102, create:
JCLSPSMC - Read only I/O records, create:
JCLSPSMD - Read only MQ records, create:
JCLSPSME - Read all remaining SMF, create:
JCLSPOTH - DCOLLECT, TMC.
JCLSPUOW - after JCLSPLTA and JCLSPLTB have run,
build PDB.ASUMUOW from CICSTRAN and DB2ACCT,
build PDB.CICS from PDB.ASUMUOW.
JCLSPCPY - Copies these datasets into PDB library:
ASUMCACH CICS ASUMUOW ASUMDB:
27. SAS Paper
Configuration and Tuning Guidelines for SAS®9 in the
Microsoft Windows Server 2008 Operating Environment
The monitoring tool that SAS recommends is the Performance Monitor (PerfMon) tool from Microsoft that comes with the Windows operating system. Hundreds of performance counters are associated with this tool, but from a SAS perspective, only the following subset of PerfMon objects and their associated counters is of primary interest:
Cache– o Data Flush Pages/sec– o Dirty Pages– o Fast Read Not Possible/sec– o Fast Read Resource Misses/sec– o Lazy Write Pages/sec– o Read Aheads/sec
Client-Side Caching– o Application Bytes Read from Cache– o Application Bytes Read from Server (Not Cached)
And Many more in the paper at:
http://support.sas.com/resources/papers/
WindowsServer2008ConfigurationandTuning.pdf
