CALCULATION OF COST OF NESTED LOOP JOINS

Some times while analyzing the SQL for tuning you might have come across scenarios whereyou feel that CBO is not making the optimized plan as it should be and you have tried all thepossible combinations of tuning the database or the SQL queries. In that scenario there are twooptions left at the performance engineer end:

1. Use hints to force optimizer to use your desired plan.2. Analyze why CBO is making different plan and what all changes are required to put CBO in

a situation so that it picks up the optimized plan on its own.

However the first option seems pretty simple but that is going to impact the performance ofthose query at different data load conditions. Suppose you have tuned it for high-end datacondition but it may not be OK for low-end data or the case may be otherwise also. So this typeof fix is recommended only in the scenario in which you are left with no other option.

Second option is to take the dump of the optimizer plans and find out how it is making the plan.What all can be changed or added so as to pick up a much more efficient and desirable plan.For that there is some detailed analysis required.

This paper will be focused on this second attribute of analysis in terms of how CBO calculatesthe cost for SQL plans so that you can make the changes that further lead optimizer to pick up adifferent plan. In this sense you will be able to find out which all things are affecting the CBOcost calculations and which can suitably be changed to make sure that optimizer is going to pickup the correct plan.

PREREQUISITES

The one of the most important prerequisites for this process is that you are using the CBO andfor CBO to calculate the cost of your plan it is essential that all your tables, indexes are wellanalyzed so as to have the accurate information in the dictionary views.

Further more to analyze a particular statement following things needs to be done:

ALTER SESSION SET SQL_TRACE=TRUE;ALTER SESSION SET EVENTS '10053 TRACE NAME CONTEXT FOREVER, LEVEL 1';

You just don’t see any of the plans which “lost out”. Unless you activate the 10053 event trace,that is. There you see all the access plans the CBO evaluated and the costs assigned to them.Event 10053 details the choices made by the CBO in evaluating the execution path for a query.It externalizes most of the information that the optimizer uses in generating a plan for a query.

HOW TO SET EVENT 10053

1. For your own sessionALTER SESSION SET EVENTS '10053 trace name context forever [, level {1|2}]'ALTER SESSION SET EVENTS '10053 trace name context off'

2. For another sessionSYS.DBMS_SYSTEM.SET_EV (<sid>, <serial#>, 10053, {1|2}, '')SYS.DBMS_SYSTEM.SET_EV (<sid>, <serial#>, 10053, 0, '')

Unlike other events, where higher levels mean more detail, the 10053-event trace at level 2produces less detail than the trace at level 1. Like the sql_trace (a.k.a. 10046 event trace), the10053-event trace is written to user_dump_dest.The trace is only generated if it the query is parsed by the cost based optimizer. Note that thisentails two conditions: the query must be (hard) parsed and it must be parsed by the CBO. If thesession, for which the 10053 trace has been enabled, is executing only sql that is alreadyparsed and is being reused, no trace is produced. Likewise, if the sql statement is parsed by therule based optimizer, trace output will consist of only the sql query, none of the otherinformation.

TRACE CONTENTS

The trace consists of 6 sections:• Query• Parameters used by the optimizer• Base Statistical Information• Base Table Access Cost• General Plans• Recosting for special features

INTERPRETATION OF TRACE FILES

Query

This part of trace contains SQL you want to trace out. But if the Query is parsed using RuleBased Optimizer then this trace file is going to be finished here only. Rule based optimizer willbe used in the following cases:

� Explicit Rule hint is provided� Optimizer_mode or optimizer_goal are set to rule� No dictionary information are present that means no tables are analyzed.

PARSING IN CURSOR #1 len=69 dep=0 uid=82 oct=42 lid=82 tim=1021693927016571 hv=552079412ad='308995f8'ALTER SESSION SET EVENTS '10053 trace name context forever, level 1'END OF STMTPARSE #1:c=0,e=126,p=0,cr=0,cu=0,mis=0,r=0,dep=0,og=4,tim=1021693927016559EXEC #1:c=0,e=176,p=0,cr=0,cu=0,mis=0,r=0,dep=0,og=4,tim=1021693927016870*** 2003-02-25 18:10:58.251QUERYexplain plan set statement_id = 'Q' forSELECT OCT.OPO_OMRPLANID, OCT.ITR_REQUIREMENT_NO, OCT.ITR_VERSION_NO, ITM.DTY_DECLTYPE_FK, ITR.LBL_BILLNO_FK, ITM.TACC_IND, ITR.COMMCLR_INDFROM OP_CURPLAN_TRANSIMPREQS OCT, OP_TRANSIMP_MOVES ITM, OP_TRANSIMP_REQS ITRWHERE OCT.OPO_OMRPLANID=:B1 AND OCT.ITR_VERSION_NO=ITM.ITR_VERSION_NO AND OCT.ITR_REQUIREMENT_NO=ITM.ITR_REQUIREMENT_NO AND ITR.VERSION_NO=ITM.ITR_VERSION_NO AND ITR.REQUIREMENT_NO=ITM.ITR_REQUIREMENT_NO

Terminology Used

Parsing in Cursor Parse/ Execlen Length of Querydep Recursive Depth of Queryuid User ID parsing the Queryoctlidtim End Time taken to parse the queryhv Hash Value of Queryad

C CPU TimeE Elapsed TimeP Parse TimeCr Blocks Read in Consistent ReadsCu Blocks Read in Current ModeMis Library Cache MissesRDep Recursive DepthOgTim Time taken to Parse or Execute the query

DESCRIPTION OF TABLES

OP_CURPLAN_TRANSIMPREQS OP_TRANSIMP_MOVES OP_TRANSIMP_REQS

ITR_REQUIREMENT_NO ITR_VERSION_NO OPO_OMRPLANID CURRENT_IND

ITR_REQUIREMENT_NOITR_VERSION_NOITEM_NOMOVE_SEQNOMOVE_TYPEDTY_DECLTYPE_FKPREFERRED_MODECREATEDBYCREATE_DATETMLASTUPDATEDBYLASTUPDATE_DATETMFAC_FROMFACILITY_FKFAC_TOFACILITY_FKFAC_DECLENDFCLTY_FKCUS_DECLARANTCODE_FKDECLARANT_NAMECOMBINE_DECL_INDBONDED_CARRIER_IDAPP_DATETMEARL_PICKUP_DATETMPRV_DECL_NOCRSNGTPT_MEANS_IDCRSNGTPT_MEANS_NATRELEASE_PARTY_CODERELEASE_PARTY_NAMEDECL_REFRCC_PRVCUSTOFFCODE_FKPRV_CUSTOMSOFF_NAMERCC_DEPCUSTOMSOFFCODE_FKDEP_CUSTOMSOFF_NAMERCC_DESCUSTOMSOFFCODE_FKDES_CUSTOMSOFF_NAMEDECL_CNTRY_DESTDECL_CNTRY_DSPTCHDECL_CNSG_NAMEDECL_CNSG_ADDR1DECL_CNSG_ADDR2DECL_CNSG_ADDR3DECL_CNSG_ADDR4POSTCODECNT_COUNTRYCODE_FKCONTACT_PERSONJOBTITLEPHONE_NODECL_REMARK_1DECL_REMARK_2DECL_REMARK_3DECL_INSTR_NODECL_INSTRITEM_NODECLARATION_NODECL_STATUS_CODETACC_INDDECL_INSTR_STATUSLAST_ISSCANC_BYLAST_ISSCANC_DATETMTACC_UPDATEDBYTACC_UPDATED_DATETMDECLARANTSEPARATE_PICKUP_DATETMCUSTOFF_DEST_CNTRYCUST_PICK_DEL_REF

REQUIREMENT_NOVERSION_NOGAT_ACTIVITYTYPE_FKLOC_DISCHARGEPORT_FKLBL_BILLNO_FKLOC_PLD_FKOST_ORGAREAID_FKIMPHAULAGESHIPMENT_TYPEVADCD_INDCREATEDBYCREATE_DATETMLASTUPDATEDBYLASTUPDATE_DATETMEQM_EQUIPMENTNO_FKROT_OWNINGTEAM_FKSHO_SHIPMENTOWNER_FKLOC_INTERMEDIATELOC_FKRESTITUTION_LOCATIONWHARFAGE_CLR_NOCUS_CONSIGNEE_FKCONSIGNEE_NAMECUS_CNPARTY1_FKCNPARTY1_NAMECUS_CNPARTY2_FKCNPARTY2_NAMEDEST_CUST_REF1DEST_CUST_REF2DEST_CUST_REF3INTERNAL_NOTES1INTERNAL_NOTES2INTERNAL_NOTES3STOWAGE_REQUESTTPT_INTERNAL_NOTES1TPT_INTERNAL_NOTES2TPT_INTERNAL_NOTES3DECL_INSTR_NOCOMMCLR_INDCOMMCLR_EXPIRY_DATETMCOMMCLR_UPDATEDBYCOMMCLR_UPDATED_DATETMCONFIRMED_INDBILLTYPEBILL_SPLIT_INDPORT_OCC_NO

PARAMETERS USED BY THE OPTIMIZER

This section of the trace the optimizer lists all the init.ora parameters that have an influence onthe access plan. The list changes from Oracle version to version. This here is the list for 9.2, thedescription of these one can look into the Oracle Documentation.

OPTIMIZER_FEATURES_ENABLE = 9.2.0OPTIMIZER_MODE/GOAL = Choose_OPTIMIZER_PERCENT_PARALLEL = 101HASH_AREA_SIZE = 1048576HASH_JOIN_ENABLED = TRUEHASH_MULTIBLOCK_IO_COUNT = 0SORT_AREA_SIZE = 524288OPTIMIZER_SEARCH_LIMIT = 5PARTITION_VIEW_ENABLED = FALSE_ALWAYS_STAR_TRANSFORMATION = FALSE_B_TREE_BITMAP_PLANS = TRUESTAR_TRANSFORMATION_ENABLED = FALSE_COMPLEX_VIEW_MERGING = TRUE_PUSH_JOIN_PREDICATE = TRUEPARALLEL_BROADCAST_ENABLED = TRUEOPTIMIZER_MAX_PERMUTATIONS = 2000OPTIMIZER_INDEX_CACHING = 0_SYSTEM_INDEX_CACHING = 0OPTIMIZER_INDEX_COST_ADJ = 100OPTIMIZER_DYNAMIC_SAMPLING = 1_OPTIMIZER_DYN_SMP_BLKS = 32QUERY_REWRITE_ENABLED = FALSEQUERY_REWRITE_INTEGRITY = ENFORCED_INDEX_JOIN_ENABLED = TRUE_SORT_ELIMINATION_COST_RATIO = 0_OR_EXPAND_NVL_PREDICATE = TRUE_NEW_INITIAL_JOIN_ORDERS = TRUEALWAYS_ANTI_JOIN = CHOOSEALWAYS_SEMI_JOIN = CHOOSE_OPTIMIZER_MODE_FORCE = TRUE_OPTIMIZER_UNDO_CHANGES = FALSE

_UNNEST_SUBQUERY = TRUE_PUSH_JOIN_UNION_VIEW = TRUE_FAST_FULL_SCAN_ENABLED = TRUE_OPTIM_ENHANCE_NNULL_DETECTION = TRUE_ORDERED_NESTED_LOOP = TRUE_NESTED_LOOP_FUDGE = 100_NO_OR_EXPANSION = FALSE_QUERY_COST_REWRITE = TRUEQUERY_REWRITE_EXPRESSION = TRUE_IMPROVED_ROW_LENGTH_ENABLED = TRUE_USE_NOSEGMENT_INDEXES = FALSE_ENABLE_TYPE_DEP_SELECTIVITY = TRUE_IMPROVED_OUTERJOIN_CARD = TRUE_OPTIMIZER_ADJUST_FOR_NULLS = TRUE_OPTIMIZER_CHOOSE_PERMUTATION = 0_USE_COLUMN_STATS_FOR_FUNCTION = TRUE_SUBQUERY_PRUNING_ENABLED = TRUE_SUBQUERY_PRUNING_REDUCTION_FACTOR = 50_SUBQUERY_PRUNING_COST_FACTOR = 20_LIKE_WITH_BIND_AS_EQUALITY = FALSE_TABLE_SCAN_COST_PLUS_ONE = TRUE_SORTMERGE_INEQUALITY_JOIN_OFF = FALSE_DEFAULT_NON_EQUALITY_SEL_CHECK = TRUE_ONESIDE_COLSTAT_FOR_EQUIJOINS = TRUE_OPTIMIZER_COST_MODEL = CHOOSE_GSETS_ALWAYS_USE_TEMPTABLES = FALSEDB_FILE_MULTIBLOCK_READ_COUNT = 16_NEW_SORT_COST_ESTIMATE = TRUE_GS_ANTI_SEMI_JOIN_ALLOWED = TRUE_CPU_TO_IO = 0_PRED_MOVE_AROUND = TRUE

BASE STATISTICAL INFORMATION

OP_TRANSIMP_REQS OP_TRANSIMP_MOVES OP_CURPLAN_TRANSIMPREQSTable stats Table:OP_TRANSIMP_REQS Alias:ITR TOTAL :: CDN: 509576NBLKS: 70091AVG_ROW_LEN: 939Column: VERSION_NO Col#: 2Table: OP_TRANSIMP_REQSAlias: ITR NDV: 5 NULLS: 0DENS: 2.0000e-01 LO: 1 HI: 5 NO HISTOGRAM: #BKT: 1#VAL: 2Column: REQUIREMEN Col#: 1Table: OP_TRANSIMP_REQSAlias: ITR NDV: 502424 NULLS: 0DENS: 1.9904e-06 LO: 1 HI:13019515 NO HISTOGRAM: #BKT: 1#VAL: 2-- Index stats INDEX NAME:OP_TRANSIMP_REQS_PKCOL#: 1 2 TOTAL :: LVLS: 2 #LB: 1183#DK: 509576 LB/K: 1 DB/K: 1CLUF: 86847

Table stats Table:OP_TRANSIMP_MOVES Alias: ITM TOTAL :: CDN: 1126964 NBLKS: 180975AVG_ROW_LEN: 1155Column: ITR_VERSIO Col#: 2 Table:OP_TRANSIMP_MOVES Alias: ITM NDV: 5 NULLS: 0 DENS:2.0000e-01 LO: 1 HI: 5 NO HISTOGRAM: #BKT: 1 #VAL: 2Column: ITR_REQUIR Col#: 1 Table:OP_TRANSIMP_MOVES Alias: ITM NDV: 376424 NULLS: 0 DENS:2.6566e-06 LO: 1 HI: 13019515 NO HISTOGRAM: #BKT: 1 #VAL: 2Column: ITR_VERSIO Col#: 2 Table:OP_TRANSIMP_MOVES Alias: ITM NDV: 5 NULLS: 0 DENS:2.0000e-01 LO: 1 HI: 5 NO HISTOGRAM: #BKT: 1 #VAL: 2Column: ITR_REQUIR Col#: 1 Table:OP_TRANSIMP_MOVES Alias: ITM NDV: 376424 NULLS: 0 DENS:2.6566e-06 LO: 1 HI: 13019515 NO HISTOGRAM: #BKT: 1 #VAL: 2-- Index stats INDEX NAME:OP_TRANSIMP_MOVES_NU1 COL#: 1 24 TOTAL :: LVLS: 2 #LB: 3651 #DK:1126964 LB/K: 1 DB/K: 1 CLUF: 198364 INDEX NAME:OP_TRANSIMP_MOVES_PK COL#: 1 2 3 TOTAL :: LVLS: 2 #LB: 3038 #DK:1126964 LB/K: 1 DB/K: 1 CLUF: 198364

Table stats Table:OP_CURPLAN_TRANSIMPREQS Alias: OCT TOTAL :: CDN: 24736 NBLKS: 84AVG_ROW_LEN: 20Column: ITR_REQUIR Col#: 1 Table:OP_CURPLAN_TRANSIMPREQS Alias: OCT NDV: 16424 NULLS: 0 DENS: 6.0887e-05 LO: 13000128 HI: 13019515 NO HISTOGRAM: #BKT: 1 #VAL: 2Column: ITR_VERSIO Col#: 2 Table:OP_CURPLAN_TRANSIMPREQS Alias: OCT NDV: 5 NULLS: 0 DENS: 2.0000e-01LO: 1 HI: 5 NO HISTOGRAM: #BKT: 1 #VAL: 2Column: ITR_VERSIO Col#: 2 Table:OP_CURPLAN_TRANSIMPREQS Alias: OCT NDV: 5 NULLS: 0 DENS: 2.0000e-01LO: 1 HI: 5 NO HISTOGRAM: #BKT: 1 #VAL: 2Column: ITR_REQUIR Col#: 1 Table:OP_CURPLAN_TRANSIMPREQS Alias: OCT NDV: 16424 NULLS: 0 DENS: 6.0887e-05 LO: 13000128 HI: 13019515 NO HISTOGRAM: #BKT: 1 #VAL: 2-- Index stats INDEX NAME:OP_CURPLAN_TRANSIMPREQS_NU1 COL#: 12 TOTAL :: LVLS: 1 #LB: 95 #DK: 23574LB/K: 1 DB/K: 1 CLUF: 9913 INDEX NAME:OP_CURPLAN_TRANSIMPREQS_PK COL#: 3 21 TOTAL :: LVLS: 1 #LB: 133 #DK: 24736LB/K: 1 DB/K: 1 CLUF: 9069_OPTIMIZER_PERCENT_PARALLEL = 0

Terminology Used

For Tables For IndexesCDN Cardinality (No of rows of the table)NBLKS The number of blocks below high water markAVG_ROW_LEN Average Row length of a row.

Index#, col# The object# of the index and the column_id of the columns. Oracle 9 brings an improvement by using the index name rather than index#.LVLS The height of the Index b-tree.#LB The number of leaf blocks.#DK The number of distinct keys of the indexLB/K The average Number of leaf blocks per keyDB/K The average Number of data blocks per keyCLUF The clustering factor of the index

BASE TABLE ACCESS COST

Now the optimizer is using this information to evaluate access plans. First the CBO looks at thedifferent possibilities and costs to access each of the tables in the SQL by itself, taking intoconsideration all applicable predicates except join predicates.Generally Optimizer uses table scan, index unique scan, index range scan, index & equal andindex fast full scan methods.

Single Table Access Paths

OP_CURPLAN_TRANSIMPREQS OP_TRANSIMP_MOVES OP_TRANSIMP_REQS

SINGLE TABLE ACCESS PATHColumn: OPO_OMRPLA Col#: 3 Table:OP_CURPLAN_TRANSIMPREQS Alias:OCT NDV: 24732 NULLS: 0 DENS:4.0433e-05 LO: 2098117 HI: 2191648 NO HISTOGRAM: #BKT: 1 #VAL: 2 TABLE:OP_CURPLAN_TRANSIMPREQS ORIGCDN: 24736 ROUNDED CDN: 1 CMPTDCDN: 1 Access path: tsc Resc: 10 Resp: 10 Access path: index (iff) Index:OP_CURPLAN_TRANSIMPREQS_PK TABLE:OP_CURPLAN_TRANSIMPREQS RSC_CPU: 0 RSC_IO: 14 IX_SEL: 0.0000e+00 TB_SEL:1.0000e+00 Access path: iff Resc: 14 Resp: 14 Skip scan: ss-sel 0 andv 5 ss cost 5 index io scan cost 0 Access path: index (index-only) Index:OP_CURPLAN_TRANSIMPREQS_PK TABLE:OP_CURPLAN_TRANSIMPREQS RSC_CPU: 0 RSC_IO: 2 IX_SEL: 4.0433e-05 TB_SEL: 4.0433e-05 BEST_CST: 2.00 PATH: 4 Degree: 1

SINGLE TABLE ACCESSPATH TABLE:OP_TRANSIMP_MOVESORIG CDN: 1126964ROUNDED CDN: 1126964CMPTD CDN: 1126964 Access path: tsc Resc:17407 Resp: 17407 BEST_CST: 17407.00PATH: 2 Degree: 1

SINGLE TABLE ACCESS PATH TABLE: OP_TRANSIMP_REQSORIG CDN: 509576 ROUNDEDCDN: 509576 CMPTD CDN:509576 Access path: tsc Resc: 6743Resp: 6743 BEST_CST: 6743.00 PATH: 2Degree: 1Table: OP_TRANSIMP_MOVESJoin index: 31106

Terminology Used

NDV Number of Distinct Values for the column

NULLS Number of rows with a null “value” for the columnDENS Density of that column(1/NDV for without having histograms on the column)LO The lowest value for the column (only for numeric columns)HI The highest value for the column (only for numeric columns)

The information regarding these values can be obtained from dba_tab_columns, if thecorresponding tables are properly analyzed.

I will describe the way it has calculated the cost for single table access paths forOP_CURPLAN_TRANSIMPREQS.

Description of individual Section:

Full Table Scan Cost

Column: OPO_OMRPLA Col#: 3 Table: OP_CURPLAN_TRANSIMPREQS Alias: OCT NDV: 24732 NULLS: 0 DENS: 4.0433e-05 LO: 2098117 HI: 2191648 NO HISTOGRAM: #BKT: 1 #VAL: 2 TABLE: OP_CURPLAN_TRANSIMPREQS ORIG CDN: 24736 ROUNDED CDN: 1 CMPTD CDN: 1 Access path: tsc Resc: 10 Resp: 10

DENS = 1/NDV => 4.0433e-05CMPTD CDN = FF* ORIG CDN => DENS* ORIG CDN => 4.0433e-05*24732 => 1 (FF =>Filter Factor, Find the details below)Tsc Resc(Table Scan Cost) = NBLKS / DB_FILE_MULTIBLOCK_READ_COUNTThis is some what rounded calculation. From here you can calculate the effective value ofDB_FILE_MULTIBLOCK_READ_COUNT being used for this database/OS.So infact effective DB_FILE_MULTIBLOCK_READ_COUNT = NBLKS / Tsc Resc = 84/10=>8.4.So it changes with the average row length. The general values which has been calculated foreffective DB_FILE_MULTIBLOCK_READ_COUNT are as follows:

Keep in mind that the thus established k factor is only used in the CBO’s estimate forthe cost of a full table scan – or fast index scan. The actual cost in I/O of a full tablescan depends on other factors besides db_file_multi_block_read_count like properextent planning and management and whether data blocks of the table are alreadypresent in the buffer pool and how many.

PREDICATES AND FILTER FACTORS

In order to understand the index access cost calculations it is necessary to discuss filter factorsand their relationship to the query’s predicates. A filter factor is a number between 0 and 1 and,in a nutshell, is a measure for the selectivity of a predicate, or, in mathematical terms, theprobability that a particular row will match a predicate or set of predicates. If a column has 10distinct values in a table and a query is looking for all rows where the column is equal to one ofthe values, you intuitively expect that it will return 1/10 of all rows, presuming an equaldistribution. That is exactly the filter factor of a single column for an equal predicate:

FF = 1/NDV = densityBoth statistics, NDV (a.k.a. num_distinct) and density, are in dba_tab_columns but the optimizeruses the value of density in most its calculations. This has ramifications, as we will see.Here is the relationship between predicates and the resulting filter factor:

WITHOUT BIND VARIABLESpredicate Filter factorc1 = value 1/c1.num_distinct4c1 like value 1/c1.num_distinctc1 > value (Hi - value) / (Hi - Lo)c1 >= value (Hi - value) / (Hi - Lo) + 1/c1.num_distinctc1 < value (value - Lo) / (Hi - Lo)c1 <= value (value - Lo) / (Hi - Lo) + 1/c1.num_distinctc1 between val1 and val2 (val2 – val1) / (Hi - Lo) + 2 * 1/c1.num_distinct

WHEN USING BIND VARIABLES

predicate Filter factorcol1 = :b1 col1.densitycol1 {like | > | >= | < | <=} :b1 {5.0000e-02 | col1.density }5col1 between :b1 and :b2 5.0000e-02 * 5.0000e-02

COMBINING PREDICATES / FILTER FACTORS

Predicate Filter factorPredicate 1 and predicate 2 => FF1 * FF2Predicate 1 or predicate 2 => FF1 + FF2 – FF1 * FF2The rules on how filter factors are calculated when predicates are combined with “and” or “or”are straight out of probability theory. Given that fact, it should be noted that probability theorystipulates that these formulas for combining probabilities are only valid if “the predicates areindependent”.Just as the basic column densities presume a uniform distribution of the distinct column values,the CBO cost calculations for a plan presume independence of the predicates. But while there issome remedy in the form of histograms if the data distribution is not uniform, there is no remedyfor cases where the predicates are not independent.For the calculation of the filter factors for ranges of string literals, the value of the literal is theweighted sum of the ASCII values of its characters. Strings of different lengths are “rightpadded” with zeros:ADAMS = 65*256 + 68*256 + 65*256 + 77*256 + 83*256 = 2.8032e+11

COLUMN STATS WITH HISTOGRAM

In the absence of histogram data, all filter factors are derived from the density of the columnsinvolved, or from fixed values if no statistics have been gathered at all. Histograms complicatethe filter factor calculations and go beyond the scope of this paper. In spite of that, we’ll take abrief look at what changes when histograms are calculated on a column and correct the recordon a couple of myths on the way. Histograms are gathered when the option

“for {all {indexed}} columns {col1 {, col2…}} {size [n|75]}” ANALYZE“for {all {indexed}} columns size n {col1 {, col2…}}” DBMS_STATS.GATHER_TABLE_STATS

is used with the analyze command or the DBMS_STATS.GATHER_TABLE_STATS procedure.Note the subtle difference in the phrasing between the two methods.

There are two types of histograms:

• Value-Based HistogramsThe number of buckets is equal to the number of distinct values and the “endpoint” of eachbucket is the number of rows in the table with that value. Oracle builds a value based histogramif the size in the analyze for a column is larger than the number of distinct values for the column.

• Height-Based Histograms

Height-based histograms place approximately the same number of values into each bucket, sothat the endpoints of the bucket are determined by how many values are in that bucket. Oraclebuilds a height based histogram if the size in the analyze for a column is smaller than thenumber of distinct values for the column.A commonly held belief is that histograms are useless, i.e. have no effect on the access plan, ifbind variables are used since the value is not known at parse time and the CBO – histogramsare only ever used by the cost based optimizer – can not determine from the histogram if itshould use an available index or not. While the latter is true, the gathering of histograms still canchange the access plan. Why and how?

Becausea) The optimizer uses the density in its filter factor calculation, not NDV.b) The density is calculated differently for columns with histograms, not simply as 1/NDVIf the density changes, the costs of plan segments and the cardinality estimates of row sourceschange and hence the entire plan may change. I have successfully exploited that aspect ofhistograms in tuning. Another popular myth is that there is no point in gathering histograms onnon-indexed columns, likely born from the assumption that the only role of a histogram is to letthe optimizer decide between a tablescan and an index access. However, the CBO uses filterfactors, derived from column densities, to calculate the costs of different access plans, andultimately choose an access plan; and filter factors are used in two places in this calculation ofaccess plan costs:

1. In the calculation of index access costs.2. In the calculation of row source cardinalities.

In the latter calculation, the filter factors of predicates on non-indexed columns do get used.What is more, the row source cardinality has ultimately the more decisive effect as it guides thecomposition of the overall access plan. In my experience, the cause for a poor access plan is

more often the incorrect estimate of the cardinality of a row source than the incorrect estimate ofan index access cost.

INDEX ACCESS COSTS

Having discussed filter factors, we are now ready to look at the other part of the single tableaccess path evaluation – the calculation of the cost of accessing the needed rows via an index.

Access path: index (iff) Index: OP_CURPLAN_TRANSIMPREQS_PK TABLE: OP_CURPLAN_TRANSIMPREQS RSC_CPU: 0 RSC_IO: 14 IX_SEL: 0.0000e+00 TB_SEL: 1.0000e+00 Access path: iff Resc: 14 Resp: 14

The formula used to calculate the cost of index fast full scan is same as that of FTS:(Level+Leaf Blocks)/ DB_FILE_MULTIBLOCK_READ_COUNT

INDEX NAME: OP_CURPLAN_TRANSIMPREQS_PK COL#: 3 2 1 TOTAL: LVLS: 1 #LB: 133 #DK: 24736 LB/K: 1 DB/K: 1 CLUF: 9069

So the cost of IFF is:iff Resc = (1+133)/10.3 =>13.09 =>14

Access path: index (index-only) Index: OP_CURPLAN_TRANSIMPREQS_PK TABLE: OP_CURPLAN_TRANSIMPREQS RSC_CPU: 0 RSC_IO: 2 IX_SEL: 4.0433e-05 TB_SEL: 4.0433e-05 BEST_CST: 2.00 PATH: 4 Degree: 1

For unique Primary key scan :

Cost = blevel+1

INDEX NAME: OP_CURPLAN_TRANSIMPREQS_PK COL#: 3 2 1 TOTAL : LVLS: 1 #LB: 133 #DK: 24736 LB/K: 1 DB/K: 1 CLUF: 9069

RSC_IO = blevel + 1 = 2

In General the cost of different types of index scans:

The cost calculations for other index access methods are:

Unique scan blevel+1Fast full scan leaf_blocks /Effective DB_FILE_MULTIBLOCK_READ_COUNTIndex-only blevel + FF*leaf_blocksIndex range scan blevel + FF*leaf_blocks + FF*clustering_factor

Conclusions that can be drawn from these cost formulae:

� Leaf_blocks contributes to all but the unique index access cost. Index compression, whereappropriate, reduces the number of leaf blocks, lowering the index access costs and cantherefore result in the CBO using an index where before it did not.

� Except for a unique index access, the height of the index (blevel) contributes negligibly tothe cost.

� The clustering factor affects only an index range scan, but then heavily, given the fact thethis is orders of magnitude bigger than LEAF_BLOCKS.

The most important thing that 10053 trace proves is:� An index is not being used, is not even considered and entered into any plan cost

calculation if its leading column is not among the predicates.

DEFAULT TABLE, INDEX, AND COLUMN STATISTICS

Remember that the rule based optimizer parses statements where none of the tables havestatistics. What if only one, or a few but not all, tables, indexes, or columns have no statistics?There are different claims for what Oracle does in that case. The most popular is that Oracleuses the rule based optimizer for that table. But parsing is not a mix and match exercise – astatement is either parsed entirely by CBO or entirely by RBO. If at least one table in the queryhas statistics (and optimizer goal is not rule) then the cost base optimizer does parse the query.Another claim is that Oracle will dynamically, at runtime, estimate statistics on the objectswithout statistics. I have not seen any evidence of that.

Let us examine what the 10053 trace shows if the statistics on the any of the tables are deletedor that particular table is not analyzed:

1. AVG_ROW_LEN for “NOT ANALYZED” tables defaults to 1002. NBLKS and hence TABLE_SCAN_COST are identical to those of the analyzed table. How

is that possible?

The answer is actually quite simple and has interesting ramifications:The NBLKS statistic for an unanalyzed table is taken from the table’s segment header and istherefore more accurate than the NBLKS statistic of an analyzed table which may be stale.The access plan of a query on analyzed tables does not change as long as the statistics and theinit.ora parameters do not change (i.e. as long as the tables are not re-analyzed), even if thetables grow or change in other significant ways. This is no longer true if one of the tables is“unanalyzed”. A change in its size (NBLKS) is immediately reflected not only in its TSC but also,as will be demonstrated shortly, in the defaults for the table’s cardinality and column densitiesand can therefore result in a change of access plan.As a corollary, if you are using DBMS_STATS to transport the statistics from a productiondatabase to a test database in order to do your SQL analysis and tuning there, watch out fortables without statistics. Since CBO uses the actual number of blocks from the segment header,in this case from the test database rather than production, you can easily get different accessplans.

3. The cardinality is a function of a mixture of actual (NBLKS) and default (AVG_ROW_LEN)values:CDN = NBLKS * (db_block_size – block overhead) / AVG_ROW_LEN

4. The statistics for unanalyzed indexes default toLVLS #LB #DK LB/K DB/K CLUF 1 25 100 1 1 800

To find the default column statistics, remember that column statistics are not listed under “BASESTATISTICAL INFORMATION” but under “SINGLE TABLE ACCESS PATH”:

The defaults for NDV and DENS do not look like nice round defaults like the ones for indexstatistics. Note also that the density is not 1/NDV. Checking the default column statistics fordifferently sized tables confirms the suspicion that the column statistics defaults, like the tablestatistics defaults, are not static, but are derived from the NBLKS value. Examining and plottingthe default column density of tables of different sizes in a scatter diagram against the tables’number of blocks shows not only a correlation but clear functional dependency:density = 0.7821*nblks-0.9992 or practically density = 0.7821 / nblks

Note that again this is an empirically derived formula. For small values of NBLKS it does notyield the exact same densities as observed in the 10053 trace. Note also that the equation ofthe correlation function between density and NBLKS is different for different db_block_sizevalues. The actual formula is not really important, but the fact that there is a dependency of thedefault column density on NBLKS and db_block_size is interesting.Similar to filter factors for range predicates with bind variables, the optimizer uses defaults formissing/unknown statistics of “not analyzed” tables, indexes, or columns. These defaults rangefrom plain static values (index statistics and avg_row_size) to actual values (NBLKS) and, as wehave seen, some values (CDN and column densities) derived from NBLKS using complicatedformulas. The analysis of these default values was done on Oracle 8.1.7. It should surprisednobody if some of the default values or calculations have changed, and will continue to change,from Oracle release to release.

GENERAL PLANS

This concludes the single table costing part of the 10053 CBO trace. The next section in the10053 event trace starts with the heading “GENERAL PLANS”. For all but the simplest SQLstatements, this section makes up the largest part of the trace. This is where the CBO looks atthe costs of all different orders and ways of joining the individual tables and come up with thebest access plan.The cost based optimizer has three join methods in its arsenal. These are the three joinmethods and their costing formulas:

1. NL - NESTED LOOP JOINJoin cost = cost of accessing outer table+ (cardinality of outer table * cost of accessing inner table )

2. SM – SORT MERGE JOINJoin cost = (cost of accessing outer table + outer sort cost)+ (cost of accessing inner table + inner sort cost)

3. HA – HASH JOINJoin cost = (cost of accessing outer table)+ (Cost of building hash table)+ (Cost of accessing inner table)We will look at the join costing of each method for our simple query.

JOIN ORDER [N]

In the GENERAL PLANS section, the optimizer evaluates every possible permutation of thetables in the query. An exception is tables in a correlated subquery where it is semanticallyimpossible to access the table(s) in the subquery before any tables of the outer query and thusnot all permutations constitute a valid plan. Apart from this situation, each permutationevaluation in the trace is given a number and is then followed by the list of the tables – with theiraliases to distinguish them if a table occurs multiple times in the query.The initial join order is chosen by ordering the tables in order of increasing computed cardinality.In this simple case, the CBO is examining the cost of accessing first theOP_CURPLAN_TRANSIMPREQS table and then joining the OP_TRANSIMP_REQS table andfinally joining the table OP_TRANSIMP_MOVES.”Now Joining:” is always the beginning of the next batch of join evaluations, introducing the tablejoining the fray – no pun intended.

Join order[1]: OP_CURPLAN_TRANSIMPREQS [OCT] OP_TRANSIMP_REQS [ITR] OP_TRANSIMP_MOVES [ITM]Now joining: OP_TRANSIMP_REQS [ITR] *******NL Join Outer table: cost: 2 cdn: 1 rcz: 12 resp: 2 Inner table: OP_TRANSIMP_REQS Access path: tsc Resc: 6743 Join: Resc: 6745 Resp: 6745Join cardinality: 509658 = outer (1) * inner (509576) * sel (1.0000e+00) [flag=0] Best NL cost: 6745 resp: 6745Join result: cost: 6745 cdn: 509658 rcz: 37Now joining: OP_TRANSIMP_MOVES [ITM] *******NL Join Outer table: cost: 6745 cdn: 509658 rcz: 37 resp: 6745 Inner table: OP_TRANSIMP_MOVES Access path: tsc Resc: 17407 Join: Resc: 8871623551 Resp: 8871623551 Access path: index (scan) Index: OP_TRANSIMP_MOVES_NU1 TABLE: OP_TRANSIMP_MOVES RSC_CPU: 0 RSC_IO: 3 IX_SEL: 5.3132e-07 TB_SEL: 2.8230e-13 Join: resc: 1535719 resp: 1535719 Access path: index (scan) Index: OP_TRANSIMP_MOVES_PK TABLE: OP_TRANSIMP_MOVES RSC_CPU: 0 RSC_IO: 3 IX_SEL: 5.3132e-07 TB_SEL: 2.8230e-13 Join: resc: 1535719 resp: 1535719Join cardinality: 1 = outer (509658) * inner (1126964) * sel (1.0427e-12) [flag=0] Using index (ndv = 509576 sel = 3.9807e-07) Best NL cost: 1535719 resp: 1535719SM Join Outer table: resc: 6745 cdn: 509658 rcz: 37 deg: 1 resp: 6745 Inner table: OP_TRANSIMP_MOVES resc: 17407 cdn: 1126964 rcz: 12 deg: 1 resp: 17407 using join:1 distribution:2 #groups:1 SORT resource Sort statistics Sort width: 2 Area size: 131072 Max Area size: 1257472 Degree: 1 Blocks to Sort: 3183 Row size: 51 Rows: 509658 Initial runs: 21 Merge passes: 5 IO Cost / pass: 4775 Total IO sort cost: 13529 Total CPU sort cost: 0 Total Temp space used: 53208000 SORT resource Sort statistics Sort width: 2 Area size: 131072 Max Area size: 1257472 Degree: 1 Blocks to Sort: 3312 Row size: 24 Rows: 1126964Initial runs: 22 Merge passes: 5 IO Cost / pass: 4968 Total IO sort cost: 14076 Total CPU sort cost: 0 Total Temp space used: 54322000 Merge join Cost: 51757 Resp: 51757HA Join Outer table: resc: 6745 cdn: 509658 rcz: 37 deg: 1 resp: 6745 Inner table: OP_TRANSIMP_MOVES resc: 17407 cdn: 1126964 rcz: 12 deg: 1 resp: 17407 using join:8 distribution:2 #groups:1 Hash join one ptn Resc: 1355 Deg: 1 hash_area: 60 (max=307) buildfrag: 3049 probefrag: 3302 ppasses: 1 Hash join Resc: 25507 Resp: 25507Join result: cost: 25507 cdn: 1 rcz: 49Best so far: TABLE#: 0 CST: 2 CDN: 1 BYTES: 12Best so far: TABLE#: 1 CST: 6745 CDN: 509658 BYTES: 18857346Best so far: TABLE#: 2 CST: 25507 CDN: 1 BYTES: 49

Join order[2]: OP_CURPLAN_TRANSIMPREQS [OCT] OP_TRANSIMP_MOVES [ITM] OP_TRANSIMP_REQS [ITR]Now joining: OP_TRANSIMP_MOVES [ITM] *******NL Join Outer table: cost: 2 cdn: 1 rcz: 12 resp: 2 Inner table: OP_TRANSIMP_MOVES Access path: tsc Resc: 17407 Join: Resc: 17409 Resp: 17409 Access path: index (scan) Index: OP_TRANSIMP_MOVES_NU1 TABLE: OP_TRANSIMP_MOVES RSC_CPU: 0 RSC_IO: 3 IX_SEL: 5.3132e-07 TB_SEL: 5.3132e-07 Join: resc: 5 resp: 5 Access path: index (scan) Index: OP_TRANSIMP_MOVES_PK TABLE: OP_TRANSIMP_MOVES RSC_CPU: 0 RSC_IO: 3 IX_SEL: 5.3132e-07 TB_SEL: 5.3132e-07 Join: resc: 5 resp: 5Join cardinality: 1 = outer (1) * inner (1126964) * sel (5.3132e-07) [flag=0] Best NL cost: 5 resp: 5SM Join Outer table: resc: 2 cdn: 1 rcz: 12 deg: 1 resp: 2 Inner table: OP_TRANSIMP_MOVES resc: 17407 cdn: 1126964 rcz: 12 deg: 1 resp: 17407 using join:1 distribution:2 #groups:1 SORT resource Sort statistics Sort width: 2 Area size: 131072 Max Area size: 1257472 Degree: 1Blocks to Sort: 3312 Row size: 24 Rows: 1126964 Initial runs: 22 Merge passes: 5 IO Cost / pass: 4968 Total IO sort cost: 14076 Total CPU sort cost: 0 Total Temp space used: 54322000 Merge join Cost: 31485 Resp: 31485HA Join Outer table: resc: 2 cdn: 1 rcz: 12 deg: 1 resp: 2 Inner table: OP_TRANSIMP_MOVES resc: 17407 cdn: 1126964 rcz: 12 deg: 1 resp: 17407 using join:8 distribution:2 #groups:1 Hash join one ptn Resc: 12 Deg: 1 hash_area: 60 (max=307) buildfrag: 1 probefrag: 3302 ppasses: 1 Hash join Resc: 17421 Resp: 17421Join result: cost: 5 cdn: 1 rcz: 24Now joining: OP_TRANSIMP_REQS [ITR] *******NL Join Outer table: cost: 5 cdn: 1 rcz: 24 resp: 5 Inner table: OP_TRANSIMP_REQS Access path: tsc Resc: 6743 Join: Resc: 6748 Resp: 6748 Access path: index (unique) Index: OP_TRANSIMP_REQS_PK TABLE: OP_TRANSIMP_REQS RSC_CPU: 0 RSC_IO: 2 IX_SEL: 1.9624e-06 TB_SEL: 1.9624e-06 Join: resc: 7 resp: 7 Access path: index (eq-unique) Index: OP_TRANSIMP_REQS_PK TABLE: OP_TRANSIMP_REQS RSC_CPU: 0 RSC_IO: 2 IX_SEL: 0.0000e+00 TB_SEL: 0.0000e+00 Join: resc: 7 resp: 7Join cardinality: 1 = outer (1) * inner (509576) * sel (1.9624e-06) [flag=0] Using index (ndv = 509576 sel = 3.9807e-07) Best NL cost: 7 resp: 7SM Join Outer table: resc: 5 cdn: 1 rcz: 24 deg: 1 resp: 5 Inner table: OP_TRANSIMP_REQS

resc: 6743 cdn: 509576 rcz: 25 deg: 1 resp: 6743 using join:1 distribution:2 #groups:1 SORT resource Sort statistics Sort width: 2 Area size: 131072 Max Area size: 1257472 Degree: 1 Blocks to Sort: 1 Row size: 37 Rows: 1 Initial runs: 1 Merge passes: 1 IO Cost / pass: 2 Total IO sort cost: 2Total CPU sort cost: 0 Total Temp space used: 0 SORT resource Sort statistics Sort width: 2 Area size: 131072 Max Area size: 1257472 Degree: 1 Blocks to Sort: 2371 Row size: 38 Rows: 509576 Initial runs: 16 Merge passes: 4 IO Cost / pass: 3557 Total IO sort cost: 8300 Total CPU sort cost: 0 Total Temp space used: 40936000 Merge join Cost: 15049 Resp: 15049HA Join Outer table: resc: 5 cdn: 1 rcz: 24 deg: 1 resp: 5 Inner table: OP_TRANSIMP_REQS resc: 6743 cdn: 509576 rcz: 25 deg: 1 resp: 6743 using join:8 distribution:2 #groups:1 Hash join one ptn Resc: 8 Deg: 1 hash_area: 60 (max=307) buildfrag: 1 probefrag: 2302 ppasses: 1 Hash join Resc: 6756 Resp: 6756Join result: cost: 7 cdn: 1 rcz: 49Best so far: TABLE#: 0 CST: 2 CDN: 1 BYTES: 12Best so far: TABLE#: 2 CST: 5 CDN: 1 BYTES: 24Best so far: TABLE#: 1 CST: 7 CDN: 1 BYTES: 49

Join order[3]: OP_TRANSIMP_REQS [ITR] OP_CURPLAN_TRANSIMPREQS [OCT] OP_TRANSIMP_MOVES [ITM]Join order[4]: OP_TRANSIMP_REQS [ITR] OP_TRANSIMP_MOVES [ITM] OP_CURPLAN_TRANSIMPREQS [OCT]Join order[5]: OP_TRANSIMP_MOVES [ITM] OP_CURPLAN_TRANSIMPREQS [OCT] OP_TRANSIMP_REQS [ITR]Join order[6]: OP_TRANSIMP_MOVES [ITM] OP_TRANSIMP_REQS [ITR] OP_CURPLAN_TRANSIMPREQS [OCT]

Final: CST: 7 CDN: 1 RSC: 7 RSP: 7 BYTES: 49 IO-RSC: 7 IO-RSP: 7 CPU-RSC: 0 CPU-RSP: 0

I am going to explain the second join method as it is the best one giving less cost. In the similarway you can go through First join method.

Join order[2]: OP_CURPLAN_TRANSIMPREQS [OCT] OP_TRANSIMP_MOVES [ITM] OP_TRANSIMP_REQS [ITR]Now joining: OP_TRANSIMP_MOVES [ITM] *******NL Join Outer table: cost: 2 cdn: 1 rcz: 12 resp: 2 Inner table: OP_TRANSIMP_MOVES Access path: tsc Resc: 17407 Join: Resc: 17409 Resp: 17409 Access path: index (scan) Index: OP_TRANSIMP_MOVES_NU1 TABLE: OP_TRANSIMP_MOVES RSC_CPU: 0 RSC_IO: 3 IX_SEL: 5.3132e-07 TB_SEL: 5.3132e-07 Join: resc: 5 resp: 5 Access path: index (scan) Index: OP_TRANSIMP_MOVES_PK TABLE: OP_TRANSIMP_MOVES RSC_CPU: 0 RSC_IO: 3 IX_SEL: 5.3132e-07 TB_SEL: 5.3132e-07 Join: resc: 5 resp: 5Join cardinality: 1 = outer (1) * inner (1126964) * sel (5.3132e-07) [flag=0] Best NL cost: 5 resp: 5

As we have set the initialization parameter as:OPTIMIZER_INDEX_CACHING = 0OPTIMIZER_INDEX_COST_ADJ = 100This shows that it will consider the scan on indexes and table to be equally costly and that is thereason this plan is not deriving the optimim cost. If this parameters are set asOPTIMIZER_INDEX_CACHING = 100 & OPTIMIZER_INDEX_COST_ADJ = 30 then the costwould have been less. I have tested both the scenarios and the details of these are as follows:

Alter session set OPTIMIZER_INDEX_CACHING = 90;Alter session set OPTIMIZER_INDEX_COST_ADJ = 30;Set autot trace expSELECT OCT.OPO_OMRPLANID, OCT.ITR_REQUIREMENT_NO, OCT.ITR_VERSION_NO, ITM.DTY_DECLTYPE_FK, ITR.LBL_BILLNO_FK, ITM.TACC_IND, ITR.COMMCLR_INDFROM OP_CURPLAN_TRANSIMPREQS OCT, OP_TRANSIMP_MOVES ITM, OP_TRANSIMP_REQS ITRWHERE OCT.OPO_OMRPLANID=:B1 AND OCT.ITR_VERSION_NO=ITM.ITR_VERSION_NO AND OCT.ITR_REQUIREMENT_NO=ITM.ITR_REQUIREMENT_NO AND ITR.VERSION_NO=ITM.ITR_VERSION_NO AND ITR.REQUIREMENT_NO=ITM.ITR_REQUIREMENT_NO;

Execution Plan---------------------------------------------------------- 0 SELECT STATEMENT Optimizer=CHOOSE (Cost=4 Card=1 Bytes=49) 1 0 NESTED LOOPS (Cost=4 Card=1 Bytes=49) 2 1 NESTED LOOPS (Cost=3 Card=1 Bytes=24) 3 2 INDEX (RANGE SCAN) OF 'OP_CURPLAN_TRANSIMPREQS_PK' (UNIQUE) (Cost=2 Card=1 Bytes=12) 4 2 TABLE ACCESS (BY INDEX ROWID) OF 'OP_TRANSIMP_MOVES' (Cost=2 Card=1 Bytes=12) 5 4 INDEX (RANGE SCAN) OF 'OP_TRANSIMP_MOVES_PK' (UNIQUE) (Cost=1 Card=1) 6 1 TABLE ACCESS (BY INDEX ROWID) OF 'OP_TRANSIMP_REQS' (Cost=2 Card=1 Bytes=25) 7 6 INDEX (UNIQUE SCAN) OF 'OP_TRANSIMP_REQS_PK' (UNIQUE)

The reason the cost is coming less here is that it is using the index scan cost in the cost calculations for those columns whichare used in the where clause and in the below mentioned our case the cost is higher due to giving equal weightage to indexscan and table scans and considering them equally costly. Also in below case it may not be caching the index in preference totable so as to have these values from the cache itsself.

If these parameters are set as 0/100 then plan that is coming as(as in our case):Alter session set OPTIMIZER_INDEX_CACHING = 0;Alter session set OPTIMIZER_INDEX_COST_ADJ = 100;Execution Plan---------------------------------------------------------- 0 SELECT STATEMENT Optimizer=CHOOSE (Cost=7 Card=1 Bytes=49) 1 0 NESTED LOOPS (Cost=7 Card=1 Bytes=49) 2 1 NESTED LOOPS (Cost=5 Card=1 Bytes=24) 3 2 INDEX (RANGE SCAN) OF 'OP_CURPLAN_TRANSIMPREQS_PK' (UNIQUE) (Cost=2 Card=1 Bytes=12) 4 2 TABLE ACCESS (BY INDEX ROWID) OF 'OP_TRANSIMP_MOVES' (Cost=3 Card=1 Bytes=12) 5 4 INDEX (RANGE SCAN) OF 'OP_TRANSIMP_MOVES_NU1' (NON-UNIQUE) (Cost=2 Card=1) 6 1 TABLE ACCESS (BY INDEX ROWID) OF 'OP_TRANSIMP_REQS' (Cost=2 Card=1 Bytes=25) 7 6 INDEX (UNIQUE SCAN) OF 'OP_TRANSIMP_REQS_PK' (UNIQUE) Cost=1 Card=1)

This is the plan we are getting as our parameters are set as 0/100 which are default settings. However if these parameters arechanged to 90/30 from 0/100 you can find the reduction in cost from 7 to 4 which is approx 75% reduction.

The rest of the join is self-explanatory.Join cardinality = card of outer*card of inner*Selectivity of inner tablecost = cost of outer + cardinality of outer * cost of innerCost = 2 + 1*3 = 5

I am skipping the portions for hash and sort merge joins as for those there are lot of work needsto be done to reach to a conclusion to find out how oracle is calculating the cost for sorting andhashing and finally merging.

Join result: cost: 5 cdn: 1 rcz: 24Now joining: OP_TRANSIMP_REQS [ITR] *******NL Join Outer table: cost: 5 cdn: 1 rcz: 24 resp: 5 Inner table: OP_TRANSIMP_REQS Access path: tsc Resc: 6743 Join: Resc: 6748 Resp: 6748 Access path: index (unique) Index: OP_TRANSIMP_REQS_PK TABLE: OP_TRANSIMP_REQS RSC_CPU: 0 RSC_IO: 2 IX_SEL: 1.9624e-06 TB_SEL: 1.9624e-06 Join: resc: 7 resp: 7 Access path: index (eq-unique) Index: OP_TRANSIMP_REQS_PK TABLE: OP_TRANSIMP_REQS RSC_CPU: 0 RSC_IO: 2 IX_SEL: 0.0000e+00 TB_SEL: 0.0000e+00 Join: resc: 7 resp: 7Join cardinality: 1 = outer (1) * inner (509576) * sel (1.9624e-06) [flag=0] Using index (ndv = 509576 sel = 3.9807e-07) Best NL cost: 7 resp: 7

cost = cost of outer + cardinality of outer * cost of inner = 5 + 1*2 = 7

MULTI-TABLE JOINS

It joins 3 tables, enough to have some permutations of join orders to consider (6), butnot so many that one gets lost following the trail. With 4 tables, there would be 24permutations, with 5 tables 120 permutations.In general, there are n! (n faculty ) possible permutations to join n tables, so the numberof permutations – and the cost and time to evaluate them – rises dramatically as thenumber of tables in the SQL increases. The init.ora parameter“optimizer_max_permutations” can be used to limit the number of permutations theCBO will evaluate.There are also plenty of predicates to give rise to many different base table accessconsiderations. But we are not going to go into those details. I only want to demonstratethe join permutations of our case as follows:

Join order[3]: OP_TRANSIMP_REQS [ITR] OP_CURPLAN_TRANSIMPREQS [OCT] OP_TRANSIMP_MOVES [ITM]Join order[4]: OP_TRANSIMP_REQS [ITR] OP_TRANSIMP_MOVES [ITM] OP_CURPLAN_TRANSIMPREQS [OCT]Join order[5]: OP_TRANSIMP_MOVES [ITM] OP_CURPLAN_TRANSIMPREQS [OCT] OP_TRANSIMP_REQS [ITR]Join order[6]: OP_TRANSIMP_MOVES [ITM] OP_TRANSIMP_REQS [ITR] OP_CURPLAN_TRANSIMPREQS [OCT]

Join order[1] and join order[2] here are completely evaluated by CBO and as soon asthese join order starts and it finds the cost of the join exceeding at anty stage than theoptimum cost till now(7 in our case for Join Order [2]), it abandons that plan there andthen. It doesn’t evaluate the plan any further. Here the cost of accessing the first tableitsself is more than 7(accessing the table OP_TRANSIMP_REQS cost 17407 andaccessing the table OP_TRANSIMP_MOVES cost 6743), so there is no point in carryforwarding and doing any join analysis on these join orders.

CONCLUSION

If you take one piece of advice from this paper then this:

� Pay close attention to the cardinality values in an explain plan. Wrong estimates bythe CBO for the cardinalities can have a devastating effect on the choice of accessplan and the performance of the SQL statement. Assuming for the moment that theestimate of cardinality for any of the table in the plan above is incorrect and too low,then that would invalidate the entire plan costs and perhaps make the optimizerchoose a different, and better, plan. Armed with the knowledge of how the CBOarrived at this number you know what to change in order to help the optimizer makea better assumption: the filter factor of the combined predicates on this table, i.e.ultimately the densities and NDVs of the predicate columns. Here are some meansof doing that:

� Using histograms on some columns and experiment with the sizes (number ofbuckets). Even if the histogram data itself is never used, the density for a columnwith a histogram changes. Generally, collecting a value-based histogram for acolumn significantly reduces its density, often by orders of magnitude, whereascollecting a height-based histogram increases the density.

� Deleting the statistics of a column – now possible with the DBMS_STATSprocedure – an index or an entire table and let the optimizer use the defaultdensities.

� “Seeding” a table with rows that artificially either increase the cardinality of acolumn (more than that of the table) and thus lower its density, or increase thecardinality of the table without changing that of the column and thus raise thecolumn’s density.

� Using brute force and setting the density. This is possible as of Oracle 8.0 withthe advent of DBMS_STATS.SET_xxx_STATS. I recommend usingexport_table_stats to export the statistics into a STATTAB table, modify thevalue(s) there, and then import the statistics back into the dictionary. Of courseyou’ll make two exports – one into a “work” statid and one into a “backup” statidso that you can restore the original statistics in case the change does not havethe desired effect.

Submitted By:

Sumit PopliTata Consultancy Servicessumitp@delhi.tcs.co.inpopli_sumit@yahoo.com