extreme querying with analytics

EXTREME QUERYING WITH ANALYTICS

DISCLAIMERblah blah NOT LIABLE blah blah blah, I NEVER SAID THAT blah blah READ THE DOCUMENTATION blah blah blah NO PROMISES blah I GET PAID BY THE WORD blah blah

Read my blog at HTTP://BLOG.SYDORACLE.COM

PARTITIONS AND WINDOWS

A ROW IN A RESULT SET WAS ALL ALONE…

IT CAN GET CHAOTIC

AGGREGATE FUNCTIONS Aggregate functions are the basis of

many Analytics

All the standard aggregates (MIN, MAX, COUNT, SUM, etc) can be used with analytic clauses.

NEW AGGREGATE FUNCTIONS Min / Max (with added KEEP)

KEEP means keep the column value for the highest ranked record.

*THE MOST POPULOUS CITY

Which of their cities has the most potential slaves ?

CITIES DATA

SYDNEY and X both have a population of 2 million

RESULTS

MIN or MAX only makes a difference if there are multiple entries of the same ORDER BY rank

AGGREGATE FUNCTIONS Min / Max (with added KEEP) Collect

Create an collection of all the individual values

A list of large cities …

RESULTS

AGGREGATE FUNCTIONS Min / Max (with added KEEP) Collect XMLAgg (in four steps)

Collect the column(s) into an XML document

1. RECORD TO XML FRAGMENT

2. 'PARENT' ELEMENT FOR FRAGMENT

3. AGGREGATE XML FRAGMENTS

4. TOP-LEVEL PARENT FOR AGGREGATION

AGGREGATE FUNCTIONS Min / Max (with added KEEP) Collect XMLAGG ListAgg

11g function to create a single VARCHAR2 value from a collection of individual VARCHAR2s

LISTAGG - 11G

AGGREGATE FUNCTIONS WITH CASE Wrap the aggregate around a CASE statement

to give more aggregation possibilities.

SELECT SUM(case when state='VIC' then pop end)

vic_pop, SUM(case when state='NSW' then pop end)

nsw_pop FROM cities;

ANALYTIC FUNCTIONS(at last)

AGGREGATE FUNCTIONS FOR ANALYTICS Dense Rank / Rank / Row Number

ANALYTIC FUNCTIONS

Smithers,Bring me a list of our highest paid employees…and the poisoned donuts.

AGGREGATE FUNCTIONS FOR ANALYTICSselect name, wage, sector, row_number() over (partition by sector order by wage desc) rn, rank() over (partition by sector order by wage desc) rnk, dense_rank() over (partition by sector order by wage desc) drnkfrom emporder by sector, wage desc;

AGGREGATE FUNCTIONS FOR ANALYTICS

SUM - WITH ANALYTICS

WARNING Using ROW_NUMBER with other analytics can confuse…

select name, wage, cum_wage from (select name, wage, sum(wage) over (order by wage desc) cwage, row_number() over (order by wage desc) rn from emp where sector = '7G') where rn < 3

NAME WAGE CUM_WAGE Homer 2OO 2OO Lenny 1OO 4OO

SOLUTION : SWITCH WINDOWING TO ROWS

AGGREGATE FUNCTIONS FOR ANALYTICS Dense Rank / Rank / Row Number NTILE

The "Snobs" and "Yobs" function

Ignore the outliers and extremes Or ignore the 'huddled masses'

NORMALLY DISTRIBUTED DATA

NTILEExclude the most common 90%

Focus on the most common 10%

AGGREGATE FUNCTIONS FOR ANALYTICS Dense Rank / Rank / Row Number NTILE Lag / Lead

Look around for the previous or next row

LAG - PERCENTAGE CHANGE OVER TIME MONTH AMOUNT PREV_AMT PERC January 340 February 340 340 .00 March 150 340 -55.88 April 130 150 -13.33 May 170 130 30.77 June 210 170 23.53 July 350 210 66.67 August 270 350 -22.86 September 380 270 40.74

LAG - IGNORE NULLS (11G) MON AMOUNT PREV_AMT ---------- ---------- ---------- January 340 February 340 340 March 150 340 April 130 150 May 170 130 June 170 July 350 170 August 270 350 September 380 270

AGGREGATE FUNCTIONS FOR ANALYTICS Dense Rank / Rank / Row Number Percent Rank Lag / Lead First / Last

Look further ahead or behind

FIRST_VALUE AND PARTITION BY select to_char(period,'Month') mon, amount, first_value(amount) over (partition by trunc(period,'Q') order by period) prev_amt from sales order by period

RESULTS MON AMOUNT PREV_AMT ---------- ---------- ---------- January 340 340 February 340 340 March 150 340 April 130 130 May 170 130 June 210 130 July 350 350 August 270 350 September 380 350

WINDOW CLAUSE Rarely needed in practice Partition By and Order By normally

enough

BEWARE THE MISSING PARTITION If you omit the PARTITION clause,

especially with in-line views , the results can be BAD

MULTIPLE LINES FOR MULTIPLE ORDERS

REGULAR QUERY WITH ANALYTIC

INCORRECT QUERYIn the inline view, the SUM analytic applies to ALL the Orders in the table.

INCLUDE A PARTITION BY CLAUSE

ADVANCED GROUPINGS

(if we have time)

TOTALS AND SUBTOTALS Rollup Grouping sets Cube

ROLLUP, ROLLUP

TOTALS AND SUBTOTALS

TOTALS AND SUBTOTALS Rollup Cube

CUBE allows combinations of columns to be totaled

CUBE - EXAMPLE

TOTALS AND SUBTOTALS Rollup Cube Grouping sets

Perform grouping across multiple columns Without the lower level totals of CUBE

GROUPING SETS

MODEL CLAUSE If you think you have a problem which

the MODEL clause solves then Go have a coffee Go have a bar of chocolate Go have a beer Go have a lie down

BUT do something else until the feeling wears off

PIVOT / UNPIVOT IN 11G