[ Michelle Kolbe]

12c Partitioning for Data WarehousesOOW 2014

Some Background

[ 12c Partitioning for Data Warehouses]

2

My Story[12c Partitioning for Data Warehouses]

• Lead BI Engineer at • Past: Data Architect and BI Developer at

Intermountain Healthcare• President of Utah Oracle Users Group• IOUG 2014-15 Board Member• Collaborate 13 & 14 Conference

Committee• Utah State MIS Advisory Council Member• University of Utah Adjunct Professor• Past Contributing Editor of IOUG Select

Journal• Past Advisory Board Member of Healthcare

Data Warehousing Association3

The Backcountry Story[12c Partitioning for Data Warehouses]

• Online retailer of outdoor products• Started in 1996 selling avalanche beacons out of a garage in Park City, Utah

• 8 Websites:• Backcountry.com• Dogfunk• Competitive Cyclist• Steep and Cheap• Whiskey Militia• Chainlove• MotoSport• Bergfreunde.de

4

Visit IOUG at the User Group Pavilion

Stop by at the User Group Pavilion in the lobby of Moscone South and catch up with the user community!

• Connect with IOUG members and volunteers• Pick up a discount to join the IOUG community of 20,000+ technologists

strong• Enter for the chance to win books from IOUG Press or a free registration to

COLLABORATE 15!

Visit us Sunday through Wednesday!

IOUG SIG Meetings at OpenWorldAll meetings located in Moscone South - Room 208

Sunday, September 28Cloud Computing SIG: 1:30 p.m. - 2:30 p.m.

Monday, September 29Exadata SIG: 2:00 p.m. - 3:00 p.m.BIWA SIG: 5:00 p.m. – 6:00 p.m.

Tuesday, September 30Internet of Things SIG: 11:00 a.m. - 12:00 p.m.Storage SIG: 4:00 p.m. - 5:00 p.m.SPARC/Solaris SIG: 5:00 p.m. - 6:00 p.m.

Wednesday, October 1Oracle Enterprise Manager SIG: 8:00 a.m. - 9:00 a.m.Big Data SIG: 10:30 a.m. - 11:30 a.m.Oracle 12c SIG: 2:00 p.m. – 3:00 p.m.Oracle Spatial and Graph SIG: 4:00 p.m. (*OTN lounge)

• Save more than $1,000 on education offerings like pre-conference workshops• Access the brand-new, specialized IOUG Strategic Leadership Program• Priority access to the hands-on labs with Oracle ACE support• Advance access to supplemental session material and presentations• Special IOUG activities with no "ante in" needed - evening networking opportunities

and more

COLLABORATE 15 – IOUG ForumApril 12-16, 2015

Mandalay Bay Resort and CasinoLas Vegas, NV

COLLABORATE 15 Call for Speakers

Ends October 10

The IOUG Forum Advantage

www.collaborate.ioug.org

Follow us on Twitter at @IOUG or via the conference hashtag #C15LV!

Why and What is Partitioning?

8

[ 12c Partitioning for Data Warehouses]

9

Partitioning Background[12c Partitioning for Data Warehouses]

• First introduced in Oracle 8i• Enables large table to be split into smaller pieces to improve• Performance• Availability• Manageability

• Queries use partition pruning to only read pertinent blocks

10

Partitioning Strategies[12c Partitioning for Data Warehouses]

• Range• List• Hash• Interval• Reference• Virtual Column• System

• Composite – Combination of Range, List or Hash

How We Use Partitioning

[ 12c Partitioning for Data Warehouses]

11

All fact tables are interval partitioned by date, some monthly, weekly, and daily partitionsLast 2 years stored on SSD storageYearly run a process to move partitions older than 2 years to slow storage and merge into bigger partitionsMost indexes are local except PK, no partial indexes yet

12

Fact Tables[12c Partitioning for Data Warehouses]

2012 – Current in weekly partitions on SSD diskPrior to 2012 in quarterly partitions on slow diskGlobal index on PK and Natural KeyLocal bitmap indexes on all FKs

13

Visits Fact Table[12c Partitioning for Data Warehouses]

A few fact tables have more than one date field that will be used frequently in queriesFor examples, for sales we care about the ship date and also the order datePartition on date used the mostGlobal index on other date

14

Special Cases[12c Partitioning for Data Warehouses]

Interval-Reference Partitioning

15

[ 12c Partitioning for Data Warehouses]

16

Interval-Reference Partitioning[12c Partitioning for Data Warehouses]

• Composite partition first by interval then reference

• Parent table creates new partitions when data arrives• Child table is automatically maintained• Partition names inherited

17

Interval-Reference Example in 11g[12c Partitioning for Data Warehouses]

create table orders( order_number number, order_date_id number, constraint orders_pk primary key(order_number))partition by range(order_date_id) INTERVAL(7)( partition p1 values less than (20140101));table ORDERS created.

18

Interval-Reference Example in 11g[12c Partitioning for Data Warehouses]

create table orderlines( orderline_id number, order_number number not null, constraint orderlines_pk primary key(orderline_id), constraint orderlines_fk foreign key (order_number) references

orders)partition by reference(orderlines_fk);ORA-14659: Partitioning method of the parent table is not supported

19

Interval-Reference Example in 12c[12c Partitioning for Data Warehouses]

--Same script as 11gtable ORDERS created.table ORDERLINES created.

20

Interval-Reference Example in 12c[12c Partitioning for Data Warehouses]

--What partitions do we have?select table_name, partition_name, high_value, intervalfrom user_tab_partitionswhere lower(table_name) in ('orders', 'orderlines');

21

Interval-Reference Example in 12c[12c Partitioning for Data Warehouses]

--Insert some data into and check partitionsinsert into orders values (1, 20131231);insert into orders values (2, 20140102);insert into orders values (3, 20140113);commit;1 rows inserted.1 rows inserted.1 rows inserted.committed. select table_name, partition_name, high_value, intervalfrom user_tab_partitionswhere lower(table_name) in ('orders', 'orderlines');

22

Interval-Reference Example in 12c[12c Partitioning for Data Warehouses]

--Insert into orderlines and check partitionsinsert into orderlines values (1, 2);commit;1 rows inserted.committed.

23

Interval-Reference Example in 12c[12c Partitioning for Data Warehouses]

--Split into subpartitionsalter table orderssplit partition for (20140104) at (20140104)into (partition p20140101, partition p20140104);table ORDERS altered. select table_name, partition_name, high_value, intervalfrom user_tab_partitionswhere lower(table_name) in ('orders', 'orderlines');

Indexes

24

[ 12c Partitioning for Data Warehouses]

25

Types of Indexes on Partitioned Tables[12c Partitioning for Data Warehouses]

• Global Non-Partitioned Index• Global Partitioned Index• Local Index

26

Partial Index[12c Partitioning for Data Warehouses]

• Index that only spans certain partitions, not all

• Works on local and global indexes• Can be overwritten at any time

27

Partial Index[12c Partitioning for Data Warehouses]

* Chart from an Oracle presentation

28

Partial Index Example[12c Partitioning for Data Warehouses]

create table orders( order_number number , col2 number , col3 number , col4 number)indexing offpartition by range(order_number)( partition p1 values less than (100) indexing on, partition p2 values less than (200) indexing on, partition p3 values less than (300) indexing on, partition p4 values less than (400) indexing on, partition p5 values less than (500) indexing on, partition p_max values less than (MAXVALUE) indexing off);table ORDERS created.

29

Partial Index Example[12c Partitioning for Data Warehouses]

Partition definitions:select table_name, partition_name, high_value, indexingfrom user_tab_partitions where table_name = 'ORDERS';

30

Partial Index Example[12c Partitioning for Data Warehouses]

Now create two LOCAL indexes; the first one is a partial index.create index orders_idx1 on orders(order_number) local

indexing partial;create index orders_idx2 on orders(col2) local;index ORDERS_IDX1 created.index ORDERS_IDX2 created.

And let’s check out how these are defined in the Index Partitions table.select index_name, partition_name, statusfrom user_ind_partitionswhere index_name in ('ORDERS_IDX1', 'ORDERS_IDX2');

31

Partial Index Example[12c Partitioning for Data Warehouses]

Now let’s create two GLOBAL indexes, the first one being a partial index.create index orders_idx3 on orders(col3)

indexing partial;create index orders_idx4 on orders(col4);index ORDERS_IDX3 created.index ORDERS_IDX4 created.

And now let’s query the indexes table for these indexes.

select index_name, status, indexingfrom user_indexeswhere index_name in ('ORDERS_IDX3',

'ORDERS_IDX4');

32

Partial Index Example[12c Partitioning for Data Warehouses]

Check segments:select segment_name, segment_type, count(*)from user_segmentswhere segment_name in ('ORDERS_IDX1', 'ORDERS_IDX2',

'ORDERS_IDX3', 'ORDERS_IDX4')group by segment_name, segment_typeorder by 1;

33

Partial Index Example[12c Partitioning for Data Warehouses]

Explain Plan for a query against orders_idx3explain plan for select count(*) from orders where

col3 = 3;select * from table(dbms_xplan.display);

Partition Maintenance

34

[ 12c Partitioning for Data Warehouses]

35

Adding or Dropping Multiple Partitions[12c Partitioning for Data Warehouses]

Now available in 12c

ALTER TABLE orders_range_part ADD PARTITION 2014 VALUES LESS THAN to_date(‘01-01-

2015’, ‘MM-DD-YYYY’), PARTITION 2015 VALUES LESS THAN to_date(‘01-01-

2016’, ‘MM-DD-YYYY’), PARTITION 2016 VALUES LESS THAN to_date(‘01-01-

2017’, ‘MM-DD-YYYY’);

36

Splitting or Merging Multiple Partitions[12c Partitioning for Data Warehouses]

Now available in 12c

Merge:ALTER TABLE orders_range_part

MERGE PARTITIONS year_2010, year_2011, year_2012, year_2013INTO PARTITION historical_data_partition;

Split:ALTER TABLE orders_range_part

SPLIT PARTITION year_2013 INTO(year_2013_q1 VALUES LESS THAN to_date(‘04-01-2013’, ‘MM-DD-YYYY’), year_2013_q2 VALUES LESS THAN to_date(‘07-01-2013’, ‘MM-DD-YYYY’), year_2013_q3 VALUES LESS THAN to_date(‘10-01-2013’, ‘MM-DD-YYYY’), year_2013_q4);

37

Splitting or Merging Multiple Partitions[12c Partitioning for Data Warehouses]

Another way to write these as a range or list of values. ALTER TABLE orders_range_part

MERGE PARTITIONS year_2010 to year_2013INTO PARTITION historical_data_partition;

ALTER TABLE orders_range_partMERGE PARTITIONS for (to_date(‘01-01-2010’, ‘MM-DD-YYYY’)), for (to_date(‘01-01-2011’, ‘MM-DD-YYYY’)), for (to_date(‘01-01-2012’, ‘MM-DD-YYYY’)), for (to_date(‘01-01-2013’, ‘MM-DD-YYYY’)),INTO PARTITION historical_data_partition;

38

Cascading Truncate or Exchange Partition

[12c Partitioning for Data Warehouses]

• For Reference partitioning• CASCADE applies to the whole tree in one single, atomic transaction• ON DELETE for FK’s required

• ALTER TABLE orders TRUNCATE PARTITION 2011_q1 CASCADE;

39

Cascading Truncate[12c Partitioning for Data Warehouses]

11g Truncate

12c Truncate

Child 1 Child 2

Parent Parent

Child 1 Child 2

3

21

1

40

Exchange Partition[12c Partitioning for Data Warehouses]

11g Exchange

12c Exchange

Child 1

Child 2

Parent Parent

Child 1

Child 2

New Parent

New Child

New Parent

New Child

41

Asynchronous Global Index Maintenance

[12c Partitioning for Data Warehouses]

• For TRUNCATE or DROP commands• Results in no waiting for global index maintenance

What does it do?• Database will keep track of what records have been orphaned• Index stays usable• When queries are run, these orphaned records are filtered out of the index• Synchronization of orphaned records can happen multiple ways:

• SYS.PMO_DEFERRED_GIDX_MAINT_JOB, runs by default at 2 am• Can manually run above job• Run ALTER INDEX REBUILD [PARTITION]• Run ALTER INDEX [PARTITION] COALESCE CLEANUP

42

Asynchronous Global Index Maintenance Example

[12c Partitioning for Data Warehouses]

Create a range partitioned table with 500 records in 5 partitions.create table orders( order_number number)partition by range(order_number)( partition p1 values less than (100), partition p2 values less than (200), partition p3 values less than (300), partition p4 values less than (400), partition p5 values less than (500), partition p_max values less than (MAXVALUE));table ORDERS created.

43

Asynchronous Global Index Maintenance Example

[12c Partitioning for Data Warehouses]

insert /*+ APPEND*/ into ordersselect level from dualconnect by level < 501;commit;500 rows inserted.committed. select count(*)from orders;COUNT(*)---------- 500

44

Asynchronous Global Index Maintenance Example

[12c Partitioning for Data Warehouses]

Now create an index on this table. When the index is first created, it will not have any orphaned records.

create index orders_idx on orders(order_number);index ORDERS_IDX created. select index_name, orphaned_entriesfrom user_indexeswhere index_name = 'ORDERS_IDX'; INDEX_NAME ORPHANED_ENTRIES--------------------------------------------ORDERS_IDX NO

45

Asynchronous Global Index Maintenance Example

[12c Partitioning for Data Warehouses]

Now we are going to truncate the partition. This statement runs super fast and the index is still valid.

alter table orders truncate partition p1 update indexes;

table ORDERS altered. select index_name, status, orphaned_entriesfrom user_indexeswhere index_name = 'ORDERS_IDX'; INDEX_NAME STATUS ORPHANED_ENTRIES----------------------------------------------ORDERS_IDX VALID YES

46

Asynchronous Global Index Maintenance Example

[12c Partitioning for Data Warehouses]

Let’s manually clean up orphaned records.exec dbms_part.cleanup_gidx();anonymous block completed select index_name, status, orphaned_entriesfrom user_indexeswhere index_name = 'ORDERS_IDX'; INDEX_NAME STATUS ORPHANED_ENTRIES----------------------------------------------ORDERS_IDX VALID NO

47

Online Partition Move[12c Partitioning for Data Warehouses]

• Partition move operations can now be done without locking the object• Allows for 24/7 availability• Can be used to move partitions with older data to slower, cheaper storage

ALTER TABLE ordersMOVE PARTITION year2010TABLESPACE old_storageUPDATE INDEXES ONLINE;

ALTER TABLE ordersMOVE PARTITION year2010COMPRESSUPDATE INDEXES ONLINE;

48

Online Partition Move[12c Partitioning for Data Warehouses]

• Compression while DML is being performed with have an impact on compression efficiency

• Best practice to reduce concurrent DML during partition move because it requires additional disk space and resources for journaling

49

A few tips[12c Partitioning for Data Warehouses]

• If you want to change an existing partitioned table to interval partitioning, you can execute this command:ALTER TABLE <table name> SET INTERVAL (numtoyminterval(1,'MONTH'));

• With interval partitioning, to change the tablespace that new partitions are stored in use:ALTER TABLE <table name> SET STORE IN (<tablespace name>);

At the end of the year, we will merge our 2012 partitions (tables & indexes)

daily to weekly or monthlyweekly to quarterly

These partitions also get moved off of SSD storage and onto slower netapp disksAny non-interval partitioned tables need 2015 partitions added

Non of this is manual. We have a script to perform these tasks automatically on Dec 31st.

50

Maintenance at Backcountry[12c Partitioning for Data Warehouses]

????Thank you.Twitter: @mekolbeEmail: mkolbe@backcountry.com

51