Smaller is Better | הבורסה לניירות ערךDBAישי גרין | ראש צוות

1. Data Compression – Why bother?

2. Physical Compression Methods

3. Logical Compression Methods

3

4

5

6

The bigger the data – the bigger the challenges

Data Compression – Why bother?

Every action takes longer

• Data Access Queries• Loading data volumes• Archiving• Maintenance (backups, fragmentation,

statistics, data replications…)• Storage constraints

Data Compression – Why bother?

1. Archiving

2. Data compression (SQL 2008)

3. Smaller data types

4. Horizontal Partitioning

5. Vertical Partitioning

Physical Compression Methods

The most efficient way to minimize data in OLTP

• Good: Archive data into a Archive Repository

• Better: Archive data to flat files or other media (BCP out)

• Best: Purge data. Retrieve data from backup if needed.

Archiving

Compression Type

1. Data Compression

• Table level

• Index level

2. Backup Compression

Compression Methods

3. Row Compression

4. Page Compression

Data compression (SQL 2008)

Row Compression

Storing fixed-length data types in variable-length storage format.

A compressed row uses 4 bits per compressed column to store the length of the data in the column.

Data compression (SQL 2008)

Data compression (SQL 2008)

13

Row Compression

Data compression (SQL 2008)

PAGE Compressiona superset of row compression. optimizes storage of multiple rows in a page, by

minimizing the data redundancy. Page compression uses prefix compression and dictionary compression.

• Prefix compression looks for common patterns in the beginning of the column values on a given column across all rows on each page.

• Dictionary compression looks for exact value matches across all columns and rows on each page.

14

14

Data compression (SQL 2008)

PAGE Compression

15

  Prefix Compression   Dictionary Compression

Data compression (SQL 2008)

16

Performance

Data compression (SQL 2008)

17

Performance

CPU IO • Better performance when IO is a bottleneck• Reduces storage needs significantly• Use advisory procedure to calculate compression

Data compression (SQL 2008)

Performance - Summary

DATA Type Storage size Range

bigint 8 bytes. –2^31–( 2,147,483,648 )through 2^31–1 (2,147,483,647integer 4 bytes. – 32,768 to 32,767smallint 2 bytes. 0 to 255tinyint 1 byte . 1 or 0numeric (p, s)

19 bytes.–10^38+1 through 10^38–1

money 8 bytes. Monetary data values from (–2^63/10000) (–922,337,203,685,477.5808) through 2^63–1 (922,337,203,685,477.5807)

float 8 bytes Floating point number data from –1.79E +308 through 1.79E+308

19

Using Smallest Data Type Available

Using Smallest Data Type Available

Examples:• Char (large fixed value) Varchar (large value)• UniCode data type NonUnicode

• Datetime SmallDateTime• Datetime Date• Datetime Time

20

Partitioning

Definition: Breakup a table or index into smaller

parts, that when stitched recreate the original object.

Types of Partitioning:1. Horizontal2. Vertical

21

• Each table maintains a similar schema, but contains fewer rows.

• There is no data overlap between the partitions.

• A logical union of partitions recreates the original logical table.

22

Horizontal Partitioning

Horizontal Partitioning

Horizontal Partitioning

24

Fast storage

Medium speed storage

Slow storage

Horizontal Partitioning

3 Types of implementing Horizontal Partitions

• Partitioned Views

• Partitioned Indexes

• Partitioned Tables

26

Partitioned view• You can have differences between the tables

the partitioned view references

• The optimizer must consider each partition separately and duplicate the query's filter against the view for all partitions in the plan

• Limits on update or delete operations

Horizontal Partitioning - until SQL 2005

Native Horizontal PartitioningSQL 2005 and on

One Object!!!• One table • One index • One Partition Function • One Partition Scheme

Vertical Partitioning

• Divide a logical table into multiple physical tables that contain fewer columns, but the same number of rows.

• To Recreate the original table, join the partitioned tables on the primary key columns.

Vertical Partitioning

Partitioning - Summary

• use Vertical Partitioning when some columns are more queried than others(columns is the Select list)

• Use Horizontal Partitioning when part of the data is queried more than others(columns is the Where clause)

DEMO

3131

Logical Compression Methods

Logical Compression

Methods1. Covering Indexes

2. Filtered Indexes

3. Sparse Columns

4. Refactoring TSQL to retrieve less data

When it comes to data

Smaller is Better!!

33