Index the obvious and not so obvious

About myself:

• Seasoned data architect/DBA with 15 years of SQL Server experience

• Independent consultant, currently Technical Delivery Manager at Government of Alberta

• Microsoft Certified System Engineer: MCSE

• Oracle Certified Professional: Oracle DBA

• IBM Certified Solution Expert: DB2 UDB

http://netdbsolutions.comhttps://twitter.com/HarryZheng

http://ca.linkedin.com/in/harryzhenghttps://www.facebook.com/Harry.H.Zheng

2

Session agenda

1. SQL Server index basics

2. Index Design

3. Common index issues

3

SQL Server index basics

Why do we need index?

• Reduce rows selected in a query as early as possible• Help join • Support sort

4

SQL Server index basics

Index Type:

• Clustered index• Non-clustered index• Unique• Index with included columns • Filtered index

Not covered in this session• Columnstore • Spatial index• XML index• Full-text index

5

SQL Server index basicsB-Tree Index structure:

6

SQL Server index basics

Get Index Information

• exec sp_helpindex

• select * from sys.indexes • select * from sys.index_columns

7

SQL Server index basics

Index Statistics Info• DBCC SHOW_STATISTICS ('demo.person', 'IX_Person_BusinessIdentityID');

8

SQL Server index basics

Index Statistics Info• The statistics allow the optimizer to determine the selectivity of

an index• A unique, single-column index always has a selectivity of 1• One index entry points to exactly one row

• Density is the inverse of selectivity• Density values range from 0 to 1• A selective index has a density of 0.10 or less• A unique, single-column index always has a density of 0.0

• When the index is composite• SQL Server maintains detailed statistics only on the leftmost column• It does compute density for each column• Assume there is an index on (col1, col2, col3)• Density is computed for• Col1• Col1 + Col2• Col1 + Col2 + Col3

9

SQL Server index basics

Clustered index:

• Leaf level is the actual data page• Non-leaf levels contains the index key columns• Clustered index scan = table scan on Heap

10

SQL Server index basics

  Clustered index - Primary key:

• Alter table Demo.Person add constraint PK_Person primary key (BusinessEntityID);

11

SQL Server index basics

Non-clustered index:

• Leaf level contains the key and include columns• Non-leaf levels contains the index key columns

12

SQL Server index basics

 Unique index:• Create unique index IX_Person_BusinessIdentityID on

demo.Person(BusinessEntityID);

• Is clustered index always unique?

13

SQL Server index basics

Index with included columns - Covering index:Create index IX_Person_LastName_FirstName on demo.Person ( LastName,FirstName) include (ModifiedDate);

Covered query:select FirstName, LastName, ModifiedDatefrom Demo.Personwhere LastName = 'Miller'and FirstName = 'Dylan'

14

SQL Server index basics

Filtered index:Create index IX_Person_persontype on demo.Person ( persontype) where persontype = 'EM';

Covered query:select * from Demo.Personwhere persontype = 'EM'

15

Index Design

General design rule:

• Clustered index:• Narrow• Unique• Static• Ever-increasing

•  Non-clustered index:• Wider often more useful than narrow • Decided by the queries that are run against the table

16

Index Design

 Help Where clause:• Queries can seek on index Where the columns in the Where

clause are a left based subset of the index key • Columns used in equality predicates before inequality

17

Index Design

Help Where clause:Case 1:

Select * from Demo.PersonWhere FirstName = 'Dylan' and LastName = 'Miller'

How to index?

18

Index Design

Help Where clause:Case 1:

Select * from Demo.PersonWhere FirstName = 'Dylan' and LastName = 'Miller'

How to index?

Create index IX_Person_FirstName_LastName on Demo.Person

( FirstName, LastName);

19

Index Design

Help Where clause:Case 2 Session 1:

Select * from Demo.PersonWhere FirstName = 'Dylan' and LastName = 'Miller'

Case 2 Session 2:Select * from Demo.PersonWhere LastName = 'Miller'

How to index?

20

Index Design

Help Where clause: Case 2 Session 1:

Select * from Demo.PersonWhere FirstName = 'Dylan' and LastName = 'Miller'

Case 2 Session 2:Select * from Demo.PersonWhere LastName = 'Miller'

How to index?Create index IX_Person_LastName_FirstName on demo.Person ( LastName, FirstName);

21

Index Design

Help Where clause: Session 3:

Select * from Demo.PersonWhere LastName = 'Miller'and ModifiedDate > '2006-09-01‘

How to index?

22

Index Design

Help Where clause: Session 3:

Select * from Demo.PersonWhere LastName = 'Miller'and ModifiedDate > '2006-09-01‘

How to index?

Create index IX_Person_LastName_ModifiedDate on demo.Person ( LastName, ModifiedDate);

23

Index Design

Help Where clause: Case 4:

Select * from Demo.PersonWhere LastName = 'Miller'and FirstName > 'Dylan'and ModifiedDate > '2006-09-01‘

How to index?

24

Index Design

Help Where clause: Case 4:

Select * from Demo.PersonWhere LastName = 'Miller'and FirstName > 'Dylan'and ModifiedDate > '2006-09-01‘

How to index?

Create index IX_Person_LastName_FirstName_ModifiedDate on demo.Person ( LastName,FirstName, ModifiedDate);

Create index IX_Person_LastName_ModifiedDate_FirstName on demo.Person ( LastName, ModifiedDate,FirstName);

25

Index Design

Help Where clause: Case 5:

Select * from Demo.PersonWhere LastName = 'Miller'Or FirstName = 'Dylan‘

How to index?

26

Index Design

Help Where clause: Case 5:

Select * from Demo.PersonWhere LastName = 'Miller'Or FirstName = 'Dylan‘

How to index?

Create index IX_Person_FirstName on demo.Person ( FirstName);Create index IX_Person_LastName on demo.Person ( LastName);

27

Index Design

Help join: Case 6:

Select A.AddressLine1,A.City, SP.Name as StateProvinceFrom Demo.Address AInner Join Demo.StateProvince SP

on SP.StateProvinceID = A.StateProvinceID

How to index?

28

Index Design

Help join: Case 6:

Select A.AddressLine1,A.City, SP.Name as StateProvinceFrom Demo.Address AInner Join Demo.StateProvince SP

on SP.StateProvinceID = A.StateProvinceID

How to index?Create index IX_Address_StateProvinceID on demo.Address ( StateProvinceID);

29

Index Design

Help Where clause: Case 7:

Select A.AddressLine1,A.City, SP.Name as StateProvinceFrom Demo.Address AInner Join Demo.StateProvince SP

on SP.StateProvinceID = A.StateProvinceIDWhere A.City = 'Bothell‘

How to index?

30

Index Design

Help Where clause: Case 7:

Select A.AddressLine1,A.City, SP.Name as StateProvinceFrom Demo.Address AInner Join Demo.StateProvince SP

on SP.StateProvinceID = A.StateProvinceIDWhere A.City = 'Bothell‘

How to index?

Create index IX_Address_City_StateProvinceID on demo.Address ( City, StateProvinceID);

31

Index Design

Help order by clause:Case 8:Select * from Demo.PersonWhere LastName = 'Miller'Order by FirstName

How to index?

32

Index Design

Help Order by clause:Case 8:

Select * from Demo.PersonWhere LastName = 'Miller'Order by FirstName

How to index?

Create index IX_Person_LastName_FirstName on Demo.Person ( LastName, FirstName);

33

Index Design

Help Order by clause:Case 9:

Select FirstName, LastName, ModifiedDate from Demo.PersonWhere LastName = 'Miller'and ModifiedDate > '2006-09-01'Order by FirstName

How to index?

34

Index Design

Help Order by clause:Case 9:

Select FirstName, LastName, ModifiedDate from Demo.PersonWhere LastName = 'Miller'and ModifiedDate > '2006-09-01'Order by FirstName

How to index?1.Create index IX_Person_LastName_ModifiedDate on demo.Person ( LastName, ModifiedDate);2.Create index IX_Person_LastName_ModifiedDate_FirstName on demo.Person ( LastName, ModifiedDate,FirstName);3.Create index IX_Person_LastName_ModifiedDate on demo.Person ( LastName, ModifiedDate) include (FirstName);

35

Common index issues

Drawbacks of index:

• Slow down DML• Disk Space• Fragmentation – page split – low page density

36

Common index issues

Index Fragmentation:• DBCC SHOWCONTIGDBCC SHOWCONTIG scanning 'Person' table...Table: 'Person' (615673241); index ID: 2, database ID: 17LEAF level scan performed.- Pages Scanned................................: 25- Extents Scanned..............................: 5- Extent Switches..............................: 4- Avg. Pages per Extent........................: 5.0- Scan Density [Best Count:Actual Count].......: 80.00% [4:5]- Logical Scan Fragmentation ..................: 12.00%- Extent Scan Fragmentation ...................: 20.00%- Avg. Bytes Free per Page.....................: 107.2- Avg. Page Density (full).....................: 98.68%DBCC execution completed. If DBCC printed error messages, contact your system administrator.

Defrag:• DBCC DBREINDEX• DBCC INDEXDEFRAG

37

Common index issues

Misconception: 

• Index is wonderful, create index to cover every query

• How about create a separate index for each commonly used columns?

38

Common index issues

Disorder: 1. Duplicate indexes• Will SQL Server use both indexes?

2. Index Hoarder• Lots of Non-Clustered indexes on a single table• >5% of Non-Clustered indexes unused• Non-Clustered indexes with 0 reads• Indexes with 7 or more columns• Clustered indexes with > 1 column

39

Common index issues

Disorder:3. Feature Phobia• Included columns SQL 2005+• Filtered indexes – SQL 2008+ Enterprise Edition

4. Self-Loathing indexes • Low fill factor, disabled indexes

5. Index-A-Phobia• Afraid to add index

40

Common index issues

Diagnostics Tools:

• sp_BlitzIndex • http://www.brentozar.com/blitzindex/

41

Common index issues

Don’t:

• Don’t index for the sake of indexing• Don’t create duplicated indexes  • Don’t create very wide indexes• Don’t just trust SSMS suggestion

Discussion:• Should we always create indexes for FK columns?

42

Common index issues

DMVs for Index Tune

• sys.dm_db_index_usage_stats • sys.dm_db_index_operational_stats• sys.dm_db_index_physical_stats

• sys.dm_db_missing_index_groups • sys.dm_db_missing_index_details • sys.dm_db_missing_index_group_stats • sys.dm_db_missing_index_columns

43

Reference

PASS Summit 2012 Sessions:

• What, Where, Why and How of Indexes (DBA-210)• Index Psychiatry Diagnose and Treat the Top 5 Disorders (AD-204)• http://sqlinthewild.co.za/index.php/2010/09/14/one-wide-index-or-multiple-n

arrow-indexes/• http://www.sqlservercentral.com/articles/Indexing/68439/

44