Presented By : M.M.Al-Farooque

http://www.linkedin.com/in/thisisshubho

(Shubho)

Topics Handling Index fragmentation Optimizing files and partitioning tables Effective use of SQL Profiler Performance Monitor Performance troubleshooting methodology 10 steps for performance optimization

Handling Index Fragmentations What is Index Fragmentation? Fragmentation occurs in index/data pages as a result of data

insert/update/delete operations in the tables. If indexes have high fragmentations, either scanning/seeking the indexes takes much time or the indexes are not used at all (Resulting in table scan).

Two types fragmentation can occur:

Internal Fragmentation: Occurs due to the data deletion/update operation in the index pages which ends up in distribution of data as sparse matrix in the index/data pages. Also results in increase of index/data pages that increase query execution time.

External Fragmentation: Occurs due to the data insert/update operation in the index/data pages which ends up in page splitting and allocation of new index/data pages that are not contiguous. That reduces performance in determining range and read-ahead operations.

Handling Index Fragmentations How to determine whether index fragmentation occurred or not?

Execute the following SQL: select object_name(dt.object_id) Tablename,si.name

IndexName,dt.avg_fragmentation_in_percent as ExternalFragmentation,dt.avg_page_space_used_in_percent as InternalFragmentation

from ( select object_id,index_id,avg_fragmentation_in_percent,avg_page_space_used_in_percent from sys.dm_db_index_physical_stats (db_id('AdventureWorks'),null,null,null,'DETAILED' ) where index_id <> 0) as dt inner join sys.indexes si on si.object_id=dt.object_id and si.index_id=dt.index_id and dt.avg_fragmentation_in_percent>10 and dt.avg_page_space_used_in_percent<75 order by avg_fragmentation_in_percent desc

ExternalFragmentation value > 10 indicates External fragmentation occurred for corresponding index

InternalFragmentation value < 75 indicates Internal fragmentation occurred for corresponding index

Handling Index Fragmentations Defragment index fragmentation

Use “ALTER INDEX ALL ON TableName RECOGNIZE” when

External Fragmentation Between 10-15 and Internal Fragmentation between 60-75

Use “ALTER INDEX ALL ON TableName REBUILD WITH (FILLFACTOR=90,ONLINE=ON)” when

External Fragmentation >15 and Internal Fragmentation < 60

Optimizing files and partitioning tables Consider creating a "User defined file group" for your database if total database size is

over 100 MB. Make this as "Default" file group.

Consider creating multiple files per "User defined file group" so that, SQL server can concurrently access those files while retrieving data

Place heavily accessed big fat tables in one filegroup and place the table's indexes and/or text/image columns in a different filegroup on different physical disks.

Because logging is more write-intensive, place the log files on different physical disk(s) than data files.

Consider assigning the "Read only" tables into a file group that is marked as "Read Only"

Consider creating "History tables" that contains archived read only data where select operations will be applied. Use SQL server Maintanence feature to populate those history tables.

Consider partitioning big fat tables into different file groups so that the table spans across different files and SQL server engine can concurrently access the table data.

Partitioning tables

Table partitioning Partitioning lets us split a table (And,Indexes)

across multiple filegroups, based upon a user specification.

Partitioning enables database engine to read/write data, and, also to calculate (Say, calculating aggregate functions() in parallel) data operations faster.

Data could be transferred for analysis purpose based upon age, so, data volume for OLTP operation remains minimal.

Partitioning enables easy/manageable/faster way of archive/backup/recovery

Partitioning tables

Following tasks are to be performed to partition a table or index

1. Add user defined file groups to the database

2. Create partition function 3. Create a partition schema 4. Create/Modify the table or index on

the partition schema.

Partitioning tables

1. Add user defined file groups to the database

ALTER DATABASE AdventureWorks ADD FILEGROUP [2003Q3]

ALTER DATABASE AdventureWorks ADD FILE (NAME = N'2003Q3', FILENAME = N'C:\AdventureWorks\2003Q3.ndf', SIZE = 5MB, MAXSIZE = 100MB, FILEGROWTH = 5MB) TO FILEGROUP [2003Q3]

Partitioning tables 2. Create Partition function

A partition function is an object that defines the boundary points for partitioning data.

Following Command creates a partition function

CREATE PARTITION FUNCTION OrderDateRangePFN(datetime) AS RANGE LEFT FOR VALUES ('20000930 23:59:59.997', '20001231 23:59:59.997', '20010331 23:59:59.997', '20010630 23:59:59.997')

Partitioning tables 3. Create Partition Scheme The partition scheme defines the file groups

that will be used with a partition function to partition the table/index

Following command creates a partition schema CREATE PARTITION SCHEME

OrderDatePScheme AS PARTITION OrderDateRangePFN TO ([2000Q3], [2000Q4], [2001Q1], [2001Q2], [PRIMARY])

Partitioning tables

4. Create the table or index on the partition schema.

With the partition function (the logical structure) and the partition scheme (the physical structure) defined, the table can be created to take advantage of them. The table defines which scheme should be used, and the scheme defines the function

ALTER TABLE Orders ADD CONSTRAINT OrdersPK PRIMARY KEY CLUSTERED ON OrderDatePScheme (OrderDate)

Partitioning tables Partitioning Index and indexed views Following command creates a partitioned

index CREATE NONCLUSTERED INDEX

indxCustomerAddress on dbo.CustomerAddress(City) on partscheme(CustomerAddressID)

In case of an existing index, drop the index first

Partitioning indexed view means partitioning the index of the view

Effective use of SQL Profiler Tips & Tricks 1. Use existing templates, but, create your own

template when in need. 2. Save profiler trace into table for further analysis 3. Export particular events (Say, TSQLs) from

profiler trace for particular analysis. 4. Create Replay trace and replay the trace on test

server to diagnose problems. 5. Capture deadlock events on production and

diagnose on test server. 6. Capture ShowPlan to include SQL plans in the

profiler. 7. Capture Table Scan events in profiler.

Performance Monitor Profiler enables us to capture long running

queries but, cannot provide the context to explain the reasons. (e.g. Same query takes 3 seconds in production but, takes 30 ms in test server, cause, it lacks context).

Performance Monitor tool (System monitor) could be used to generate a counter log that can be used to provide the profiler trace with the context.

Performance Monitor gathers statistical data related to hardware and software metrics.

Performance Monitor Windows has lots of built in objects with

their performance counters.

When installation, Performance counters for SQL server also get installed.

Create a Performance Counter log by incorporating the following common performance counters: (Next slide)

Performance Monitor Include the following common counters

Network Interface\Output Queue length Processor\%Processor Time SQL Server:Buffer Manager\Buffer Cache Hit Ratio SQL Server:Buffer Manager\Page Life Expectancy SQL Server:SQL Statistics\Batch Requests/Sec SQL Server:SQL Statistics\SQL Compilations SQL Server:SQL Statistics\SQL Re-compilations/Sec

Performance troubleshooting methodology SQL server dynamically generates different query

plans based on: --Volume of Data --Statistics --Index variation --Parameter value in TSQL --Load on server So, when diagnosing any SQL performance

related problem , a methodology should be followed to simulate the production environment in Test server.

Performance troubleshooting methodology Performance troubleshoot methodology

--Capture trace at production server and include ShowPlan in the trace. Do the same in the Test server and compare the plans.

--Create Replay Trace in production and replay the trace on test server to create a similar load.

--In the Database Engine Tuning Advisor too, use the production trace as the workload to view tuning suggestions.

Performance troubleshooting methodology --Take performance counter log in the

production server and correlate it with the SQL profiler trace to diagnose the bottleneck.

--Analyze the Query plans in the production SQL profiler trace to diagnose whether indexes are properly utilized in the SQL’s

--Execute the TSQL (Provider earlier) to view the External/Internal fragmentation values for indexes in the target database.

10 steps for performance optimization 1. Make sure that, indexing is properly done in the database (All tables

has primary keys, and, tables has appropriate non-clustered indexes on columns which

 -Are used as search criteria fields-Are used to join other tables-Are used as foreign key fields-Have low selectivity-Are used in the ORDER BY clause

While creating indexes, create appropriate "Covering indexes" involving frequently accessed fields

Use "SQL server management studio" to view the query execution plan and identify the scopes for improvement in the SQLs

Use "SQL server tuning advisor" to get help from SQL server in creating indexes.

10 steps for performance optimization  2. Consider moving your SQL's from application to Stored

Procedures/Views if there is any  3. Re-factor the SQL's used in Stored procedures/Views/Triggers

and apply the best practices to write optimized queries

4. Consider using the "Full text search" feature to perform search on textual columns.

5. Re-build indexes if fragmentation occurs.

6. Create "Indexed Views" with expensive select queries that internally saves result sets on queries

  7. Consider refactoring your table design and apply De-

normalizations to improve select operations.

10 steps for performance optimization 8. Consider creating different "User defined file groups"

and use these based upon

-Frequency of table access-Read/write nature of tables-Types of objects (Tables/Index)

  9. Consider creating "History tables" that contains archived

read only data where select operations will be applied. Use SQL server Maintenance feature to populate those history tables.

  10. Consider partitioning big fat tables into different file

groups so that the table spans across different files and SQL server engine can concurrently access the table data.