בעיות – אין משתמשים אין

Meir DudaiSQL Server MVP

Valinor

Content slide

When do I need to start worrying?

100’s of queries/second

1000’s of DMLs/second

VLDB’s

When do I need to start worrying?

Application was designed and developed poorly…

When do I need to start worrying?

You have just migrated from other platform

What can I do?

Throw in more hardware – not very useful

What can I really do?

Improve application design

Use database enhancements and features

Change system architecture

Who Am I?

Meir Dudai MVP SQL Server consultant Editor of SQLServer.co.il

How to increase concurrency without

changing your application

How to adjust your architecture to

achieve better concurrency

Some basics

Magic settings

– RCSI

– Partition-level lock escalation

– Page Latches

Architectural design

– Bulk Load

– StreamInsight

– Caching

– Offloading Traffic

– Denali HADR

Common Concurrency Issues with Applications

Wrong transaction isolation level

Incorrect locking granularity

Missing Indexes

Accessing large dataset leading to lock escalation

Long running transactions– User interactions in active transaction

Accessing objects in reverse order across transactions

Some basics

Magic settings

– RCSI

– Partition-level lock escalation

– Page Latches

Architectural design

– Bulk Load

– StreamInsight

– Caching

– Offloading Traffic

– Denali HADR

GreenRoad Case Study:RCSI

GreenRoad is an Israeli startup improving driving behavior for fleets and consumers

Works as SAAS

More than 400 inserts/sec

Users running queries and reports frequently

Method used to increase concurrency:RCSI: Read Committed Snapshot Isolation

Row-1

Tran2 (Select)Tran1 (Update)

X-Lock S-Lock BlockedRow-1

Reader Writer Blocking

Data Page

Read Committed Snapshot Statement-Level ‘Snapshot Isolation’

New “flavor” of read committed– Turn ON/OFF on a database

Readers see committed values as of beginning of statement

• Writers do not block Readers

• Readers do not block Writers

• Writers do block writers

Can greatly reduce locking / deadlocking without changing applications

Demo

Lock Escalation

HOBT

Page Page Page

Row Row Row

T1: IX

T1: IX

T1: XT1: X

T1: XT1: X

T1: XT1: X

T1: XT1: X

T1: X

Lock Escalation

Lock EscalationConverting finer-grain locks to coarse grain locks.– Row to Table– Page to Table.

Benefits– Reduced locking overhead– Reduces Memory requirement

Triggered when– Number of locks acquired on a rowset > 5000– Memory pressure

Partitioned Tables and Indexes

SQL Server 2005 introduced partitioning, which some customers use to scale a query workload– Another common use is to streamline maintenance and enable

fast range inserts and removals from tables

FG1 FG2 FG3

PartitionedTable

Partition 1 Partition 2 Partition 3

Lock escalation on partitioned tables reduces concurrency as the table lock locks ALL partitions

Only way to solve this in SQL Server 2005 is to disable lock escalation

Lock Escalation: The Problem

IXX

FG1 FG2 FG3

PartitionedTable

Partition 1 Partition 2 Partition 3

Query 1ESCALATE

Query 2

update update

Lock Escalation: The Solution

SQL Server 2008 allows lock escalation to the partition level, allowing concurrent access to other partitions

Escalation to partition level does not block queries on other partitions

IX

XFG1 FG2 FG3

PartitionedTable

Partition 1 Partition 2 Partition 3

Query 1ESCALATE

Query 2

update update

Demo

Some basics

Magic settings

– RCSI

– Partition-level lock escalation

– Page Latches

Architectural design

– Bulk Load

– StreamInsight

– Caching

– Offloading Traffic

– Denali HADR

Silent Blocker: Page Latches

Page Header

R1

R2

R3

Free Offset

What is a Pagelatch?

Access patterns leading to latch contention– Clustering index has a increasing primary key with high

rate of inserts – Relatively small tables with a lot of read/write/delete– Indexes with low cardinality in the key with high rate of

inserts– Allocation Structures (e.g. SGAM)

– Observed with 16+ concurrent threads

Silent Blocker: Page LatchesWhat can you do?– Cluster on some other column– Partition the table on some other attribute (computed

column (hash))– Add another major key column– Change the application

Demo

Some basics

Magic settings

– RCSI

– Partition-level lock escalation

– Page Latches

Architectural design

– Bulk Load

– StreamInsight

– Caching

– Offloading Traffic

– Denali HADR

Codename Lyngby Case Study: Power of bulk inserts

Security/audit system

50GB of data inserted daily

Data is kept for one year and should be available for queries and reports at any time

Method used to increase concurrency:Bulk inserts

Benefits of Bulk Inserts

Improved concurrency

Allow application-database decoupling

In Lyngby case – 50GB a day!

http://www.tapuz.co.il/blog/userBlog.asp?FolderName=Hygge

Bulk Inserts & RCSI: SQLCAT Benchmark

What happens when you query a table while bulk inserts are running?

Bulk Inserts & RCSI: SQLCAT Benchmark

And with RCSI?

Bulk loading does not affect the size of the version store in tempdb under RCSI

Keep in mind: RCSI adds 14 noncompressible bytes into every row in every table

Bottom line: RCSI works extremely well with bulk inserts and achieves improved concurrency

More details about this benchmark:

http://sqlcat.com/technicalnotes/archive/2009/04/06/bulk-loading-data-into-a-table-with-concurrent-queries.aspx

StreamInsight:Complex Event Processing

• SQL Server 2008 R2 feature• Processing and querying of event data

streams• Data queried while “in flight”• May involve multiple concurrent event

sources• Works with high data rates• Aims for near-zero latency

Isn’t This Just a Database Application?

Database CEP

Queries Ad hoc on stored data Continuous standing queries

Latency Seconds Milliseconds

Data Rate Hundreds per Second Tens of thousands per second

request

response

Eventoutput streaminput

stream

Caching

• Your database can get some rest...

Microsoft AppFabric

Other solutions

• Memcache−Requires application change

• 3rd party−Usually transparent

Tapuz Case Study: Offloading Traffic

Tapuz is the leading user content based website in Israel– 1,200 forums– 74,000 blogs– 500,000 visitors each month

More than 4000 queries/sec

Method used to increase concurrency:Offloading using Transactional Replication

Database Scale Out

Queries scaled out (often geographically) similar to reporting cases

Databases replicate reciprocally and are writable

Redundancy provides fault tolerance and lowers maintenance downtime

Online upgrades possible

Denali HADR• V.Next, code named “Denali”

offers a brand new HADR feature• High-availability and disaster

recovery feature, using up to 4 replicas

• Replicas can be synchronous or asynchronous −CTP supports only single

asynchronous replica

Denali HADR

• Coming up next!

Q&A

Related sessionsMonday

• 14:00-15:15 − New High Availability Solution in SQL

Server “Denali”− Justin Erickson & Oren Bouni

Tuesday• 09:30-10:45

− Mission Critical: Improving HighAvailability and ReliabilityAssaf Fraenkel

• 12:30-13:30 − Designing High Performance I/O System

with SQL Server− Dubi Lebel

• 11:15-12:15 − Troubleshoot Like a Pro with the

Microsoft SQL Server 2008 Tools − Guy Glantser

Open your mind to uncommon features and technologies in SQL Server and try themUsing these features, amazing concurrency can be achieved by any DBA using SQL Server!

ופייסבוק משובים

השלמה- מירב

Recommended Courses: SQL 2008 • Maintaining a Microsoft SQL Server 2008

Database

• 2008 Implementing a Microsoft SQL Server Database

! מיוחד מבצע ועכשיומקורסי לאחד כאן SQLהרשם המוצעים

: משלימה הסמכה בחינת קנה•70-432

: 2008, TS Microsoft SQL Server Implementation and Maintenance

•70-433 : 2008, TS Microsoft SQL Server Database Development

- ל שנתי מנוי ללא TechNetוקבל חוזרת לבחינה ואפשרותעלות

, פנה והרשמה נוספים המוסמכות לפרטים למכללות

ופייסבוק משובים

השלמה- מירב