how to manage and monitor large sql server estates

Large SQL Server EstatesHow to manage and monitor them…

Web: www.sqlwatchmen.comEmail: Jim.Murphy@sqlwatchmen.comTwitter: @SQLMurph

Jim MurphyCEOSQLWatchmen, LLC.

Who is SQLWatchmen?

§ DBA Services – performance tuning,

disaster recovery, maintenance

§ SQL Programming - views, SPs, triggers

§ Integration Services - SSIS for trying

systems together

§ Reporting - SSRS for custom analytics

to fill business needs

§ Cloud Migration

§ Tech company in Austin, TX

§ Can serve as Remote / Fractional DBAs

§ SQL Server Specialists (We work alongside

general tech consulting firms)

§ Compliance (HIPAA, SOX, SOC2)

How can we assist you?

2

Only Scratching The Surface

• There is so much we have to skip due to timing.• Discussing two aspects of large estates:

• Managing• Monitoring

• We’ll discuss DIY options along the way.

3

WHAT’S ALL THIS NOISE

TIPS & TRICKS

PITFALLS & AVOIDING THEM

CONCLUSION

DEMO

4

5

PITFALLS FOR DBA’S WHEN MANAGING LARGE SQL SERVER ESTATES

Basics - Alerts

Systems Online?

• If not, how are you alerted?• If SQL Agent stopped, no alerts!• DIY Solutions.

6

Basics – Estate View

• Dashboard needed!• DIY Solutions.• This is a not a small project!

7

Basics - Repository

This is a large ODS/DW like project!

• Collect local data.• Sweep data to central monitoring repository. (SSIS?)• Interpret results.• Make data accessible.

8

Maintenance Issues - Backups

• Are they working?• Does one instance have backups disabled?• DIY Solutions.

• Hope. Pray.

9

Maintenance Issues – Disk Space

• Disk Space Danger• Is daily trigger of >80% good enough?• Is every 5 mins too much?• PowerShell?

• Failed jobs or long running jobs• Email overload.

10

11

Management - Compliance

• Change Control and settings.• How to ensure no one changes settings?• How to enforce new instances to follow company standard?• DIY Solutions.

12

13

14

Performance – Does it really matter?

Problem: Application Hourglasses & Time-outs.

§ The database is the foundation of the entire application or corporate stack. If the database is slow…

15

Performance - Waitstats• Waitstats (sys.dm_os_wait_stats)

• http://download.microsoft.com/download/4/7/a/47a548b9-249e-484c-abd7-29f31282b04d/Performance_Tuning_Waits_Queues.doc

• Identify if system bottleneck is:• CPU• DISK• MEMORY• NETWORK

• … Then locate queries consuming those resources and tune them.

16

Performance - Perfmon• What to monitor in Perfmon.exe (Performance Monitor):

• Processor - % Processor Time.• System – Processor Queue Depth.• SQL Statistics - Batch Requests/s. (throughput).• SQL Statistics – Compilations and Recompilations.• Memory Manager – Memory Grants Pending • Physical Disk – Avg. Disk Sec/Read and /Write.• Network Interface – Bytes Total/sec.

• What NOT to monitor• PLE < 300• Disk Queue Length

17

Performance - Queries

• Find queries consuming most of the resources.• Query is a long query and I can’t paste it here. If you want it, go to

https://www.sqlwatchmen.com/contact-us and let me know you want the “worst query” script and I’ll send it to you.

18

Demo

19

PRO TIP

Pay attention to data collection FREQUENCY. If your average is over 1 min while SAN Admin’s average is over 1

hour, they may not see the spikes you report. [ ]What you see What they see

20

Performance Solutions

DIY: How to do this across 100 SQL Servers?

• Daily job/email?• Job to save query results to local tables.• Sweep changes to central server.• Baseline – What is normal?• Performance Dashboard Reports (add-in to SSMS).

21

22

WHAT IS NOISE AND HOW TO AVOID IT

23

Noisy SQL Servers

Alerts must be:• Actionable.• As infrequent as possible.

Using a service can help:• PagerDuty (integrates with Jira/others).• SQLWatchmen uses OpsGenie and Redmine.

24

Conclusion

Managing lots of SQL Server installs:• There are some great Enterprise tools PBM/CMS/PowerShell.

Monitoring lots of SQL Server installs:• DIY is complicated.• DIY is the right choice for some.• For everyone else:

• Monitoring software solves these problems!

25

26

Build Buy

Large SQL Server EstatesHow to manage and monitor them…

Web: www.sqlwatchmen.comEmail: Jim.Murphy@sqlwatchmen.comTwitter: @SQLMurph

Jim MurphyCEOSQLWatchmen, LLC.

Thank You!