management group checkup a database perspective kevin holman premier field engineer microsoft

Management Group Management Group CheckupCheckup

A Database PerspectiveA Database PerspectiveKevin Holman

Premier Field Engineer

Microsoft

Overview

My management group appears to be working fine. Why do I need to dig any deeper?

You are wasting time, space, bandwidth, and money.

If you plan to grow agent count or MP’s – you need this.

You need a way to monitor impact of a new MP.

You don’t want to spend time figuring out why things aren’t working, or slow.

Overview

But, I don’t have time!

#1 thing I hear

We spend more time down the road fixing things

Top 5% rule

Once optimized, very little work required

Make a plan, or plan to fail.

Overview (cont)

Operational Database is the easiest place to go to find performance issues and optimization possibilities within the management group

SQL configuration settingsLarge tables don’t lie.Kill that config churn.Moar Tuning please.Moar Tuning please. Yes. Moar.

SQL config settings

Limit SQL MemoryDebatedMore important on clustersDefault is unrestricted (2TB)Leave 1-4GB for the host OSUse perfmon to ensure you don’t have a problem

Memory \ Available Mbytes

Memory \ Pages/sec

Logical Disk \ Avg disk sec/Write/Read \ C:

SQL Config

Autogrow

Option 1: Disable autogrow on the OpsDB

Option 2: Enable autogrow on the OpsDB

Set growth size on DB to 500MB

Set growth size on Log to 100MB

Cap max growth to 80% of dedicated volume size, or 50-100GB, whichever is less.

SQL Config

Version

Use 64bit OSUse 64bit SQLSQL 2008 SP1 is preferredEnterprise edition is preferred but not required

Online indexing without blocking DB

Errors at night during maintenance from maintenance jobs or on Management servers

Especially on Warehouse

SQL Config

Collation

Double check this!SQL_Latin1_General_CP1_CI_ASNo other collations are supported.Must be the collation for the Instance and DB.Think about DB moveshttp://support.microsoft.com/kb/958979

http://support.microsoft.com/kb/958979

SQL Config

DB Settings

SIMPLE recovery model unless using DB replication. DB and logs on different spindle array sets which meet or exceed the R2 sizing helper.50% free space in the OpsDB at all times (or more!)Don’t run maintenance that conflicts with our built-in maintenance.

SQL Config

TempDB

Set initial size to 10-20% of the DB size to keep growths from having to occur when needed.Set growths to 500MB/100MB max (see link).Can split file into multiple files (one equally sized file per logical CPU) for better performance

http://msdn.microsoft.com/en-us/library/ms175527.aspx


SQL Config

PartitionsAlignment

Causes split I/O which is a significant performance hit

Not an issue for 2008 and newer. Affects 2003 OShttp://support.microsoft.com/kb/929491http://msdn.microsoft.com/en-us/library/dd758814(SQL.100).aspx

Allocation unit SizeSet when the partition is formatted

Use 64k for SQL DB and log partitions


http://msdn.microsoft.com/en-us/library/dd758814(SQL.100).aspx


Large Tables

They tell the story in most cases

Daily perf tables should be at the top in most cases. If not - investigate

Large Tables

Query:

SELECT top 20 so.name, si.rowcnt as row_count, 8 * Sum(CASE WHEN si.indid IN (0, 1) THEN si.reserved END) AS data_kb, Coalesce(8 * Sum(CASE WHEN si.indid NOT IN (0, 1, 255) THEN si.reserved END), 0) AS index_kb, Coalesce(8 * Sum(CASE WHEN si.indid IN (255) THEN si.reserved END), 0) AS blob_kb FROM dbo.sysobjects AS so JOIN dbo.sysindexes AS si ON (si.id = so.id) WHERE 'U' = so.type GROUP BY so.name, si.rowcnt ORDER BY data_kb DESC

Large Tables

LocalizedText or PublisherMessages“Mostly” resolved in R2R2 still requires cleanup if using SCCM MP or other conversion MP’s which create events from scriptMight not get 100%Cleanup at 1 million rows or moreTakes 20 minutes per million rows (generally)

http://blogs.technet.com/b/kevinholman/archive/2008/10/13/does-your-opsdb-keep-growing-is-your-localizedtext-table-using-all-the-space.aspx

Large Tables

Event_xx

Very common to see this be very largeEvents are of little valueWe over-collect many uninteresting eventsSee tuning section for remediation

Large Tables

Statechangeevent

Very common to see this be very largeWay too many state changesMonitor flip-flopHealthService restartsCleanup old onesSee tuning section for remediation

Large Tables

PerformanceSignatureData

From Self-Tuning Threshold (Rules)Baseline collection dataNot used except for troubleshootingWill keep 1-2 business cycles of last-inserted dataDisable them.

Large Tables

RecursiveMembership

Holds group membership dataLarge when:

Too many groups

Groups have too large of membership

Don’t create “mega-groups” which contain redundant typesUse Windows Computer in most casesStay under 1000 groups

Tuning

More than Alerts!

Every insert countsDiscovery changes (Config churn)Monitor State changesPerformance collectionEvent collectionAlertsPerformance signature data collection

Affects RMS, MS, OpsDB, Warehouse

Tuning

Discovery changesCritical in large environmentsHurts the OpsDB and WarehouseCreates lots of insert activity and is often the largest data payloadCauses RMS and DB performance issues

http://blogs.technet.com/b/kevinholman/archive/2009/10/05/what-is-config-churn.aspx



Tuning

DEMO

Report: Top Discoveries

Tuning

Monitor State ChangesMost common issue causing performance problems in customer environmentsRMS has to constantly recalculate stateCauses failures in Warehouse for maintenanceBadly written monitorsHealth Service restartingOveruse of Maintenance modeClean up old statechanges

Tuning

Monitor State Changes

http://blogs.technet.com/b/kevinholman/archive/2009/12/21/tuning-tip-do-you-have-monitors-constantly-flip-flopping.aspx

http://blogs.technet.com/b/kevinholman/archive/2009/11/24/writing-monitors-to-target-logical-or-physical-disks.aspx

http://blogs.technet.com/b/kevinholman/archive/2009/12/21/the-new-and-improved-guide-on-healthservice-restarts-aka-agents-bouncing-their-own-healthservice.aspx










Tuning

DEMO

Report: Top StateChanges

Tuning

Performance Collection

Accounts for a LOT of insert activityDon’t collect what you don’t need.Just because an MP has it enabled – doesn’t mean you need it.Data retention costs money.Top 5% usually account for most spaceMore critical in large environments

Tuning

Performance Collection

Top 20 performance insertions by perf object and counter name:

select top 20 pcv.ObjectName, pcv.CounterName, count (pcv.countername) as Total from performancedataallview as pdv, performancecounterview as pcv where (pdv.performancesourceinternalid = pcv.performancesourceinternalid) group by pcv.objectname, pcv.countername order by count (pcv.countername) desc

Tuning

DEMO

Report: Top Performance Collections

Tuning

Event CollectionVery common to see this be very largeEvents are of little valueWe over-collect many uninteresting eventsAccounts for a lot of inserts into the databaseOften a top consumer in the Warehouse

http://blogs.technet.com/b/kevinholman/archive/2009/11/25/tuning-tip-turning-off-some-over-collection-of-events.aspx




Tuning

DEMO

Report: Top Events

Tuning

AlertsDon’t ignore/disable Script Errors and WMI failuresUse published queries and reportsLook at by Repeat count and by Alert countTop consumers cause most DB impactUse KnowledgeUse ReSearch This!http://www.systemcentercentral.com/PackCatalog/PackCatalogDetails/tabid/145/IndexId/21716/Default.aspx

http://www.systemcentercentral.com/PackCatalog/PackCatalogDetails/tabid/145/IndexId/21716/Default.aspx


Tuning

DEMO

Report: Top Alerts

Tuning

Performance Signature DataOften ignoredCan add up in most cases, and be extreme in some casesData is of little valueRecommendation is to limit STT’s only where appropriate and non-critical.

http://blogs.technet.com/b/kevinholman/archive/2007/10/18/useful-operations-manager-2007-sql-queries.aspx



Tuning

DEMO

Report: Top impact by Management Pack

Summary

Keep SQL set to best practicesMemory, Autogrow, Collation, TempDB, Partitions

Large TablesQuick spot-check to make sure all is in order

Tune your environmentIt’s not that hardBe methodicalTop 5% - low hanging fruitDon’t focus only on alerts

Resource links

http://blogs.technet.com/kevinholman/archive/2007/10/18/useful-operations-manager-2007-sql-queries.aspx

http://blogs.technet.com/kevinholman/archive/2008/04/12/what-sql-maintenance-should-i-perform-on-my-opsmgr-databases.aspx




















© 2009 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions,

it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

management group checkup a database perspective kevin holman premier field engineer microsoft

Documents