Windows Azure DiagnosticsLogging and Monitoring in the Cloud

Matthew KernerProgram Manager, Windows Azure

PDC09-SVC15

Diagnostics: Single Server vs. the CloudSingle Server> Static Environment

> Single well-known instance> Traceable local transactions

> Local Access Feasible> All in one TS session> Data & tools co-located> In-Place Changes

Cloud> Dynamic Environment

> Multi-instance, elastic capacity

> Distributed transactions> Local Access Infeasible

> Many nodes> Distributed, scaled-out

data> Service Upgrades

Windows Azure Diagnostics> SDK component providing distributed

monitoring & data collection for cloud apps

> Support Standard Diagnostics APIs> Cloud-Friendly

> Manage multiple role instances centrally> Scalable

> Built on Windows Azure Storage & used by scale-out Windows Azure platform components

> Developer In Control> What to collect & when to collect it

Hello World

Windows Azure Diagnostics

Demo

Windows Azure Diagnostics

Role

Role Instance

Diagnostic Monitor

Configuration

Quota enforcement

Local directory storage

Data collection

(traces, logs, crash dumps)

Windows Data

SourcesIIS Logs & Failed Request

LogsPerf Counters

Windows Event Logs

Windows Azure Diagnostics

Role

Role Instance

Diagnostic Monitor

Local directory storage

Request upload

Windows Azure Storage

Scheduled or on-demand upload

Windows Data

Sources

Windows Azure Diagnostics

Windows AzureHosted Service

DevelopmentFabric

DevelopmentFabric

Windows Azure Diagnostics

Windows AzureHosted Service

Controller Code

Desktop Diag ApplicationDiagnosti

c Manager

Configure

HOW-TO

Activate Windows Azure DiagnosticsGenerate DataEnable Local BufferingTransfer to Windows Azure Storage

Sample: Activate WA Diagnostics

// This is done for you automatically by // Windows Azure Tools for Visual Studio

// Add a reference to Microsoft.WindowsAzure.Diagnosticsusing Microsoft.WindowsAzure.Diagnostics; // Activate diagnostics in the role's OnStart() methodpublic override bool OnStart(){ // Use the connection string contained in the // application configuration setting named // "DiagnosticsConnectionString” // If the value of this setting is // "UseDevelopmentStorage=true" then will use dev stg DiagnosticMonitor.Start("DiagnosticsConnectionString"); ...}

Sample: Web.Config Changes

<!– This is automatically inserted by VS. The listener routes System.Diagnostics.Trace messages to Windows Azure Diagnostics.--><system.diagnostics> <trace> <listeners> <add type="Microsoft.WindowsAzure.Diagnostics.DiagnosticMonitorTraceListener, Microsoft.WindowsAzure.Diagnostics, Version=1.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35" name="AzureDiagnostics"> <filter type="" /> </add> </listeners> </trace></system.diagnostics>

Sample: Generate Diagnostics Datastring myRoleInstanceName = RoleEnvironment.CurrentRoleInstance.Id;

// Trace with standard .Net tracing APIsSystem.Diagnostics.Trace.TraceInformation( "Informational trace from " + myRoleInstanceName); // Capture full crash dumpsCrashDumps.EnableCollection(true);// Capture mini crash dumpsCrashDumps.EnableCollection(false);

Sample: Enable Local Data Buffering// Managed traces, IIS logs, failed request logs, // crashdumps and WA diags internal logs are buffered // in local storage by default. Other data sources must be // added explicitlyDiagnosticMonitorConfiguration diagConfig = DiagnosticMonitor.GetDefaultInitialConfiguration();

// Add performance counter monitoringPerformanceCounterConfiguration procTimeConfig = new PerformanceCounterConfiguration();

// Run typeperf.exe /q to query for counter namesprocTimeConfig.CounterSpecifier = @"\Processor(*)\% Processor Time";procTimeConfig.SampleRate = System.TimeSpan.FromSeconds(1.0);

diagConfig.PerformanceCounters.DataSources.Add(procTimeConfig);

// Continued on next slide...

Sample: Enable Local Data Buffering// Continued from previous slide... // Add event collection from the Windows Event Log// Syntax: <Channel>!<xpath query> // http://msdn.microsoft.com/en-us/library/dd996910(VS.85).aspx diagConfig.WindowsEventLog.DataSources.Add("System!*");

// Restart diagnostics with this custom local buffering // configurationDiagnosticMonitor.Start( "DiagnosticsConnectionString", diagConfig);

Sample: Web.Config Changes<!-- You can optionally enable IIS failed request tracing. This has some performance overhead A service upgrade is required to toggle this setting.--><system.webServer> <tracing> <traceFailedRequests> <add path="*"> <traceAreas> <add provider="ASP" verbosity="Verbose" /> <add provider="ASPNET" areas="Infrastructure,Module,Page,AppService" verbosity="Verbose" /> <add provider="ISAPI Extension" verbosity="Verbose"/> <add provider="WWW Server" verbosity="Verbose"/> </traceAreas> <failureDefinitions statusCodes="200-599"/> </add> </traceFailedRequests> </tracing></system.webServer>

Sample: Scheduled Data Transfer

// Start off with the default initial configurationDiagnosticMonitorConfiguration dc = DiagnosticMonitor.GetDefaultInitialConfiguration();

dc.WindowsEventLog.DataSources.Add("Application!*");

dc.WindowsEventLog.ScheduledTransferPeriod = System.TimeSpan.FromMinutes(5.0);

DiagnosticMonitor.Start("DiagnosticsConnectionString", dc);

Sample: On-Demand Data Transfer// On-Demand transfer of buffered files.// This code can live in the role, or on the desktop,// or even in another service.var ddm = new DeploymentDiagnosticManager( storageAccount, deploymentID);var ridm = ddm.GetRoleInstanceDiagnosticManager( roleName, roleInstanceName);var dataBuffersToTransfer = DataBufferName.Logs;OnDemandTransferOptions transferOptions = new OnDemandTransferOptions();transferOptions.From = DateTime.MinValue;transferOptions.To = DateTime.UtcNow;transferOptions.LogLevelFilter = LogLevel.Critical;Guid requestID = ridm.BeginOnDemandTransfer( dataBuffersToTransfer, transferOptions);

Storage Considerations> Standard WA Storage costs apply for

transactions, storage & bandwidth> Data Retention

> Local buffers are aged out by the Diagnostic Monitor according to configurable quotas

> You control data retention for data in table/blob storage

> Query Performance on Tabular Data> Partitioned by high-order bits of the tick

count> Query by time is efficient> Filter by verbosity level at transfer time

Feature Summary> Local data buffering

> Configurable trace, perf counter, Windows event log, IIS log & file buffering

> Local buffer quota management> Query & modify config from the

cloud or from the desktop per role instance

> Transfer to WA Storage> Scheduled & on-demand> Filter by data type, verbosity &

time range> Transfer completion notification> Query & modify from the cloud

and from the desktop per role instance

Data SourceDefault Configuration How to Configure

Format

Trace logsEnabled, stored locally

Diag API, Trace listener Table

Performance Counters Disabled Diag API TableWindows Event Logs Disabled Diag API Table

Infrastructure LogsEnabled, stored locally Diag API Table

IIS LogsEnabled, stored locally Diag API, Web.config Blob

IIS Failed Request Logs Disabled Diag API, Web.config BlobApplication Crash Dumps Disabled Diag API, Crash API BlobArbitrary Logs & Files Disabled Diag API Blob

> Under the hood> Role runs in Performance Log Users group> Coming Soon: IIS Logs generated in role’s local data directory

Common Diagnostic Tasks> Performance measurement> Resource usage> Troubleshooting and debugging> Problem detection> Quality of Service Metrics> Capacity planning> Traffic analysis (users, views, peak times)> Billing> Auditing

Performance Measurement Resource UsageTroubleshooting

Windows Azure Diagnostics

Demo

http://diags.cloudapp.net

Controller

DiagController.exeDiagnosti

c Manager

PowerShell scripts

SurveyResults

SurveyWeb Role

Diagnostic Monitor

My Utility Classes

Desktop Cloud

My Utility

Classes

Windows Azure Storage

Table

Blob

Diags

Q&A

Windows Azure Diagnostics> SDK component providing distributed

monitoring & data collection for cloud apps

> Standard diagnostics APIs> Cloud-Friendly and Scalable> Developer In Control> Available now in the WA SDK

Flip Camera/BlueTrack Mouse

YOUR FEEDBACK IS IMPORTANT TO US! Please fill out session evaluation

forms online atMicrosoftPDC.com

Learn More On Channel 9> Expand your PDC experience through

Channel 9

> Explore videos, hands-on labs, sample code and demos through the new Channel 9 training courses

channel9.msdn.com/learnBuilt by Developers for Developers….

© 2009 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.