lync storage service front end server replication for ha (within a pool) cdr and qoe adaptors...

Monitoring Microsoft Lync Server 2013: Getting the Most out of Monitoring Service Data

Sasa JuratovicJens Trier Rasmussen

OUC-B323

IntroductionsJens Trier RasmussenSenior Program ManagerHave been working in the UC space for more than 25 years and with Microsoft products since Exchange V4.0 beta 2 and LCS 2003With Microsoft for 14 years; 9 years with MCS and rest in Lync PG.EMEA basedEnjoys good food and wine, and hence also need to enjoy table tennis and road bike cycling to keep me fit

Sasa JuratovicSenior Program ManagerJoined Microsoft in 2001Joined Lync Product Group in 2005Working in Unified Communications and Messaging space for the last 15 yearsEMEA basedEnjoys good BBQ and beer, tennis and skiing to keep fit

Jens Trier Rasmussen

Sasa Juratovic

Session objectives and takeawaysSession objectivesDescribe the purpose of the Monitoring service What data can you gather from the Monitoring server dataHow can you use this data to answer deployment questionsHow can you use the data to assist in troubleshooting your environmentWhat other tools can you leverage beyond the built-in reports

TakeawaysWhy is the Monitoring service a required component in every Lync DeploymentLearn how to interpret the data in the monitoring server reportsLearn how to turn the monitoring server data into a powerful troubleshooting tool

Monitoring service foundations

Can you fix it before you diagnose it?

Monitoring server foundationsWhat is new in Monitoring Server architectureThe process of installing and configuring monitoring has been simplified in Lync Server 2013Prior versions required a separate Monitoring Server roleThe Monitoring Server role has been eliminatedThe monitoring service (in the form of "unified data collection agents") has been collocated into all Front End serversDecreases the number of server roles required when implementing Lync Server 2013Reduces the complexity of Lync Server 2013 setup and administration

What is new in Monitoring Server ReportsNew voice quality reports - "Media Quality Comparison Report", "Conference Join Time Report"Improved reports for analyzing and troubleshooting both video and application sharing sessionsImproved report performance

Monitoring server foundationsConfigure monitoring storeOpen your pool properties and enable MonitoringSelect the SQL store (and optional mirror)Publish your topologyTopology Builder will handle installing the databases

How many monitoring stores do I need?Use single / consolidated monitoring store unless …It’s very large deployment (more than 200K users)Local legal and compliance rules mandate local storeRemote sites are connected via very poor network linkLync Server sites need their own custom monitoring configurations

Monitoring server foundationsDeploy monitoring reportsOpen the Lync deployment wizardClick on “Deploy Monitoring Reports”Specify the Monitoring database and SSRS instance that will home the reports

Monitoring administrationWhat can be configured (CDR)Enable/disable monitoring of CDRsEnable/disable purging of CDRsLength of time to keep CDRsLength of time to keep error reports

What can be configured (QoE)Enable/disable monitoring of QoE dataEnable/disable purging of QoE dataLength of time to keep QoE data

Note: CDR and QoE can also be configured using PowerShell

Data collectionAt the end of each media session end point sends QoE data to home pool (and in turn to QoE database)All media end points have these capabilities – Lync Clients, Mediation Server, AVMCU, EdgeLook for VQReportEvent in Lync client UccApilog

Monitoring service components

Async Interface

Lync Storage Service

Front End Server

Replicationfor HA

(Within a pool)

CDR and QoE Adaptors

CDR/QoE Data

UDC Agent

Async DB

SQL

Transformation of QoE Report into information

QoE Information• QoE Database Schema - http://

technet.microsoft.com/en-us/library/gg398687.aspx• Needed to write custom SQL queries

• Report information inserted into a number of tables, important ones are:• AudioStream Table http://technet.microsoft.com/en-us/library/gg425961.aspx • MediaLine Table http://technet.microsoft.com/en-us/library/gg425920.aspx • Session Table http://technet.microsoft.com/en-us/library/gg398635.aspx

• Session can have one or more media modalities with each modality having two streams

• Important elements:• Caller, Callee• User Agent Type, Location, Network connectivity type• Media quality data

ClassifiedPoorCall column in MediaLine table• Logical OR of (from AudioStream Table) ...• RatioConcealedSamplesAvg > 7% (0.07)• The ratio of auto-generated audio data over real speech data, i.e. audio data is

delayed or missed, due to network connectivity issues. • PacketLossRate > 10% (0.1)• Average packet loss rate during the call

• JitterInterArrival > 30ms• Maximum network jitter during the call

• RoundTrip > 500ms• Round trip time from RTCP statistics

• DegradationAvg > 1.0• The amount the Network MOS was reduced because of jitter and packet loss

What can I get from these reports?

Monitoring

Reports

Monitoring

Dashboard

Diagnostic logs“Thanks for reporting your issue – do you have logs with that?”Both diagnostic codes and media quality are collected and reported Correlate the user’s reported issue with the associated session reported via CDR/QoE Provides an objective view of the experience

Trend analysisUnderstand how to determine the scope of the issueWho else is impacted?How often does this happen?Why is it happening?

Investigating audio quality issues

Media Paths

AV MCUCAS

CAA

Remote

Internal Internal

Mediation Server

PSTN Gateway

Guest

Media Path

ExUM

Edge

Three Call Quality Dimensions

1. Server Plant2. Endpoints3. Last Mile

Server Plant0: Server Health1: Server to Server2: Mediation to IP PSTN GW3:IP PSTN GW Health

AV MCUCAS

CAA

Mediation Server

PSTN Gateway

Endpoints0: Device1: System 2: Media Path3: Media Transport

AV MCU

Remote

Internal Internal

Mediation Server

PSTN Gateway

Last Mile

0: Wired1: Wireless

AV MCU

Internal Internal

Mediation Server

In the box Reports• Device• Information about device performance

• Server Performance Report• Calls to/From Servers

• Media Quality Summary Report shows you calls per call path• Client to Client, Client to AVMCU, MS to AV-MCU, Client to MS, MS to

GWY, Other

In the box Reports• Media Quality Comparison Report let you

compare calls between location and network connection type• External/Internet and Wired/Wireless and VPN

• Call Leg Media Quality Report shows you server to server calls• MS <-> AV-MCU and MS <-> PSTN Gateway

• Location• Per subnet

Device

Server Performance

Media Quality Summary

Media Quality Comparison

Call Leg Media Quality

Location

Poor Audio Call Scenario

End-User Reports to Helpdesk• I was hosting a conference the other day

and the audio was terrible• I called my wife and her voice was choppy

Troubleshooting Strategy• Make sure helpdesk provides adequate

information for you to troubleshoot• Issue classification, PC/device information, when/where/who

• One off or common patterns?• Could be device or PC of individual user(s)?• Could be common pattern across users?

• Start with in the box Monitoring Server reports

• If required use Views or Custom Queries against QoE Information• SQL Management Studio

Get the base information• Use User Activity Report for the two users

and locate the calls• Determine caller subnet and callee subnet and endpoint

Understand the topology and paths • U5 in conference

• S3 > R3 > R4 > R6 > S5

• U3 PSTN call• S2 > R2 > R4 >R5 > S6

• Hypothesis• R4 has a problem, because

both reported paths crosses R4

U1 U2 U5U3

S1 S2 S3

S5 S6

R1 R2

R3

R4

R5R6

AVMCU1 MS1

S4

U6

Troubleshooting – Location Report• Caller Subnet report for relevant

subnets• require subnets defined in Lync

Troubleshooting – Media Quality Summary Report• Look for users on

relevant subnets talking to MS and AV-MCU• PSTN Calls (Non-Bypass): UC Leg• UC Conference Sessions• Look at individual calls

• NB: Not easy to get the subnet

Troubleshooting - QoEReportsEndpointSubnetViewSELECT TOP 1000 [SessionTime]

,[CallerSubnet],[DegradationAvg],[JitterInterArrival],[PacketLossRate],[RoundTrip],[RatioConcealedSamplesAvg],[CallNetworkType]

FROM [QoEMetrics].[dbo].[QoEReportsEndpointSubnetView] ORDER by CallerSubnet,SessionTime

• NB: Does not have the callee endpoint

QoEReportsEndpointSubnetView sampleSessionTime CallerSubnet Degradation

AvgJitterInter

ArrivalPacketLoss

RateRoundTrip RatioConcealed

SamplesAvgCallNetworkType

2013-05-08 x.y.z.0 NULL 0 0.0000 1 NULL 0

2013-05-08 x.y.z.0 0.07 2 0.0000 0 0.00 0

2013-05-08 x.y.z.0 0.11 2 0.0000 1 0.00 0

2013-05-08 x.y.z.0 NULL 0 0.0000 0 NULL 0

2013-05-08 x.y.z.0 0.12 1 0.0000 2 0.00 0

2013-05-09 x.y.z.0 NULL 3 0.0000 11 0.00 0

2013-05-12 x.y.z.0 NULL 0 0.0000 NULL NULL 0

2013-05-13 x.y.z.0 NULL 1 0.0000 10 0.00 0

2013-05-13 x.y.z.0 NULL 2 0.0988 NULL 0.00 0

2013-05-14 x.y.z.0 NULL 0 0.0391 4 NULL 0

2013-05-14 x.y.z.0 NULL 0 0.1350 1 NULL 1

Troubleshooting - Custom Query• All audio streams from wired clients to AV-

MCU and MS servers• S2 and S3 should show poor calls per our hypothesis

ReportDate CallerSubnet PacketLossRate PacketLossRateMax

ClassifiedPoorCall CallerUri CalleeEP

2013-05-08 S2 0.217 0.645 1 sip:u3@contoso.dk MS1

2013-05-08 S2 0.189 0.453 1 sip:u3@contoso.dk MS1

2013-05-08 S1 0.001 0.045 NULL sip:u1@contoso.dk AVMCU1

2013-05-08 S1 0.001 0.065 NULL sip:u1@contoso.dk MS1

2013-05-13 S3 0.163 0.487 1 sip:u5@contoso.dk AVMCU1

2013-05-13 S1 0.008 0.039 NULL sip:u2@contoso.dk MS1

2013-05-13 S3 0.223 0.352 1 sip:u5@contoso.dk AVMCU1

You have a problem – how to get well?

Network Classifications

Inside/Outside

Managed• Corporate network• Quality expected to be good• SLA can imposedUn-managed• Internet• Hotel & coffee shop wireless• Low quality tolerated &

expected• SLA can’t be imposed

Endpoint ‘Last Mile’

Wired• Quality expected to be good• Used for client and server to

server communicationWireless• Quality also expected to be

good, but harder to actually implement

• Used for client to server communication

Strategy• Establish base line and see how you are

trending during remediation• Look at getting the biggest bang for the

buck• Start with wired server to server

• Low hanging fruits• User education on wired vs. wireless

Custom Query showing statistics per media path type

Type Date        All Streams         

Poor Streams       

Poor Streams Ratio

AVMCU-MS 5/7/2013 42.554 7.353 17.3

MS-GW 5/7/2013 111.857 6.927 6.2

Wired P2P 5/7/2013 40.200 5.726 14.2

Wired AVMCU/MS 5/7/2013 65.153 6.944 10.7

Wired Other 5/7/2013 146 9 6.2

VPN 5/7/2013 130 22 16.9

External 5/7/2013 166.113 41.549 25

Wireless P2P 5/7/2013 17.876 3.876 21.6

Wireless AVMCU/MS 5/7/2013 22.398 4.807 21.5

Wireless Other 5/7/2013 200 12 6

TOTAL* 466.627 77.225 16.5%

Example: Wireless drill-down • Wireless streams to AVMCU or MS

ReportDate CallerSubnet PacketLossRate PacketLossRateMax

ClassifiedPoorCall CallerUri CalleeEP

2013-05-08 S4 0.417 0.645 1 sip:u6@contoso.dk MS1

2013-05-08 S4 0.289 0.543 1 sip:u6@contoso.dk MS1

2013-05-13 S4 0.363 0.687 1 sip:u6@contoso.dk AVMCU1

Wireless remediation• Delivering Lync 2013 Real-Time

Communications over Wi-Fi• http://www.microsoft.com/en-us/download/details.aspx?id=36494

• Networking Guide: Network Planning, Monitoring, and Troubleshooting with Lync Server• http://www.microsoft.com/en-us/download/details.aspx?id=39084

Summary

Session objectives and takeawaysSession objectivesDescribe the purpose of the Monitoring service What data can you gather from the Monitoring server dataHow can you use this data to answer deployment questionsHow can you use the data to assist in troubleshooting your environmentWhat other tools can you leverage beyond the built-in reports

TakeawaysWhy is the Monitoring service a required component in every Lync DeploymentLearn how to interpret the data in the monitoring server reportsLearn how to turn the monitoring server data into a powerful troubleshooting tool

Related contentNetworking Guide: Network Planning, Monitoring, and Troubleshooting with Lync Server

http://www.microsoft.com/en-us/download/details.aspx?id=39084

Sample querieshttp://blogs.technet.com/b/jenstr/

Breakout SessionsOUC-B204 Network Design and Deployment Strategies to Ensure Success for Lync 2013 Enterprise VoiceOUC-B303 Designing for High Availability and Disaster Recovery in Microsoft Lync 2013OUC-B305 Enterprise Network Requirements for Microsoft Lync Server 2013OUC-B324 Planning & Deploying Your Enterprise Voice OUC-B331 Voice Interoperability FundamentalsOUC-B332 Planning and Deploying Conferencing in Microsoft Lync Server 2013OUC-B334 Migration and Coexistence with Lync Server 2013

Find Us Later At the Lync Demo Booths

msdn

Resources for Developers

http://microsoft.com/msdn

Learning

Microsoft Certification & Training Resources

www.microsoft.com/learning

TechNet

Resources

Sessions on Demand

http://channel9.msdn.com/Events/TechEd

Resources for IT Professionals

http://microsoft.com/technet

Evaluate this session

Scan this QR code to evaluate this session.

© 2013 Microsoft Corporation. All rights reserved. Microsoft, Windows and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Appendix

Tips• User agent type (UAType) shows type of

clientSELECT TOP 1000 [UAType] ,[UAName] ,[UACategory] FROM [QoEMetrics].[dbo].[UserAgentDef] ORDER BY UAType

• ExamplesUAType

Type of client

1 Mediation Server

2 AV-MCU

4 OC and Lync desktop client

16398 Lync Mobile 2013 for Windows Phone

16403 Lync Windows Store App

Tips

• CallerInside and CalleeInside shows if the caller/callee was remote

• CallerInside = 0 – Caller was remote• CallerInside = 1 – Caller was not remote

NetworkConnectionDetailKey

NetworkConnectionDetail

2 Ethernet

1 Wifi

0 wired

• Network Connection type defines the connection typeSELECT TOP 1000 [NetworkConnectionDetailKey] ,[NetworkConnectionDetail] FROM [QoEMetrics].[dbo].[NetworkConnectionDetail]