Microsoft Exchange 2013 Sizing An Information Behind the Guidance

Microsoft Exchange 2013 Sizing An Information Behind the GuidanceAbout Me Charles DerberHave been working on Microsoft Messaging for ALMOST a Decade

Experienced on Planning / Implementing / Migrations / Designing / Consulting for Messaging Enterprise & Hosting Solutions

Also experienced on Multiple Messaging Vendor like Microsoft, IBM, Linux etc. but gained much expertise in Exchange due to exposure and interest on Microsoft Products

Speaker @Mumbai IT Professional User Group & I do blog @ http://charlesgate86.wordpress.com AgendaIn the beginning Road to SizingArchitectural Changes Impact on sizingSizing Server role specific Deep drive - MBX / CAS / Active DirectoryExchange Roles Sizing Storage / Networks / Virtualization / HardwareExchange 2013 Sizing tools Considering Exchange 2013 VirtualizationSizing Demo

Sizing is both a Science & an Art form.

GuidanceWhere it is coming from? Pre-release environment!!(Test labs / Dog Food(emails are used in a way where it is not intended to use / Yes multi profile) / MSIT / Customer / Field work)ResourcesTechNet / MS Exchange Team Blog / Experts BlogsThird Party Tools

What is not possible?Doesnt cover everything because it cannot cover everything - but will provide some base line to start with

Third party solution - BB / SMTP Relay / Applications

Third Partner dependent - Every one has different architect / functionality

Hardware support

Needs lab testing for your environment / consider failure domains - High AvailabilitySizing ProcessRead / Understand sizing, scalability, capacity guidance

Documentation @TechNet / Exchange Team Blog

Collect Data on existing deployment

User profiles (message sent + received per day)Average message size

Sizing ProcessDefine constraint

No of database copies Backup requirement Storage architectureSafety Net duration Virtualization Growth Plans 3rd Party Products

Sizing Process4. Input profile data and design constraints into calculator tool

Always use latest calculator

5. Consider impact of various option provided by results via Calc.

CostRebuilt timesImpact on High Availability

Sizing Process6. Finalize Design

Storage calculator provides configuration scriptsArchive the calculator as documentation of the sizing process

Architectural Changes / FeaturesExchange Server Roles

CAS Stateless OWA / OAB / OA / EAS / EWS / Mail Flow MBX PF HA / Managed Store / IOPS / Safety Net & Others

Balanced use of Hardware CPU / Memory / DiskMinimum Systems RequirementRAM 8GB Mailbox or Multirole/Mailbox+CAS4GB CAS

CPU Follow published OS guidelines

Disk Space30GBRAID vs JBOD, DAS vs SAN, Oh My!Where is your personal/precious email JBOD

Google email account is stored without RAIDMicrosoft's Office 365 cloud service utilizes JBOD for storage Microsoft's own internal deployment of Exchange 2010 has over 180,000 users on a JBOD infrastructureMicrosoft's TerraServer uses JBOD to store all of the data presented through that web site

Microsoft therefore recommends that if you are considering utilization of drives in a JBOD architecture that you do choose to do so with the 7.2K RPM SAS drives rather than SATAWhat Storage should I choose ?Even I can't tell you which storage architecture to choose

Microsoft supports SAN and DAS+RAID along with DAS+JBOD because no single storage architecture is right for every customer out there.

Unless you are in a situation where money is no object and will continue to be no object, you must seriously evaluate the storage solutions available to you Sizing Exchange 2013 Mailbox Role

Mailbox Role - ImpactIncreased resource usage

Cache sizing is different

New content indexing architecture impacts performance

By default UM enabled on MBX

Managed availability Mailbox Role - StorageMailbox Database, content indexes, log space

20% database overhead is now 0%

Content Index size is now 20% of EDBPlus space for additional index set per volume (master merge)Note that impact of space for master merge is reduced with multiple DBs per-volumeMailbox Role Storage BandwidthBDM in 2013 now consuming ~1MB/sec/DB copy, significant reduction from 2010Mailbox Role Transport StorageTransport queue database takes advantage of ESE IO improvements to reduce IOPS

Microsoft production observations show ~1 DB IO per 75KB message

Low IOPS suggest that placing transport queue on system/install volume is now feasible in many scenarios

Significant transport throughput benefit seen from a protected write cache disk controller, set to 100% write cache

Transport capacity requirements include queue and Safety NetMailbox Role IOPSExchange 2013 reduced IOPS requirements (~33% reduction compared to 2010)

Checkpoint depth is now consistent for all DB copies, so IOPS requirements are the same

Mailbox Role CPUMessages sent or received per mailbox per dayMcycles per User, Active DB Copy or Standalone (MBX only)Mcycles per User, Active DB Copy or Standalone (Multi-Role)Mcycles per User, Passive DB Copy502.132.660.691004.255.311.371506.387.972.062008.5010.632.7425010.6313.283.4330012.7515.944.1135014.8818.594.8040017.0021.255.4845019.1323.916.1750021.2526.566.85Note: Baseline platform for CPU guidance changed in 2013. Mcycle requirements in 2010 and 2013 cannot be directly compared.Mailbox Role CPUHyperthreading (SMT) Fantastic Technology increases CPU throughput DONT USE IT!

.Net framework allocates memory per core heaps

Significant impact to some Exchange service memory footprintsMailbox Role MemoryMemory on the Mailbox role sized based on ESE cache requirements

Cache requirements have remained constant from 2010

Overall cache sized to 25% of RAM, so guidance (based on total system memory) is 4x of 2010 cache sizing recommendation

Multi-role servers require additional memory for CAS based on user concurrency during worst-case failure

Messages sent or received per mailbox per dayMailbox role memory per active mailbox (MB)501210024150362004825060300723508440096450108500120Mailbox Role MemoryMemory is allocated within heaps

.NET garbage collector has different modes which optimize for different allocation scenarios

Workstation GCUses common heap and cleanup process (can be concurrent or not)

Server GCAllocates a heap and thread per logical proc

Server GC results in dramatically larger memory requirements at rest when SMT is enabledMailbox Role MemoryBest practice to install KB 2803754 or 2803755http://support.microsoft.com/kb/2803754 (Windows Server 2008 R2)http://support.microsoft.com/kb/2803755 (Windows Server 2012)

Reduces memory consumption in each store workerNo impact to sizing guidanceMemory is available for use by other processes

Decreases CPU spent in .NET garbage collector

Benefits Mailbox & multi-role

Enable by installing hotfix and setting regkey:HKLM\Software\Microsoft\.NETFramework\DisableRetStructPinning (REG_DWORD) = 1Mailbox Role NetworkSeeding of multiple database per disk may require increased bandwidth between servers

Avoid bottlenecking on network

Plan for reseed operations, particularly in JBOD deployments

10GB Ethernet expected to become more common for Exchange infrastructureCost has dropped, many customers are standardizing on 10GB Ethernet in their datacentersMailbox Role Unified MessagingUM is sized using a concurrency model you cannot have a dedicated server role as UM

Plan for a maximum of 100 concurrent calls per serverNote that UM in 2013 is a component of the Mailbox role, may need to adjust user distribution to optimize UM utilization/concurrency

Voicemail transcription is a heavy consumer of CPU

Plan for 1 CPU core per concurrent transcription If server is CPU starved, voicemail transcription may be skipped (voicemail delivered without transcription)

Sizing Exchange 2013 CAS Role

CAS Role - ImpactLow resource utilization makes multi-role deployment (or virtualization) attractive

CAS is intelligent director of traffic work

Shift of processing resources from LB layer to CAS may negate new resource demand

CAS Role ProcessorCAS CPU is sized using a percentage of Mailbox CPU active user requirements

2013 CAS requires 25% of Mailbox active-user mcycles, down from 75% in 2010

Given significant reduction, ensure that enough CAS servers are deployed to handle failures and provide high availability

CAS Role MemoryCAS memory is sized using a simple formula of 2GB + 2GB per-CPU core.

The per-core value assumes utilized CPU cores at peak (worst case failure), so this can get a little complicated

Kemp Hardware Load Balancer & MS Exchange

Why Hardware Load Balancer?Microsoft RecommendsService Aware, Pre-auth, SSO for Exchange & LyncEvenly distributes load(Http/Mapi{Exchange 2010}/Https) between CAS serversCA-FE now a stateless proxy serverSSL Acceleration & Reverse Proxy Site HA & Geo Auto-failoverKemp HLB is Microsoft Certified & Few others too!!!Engineered especially for Microsoft Exchange & LyncLync / Application servers load balances too not limited to ExchangePlay around with Trial Virtual Load Balancer from Kemp to experience the functionality and its features.

Active DirectoryRecommend deploying 1 AD GC core for every 8 Mailbox cores handling active load (assuming 64-bit GCs)

Size memory such that the entire NTDS.DIT can be contained within RAM for optimal query performanceExchange 2013 Diagnostic & PerfmonLogs:Microsoft\Exchange Server \V15\Logging\Diagnostics\DailyPerformanceLogs folderMicrosoft\Exchange Server\V15\Logging\Diagnostics\PerformanceLogsToBeProcessed

MS Exchange Health Management Service

Works towards assessing the health of Exchange components OWA / EAS / EWS / ECP

It also captures information about activities such as the Calendar Repair Assistant, Mailbox Assistants, Mailbox Replication service, Migration service, OAB generation, and so on V15\Logging rootExchange 2013 Diagnostic & PerfmonMost of the logs generated in CSV which are helpful in troubleshooting issues at times root cause.

It fills the Exchange drive when not monitored use data collector set to create resource policy and delete or follow Removing Old Exchange 2013 Log Files

Diagnostic logging 70 New Added & 81 Old Removed

Removal of events for Public Folders and the addition of Front End Transport server events seem to be the biggest change. The ExchangeSA\OALGenerator one, commonly used to troubleshoot Offline Address Book issues, has been removed as wellExchange 2013 - Sizing Tools

Sizing Tools Exchange Role CalculatorRole Requirements Calculator turns published sizing guidance into a modeling toolTry out various failure scenariosUnderstand the impact of different hardware & storage choicesProvides scripts for DAG, database & copy creationMany new featuresCAS sizingTransport storage sizingMultiple databases per-volume (JBOD) supportHigh availability architecture improvements

http://aka.ms/E2013Calc Sizing Tools Exchange Role Calculator

Sizing Tools Exchange Profile AnalyzerHelps administrators collect estimated statistical information from a single mailbox store or across an Exchange Server organization

Can only be run on Exchange 2003(SP2) & 2007Exchange View-Only Administrator permission to runRun during non-business hours to avoid performance issues(on CAS) if anyImprove capacity planning method

Sizing Tools Exchange Profile Analyzer

Sizing Tools Collectlogs.vbsCalculate Exchange transaction logs generated for each hour in the day

A simple VBS script that will collect all files in a folder and output it to a log file

Use Task Scheduler to execute this script at certain intervals in the day (e.g. every 15 minutes)

Sizing Tools EmailStatistics.ps1Script parses messaging tracking logs of your Exchange Servers and then tabulates the information into a CSV file

Demo

Messages Received per Mailbox Per Day = Received Total Messages Sent per Mailbox Per Day = Sent Unique Total Average Message Size = Average of Received MB Total & Sent Unique MB TotalSizing Tools Processor Query ToolQuickly look up SPECint benchmark for a particular processor type

Takes average across multiple vendor submissions

Provides specific value to insert into Role Requirements Calculator

http://aka.ms/ExProcQueryTool

Sizing Tools Jetstress 2013Jetstress 2013 released March 2013

Validates that the physical deployment is capable of meeting specific performance requirements

Validates that the storage design is capable of meeting specific performance requirements

Finds weak components prior to deploying in production

Proves storage and I/O stability

http://aka.ms/Jetstress2013Sizing Tools Jetstress 2013Jetstress is dumb tool that is used worldwide by thousands of Exchange professionals

When the Jetstress test reports a failure, Jetstress has not failed

Sizing Tools Exchange Client Bandwidth CalculatorThis Calculator has been designed to profile the network bandwidth requirements of both Exchange on-premises and Exchange Online (Office 365) workloads.

Hub site consolidation planning

Exchange service externalisation planning

Cloud requirements planning for Exchange Online (Office 365)

Demo

Exchange 2013 Virtualization SizingSizing methods for Exchange 2013 for physical and virtual hardware is the same

However you need to consider the impact of a shared virtual infrastructure

~10% for Hyper-V (follow vendors guidance for other hypervisors)

Co-locating Exchange with other virtual machines and workloads

Use resource reservation options to ensure that Exchange gets what it needs

Exchange 2013 Virtualization Supportability Supportability

Windows Server 2008 R2Hyper-V Server 2008 R2Windows Server 2012Hyper-V Server 2012Third-party hypervisors (SVVP)

Exchange RolesBoth Exchange 2013 roles supported for virtualizationLive migration & Host based clustering is supported

Exchange 2013 Not SupportedDynamic Memory

Memory over commit / CPU over subscription

Hyper V snapshots

Differencing / delta disk

Apps on the parent OS

Exchange 2013 Not SupportedHyper-V replica

Replica provides DR for any VM via log shipping to a remote hypervisorUse Data Availability (DAG) to obtain better HA & DR with Exchange 2010 & 2013Replica makes sense for apps that dont have DR capability built-in to the product

Windows Azure Infrastructure Services, AWS or similar

Exchange 2013 Dynamic Memory ImpactQuestions?Data

Storage Subsystem

Background Database Maintenance

Transactional I/O

Extensible Storage Engine(ESE)

Windows I/O Manager

Device Drivers

Jetstress Application

Auto tuning

Thread Dispatcher

Background Log Checksummer

Offline Log & Database Checksummer

Windows Operating System

Hardware

Windows Performance Counters

Performance Data

Reporting and Verification