storage capacity enforcement 2020 - microsoft dynamics
TRANSCRIPT
Storage Capacity Enforcement
Understand the effects of capacity enforcement within Microsoft Dataverse and how to maintain
storage to prevent capacity enforcement.
2020
Download the latest version of the Playbook here: https://aka.ms/CapacityEnforcementPlaybook
PURPOSE
This playbook is intended to help customers identify their current capacity usage, understand how their storage is
calculated, and develop an ongoing plan to manage capacity to prevent environment enforcements due to over-
capacity issues.
THE SITUATION
Microsoft has
implemented additional
enforcements to storage
capacity for Microsoft
Dataverse environments.
BASED ON EXPERIENCES TO DATE, CAPACITY PLANNING AND MANAGEMENT DEPENDS ON
THESE FACTORS:
Understanding the enforcement
Assessing your storage plan
Monitoring your consumption
Identifying retention policies and managing data to those policies
Developing an archival strategy to retain older, rarely used data
Having a plan to account for an increase in data
SUCCESS
In summary, this guide will help you:
Understand storage capacity
Prevent capacity overage and enforcement
Determine your current storage capacity and consumption
Identify what is consuming capacity
Review your environment strategy
Reduce and maintain storage capacity
Audiences:
Partners
Customers
Business
Technical
INSIDE THIS PLAYBOOK:
Chapter 2
EXPLORE
Learn more about
your storage
capacity
Chapter 3
IMPLEMENT
Create and
implement a plan
to reduce and
manage storage
2
Appendix
Available
resources
Chapter 1
INITIATE
Understand
capacity
enforcement
and how it will
affect your
organization
Chapter 2
EXPLORE
Learn more about your
storage needs and storage
capacity
Review your current storage
capacity and consumption• Power Platform admin center
• Download all tables
Understand current application
lifecycle management (ALM)
practices
• Environment needs
• Data within environments (full
vs. minimal)
• Data retention needs and definition
Resources
• New Microsoft Dataverse storage
capacity
• Legacy storage capacity
• View self-service storage capacity
Understand capacity
enforcement and how it will
affect your organization
Learn about storage capacity• Power Platform admin center
• Capacity calculation
• Environment storage
Why is this important?
• Enforcement will limit some admin
operations
• Help customers maintain a healthy
storage plan
• Identify potential inefficiencies
leading to increased storage
consumption
Resources
• What's new about storage
• What happens when my
organization exceeds storage
entitlements?
Chapter 1
INITIATEChapter 3
IMPLEMENT
Create and implement a
plan to reduce and manage
storage
Refine ALM practices• Number of environments
• Full vs. minimal environments
• Update retention definitions
Manage storage capacity• Data archival
• Maintaining data within database,
log, and file storage
• Capacity add-ons
Resources
• Capacity add-ons
Storage Capacity EnforcementChapter 1: Initiate
Understanding storage capacity
StorageAll usage and entitlement
categories under one label, inclusive of all record types
Legacy
Storage models
Optimizing data management for database
storage, attachments, and audit logs
Technical
New
Storage capacity management: usage and entitlement across these
three categories:
Database capacityStore and manage
entity definitions and data
File capacityManage attachments, files,
photos, and videos
Log capacityRecord data changes over time
for analysis & reporting purposes
More Info: Licenses for the new storage modelMore Info: Licenses for the legacy storage model
Storage models
Two different storage models—new capacity
and legacy—are determined by your license
Some of the benefits of this change include:
• Scalability with purpose-built storage
management solutions
• Ability to enable new business scenarios
• Reduced need to free up storage space
• Support for a variety of data types
• Additional default and full user entitlements
• Flexibility to create new environments
Two versions of storage reporting
Business
Technical
Legacy Model New Model
Capacity Entitlements Capacity Entitlements
Default capacity
(Tenant)Accrued entitlement (per
full enterprise USL)
Default capacity (Tenant)
Accrued entitlement(per full enterprise USL)
Benefit
Storage 10 GB
5 GB
for every 20 full user
licenses
Microsoft Dataverse
Database Capacity10 GB 250 MB Entitlement per USL
Microsoft Dataverse File
Capacity20 GB 2 GB
Additional default and full
USL entitlement
Microsoft Dataverse Log
Capacity2 GB n/a
Additional default
entitlements
Add-on for production and non-production
environmentsCreate environments with available storage
Environment
type
Number of
Environments
Additional environments can be created when at least 1 GB of database capacity is available.
No need to purchase additional environments.
Non-
production1
Production 1
Storage models
Capacity entitlements
Business
Technical
Power Apps Power Automate and Power Virtual Agents Licensing Guide
Dynamics 365 Licensing Guide
Power Platform admin center
The Power Platform admin center is the tool
you should use to maintain visibility of storage
and consumption
Business
Technical
Administer Power Platform
Storage capacity
Capacity availability is defined by the user
SKUs purchased, self-service licensing, and
additional add-ons
Org (tenant) default
The capacity is defined by the product licensing
purchased
User licenses
With every user license, your organization gets an
additional amount of capacity
Additional capacity
Any additional capacity add-ons purchased
Business
Technical
View self-service storage capacity
Capacity overage and enforcement
Capacity overage
You should be within limits for your entitled
capacity for database, log, and file
Storage consumption is rolled up to a tenant level.
It is the sum of all environment storage within a
tenant. Your tenant needs to stay within their limits
to avoid capacity enforcement.
Overage doesn’t specifically mean that your tenant
will experience enforcement. However, you should
always evaluate your usage to stay within your
entitled capacity.
Business
Technical
Capacity deficiency
File
Log
Database
File deficiency can be covered by Log or Database
Log deficiency can be covered by Database
Database deficiency is always deficient
Business
Technical
Overage scenarios
Database storage is over capacity
• 10 GB over in database usage
• 200 GB available in file storage
Type Entitled Consumed
Database 100 GB 110 GB
Log 10 GB 5 GB
File 400 GB 200 GB
Log storage is over capacity
• 10 GB over in log usage
• 5 GB available in database storage
Type Entitled Consumed
Database 100 GB 95 GB
Log 10 GB 20 GB
File 400 GB 200 GB
DEFICIENT
Below are examples of overage scenarios that illustrate if a
tenant is deficient in capacity or not
DEFICIENT
Technical
Example storage capacity scenarios, overage enforcement
Overage scenarios
File storage is over capacity
• 90 GB over in file storage
• 85 GB available (80 GB database + 5
GB log)
Log storage is over capacity
• 10 GB over in log usage
• 20 GB available in database storage
DEFICIENT Not Deficient
Type Entitled Consumed
Database 100 GB 20 GB
Log 10 GB 5 GB
File 200 GB 290 GB
Type Entitled Consumed
Database 100 GB 80 GB
Log 10 GB 20 GB
File 400 GB 200 GB
Below are examples of overage scenarios that illustrate if a
tenant is deficient in capacity or not
Technical
Example storage capacity scenarios, overage enforcement
Capacity enforcement
Once you are deficient in capacity, you will no longer be able to
complete certain admin operations
Business
Technical
Storage Capacity EnforcementChapter 2: Explore
Review currentconsumption
Power Platform admin center
Functionality within the Power Platform admin center will allow you to
review your current consumption
Technical
Administer Power Platform
Storage capacity
View all environments and their total file, log,
and database consumption
File
• Attachment
• AnnotationBase
• Any custom or out-of-the-box entity that
has fields of datatype file or image (full
size)
• Any entity that is used by one or more
installed Insights applications and ends in
– Analytics
Log
• AuditBase
• PlugInTraceLogBase
Database
• All other entities are counted for your
database
Technical
Storage capacity tab
Environment usage
View detailed information of consumption of
each storage type
• Actual database usage
• Top database tables and their growth over time
• Actual file usage
• Top files tables and their growth over time
• Actual log usage
• Top tables and their growth over time
Technical
Review application lifecyclemanagement practices
Environment planning
Dev(s) QA UAT Pre-prod Production
Create a storage plan by understanding environment needs based on
ALM strategy
Business
Technical
Establishing an Environment Strategy for Microsoft Power Platform
Environment planning
Create a storage plan by understanding environment needs based on
ALM strategy
Art. 6 GDPR – Lawfulness of processing
–
Development • Sample data set
QA • Sample data set
User testing • Near production data set
Technical
Environment planning
A storage plan example (database storage)
• Dev environments = minimal data
• QA environments = enough data to validate current scenarios
• UAT (user acceptance testing) environments = sufficient data to allow end-to-end testing by users
• Training/staging environments
• Training environments = significant data to ensure all user scenarios are adequately represented
• Staging environments = no application data
• Used as a production-equivalent placeholder of the production solution
Dev
Plan – 6 GB
Solution Dev 1* - 1.5
GB
Solution Dev 2* - 1.5
GB
Solution Base - 3 GB
QA
Plan – 11.5 GB
QA Prod* - 4 GB
QA Dev - 4 GB
Integration Test* - 3.5
GB
UAT
Plan – 52 GB
UAT Prod* - 25 GB
UAT Dev - 27 GB
Pre-prod
Plan – 78 GB
Training* - 75 GB
Staging* - 3 GB
Production
PROD - 80 GB
Technical
Environment planning
A storage plan example (database storage)
Dev
Plan – 6 GB
Solution Dev 1* - 1.5
GB
Solution Dev 2* - 1.5
GB
Solution Base - 3 GB
QA
Plan – 11.5 GB
QA Prod* - 4 GB
QA Dev - 4 GB
Integration Test* - 3.5
GB
UAT
Plan – 52 GB
UAT Prod* - 25 GB
UAT Dev - 27 GB
Pre-prod
Plan – 78 GB
Training* - 75 GB
Staging* - 3 GB
Production
PROD - 80 GB
• Example planned storage = 227.5 GB
• * indicate optional instances = 113.5 GB
• A clear plan allows for proper capacity purchasing to prevent capacity enforcement
• Clearly understand and define your approximate storage needs BEFORE you start deploying a complex environment
model
• Ensure your storage plan includes growth over time and enough additional capacity to address potentially unforeseen
needs (POCs, significant unplanned data growth, and so on)
Technical
Data retention
Retention policy
Understand what data you need to keep and for how long
Business
Technical
While determining your data retention policies, it’s important to
classify your data
Business
Technical
Master
Master data are static objects in
your system like Accounts,
Contacts, Products. This data set
is, usually, smaller than
operational and transactional
data set.
Retention advice/example:
- As long as related operational
data exists
- X years according to external
regulations or internal policy
Operational
Operational data is module
related data such as Sales,
Customer Service, etc. As an
example, these could be
Opportunity, Lead, Case, Project,
etc.
Retention advice/example:
- Closed more than 12 months
ago or created in last 24 months
- X years according to external
regulations or internal policy
Transactional
Transactional data are
interactions, in most cases
activities, related with master and
operational data. This is usually
the largest data set within an
instance.
Retention advice/example:
- Created within 2 years or more
- To seed Sales Insights
algorithms, suggestion is, at least,
120 days since closed/modified
- X years according to external
regulations or internal policy
Move older data, defined by retention policies, from Microsoft
Dataverse to a different storage repository
Business
Technical
Prevent excessive growth
Certain Dynamics 365 apps settings can help prevent unnecessary storage growth
Email tracking
Maximum file size
Auditing
Automatically delete completed real-time workflow jobs
Delete Async Plugin Logs if plugin succeeds, save Dynamics 365 storage space (MVP)
Technical
Creating a plan
Reduce Maintain Expand
Business
Technical
Business
Technical
Storage Capacity EnforcementChapter 3: Implement
Reduce current consumption
With an environment plan, evaluate current environments and make
the necessary changes
Technical
Reduce current consumption
After defining a retention policy and implementing an archival
solution (if applicable), reduce your consumption
File Storage
Remove email attachments using
Advanced Find
Remove email messages using a bulk
deletion job
Remove notes with attachments
using Advanced Find
Remove notes with attachments
using a bulk deletion job
Log Storage
Delete audit logs
Delete plug-in trace logs using bulk
deletion job
Database Storage
Delete bulk email and workflow
environments using a bulk deletion
job
Evaluate and delete suspended
workflows
Remove bulk duplicate detection
jobs and associated copies of
duplicate records
Delete bulk import environments
using a bulk deletion job
Delete bulk deletion job
environments using a bulk deletion
job
Remove unrequired entities and
fields from Relevance Search
Delete imported records
Technical
Maintain storage consumption
Continuously maintaining your data prevents uncontrolled storage
growth and major data purges, which can take time to complete.
Technical
Unexpected storage growth occurs after releases or data migrations
Table size (GB)
Table name 30-Aug 6-Sep
WoW
Change 13-Sep
WoW
Change 20-Sep
WoW
Change
4 Week
Growth
ImportDataBase 0.76 18 17.24 65.34 47.34 21.5622 -43.7778 20.8022
msdyn_entityrankingruleBase 0.17 2 1.83 3.74 1.74 1.6456 -2.0944 1.4756
msdyn_playbookinstanceBase 0.2 1 0.8 1.1 0.1 0 -1.1 -0.2
msdyncrm_leadentityfieldBase 0.15 0.22 0.07 0.4598 0.2398 0.708092 0.248292 0.558092
msevtmgt_eventvendorBase 0.16 10 9.84 16.5 6.5 10.89 -5.61 10.73
msdyn_sessiontemplateBase 0.19 1 0.81 0.44 -0.56 0.9196 0.4796 0.7296
msfp_projectBase 0.19 0.75 0.56 1.32 0.57 3.1944 1.8744 3.0044
msdyn_flowcardtypeBase 0.16 1.1 0.94 2.178 1.078 5.03118 2.85318 4.87118
Technical
Purchase additional storage capacity
Purchase additional storage capacity
After completing your environment planning, you may need to
purchase more capacity add-ons
Capacity add-ons
Business
Technical
Storage Capacity EnforcementAppendix
Below are the communications sent to customers regarding overage
and the capacity enforcements
2/28/2020 – MC205095
6/22/2020 – Targeted emails to customers over or approaching
limit
6/22/2020 –
A1: MC216987
A2: MC216988
B1: MC216989
B2: MC216990
7/21/2020 – MC218982
Message Center
Business
Technical
What are some of the questions commonly being asked?
New Storage FAQ
Legacy Storage FAQ
Business
Technical
Thank you!
This playbook is provided “as-is.” Information and views expressed in this playbook, including URL and other Internet Web
site references, may change without notice. You bear the risk of using it.
Some examples are for illustration only and are fictitious. No real association is intended or inferred.
This playbook does not provide you with any legal rights to any intellectual property in any Microsoft product. You may copy
and use this playbook for your internal, reference purposes.
© 2020 Microsoft Corporation. All rights reserved.