default 3 gb user space 2 gb per-process space 2 gb system space 3 gb per-process space 1 gb system...
TRANSCRIPT
![Page 1: Default 3 GB user space 2 GB Per-Process Space 2 GB System Space 3 GB Per-Process Space 1 GB System Space](https://reader038.vdocuments.net/reader038/viewer/2022102814/5518d191550346881f8b5d7f/html5/thumbnails/1.jpg)
![Page 2: Default 3 GB user space 2 GB Per-Process Space 2 GB System Space 3 GB Per-Process Space 1 GB System Space](https://reader038.vdocuments.net/reader038/viewer/2022102814/5518d191550346881f8b5d7f/html5/thumbnails/2.jpg)
Mysteries of Windows Memory Management Revealed
Mark RussinovichTechnical FellowWindows Azure
(created jointly with Dave Solomon)
![Page 3: Default 3 GB user space 2 GB Per-Process Space 2 GB System Space 3 GB Per-Process Space 1 GB System Space](https://reader038.vdocuments.net/reader038/viewer/2022102814/5518d191550346881f8b5d7f/html5/thumbnails/3.jpg)
About MeTechnical Fellow, Windows Azure, MicrosoftCofounder and chief software architect of Winternals Software Coauthor of Windows Internals book series
With Dave Solomon
Coauthor of Sysinternals Administrator’s Reference (Q1 2011?)
With Aaron Margosis
Author of Zero Day, A Novel (March 2011)Author of Windows Sysinternals tools
Home of blog and forums
![Page 4: Default 3 GB user space 2 GB Per-Process Space 2 GB System Space 3 GB Per-Process Space 1 GB System Space](https://reader038.vdocuments.net/reader038/viewer/2022102814/5518d191550346881f8b5d7f/html5/thumbnails/4.jpg)
Goals
Deep dive on:Process virtual and physical memory usageOperating system virtual and physical memory usage
Crisply define memory-related terminologyHighlight tools that reveal memory usageDescribe ‘dark spots’ in memory analysis counters and tools
![Page 5: Default 3 GB user space 2 GB Per-Process Space 2 GB System Space 3 GB Per-Process Space 1 GB System Space](https://reader038.vdocuments.net/reader038/viewer/2022102814/5518d191550346881f8b5d7f/html5/thumbnails/5.jpg)
Agenda
Virtual MemoryAddress Space UsageProcess CommitSystem Commit
Physical MemoryWorking SetsPaging Lists
Hard to Track Memory
![Page 6: Default 3 GB user space 2 GB Per-Process Space 2 GB System Space 3 GB Per-Process Space 1 GB System Space](https://reader038.vdocuments.net/reader038/viewer/2022102814/5518d191550346881f8b5d7f/html5/thumbnails/6.jpg)
Tools We’ll Use
Task ManagerSysinternals Process ExplorerSysinternals Vmmap
Process virtual and physical memory usageSysinternals Rammap
System physical memory usageSysinternals Testlimit
Test program to leak different kinds of memory
Sysinternals tools are free at www.sysinternals.com
![Page 7: Default 3 GB user space 2 GB Per-Process Space 2 GB System Space 3 GB Per-Process Space 1 GB System Space](https://reader038.vdocuments.net/reader038/viewer/2022102814/5518d191550346881f8b5d7f/html5/thumbnails/7.jpg)
Virtual Memory
![Page 8: Default 3 GB user space 2 GB Per-Process Space 2 GB System Space 3 GB Per-Process Space 1 GB System Space](https://reader038.vdocuments.net/reader038/viewer/2022102814/5518d191550346881f8b5d7f/html5/thumbnails/8.jpg)
Memory Management Fundamentals
Windows has demand-paged memory managementProcesses “demand” memory as neededThere is no swapping
A page is 4 KB (8 KB on Itanium)Large pages are available for improved TLB usage
x86: 4 MBX64 and x86 PAE: 2 MBItanium: 16 MB
There is NO (will, almost no) connection between virtual memory and physical memory
![Page 9: Default 3 GB user space 2 GB Per-Process Space 2 GB System Space 3 GB Per-Process Space 1 GB System Space](https://reader038.vdocuments.net/reader038/viewer/2022102814/5518d191550346881f8b5d7f/html5/thumbnails/9.jpg)
32-bit x86 Address Space32-bits = 2^32 = 4 GB
/3GB and /USERVA can extend process address up to 3 GBProcess must be marked “large address space aware” to use memory above 2 GB
Default 3 GB user space
2 GBPer-Process
Space
2 GBSystemSpace
3 GBPer-Process
Space
1 GBSystemSpace
![Page 10: Default 3 GB user space 2 GB Per-Process Space 2 GB System Space 3 GB Per-Process Space 1 GB System Space](https://reader038.vdocuments.net/reader038/viewer/2022102814/5518d191550346881f8b5d7f/html5/thumbnails/10.jpg)
64-bit Address Spaces64-bits = 2^64 = 17,179,869,184 GB
x64 today supports 48 bits virtual = 262,144 GB = 256 TBIA-64 today support 50 bits virtual = 1,048,576 GB = 1024 TB64-bit Windows supports 44 bits = 16,384 GB = 16 TB
x64(AMD64& Intel 64)
IA-64
8 TBSystemSpace
7 TBPer-Process
Space
7 TBSystemSpace
8 TBPer-Process
Space
![Page 11: Default 3 GB user space 2 GB Per-Process Space 2 GB System Space 3 GB Per-Process Space 1 GB System Space](https://reader038.vdocuments.net/reader038/viewer/2022102814/5518d191550346881f8b5d7f/html5/thumbnails/11.jpg)
Virtual Address Space Components
Address space breakdownPrivate (e.g. process heap)
Reserved or committed
Shareable (e.g. EXE, DLL, shared memory, other memory mapped files)
Reserved or committed
Free (not yet defined)Performance counters available:
Private Bytes – committed private memoryVirtual Bytes – total of shareable+private (including reserved)No separate counters for Shareable or Reserved or Free
![Page 12: Default 3 GB user space 2 GB Per-Process Space 2 GB System Space 3 GB Per-Process Space 1 GB System Space](https://reader038.vdocuments.net/reader038/viewer/2022102814/5518d191550346881f8b5d7f/html5/thumbnails/12.jpg)
Why Reserve Memory?
Reserved memory lets an application lazily commit contiguous memory Used for stack and heap expansion
Before Expansion
Committed
Reserved
After Expansion
Committed
Reserved
ThreadStack
StackGrowsDown
GuardGuard
![Page 13: Default 3 GB user space 2 GB Per-Process Space 2 GB System Space 3 GB Per-Process Space 1 GB System Space](https://reader038.vdocuments.net/reader038/viewer/2022102814/5518d191550346881f8b5d7f/html5/thumbnails/13.jpg)
Viewing Address Space Breakdown
Task Manager only lets you see private bytes Before Vista: column called “VM Size”Vista and later: column called “Commit Size”
Process Explorer shows both virtual size and private bytes
Add 2 columns to process listVirtual SizePrivate Bytes
Run Testlimit twice Testlimit -rTestlimit -m
Note: if on 64-bit Windows, 32-bit Testlimit can grow to 4GB
![Page 14: Default 3 GB user space 2 GB Per-Process Space 2 GB System Space 3 GB Per-Process Space 1 GB System Space](https://reader038.vdocuments.net/reader038/viewer/2022102814/5518d191550346881f8b5d7f/html5/thumbnails/14.jpg)
Understanding Process Address Space Usage
Most virtual memory problems are due to a process leaking private committed memory
Heap, GC heap, language heaps (CRT)Private Bytes only tells part of the story
Doesn’t account for shareable memory that’s not shared (e.g. DLLs loaded only by this process)Fragmentation can be an issue
Address space can effectively be exhausted prematurely
Basic performance counters don’t provide enough information to troubleshoot
FragmentedAddressSpace
![Page 15: Default 3 GB user space 2 GB Per-Process Space 2 GB System Space 3 GB Per-Process Space 1 GB System Space](https://reader038.vdocuments.net/reader038/viewer/2022102814/5518d191550346881f8b5d7f/html5/thumbnails/15.jpg)
Viewing Processes with VMMap
VMMap shows detailed breakdown of process address space:
Private process memoryCopy-on-write Private (VirtualAlloc)Heap and GC HeapStack
Shareable process memoryShareable – shareable memoryMapped File – memory mapped files
Page table – page table pages
Note that “shareable” types can have private commitmentRead/write pages in shared memoryCopy-on-write pages
![Page 16: Default 3 GB user space 2 GB Per-Process Space 2 GB System Space 3 GB Per-Process Space 1 GB System Space](https://reader038.vdocuments.net/reader038/viewer/2022102814/5518d191550346881f8b5d7f/html5/thumbnails/16.jpg)
Viewing FragmentationFragmentation is visible by selecting Options->Show Free Regions, selecting the Free type, and sorting by size
Largest free block is largest allocation possible
Clickable fragmentation map in View->Fragmentation ViewRun testlimit -t on 64-bit Windows
Threads need 256 KB 64-bit stack and 1 MB 32-bit stack
![Page 17: Default 3 GB user space 2 GB Per-Process Space 2 GB System Space 3 GB Per-Process Space 1 GB System Space](https://reader038.vdocuments.net/reader038/viewer/2022102814/5518d191550346881f8b5d7f/html5/thumbnails/17.jpg)
File MappingsFile mapping enables an application to read and write file data through memory operationsFile mappings are used for
Image (.EXE and .DLL) loading: “Image” in VMMapData files access (e.g. NLS files): “Mapped File” in VMMap“Pagefile-backed” shared memory: “Shareable” in VMMap
Entire file doesn’t have to be mappedAllows for “windows” into the file
AddressSpace
Database.db
![Page 18: Default 3 GB user space 2 GB Per-Process Space 2 GB System Space 3 GB Per-Process Space 1 GB System Space](https://reader038.vdocuments.net/reader038/viewer/2022102814/5518d191550346881f8b5d7f/html5/thumbnails/18.jpg)
Tracing File Mapping with Process MonitorProcmon can trace image loader activity
![Page 19: Default 3 GB user space 2 GB Per-Process Space 2 GB System Space 3 GB Per-Process Space 1 GB System Space](https://reader038.vdocuments.net/reader038/viewer/2022102814/5518d191550346881f8b5d7f/html5/thumbnails/19.jpg)
VMMap Differencing
Press F5 to refresh the viewVMMap keeps all snapshots
Use the timeline to select snapshots to compare
![Page 20: Default 3 GB user space 2 GB Per-Process Space 2 GB System Space 3 GB Per-Process Space 1 GB System Space](https://reader038.vdocuments.net/reader038/viewer/2022102814/5518d191550346881f8b5d7f/html5/thumbnails/20.jpg)
Tracing with VMMap
You can launch a process with profiling
Detours tracks virtual and heap activity
![Page 21: Default 3 GB user space 2 GB Per-Process Space 2 GB System Space 3 GB Per-Process Space 1 GB System Space](https://reader038.vdocuments.net/reader038/viewer/2022102814/5518d191550346881f8b5d7f/html5/thumbnails/21.jpg)
The System Commit LimitSystem committed virtual memory must be backed either by physical memory or stored in the paging file
Sum of (most of) physical memory and current paging files
Allocations charged against the system commit limit:
Process private bytes Pagefile-backed shared memory Copy-on-write pagesRead/write file pagesSystem paged and nonpaged code and data
When limit is reached, virtual memory allocations fail
Processes may crash (or corrupt data)
![Page 22: Default 3 GB user space 2 GB Per-Process Space 2 GB System Space 3 GB Per-Process Space 1 GB System Space](https://reader038.vdocuments.net/reader038/viewer/2022102814/5518d191550346881f8b5d7f/html5/thumbnails/22.jpg)
Changing the System Commit Limit
You can increase the system commit limit by adding RAM or increasing the pagefile sizeThe system commit limit can grow if paging files configured to expand
So the system commit limit might be the current limit, not the maximumDefault configuration (“System Managed”):
Minimum: 1.5x RAM if RAM < 1 GB; RAM otherwiseMaximum: 3x RAM or 4 GB, whichever is larger
Maximum system commit limit should be based on system commit peak for extreme workload
![Page 23: Default 3 GB user space 2 GB Per-Process Space 2 GB System Space 3 GB Per-Process Space 1 GB System Space](https://reader038.vdocuments.net/reader038/viewer/2022102814/5518d191550346881f8b5d7f/html5/thumbnails/23.jpg)
Viewing System Commit UsagePerformance Counters:
Committed BytesCommit Limit
Task Manager XP: commit charge labeled “PF Usage”Vista: commit charge labeled “Page File”Win7: commit charge labeled “Commit”Vista and Win7 show commit limit after slash
![Page 24: Default 3 GB user space 2 GB Per-Process Space 2 GB System Space 3 GB Per-Process Space 1 GB System Space](https://reader038.vdocuments.net/reader038/viewer/2022102814/5518d191550346881f8b5d7f/html5/thumbnails/24.jpg)
Viewing the System Commit LimitProcess Explorer shows commit charge (with history), commit limit, and commit peak
No built-in tool shows peak any more
![Page 25: Default 3 GB user space 2 GB Per-Process Space 2 GB System Space 3 GB Per-Process Space 1 GB System Space](https://reader038.vdocuments.net/reader038/viewer/2022102814/5518d191550346881f8b5d7f/html5/thumbnails/25.jpg)
Exhausting the System Commit Limit
On 32-bit system, run “Testlimit –m” multiple times until system commit limit exhaustedOn 64-bits, “Testlimit64 –m” will exhaust the system commit limit before its address space:
![Page 26: Default 3 GB user space 2 GB Per-Process Space 2 GB System Space 3 GB Per-Process Space 1 GB System Space](https://reader038.vdocuments.net/reader038/viewer/2022102814/5518d191550346881f8b5d7f/html5/thumbnails/26.jpg)
Sizing the Paging FileIf you enough RAM to support your commit needs, why even have one?
System can page out unused, modified private pages vs keeping them in RAMMore RAM available for useful stuff
Many recommendations use a formula based on RAM (1.5x, 2x, etc.)
Actually, the more RAM, the smaller the paging file neededShould be based on workload usage of committed virtual memory
Look at commit peak after workload has runPre-Vista: Task ManagerVista+: Process ExplorerApply a formula to that to give buffer (1.5x or 2x)Make sure it’s big enough to hold a kernel crash dump
![Page 27: Default 3 GB user space 2 GB Per-Process Space 2 GB System Space 3 GB Per-Process Space 1 GB System Space](https://reader038.vdocuments.net/reader038/viewer/2022102814/5518d191550346881f8b5d7f/html5/thumbnails/27.jpg)
Physical Memory
![Page 28: Default 3 GB user space 2 GB Per-Process Space 2 GB System Space 3 GB Per-Process Space 1 GB System Space](https://reader038.vdocuments.net/reader038/viewer/2022102814/5518d191550346881f8b5d7f/html5/thumbnails/28.jpg)
Working Set List
All the physical pages “owned” by a processE.g. the pages the process can reference without incurring a page fault
A process always starts with an empty working setIt then incurs page faults when referencing a page that isn’t in its working setMany page faults may be resolved from memory
Working Set
newer pages older pages
![Page 29: Default 3 GB user space 2 GB Per-Process Space 2 GB System Space 3 GB Per-Process Space 1 GB System Space](https://reader038.vdocuments.net/reader038/viewer/2022102814/5518d191550346881f8b5d7f/html5/thumbnails/29.jpg)
Working Set
Each process has a default working set minimum and maximum
Can change with SetProcessWorkingSetWorking set minimum controls maximum number of locked pages (VirtualLock)Minimum is also reserved from RAM as a guarantee to the processWorking set maximum is ignored
If there’s ample memory, process working set represents all the memory it has referenced (but not freed)
If memory is tight, working sets get trimmed
![Page 30: Default 3 GB user space 2 GB Per-Process Space 2 GB System Space 3 GB Per-Process Space 1 GB System Space](https://reader038.vdocuments.net/reader038/viewer/2022102814/5518d191550346881f8b5d7f/html5/thumbnails/30.jpg)
Working Set Replacement
When memory manager decides the process is large enough, it give up pages to make room for new pagesLocal page replacement policy
Means that a single process cannot take over all of physical memory unless other processes aren’t using itPage replacement algorithm is least recently accessed (pages are aged when available memory is low)
Working Set
To standby or modified
page list
![Page 31: Default 3 GB user space 2 GB Per-Process Space 2 GB System Space 3 GB Per-Process Space 1 GB System Space](https://reader038.vdocuments.net/reader038/viewer/2022102814/5518d191550346881f8b5d7f/html5/thumbnails/31.jpg)
Working Set BreakdownConsists of 2 types of pages:
Shareable (of which some may be shared)Private
Four performance counters available:Working Set Shareable
Working Set Shared (subset of shareable that are currently shared)
Working Set PrivateWorking Set Size (total of WS Shareable+Private)
Note: adding this up for each process overcounts shared pages
Caveats:Working set does not include trimmed memory that is still cachedShareable working set should be viewed as “private” if it’s not shared
![Page 32: Default 3 GB user space 2 GB Per-Process Space 2 GB System Space 3 GB Per-Process Space 1 GB System Space](https://reader038.vdocuments.net/reader038/viewer/2022102814/5518d191550346881f8b5d7f/html5/thumbnails/32.jpg)
Viewing Working Set with Task Manager
Displays private working set sizeCalls it “Memory (Private Working Set)”
![Page 33: Default 3 GB user space 2 GB Per-Process Space 2 GB System Space 3 GB Per-Process Space 1 GB System Space](https://reader038.vdocuments.net/reader038/viewer/2022102814/5518d191550346881f8b5d7f/html5/thumbnails/33.jpg)
Viewing Working Set with Process ExplorerProcess Explorer shows all the performance counters
Virtual BytesPrivate BytesWS Shareable BytesWS Shared BytesWS Private Bytes
Run Testlimit three times:Testlimit -r 1024 -c 1Testlimit -m 1024 -c 1Testlimit -d 1024 -c 1
Note how working set numbers don’t at all represent the process virtual memory usage
![Page 34: Default 3 GB user space 2 GB Per-Process Space 2 GB System Space 3 GB Per-Process Space 1 GB System Space](https://reader038.vdocuments.net/reader038/viewer/2022102814/5518d191550346881f8b5d7f/html5/thumbnails/34.jpg)
Viewing the Working Set with VMMapVmmap shows working set size of each component of address spaceAlso shows locked pagesCopy-on-write pages will show up as Private WS in shareable regions
![Page 35: Default 3 GB user space 2 GB Per-Process Space 2 GB System Space 3 GB Per-Process Space 1 GB System Space](https://reader038.vdocuments.net/reader038/viewer/2022102814/5518d191550346881f8b5d7f/html5/thumbnails/35.jpg)
Physicalmemory
Page 3
Page 1
How Copy-On-Write Works:Before
ProcessAddressSpace
Orig. Data
ProcessAddressSpace
Orig. Data
Page 2
![Page 36: Default 3 GB user space 2 GB Per-Process Space 2 GB System Space 3 GB Per-Process Space 1 GB System Space](https://reader038.vdocuments.net/reader038/viewer/2022102814/5518d191550346881f8b5d7f/html5/thumbnails/36.jpg)
ProcessAddressSpace
Physicalmemory
How Copy-On-Write Works:After
ProcessAddressSpace
Orig. Data
Page 3
Page 1
Page 2
Copy of page 2
Mod’d. Data
![Page 37: Default 3 GB user space 2 GB Per-Process Space 2 GB System Space 3 GB Per-Process Space 1 GB System Space](https://reader038.vdocuments.net/reader038/viewer/2022102814/5518d191550346881f8b5d7f/html5/thumbnails/37.jpg)
Managing Physical Memory
System keeps unassigned physical pages on one of several lists
Free page listModified page listStandby page lists (8 as of Vista & later)Zero page listROM page listBad page list - pages that failed memory test at system startup
Lists are implemented by entries in the “PFN database”
Maintained as FIFO lists or queues
![Page 38: Default 3 GB user space 2 GB Per-Process Space 2 GB System Space 3 GB Per-Process Space 1 GB System Space](https://reader038.vdocuments.net/reader038/viewer/2022102814/5518d191550346881f8b5d7f/html5/thumbnails/38.jpg)
Paging Dynamics
New pages are allocated to working sets from the top of the free or zero page listPages released from the working set due to working set replacement go to the bottom of:
The modified page list (if they were modified while in the working set)The standby page list (if not modified)
Decision made based on “D” (dirty = modified) bit in page table entry
Association between the process and the physical page is still maintained while the page is on either of these lists
![Page 39: Default 3 GB user space 2 GB Per-Process Space 2 GB System Space 3 GB Per-Process Space 1 GB System Space](https://reader038.vdocuments.net/reader038/viewer/2022102814/5518d191550346881f8b5d7f/html5/thumbnails/39.jpg)
Standby and Modified Page Lists
Modified pages go to modified (dirty) listAvoids writing pages back to disk too soon
Unmodified pages go to standby (clean) listsThey form a system-wide cache of “pages likely to be needed again”
Pages can be faulted back into a process from the standby and modified page listThese are counted as page faults, but not page reads
![Page 40: Default 3 GB user space 2 GB Per-Process Space 2 GB System Space 3 GB Per-Process Space 1 GB System Space](https://reader038.vdocuments.net/reader038/viewer/2022102814/5518d191550346881f8b5d7f/html5/thumbnails/40.jpg)
Modified Page Writer
When modified list reaches certain size, modified page writer system thread is awoken to write pages out
Also triggered when memory is overcommitted (too few free pages)Does not flush entire modified page list
Two system threadsOne for mapped files, one for the paging file
Pages move from the modified list to the standby list
E.g. can still be soft faulted into a working set
![Page 41: Default 3 GB user space 2 GB Per-Process Space 2 GB System Space 3 GB Per-Process Space 1 GB System Space](https://reader038.vdocuments.net/reader038/viewer/2022102814/5518d191550346881f8b5d7f/html5/thumbnails/41.jpg)
Free and Zero Page Lists
Free Page ListUsed for page readsPrivate modified pages go here on process exitPages contain junk in them (e.g. not zeroed)On most busy systems, this is empty
Zero Page ListUsed to satisfy demand zero page faults
References to private pages that have not been created yet
When free page list has 8 or more pages, a priority zero thread is awoken to zero themOn most busy systems, this is empty too
![Page 42: Default 3 GB user space 2 GB Per-Process Space 2 GB System Space 3 GB Per-Process Space 1 GB System Space](https://reader038.vdocuments.net/reader038/viewer/2022102814/5518d191550346881f8b5d7f/html5/thumbnails/42.jpg)
Paging Dynamics
StandbyPageLists
ZeroPageList
FreePageList
WorkingSets
page read from disk or kernel allocations
demand zero page faults
working set replacement
ModifiedPageList
modifiedpagewriter
zeropage
thread
“soft”pagefaults
BadPageList
Private pages at process exit
“global valid” faults
![Page 43: Default 3 GB user space 2 GB Per-Process Space 2 GB System Space 3 GB Per-Process Space 1 GB System Space](https://reader038.vdocuments.net/reader038/viewer/2022102814/5518d191550346881f8b5d7f/html5/thumbnails/43.jpg)
Viewing the Paging Lists with Task Manager
XP/2003:Available = Standby + Zero + FreeSystem Cache = Standby + Modified + System Working Set
Vista/Server 2008:Replaced Available with Free
Free + Zero list
System Cache relabeled Cached
Windows 7/Server 2008 R2Available put back
![Page 44: Default 3 GB user space 2 GB Per-Process Space 2 GB System Space 3 GB Per-Process Space 1 GB System Space](https://reader038.vdocuments.net/reader038/viewer/2022102814/5518d191550346881f8b5d7f/html5/thumbnails/44.jpg)
Viewing the Paging Lists with Process Explorer
Process Explorer shows each paging listClick View->System Information
![Page 45: Default 3 GB user space 2 GB Per-Process Space 2 GB System Space 3 GB Per-Process Space 1 GB System Space](https://reader038.vdocuments.net/reader038/viewer/2022102814/5518d191550346881f8b5d7f/html5/thumbnails/45.jpg)
Total Process Private Memory Usage
Working Set size does not include:
Private memory on standby or modified listsPage tables
Rammap shows this on Processes tab
![Page 46: Default 3 GB user space 2 GB Per-Process Space 2 GB System Space 3 GB Per-Process Space 1 GB System Space](https://reader038.vdocuments.net/reader038/viewer/2022102814/5518d191550346881f8b5d7f/html5/thumbnails/46.jpg)
Viewing Memory Usage with Rammap
In addition to showing size of paging lists, shows usage breakdown:
Process privateMapped fileShared memoryPage tablesPaged poolNonpaged poolSystem PTESession privateMetafileAWEDriver lockedKernel stack
![Page 47: Default 3 GB user space 2 GB Per-Process Space 2 GB System Space 3 GB Per-Process Space 1 GB System Space](https://reader038.vdocuments.net/reader038/viewer/2022102814/5518d191550346881f8b5d7f/html5/thumbnails/47.jpg)
Prioritized Standby Lists
In Vista & later, there are 8 prioritized standby listsPages are removed from lowest priority list first
Low memory priority process will keep re-using low priority pagesHigher priority information remains cached
0
1
2
3
4
5
6
7
Prioritized Standby Lists
Pages removed
Pages added
![Page 48: Default 3 GB user space 2 GB Per-Process Space 2 GB System Space 3 GB Per-Process Space 1 GB System Space](https://reader038.vdocuments.net/reader038/viewer/2022102814/5518d191550346881f8b5d7f/html5/thumbnails/48.jpg)
SuperFetch™Superfetch proactively repopulates RAM with the most useful data
Sets priority of pages to optimal value, based the page history and other analysis that it performsTakes into account frequency of page usage, usage of page in context of other pages in memoryAdapts application launch patterns, in chunks of 8 hours (times a day) and weekend vs weekday
Scenarios SuperFetch improves includeResume from hibernate and suspendFast user switchingPerformance after infrequent or low priority tasks executeApplication launch
Windows 7: Disabled if the OS is booted of an SSD
![Page 49: Default 3 GB user space 2 GB Per-Process Space 2 GB System Space 3 GB Per-Process Space 1 GB System Space](https://reader038.vdocuments.net/reader038/viewer/2022102814/5518d191550346881f8b5d7f/html5/thumbnails/49.jpg)
Memory Priority
Each thread has its own memory priority5: normal1: low
This determines which standby list is used for the page (when/if it arrives on the standby list)Thread priority comes from process memory priority
Can be changed for process or individual threadSetPriorityClass or SetThreadPriority “background mode”
![Page 50: Default 3 GB user space 2 GB Per-Process Space 2 GB System Space 3 GB Per-Process Space 1 GB System Space](https://reader038.vdocuments.net/reader038/viewer/2022102814/5518d191550346881f8b5d7f/html5/thumbnails/50.jpg)
Standby List Population
Priority 7 come from a static set (pre-trained at Microsoft)
Pre-populated at each bootIncludes pages related to user input that requires fast responsiveness (right-click, desktop properties, control panel, start menu, etc.)
Priority 6 are pages that SuperFetch considers important, or useful (will rarely get repurposed)Priority 5 are standard user pages (memory priority 5)Priority 1 are low priority user pages (memory priority 1)Priority 0-4 may be Superfetch decayed, cache manager read-ahead and pagefault clustering
![Page 51: Default 3 GB user space 2 GB Per-Process Space 2 GB System Space 3 GB Per-Process Space 1 GB System Space](https://reader038.vdocuments.net/reader038/viewer/2022102814/5518d191550346881f8b5d7f/html5/thumbnails/51.jpg)
How Much of the Standby List has Been Consumed?
RAMMap shows the amount of memory repurposed off each standby list since boot:
![Page 52: Default 3 GB user space 2 GB Per-Process Space 2 GB System Space 3 GB Per-Process Space 1 GB System Space](https://reader038.vdocuments.net/reader038/viewer/2022102814/5518d191550346881f8b5d7f/html5/thumbnails/52.jpg)
What File Data is In the Standby Lists? Viewing Cached Files with Rammap
![Page 53: Default 3 GB user space 2 GB Per-Process Space 2 GB System Space 3 GB Per-Process Space 1 GB System Space](https://reader038.vdocuments.net/reader038/viewer/2022102814/5518d191550346881f8b5d7f/html5/thumbnails/53.jpg)
Do You Have Enough Memory?There’s no sure-fire rule or counter to tell if you if you have enough memoryThe general rule: available memory remains generally low
Use Perfmon to monitor available memory over timeUse Process Explorer, or on Vista and later, Task Manager, to monitor physical memory usageUse Process Explorer, or Task Manager to see instantaneous value of available memoryWatch in Process Monitor for excessive reads from paging file
![Page 54: Default 3 GB user space 2 GB Per-Process Space 2 GB System Space 3 GB Per-Process Space 1 GB System Space](https://reader038.vdocuments.net/reader038/viewer/2022102814/5518d191550346881f8b5d7f/html5/thumbnails/54.jpg)
Tracing Paging with Procmon
Procmon distinguishes paging I/Os in the details column
Can set filter for “detail contains paging”
I/O to Pagefile.sys excluded by defaultEnable advanced output or remove exclude filterExcessive reads from paging file indicates need more RAM
![Page 55: Default 3 GB user space 2 GB Per-Process Space 2 GB System Space 3 GB Per-Process Space 1 GB System Space](https://reader038.vdocuments.net/reader038/viewer/2022102814/5518d191550346881f8b5d7f/html5/thumbnails/55.jpg)
Hard to Track Memory
![Page 56: Default 3 GB user space 2 GB Per-Process Space 2 GB System Space 3 GB Per-Process Space 1 GB System Space](https://reader038.vdocuments.net/reader038/viewer/2022102814/5518d191550346881f8b5d7f/html5/thumbnails/56.jpg)
Hidden Cost of Reserved MemoryMemory Manager charges for page tables for reserved address space not yet committed
Charged against process private bytes (and therefore system commit limit)Cannot track this down
Experiment:Testlimit64 –r 100000 –c 10
Testlimit64 –rReserves ~8192GBPrivate bytes grows to >16GB!
![Page 57: Default 3 GB user space 2 GB Per-Process Space 2 GB System Space 3 GB Per-Process Space 1 GB System Space](https://reader038.vdocuments.net/reader038/viewer/2022102814/5518d191550346881f8b5d7f/html5/thumbnails/57.jpg)
Cost of Reserved Memory
Virtual Address Space (VAD) descriptors come from nonpaged poolExample:
Testlimit64 –r results in 640mb of nonpaged pool usage for VADsPoolmon shows this:
![Page 58: Default 3 GB user space 2 GB Per-Process Space 2 GB System Space 3 GB Per-Process Space 1 GB System Space](https://reader038.vdocuments.net/reader038/viewer/2022102814/5518d191550346881f8b5d7f/html5/thumbnails/58.jpg)
Shared Memory
Shared memory is backed by virtual memoryEither paging file (if there is one), else physical memory
However, amount created not charged to process commit limit
Therefore, a shared memory VM leak is hard to track down
![Page 59: Default 3 GB user space 2 GB Per-Process Space 2 GB System Space 3 GB Per-Process Space 1 GB System Space](https://reader038.vdocuments.net/reader038/viewer/2022102814/5518d191550346881f8b5d7f/html5/thumbnails/59.jpg)
Demo: Shared Memory Leak
Testlimit –s 1000 –c 3This creates a 3 GB shared memory section
Note that process virtual and private bytes do not include this value
And only virtual bytes rise when process maps the section into its address space
![Page 60: Default 3 GB user space 2 GB Per-Process Space 2 GB System Space 3 GB Per-Process Space 1 GB System Space](https://reader038.vdocuments.net/reader038/viewer/2022102814/5518d191550346881f8b5d7f/html5/thumbnails/60.jpg)
VirtualLock Locked PagesNo special privilege is required to VirtualLock pages (as of Vista)
Allows process to allocate non-paged memory
Locked memory has a major impact on the system:Overrides memory management policiesPrevents contiguous physical memory allocationCan prevent hibernation
Testlimit -l 1024 –c 1DWM locks memory:
![Page 61: Default 3 GB user space 2 GB Per-Process Space 2 GB System Space 3 GB Per-Process Space 1 GB System Space](https://reader038.vdocuments.net/reader038/viewer/2022102814/5518d191550346881f8b5d7f/html5/thumbnails/61.jpg)
Driver Locked Pages and AWE pagesVPC and Hyper-V use driver locked pages
Counts against system commit, but not otherwise detectableNo way to track back to owner
![Page 62: Default 3 GB user space 2 GB Per-Process Space 2 GB System Space 3 GB Per-Process Space 1 GB System Space](https://reader038.vdocuments.net/reader038/viewer/2022102814/5518d191550346881f8b5d7f/html5/thumbnails/62.jpg)
AWE Memory
Address Windowing Extension (AWE) allows processes to directly control physical memory
Way to override the memory manager’s caching algorithmsCan be more than can be contained in the address space
AWE memory is not accounted to the owning process
Thus, available physical memory drops, but no process accounts for usage
![Page 63: Default 3 GB user space 2 GB Per-Process Space 2 GB System Space 3 GB Per-Process Space 1 GB System Space](https://reader038.vdocuments.net/reader038/viewer/2022102814/5518d191550346881f8b5d7f/html5/thumbnails/63.jpg)
Demo: AWE Memory Leak
Testlimit -a 1000 -c 1Allocates 1000 MB AWE memory
![Page 64: Default 3 GB user space 2 GB Per-Process Space 2 GB System Space 3 GB Per-Process Space 1 GB System Space](https://reader038.vdocuments.net/reader038/viewer/2022102814/5518d191550346881f8b5d7f/html5/thumbnails/64.jpg)
More Information
More information in Windows Internals, 5th EditionMemory Management chapter
MSDN memory management API documentationThe best way to gain an understanding of memory manager behavior is to experiment and observeCome to my other sessions:
Inside Windows Azure: 10:30Case of the Unexplained: 12:30
Sysinternals Primer: 9:30Aaron Margosis and Tim Reckmeyer
![Page 65: Default 3 GB user space 2 GB Per-Process Space 2 GB System Space 3 GB Per-Process Space 1 GB System Space](https://reader038.vdocuments.net/reader038/viewer/2022102814/5518d191550346881f8b5d7f/html5/thumbnails/65.jpg)
Session Evaluations
Tell us what you think, and you could win!
All evaluations submitted are automatically entered into a daily prize draw*
Sign-in to the Schedule Builder at http://europe.msteched.com/topic/list/ * Details of prize draw rules can be obtained from the Information Desk.
![Page 66: Default 3 GB user space 2 GB Per-Process Space 2 GB System Space 3 GB Per-Process Space 1 GB System Space](https://reader038.vdocuments.net/reader038/viewer/2022102814/5518d191550346881f8b5d7f/html5/thumbnails/66.jpg)