Download - Memory management in sql server
Customer Service & Support
1. Memory Manager: Windows NT
• Virtual memoryo Address Space Layouto Virtual Memory Managero 32-bit Virtual Addresseso Address Translation
• Memory Pool• 4GT Tuning
o /3GB Switch (Two slides)o Effects of /3GB Tuningo /USERVA Switch
• PAEo Using /3GB & PAE together
• AWE• 32-bit vs 64-bit Virtual Memory
Memory Manager• As the size of applications and the operating systems
that run grow larger and larger, so do their demands on memory.
• Consequently, all modern operating systems provide a form of virtual memory to applications.
• Windows Server 2003 onwards, the applications ported to it, eventually require even more memory than they did on the last operating system on which they ran.
• Even applications being written exclusively for Windows NT will be written with the future in mind and will no doubt take advantage of all the memory that is available to them.
Virtual Memory• Address Space Layout• Virtual-Memory Manager (VMM)• 32-bit Virtual Addresses• Address Translation
Address Space Layout• Windows provides a page-based virtual memory management
scheme that allows applications to realize a 32-bit linear address space for 4 GB of memory.
• Each application can address 2 GB of available memory, regardless of how much physical memory actually exists.
• Windows employs the PC's hard disk as the memory-backing store, and has a practical limit imposed only by the available disk space.
Virtual Memory Manager
• Enables Windows to provide virtually unlimited memory to the system. • Is constructed upon a page-based memory management scheme that
divides all of memory into equal chunks called pages.• Each page is 4096 bytes (4 KB) in size with no discrimination applied as
to how a page is used. • Everything in Windows—code, data, resources, files, and dynamic
memory—is implemented using pages of physical memory.• Employs the hard disk to store unutilized pages of memory in one or
more files called pagefiles.• Pagefiles represent pages of data that are not currently being used, but
may be needed spontaneously at any time.• Pagefiles in Windows are dynamic in size, allowing them to grow as the
demands for pages of memory grow. • Is able to make pages of memory available to applications on demand
by swapping pages to and from pagefiles.• Provides much more virtual memory than the available physical
memory.
32-bit Virtual Addresses
• Windows implement a virtual memory system based on a flat (linear) 32-bit address space.o 32 bits of address space translates into 4 GB of virtual memory.
• To Windows, the 32-bit virtual address is nothing more than a placeholder of information used to find the actual physical address.o Windows separates each 32-bit virtual address into three groups:
• Two containing 10 bits and one containing 12 bits.o Each group of bits is then used independently as an offset into a specific page of
memory.
Address Translation• User applications reference 32-
bit virtual addresses. • Using data structures that the
memory manager creates and maintains, the CPU translates virtual addresses into physical addresses.
• Virtual addresses are not mapped directly to physical ones.
• Instead, each virtual address is associated with a system-space structure called a page table entry (PTE), which contains the physical address to which the virtual one is mapped.
Memory Pools• Paged Pool
o Is a region of virtual memory in system space that can be paged in and out of the system.
o Is accessible from any process context.
o Device drivers that do not need to access the memory from DPC/dispatch level or above can use paged pool.
• Nonpaged Poolo Consists of ranges of system virtual
addresses that are guaranteed to reside in physical memory at all times.
o Can be accessed at any time (from any IRQL level and from any process context) without incurring a page fault.
o Page faults cannot be satisfied at DPC/dispatch level or above.
4GT Tuning• Operating systems based on Windows NT
technology provide applications with a 4 GB virtual address space.
• The virtual address space is divided such that 2 GB is available to the application and the other 2 GB is available only to the system.
• The 4GT RAM Tuning feature:o Increases the user-mode memory available to the application to 3 GB.o Reduces the amount available to the system to 1 GB.
• The 4GB Tuning feature is enabled by adding /3GB Switch to the Boot.ini file.
/3GB Switch• /3GB is a switch used within the Boot.ini to:
o Increase the size of a user process address space from 2 GB to 3GB.o Reduce the Kernel space from 2 GB to 1 GB.
• The application .exe must be flagged as a 3-GB-aware application.
/3GB Switch
Effects of /3GB Tuning• When you use the /3GB switch, the available
address space is reduced to only 1 GB of operating system space.
• This 1 GB is reserved to load the operating system kernel mode modules, all the page table entries, and the pools.
• As a result, the operating system will have only a fraction of kernel address space to store PTEs, Pool Memory and System Cache.
• This could cause potential performance issues for any kernel-mode component.
/USERVA Switch• Windows 2003 Servers and
Windows XP SP1 onwards, a new switch /USERVA was incorporated to work in conjunction with the /3GB switch.
• You can use the /USERVA switch for more precise tuning of user and kernel virtual memory space in the Windows Server 2003 family.
• The use of the /USERVA switch with the /3GB switch in the Boot.ini file helps tune the user-mode space to a value between 2 GB and 3 GB.
• The difference is given back to the kernel mode.
/3GB SWITCH IS NOT SUPPORTED ON
64 BIT SYSTEMS
Summary of 4GT Tuning• 4GT allows memory intensive Win32 applications to increase its virtual address
space beyond 2GB o Causes kernel virtual address space to be reduced to 1GB
• This feature is enabled byo On pre-Vista OSs use /3GB switch in boot.ini
• If PAE mode is enabled, access to RAM is limited to 16GBo On Vista and later OSs use “bcdedit/set IncreaseUserVa3072”
• If PAE mode is enabled, all of 64GB of RAM is accessible• Only applications linked with /LARGEADDRESSAWARE flag can use this extra
virtual address spaceo All other applications can only use 2GB
• /USERVA enables fine tuning of UVAS between 2GB and 3GBo Used in tandem with the /3GB switch
• Reduces the following kernel virtual address rangeso Maximum Paged Pool to 256MB o Maximum Non-Pages Pool to 128MB
• Supported on the following Windows 2003 Server version of the OSo Standard Edition, Enterprise Edition, DataCenterEdition
Physical Address Extension (PAE)
• Physical Address Extension (PAE) allows up to 64 GB of physical memory.
• PAE maps up to 64 GB of memory into a 32-bit (4 GB) virtual address space.
• PAE is supported in:o Windows XP Professionalo Windows 2000 Advanced Server
and Datacenter Servero Windows Server 2003, Enterprise
Edition and Datacenter Edition.
Using /3GB & /PAE Together
• Using /3GB and/or /PAE changes the amount of resources that are dedicated for various operations.o Using /3GB reduces the amount of nonpaged pool from 256MB to
128MB.
• Using /3GB and /PAE together:o Provides some customers the most optimum performance for their
servers.o Is required when using applications such as Exchange Server.
Address Windowing Extensions
(AWE)• Is a set of APIs that allows an
application to quickly manipulate physical memory greater than 4GB.
• Allows applications to directly address huge amounts of memory while continuing to use 32-bit pointers.
• Allows applications to have data caches larger than 4GB.
• Uses physical nonpaged memory and window views of various portions of this physical memory within a 32-bit virtual address space.
32-bit vs. 64-bit Virtual Memory
2. Memory Manager: SQLOS
• NUMA (Non-Uniform Memory Architecture)• BP and MTL ?• Memory Types• Memory Pressure
SQLOS• Memory Management• Scheduling• Exception handling
DeadlockMonitor
SQ
LO
S H
ostin
g A
PI
Scheduling
Memory Manager
BufferPool
LockManager
Synchronization
ResourceMonitor
LazyWriter
SQLOS API
I/O
SQ
LO
S A
PI
Exte
rnal C
om
pon
en
ts (CL
R/M
DA
C)
SQLOS
SchedulerMonitor
= thread
Parser OptimizerSQL
ManagerDatabase Manager
Query Executor
Query Processor
Transaction Services
Lock Manager
File Manager
Buffer Manager
Utilities: Bulk Load DBCCBackup/Restore
Access Methods Managers for:
Row OperationsIndexesPagesAllocationsVersions
Storage Engine
Protocols
“External” Components
• Three external components in SQL address spaceo Linked serverso COM objects (sp_OA*)o Extended stored procs
• Linked Server and COM objects always use MemToLeave. Xprocs may use either BP or MTL depending upon the call to srv_alloc()
NUMA (Non-Uniformed Memory Access)
Memory
CPU 0 CPU 1 CPU 2 CPU 3
Memory
CPU… CPU… CPU… CPU n
Local Memory Access
Foreign Memory
Access
4x local Local Memory Access
Non-Uniform Memory Architecture (NUMA)
• Reduces load on memory placed by multiple high speed CPUs
• Groups of CPUs organized into nodes• Nodes have local memory, cache and I/O controllers• Accessing memory in other nodes involves higher latency• NUMA Ratio is ratio of local vs. remote memory access
time• Cache Coherency maintained by hardware• BIOS Static Affinity Resource Table (SART) table
describes NUMA layout to Windows• OS support for NUMA• Schedules threads within same process on CPUs local to
a node• Memory allocation requests from local node
What is Soft-NUMA• Activates custom NUMA configuration on
top of any of hardware
• Registry settings control final SoftNUMA
configuration
• Provides greater performance, scalability,
and manageability on SMP as well as on
real NUMA hardware
[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Microsoft SQL Server\90\NodeConfiguration\Node0]"CPUMask"=dword:0000000F]HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Microsoft SQL Server\90\NodeConfiguration\Node1["CPUMask"=dword:00000030]HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Microsoft SQL Server\90\NodeConfiguration\Node2["CPUMask"=dword:000000C0
[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Microsoft SQL Server\MSSQL.1\MSSQLServer\SuperSocketNetLib\Tcp\IPAll]"TcpPort"="1433]0x3[,2000]0x4[""TcpDynamicPorts"="""DisplayName"="Any IP Address"
Soft-NUMA Configuration Example
BufferPool and MTL?• How is Buffer Pool size calculated?• What happens to the rest of the VAS?• Roughly: MTL = (MaxWorkerThreads * StackSize) + DefautReservationSize
Bpool = Min(max server memory,Min(RAM,(VAS-MTL)))
Memory Types• Internal Virtual Memory (VAS)• External Virtual Memory(physical RAM +Swap)• Internal Physical Memory(Memory Used by the
application)• External Physical memory (PhysicalRAM)
Memory Pressure• Internal VAS Pressure• External VAS Pressure• Internal Physical Memory Pressure• External Physical memory Pressure
Internal Virtual Memory Pressure
• Running low on VAS due to fragmentation (a lot of VAS is available but in small blocks) and/or consumption (direct allocations, DLLs loaded in SQL Server VAS, high number of threads).
• SQL Server detects this condition and may release reserved regions of VAS, reduce buffer pool commit target, and start shrinking caches.
Internal Virtual Memory Pressure
Troubleshooting• Identify major memory consumers inside
SQL Server. • Verify server configuration.• Further actions depend on the investigation:
check for workload; possible design issues; other resource bottlenecks.
External Virtual Memory Pressure
• Running low on space in the system page file(s). This may cause the system to fail memory allocations, as it is unable to page out currently allocated memory. This condition may result in the whole system responding very slowly or even bring it to a halt.
External Virtual Memory Pressure
Troubleshooting
• Increase swap file size.• Check for major physical memory consumers• Find major system memory consumers.• Attempt to eliminate (if possible).
Internal Physical Memory Pressure
• SQL Server detects high memory consumption internally, causing redistribution of memory between internal components.
• Internal memory pressure may be a result of:
• Responding to the external memory pressure (SQL Server sets lower memory usage caps).
• Changed memory settings (e.g. ‘max server memory’).
• Changes in memory distribution of internal components (due to high percentage of reserved and stolen pages from the buffer pool).
Internal Physical Memory Pressure
Troubleshooting
• Identify major memory consumers inside SQL Server.
• Verify server configuration.• Further actions depend on the investigation:
check for workload; possible design issues; other resource bottlenecks
External Physical Memory Pressure
• Physical memory (RAM) running low. This causes the system to trim working sets of currently running processes, which may result in overall slowdown.
• SQL Server detects this condition and, depending on the configuration, may reduce the commit target of the buffer pool and start clearing internal caches.
External Physical Memory Pressure
Troubleshooting
• Find major system memory consumers.• Attempt to eliminate (if possible).• Check for adequate system RAM and consider
adding more RAM
Resources• Blogs
o http://blogs.msdn.com/slavao o http://
blogs.msdn.com/psssql/archive/2007/10/18/do-i-have-to-assign-the-lock-privilege-for-local-system.aspx o http://blogs.msdn.com/slavao/archive/2005/02/19/376714.aspx o http://blogs.msdn.com/slavao/archive/2005/08/18/453354.aspx o http://blogs.msdn.com/slavao/articles/441058.aspx o http://blogs.msdn.com/sqlprogrammability/
• External Links:o http://lse.sourceforge.net/numa/faq o http://www.modhul.com/2007/11/10/optimising-system-memory-for-sql-server-part-i/ o http://sqlserver.ro/blogs/cristians_blog/archive/2007/10/02/the-return-of-the-ring-buffers.aspx o http://lse.sourceforge.net/numa/faq/system_descriptions.html o http://lse.sourceforge.net/numa/faq/
• Books:o Eldad Eilam: Reversing: Secrets of Reverse Engineeringo Ken Henderson: SQL Server 2005 Practical Troubleshooting The Database Engine o Kalen Delaney: Inside Microsoft SQL Server 2005 The Storage Engine
• KBAs:o http://support.microsoft.com/kb/918483o http://support.microsoft.com//kb/271624o http://support.microsoft.com//kb/907877
?
Thank you!