Debug C Run-Time & Memory Management

Chapter 17, Debugging Applications for .NET and Windows by John Robbins

March 19th 2004

Divyanand M. Kutagulla,

SU, Dept of EECS,

dkutagulla@hotmail.com

Main Topics The Debug C Runtime Library (The DCRT)

How to setup use the DCRT Useful DCRT functions

Memory Management OS heaps

Useful VS.NET Compiler switches

The Debug C Runtime Library (The DCRT)

References

1. Robbins’ Book

2. Platform SDK

Overview The Debug C Run-Time Library DCRT Features Setting up and Using the DCRT Useful DCRT functions DCRT library versions An Important Note

The Debug C Run-Time Library The Debug version of the C/C++ runtime on

the Microsoft platforms ‘Memory debugging’ for C/C++ Has been there since Visual C++ 4… a.k.a. DCRT (say it as: “D-Cert”)

DCRT Features Heap Memory tracking support

Allocations De-allocations Memory underwrites, overwrites Memory Leaks

Reporting Subsystem Trace of the app’s memory behavior

Heap Memory Tracking Support Memory allocations done through new, malloc, calloc

Memory underwrites, overwrites Reported through an assertion in the DCRT

Memory Leaks Reported at app exit though a call to

OutputDebugString MFC reports memory leaks this way…

Reporting Subsystem Reporting Subsystem (trace)

_RPTn & _RPTFn macros n is a number from 1 to 4, shows how many

parameters to report Can redirect the reports to any destination (a

logfile, console etc) Assertion support through the ASSERT macro

Violates the CROA (Cardinal rule of assertions) by destroying the last error code...

Setting up & Using the DCRT DCRT Source Files DCRT Setup DCRT Flags Useful DCRT functions

DCRT Source Files Optionally installed during VS.NET

installation If CRT source code installation is selected Install Path: <VS. NET inst dir>\vc7\crt\src\

<VS.NET inst dir>: VS.NET installation directory E.g. C:\Program Files\Microsoft Visual

Studio .NET 2003\Vc7\crt\src

DCRT Source Files (cont’d)DBGNEW.CPP The debug version of global new

operator

DBGDEL.CPP The debug version of global delete operator

DBGHEAP.C The debug heap handling functions

DBGHOOK.C The stub memory allocation hook functions

DCRT Source Files (cont’d)DBGINT.H Internal debug headers and functions

DBGRPT.C Debug reporting functions (the reporting subsystem)

CRTDBG.H The header file that the client includes to start using the DCRT

DCRT Setup #define the _CRT_DBG_MAP_ALLOC

macro before any #includes in the header. This will cause all the calls to memory

allocation/de-allocation routines to be redirected to special versions of these routines

The special versions record the source file name and the line number in that file where the call occurred __FILE__ & __LINE__ macros

DCRT Setup (cont’d) After all the header files have been

‘#include’d, ‘#include’ the CRTDBG.H header.

Turn on the DCRT heap code: Done through calling the _CrtSetDbgFlag() with

the appropriate flag arguments to configure the DCRT Turn on the memory leak reporting Turn on memory checking on each Memory

allocation Etc…

DCRT Flags_CRTDBG_ALLOC_MEM_DF Turns on debug heap allocations. (on

by default)

_CRTDBG_CHECK_ALWAYS_DF Turn on checking on every allocation/de-allocation

_CRTDBG_LEAK_CHECK_DF Turn on check for memory leaks at the end of the program

_CRTDBG_CHECK_CRT_DF The memory allocated by the CRT for its internal use is also considered when checking for memory leaks memory state differences ( generally this flag is not turned on)

DCRT Flags (cont’d)_CRTDBG_DELAY_FREE_MEM_DF

Code example shows the different DCRT flags and how to set/unset the DCRT flags

The DCRT will flag the memory to be freed but will not actually de-allocate it. It flags the memory by filling the memory with special value: 0xDD. The DCRT checks that this memory blocks are not subsequently accessed

Very useful when we want to simulate low memory conditions on the application

DCRT Bug!!!If we allocate memory through the new operator, the memory leak report does not show the correct source file location of the leaked allocation. (sample report below is from the previous code example)

Detected memory leaks!Dumping objects ->c:\program files\microsoft visual studio .net2003\vc7\include\crtdbg.h (689) : {57} normal block at 0x00320800, 28 bytes long.Data: < myclass > CD CD CD CD 6D 79 63 6C 61 73 73 00 CD CD CD CD …< OUTPUT TRUNCATED FOR CLARITY >…Object dump complete.

Note DCRT BUG!!!: The leak report for the ‘new’ed memory seems to come from crtdbg.h although the error is not in this file..

Why the DCRT Bug??? The new operator is defined as an inline in

CRTDBG.H :inline void * __cdecl operator new(size_t s){ return ::operator new(s, _NORMAL_BLOCK, __FILE__,

__LINE__); }

But it is not expanded inline at point of call when _DEBUG is defined (i.e. when we do a Debug build)!!!

Useful DCRT functions

References

1. Robbins’ book

2. MSDN

Overview Memory allocation functions Memory Integrity Check functions Parameter Validation Functions Memory State Validation Functions DCRT Hooks

Memory allocation/deallocation Memory dumping

DCRT Debug Report Functions

Memory allocation functions _malloc_dbg: C style Memory Allocation

Handles calls to malloc & new in Debug builds new: C++ style memory allocation

Calls to new are passed to _nh_malloc_dbg defined in DBGHEAP.C

Debugging support offered: (example block layout) Buffers on either side of user portion of block File name and line number of the allocations Allocation Type information : used for leak detection and

state reporting

Memory Block Allocation Types _NORMAL_BLOCK: The Default block

allocated by call to malloc/new. Only keeps track of the file and line information. Raw memory dump

_CLIENT_BLOCK: Can be customized by the application to keep track of additional information that may be output when memory is dumped

Memory Block Allocation Types _CRT_BLOCK: Used for internal CRT use _FREE_BLOCK: These blocks are only

marked as free and not removed from the allocation list when de-allocated Filled with 0xDD

_IGNORE_BLOCK When The DCRT heap is turned off in the middle the program, the memory allocated is stored in allocation as these blocks

Memory Integrity Check functions Memory integrity _CrtCheckMemory():

Assert(_CrtCheckMemory()) tracks down underwrites and overwrites as close as they occur, also checks that freed pointers are not used with out being reinitialized.

When the _CRTDBG_CHECK_ALWAYS_DF is set, this Function is called after every memory allocation / de-allocation.

Code Example

Parameter Validation Functions Validity of Memory: can be used as

debugging parameter validation functions _CrtIsValidHeapPointer _CrtIsMemoryBlock _CrtIsValidPointer

Memory State Validation Functions Memory State Functions: Keep track of

Memory state, Can use these to track memory leaks or to track memory consumption _CrtMemCheckpoint _CrtMemDifference _CrtMemDumpStatistics(Code Example)

DCRT Hooks Hook into the memory allocation code stream

to see each allocation call (Code Example) Can’t do the same for deallocations though…

Hook memory dump functions and use them to enumerate client blocks (memory that the app allocates) (Code Example) Can generate more useful memory dump

messages Can have subtypes of a client block for more

dumping granularity

DCRT Debug Report Functions Used to control the DCRT reporting subsystem Can redirect the Debug heap Report output to a

file or a the debug window or a message box _CrtSetReportMode _CrtSetReportFile _CrtSetReportHook _CrtSetReportHook2(Code Example)

DCRT library versions Two major categories:

Debug and Release In each category there are three types:

Single Threaded static library Multithreaded static library Multithreaded DLL

In total, six versions of the C runtime are available…

An Important Note… Use the same version of the C runtime library

across all the modules of your application Else … can end up allocating memory using one

version of the library and deallocating it using another version

“Different versions of the DCRT running at the same time have multiple versions of the heap memory tracking the same code” - Robbins

Memory Management

References:

1. Robbins’ Book

2. Platform SDK

Overview Windows Processes Process Partitions Process Heaps

Windows Processes… 32 bit processes => Virtual address space

(VAS) of 4GB (8TB on 64 bit Windows) Addressable by all the processes’ threads Page Map translates all the processes’ Virtual

Addresses (VAs) to Physical Addresses (PAs) Maintained by Windows, accessed each time a

thread makes a memory reference

32 bit VAS Partitions Default 4 GB of process space:

2 GB application partition in low memory (0x00000000 thro’ 0x7FFFFFFF)

2GB system partition (0x80000000 thro’ 0xFFFFFFFF) Can change this allocation to 3GB application

partition and 1 GB system partition Add the /3GB switch to Boot.ini file For I/O intensive applications (e.g. DBMS) running on

server systems

Process Heaps Block of one or more pages in the VAS of the

process Reside in the Application Partition Created by the processes themselves Use a different set of functions (windows API

functions) to manage this memory

Useful VS.NET Compiler Switches

References:

1. Robbins’ Book

2. VS.NET Online Help System

Overview Run Time Check Switches (/RTCx) Buffer Security Switch /GS

Run Time Check Switches (/RTCx) /RTCc: Enable data value truncation checks

(e.g. a 16 bit value assigned to a 8 bit value) Need to select the “Smaller Type Check” option

/RTCs: Enable stack frame runtime checking All local variables initialized to OxCC (breakpoint

instruction in the x86) Stack Frame corruption: mismatched calling

conventions (calling a _stdcall function with a function pointer having a _cdecl convention).

Run Time Check Switches /RTCu: Check for uninitialized variables and

their usage. Can avoid this by compiling this with /W4 (warning

level 4) and /Wx treat all warnings as errors

/RTC1: Shorthand for combining /RTCs and /RTCu

Note: The /RTCx switches will not allow any optimizations. These switches are only valid in a debug build.

Run Time Check Switches

Buffer Security Check Switch /GS Must be turned on for both Debug and Release

builds Ensures that the function return address is not

modified. A security cookie XORed with the return address is stored

before the return address This value is again recomputed at function exit and

checked against the original value. If there is a change an error box pops up… (this behavior can be overridden)

Buffer Security Check Switch Implemented in the Microsoft C runtime files:

SECINIT.C SECHOOK.C SECFAIL.C

Provided to allow user to override buffer overrun response…

Customizing Buffer Overrun Response

Default behavior of Buffer overrun A message box is displayed and the program

terminates Can override the above behavior

Hook a user defined processing function to _set_security_error_handler

(Code Example)

Done at last…WHEW!!!

Man…

DCRT is really neat !!!

The End

Questions?