examine small memory dump files

31
Examine Small memory dump files A small memory dump file records the smallest set of useful information that may help identify why your computer has stopped unexpectedly. This option requires a paging file of at least 2 megabytes (MB) on the boot volume. On computers that are running Microsoft Windows 2000 or later, Windows create a new file every time your computer stops unexpectedly. A history of these files is stored in a folder. This dump file type includes the following information: The Stop message and its parameters and other data A list of loaded drivers The processor context (PRCB) for the processor that stopped The process information and kernel context (EPROCESS) for the process that stopped The process information and kernel context (ETHREAD) for the thread that stopped The Kernel-mode call stack for the thread that stopped The small memory dump file can be useful when hard disk space is limited. However, because of the limited information that is included, errors that were not directly caused by the thread that was running at the time of the problem may not be discovered by an analysis of this file. If a second problem occurs and if Windows creates a second small memory dump file, Windows preserves the previous file. Windows gives each file a distinct, date-encoded file name. For example, Mini022900-01.dmp is the first memory dump file that was generated on February 29, 2000. Windows keeps a list of all the small memory dump files in the %SystemRoot%\ Minidump folder. Configure the dump type To configure startup and recovery options to use the small memory dump file, follow these steps.

Upload: rachitlohani4231

Post on 07-Apr-2015

142 views

Category:

Documents


2 download

TRANSCRIPT

Page 1: Examine Small Memory Dump Files

Examine Small memory dump files

A small memory dump file records the smallest set of useful information that may help identify why your computer has stopped unexpectedly. This option requires a paging file of at least 2 megabytes (MB) on the boot volume. On computers that are running Microsoft Windows 2000 or later, Windows create a new file every time your computer stops unexpectedly. A history of these files is stored in a folder.

This dump file type includes the following information: The Stop message and its parameters and other data A list of loaded drivers The processor context (PRCB) for the processor that stopped The process information and kernel context (EPROCESS) for the process that

stopped The process information and kernel context (ETHREAD) for the thread that

stopped The Kernel-mode call stack for the thread that stopped

The small memory dump file can be useful when hard disk space is limited. However, because of the limited information that is included, errors that were not directly caused by the thread that was running at the time of the problem may not be discovered by an analysis of this file.

If a second problem occurs and if Windows creates a second small memory dump file, Windows preserves the previous file. Windows gives each file a distinct, date-encoded file name. For example, Mini022900-01.dmp is the first memory dump file that was generated on February 29, 2000. Windows keeps a list of all the small memory dump files in the %SystemRoot%\Minidump folder.

Configure the dump typeTo configure startup and recovery options to use the small memory dump file, follow these steps.

Note Because there are several versions of Microsoft Windows, the following steps may be different on your computer. If they are, see your product documentation to complete these steps.

1. Click Start, point to Settings, and then click Control Panel. 2. Double-click System. 3. Click the Advanced tab, and then click Settings under Startup and Recovery.4. In the Write debugging information list, click Small memory dump (64k).

To change the folder location for the small memory dump files, type a new path in the Dump File box (or in the Small dump directory box, depending on your version of Windows).

Tools to read the small memory dump file

Page 2: Examine Small Memory Dump Files

You can load small memory dump files by using the Dump Check Utility (Dumpchk.exe). You can also use Dumpchk.exe to verify that a memory dump file has been created correctly. The Dump Check Utility does not require access to debugging symbols. The Dump Check Utility is included with the Microsoft Windows 2000 Support Tools and the Microsoft Windows XP Support Tools.

For additional information about how to use the Dump Check Utility in Windows 2000 and in Windows NT, click the following article number to view the article in the Microsoft Knowledge Base: 156280  (http://support.microsoft.com/kb/156280/ ) How to use Dumpchk.exe to check a memory dump file For additional information about how to use the Dump Check Utility in Windows XP, click the following article number to view the article in the Microsoft Knowledge Base: 315271  (http://support.microsoft.com/kb/315271/ ) How to use Dumpchk.exe to check a memory dump file Note The Dump Check Utility is not included in the Microsoft Windows Server 2003 Support Tools. To obtain the Dump Check Utility if you are using Microsoft Windows Server 2003, download and install the Debugging Tools for Windows package from the following Microsoft Web site: http://www.microsoft.com/whdc/devtools/debugging/default.mspx (http://www.microsoft.com/whdc/devtools/debugging/default.mspx) You can also read small memory dump files by using the WinDbg tool or the KD.exe tool. WinDbg and KD.exe are included with the latest version of the Debugging Tools for Windows package.This Web page also provides access to the downloadable symbol packages for Windows. To use the resources, create a folder on the disk drive where the downloaded local symbols or the symbol cache for symbol server use will reside. For example, use C:\Symbols. You can use the following symbol path with all the commands that are described in this article: SRV*c:\symbols*http://msdl.microsoft.com/download/symbolsIf you download the symbols to a local folder, use the path of that folder as your symbol path.

For more information about the dump file options in Windows, click the following article number to view the article in the Microsoft Knowledge Base: 254649  (http://support.microsoft.com/kb/254649/ ) Overview of memory dump file options for Windows Server 2003, Windows XP, and Windows 2000

Back to the top

Install the debugging tools

To download and install the Windows debugging tools, visit the following Microsoft Web site: http://www.microsoft.com/whdc/devtools/debugging/default.mspx (http://www.microsoft.com/whdc/devtools/debugging/default.mspx)

Page 3: Examine Small Memory Dump Files

Select the Typical installation. By default, the installer installs the debugging tools in the following folder: C:\Program Files\Debugging Tools for Windows

Open the dump file

To open the dump file after the installation is complete, follow these steps:1. Click Start, click Run, type cmd, and then click OK.2. Change to the Debugging Tools for Windows folder. To do this, type the

following at the command prompt, and then press ENTER:

cd c:\program files\debugging tools for windows

3. To load the dump file into a debugger, type one of the following commands, and then press ENTER:

windbg -y SymbolPath -i ImagePath -z DumpFilePath

kd -y SymbolPath -i ImagePath -z DumpFilePath

The following table explains the use of the placeholders that are used in these commands.Collapse this tableExpand this tablePlaceholder Explanation

SymbolPath

Either the local path where the symbol files have been downloaded or the symbol server path, including a cache folder. Because a small memory dump file contains limited information, the actual binary files must be loaded together with the symbols for the dump file to be correctly read.

ImagePathThe path of these files. The files are contained in the I386 folder on the Windows XP CD-ROM. For example, the path may be C:\Windows\I386.

DumpFilePath The path and file name for the dump file that you are examining.

Sample Commands

You can use the following sample commands to open the dump file. These commands assume the following:

The contents of the I386 folder on the Windows CD-ROM are copied to the C:\Windows\I386 folder.

Your dump file is named C:\Windows\Minidump\Minidump.dmp.

Sample 1: kd -y srv*c:\symbols*http://msdl.microsoft.com/download/symbols -i c:\windows\i386 -z c:\windows\minidump\minidump.dmpSample 2. If you prefer the graphical version of the debugger instead of the command line version, type the following command instead: windbg -y srv*c:\symbols*http://msdl.microsoft.com/download/symbols -i c:\windows\i386 -z c:\windows\minidump\minidump.dmp

Page 4: Examine Small Memory Dump Files

Back to the top

Examine the dump file

There are several commands that you can use to gather information in the dump file, including the following commands:

The !analyze -show command displays the Stop error code and its parameters. The Stop error code is also known as the bug check code.

The !analyze -v command displays verbose output. The lm N T command lists the specified loaded modules. The output includes the

status and the path of the module.

Note The !drivers extension command displays a list of all drivers that are loaded on the destination computer, together with summary information about their memory use. The !drivers extension is obsolete in Windows XP and later. To display information about loaded drivers and other modules, use the lm command. The lm N T command displays information in a format that is similar to the old !drivers extension.

For help with other commands and for complete command syntax, see the debugging tools Help documentation. The debugging tools Help documentation can be found in the following location: C:\Program Files\Debugging Tools for Windows\Debugger.chmNote If you have symbol-related issues, use the Symchk utility to verify that the correct symbols are loaded correctly. For additional information about using Symchk, click the following article number to view the article in the Microsoft Knowledge Base: 311503  (http://support.microsoft.com/kb/311503/ ) Use the Microsoft Symbol Server to obtain debug symbol files

Simplify the commands by using a batch file

After you identify the command that you must have to load memory dumps, you can create a batch file to examine a dump file. For example, create a batch file and name it Dump.bat. Save it in the folder where the debugging tools are installed. Type the following text in the batch file: cd "c:\program files\debugging tools for windows"

kd -y srv*c:\symbols*http://msdl.microsoft.com/download/symbols -i c:\windows\i386 -z %1When you want to examine a dump file, type the following command to pass the dump file path to the batch file: dump c:\windows\minidump\minidump.dmp

System requirements

ADPlus has the following minimum requirements: Windows NT 4.0 Service Pack 4 or later

Page 5: Examine Small Memory Dump Files

Windows Scripting Host version 5.6 or later At least 10 megabytes (MB) of free space on the disk or network share where

ADPlus will put the output files Microsoft Debugging Tools for Windows installed

New features for ADPlus Version 6.0

ADPlus V6.0 has been completely rewritten. The tool has new switches and new capabilities. You can now configure the tool through an external configuration file. You can view updated information about the new features and switches in the debugger help file (Debugger.chm) that is included in the Microsoft Windows Debuggers package. To obtain the package, visit the following Microsoft Web site:http://www.microsoft.com/whdc/devtools/debugging/default.mspx (http://www.microsoft.com/whdc/devtools/debugging/default.mspx) Debugger.chm is located in the same folder as ADPlus.vbs. To locate the documentation for ADPlus, click the Contents tab, and then click through the following items:

Using Debugging Tools for Windows Crash Dump Files User-Mode Dump Files Creating a User Mode Dump File ADPlus

You can also find documentation for ADPlus by clicking the Index tab. Type ADPlus in the keyword text box.

What does ADPlus do?

ADPlus is console-based Microsoft Visual Basic script. It automates the Microsoft CDB debugger to produce memory dumps and log files that contain debug output from one or more processes. Each time that you run ADPlus, debugging information (memory dumps and text files that contain debug information) is put in a new, uniquely named folder (such as C:\Temp\Crash_Mode__Date_01-22-2001__Time_09-41-08AM) on the local file system or on a remote network share. Additionally, each file that ADPlus creates has a unique name (such as PID-1708__Inetinfo.exe__Date_01-22-2001__Time_09-41-08AM.log) to avoid overwriting older files with newer ones.

ADPlus works with any user mode process or service such as Internet Information Services (IIS), Microsoft Transaction Server (MTS), or Microsoft COM+ applications.

The following are some of the features of ADPlus: ADPlus uses the latest Microsoft debuggers for improved features, speed, and

reliability. When ADPlus is dumping memory for multiple processes, it does so

asynchronously so that each process is frozen and dumped at the same time. This method can provide an effective "snapshot" of the whole application at the time that ADPlus was run. You must capture all the processes that compose an

Page 6: Examine Small Memory Dump Files

application, and all the processes that the application uses at the same time, to capture the state of the application at the time that the problem occurs. This is especially important for applications that make remote procedure calls to other processes.

ADPlus has a command-line interface. Because ADPlus does not have a graphical user interface, you can run it in quiet mode (to suppress dialog boxes) from a remote command shell (a command shell that is remoted out by using Remote.exe). In quiet mode, errors appear in the console and are written to the event log. For more information about how to run ADPlus from a remote command shell, see the "Usage Scenarios" section of this article.

If you use the -notify switch when ADPlus monitors for crashes, and the Windows Messenger service is started, ADPlus can alert a user or computer of a crash through the Windows Messenger service.

When ADPlus monitors a process in crash mode, if a crash occurs, ADPlus sends important information about the type of crash to the event log.

ADPlus supports XCOPY deployment. If you install the debuggers package that is included with ADPlus on a test computer, you can copy the folder where the debuggers were installed to another computer. Additionally, ADPlus does not require that you register any custom Component Object Model (COM) components on the system. Because of this, you can use ADPlus on production servers that have a locked-down software configuration. To remove ADPlus, delete the folder where it was installed or copied to.

When should you use ADPlus?

ADPlus is intended to provide Microsoft PSS support professionals with debugging information that they must have to isolate the cause of problems that occur in complex environments.Use ADPlus to capture debugging information if you are experiencing the following problems:

Processes that stop responding. Processes that consume 100 percent CPU on a single processor computer, 50

percent CPU on a dual processor computer, 25 percent CPU on a quad processor computer, and so on.

Processes that crash or shut down unexpectedly.

When should you not use ADPlus?

Do not use ADPlus in the following situations: If you must troubleshoot a program or process that quits unexpectedly during

startup. You can only use ADPlus for processes that start successfully. To troubleshoot processes that quit unexpectedly during startup, User Mode Process Dump may be a better solution. For more information about User Mode Process Dump, click the following article number to view the article in the Microsoft Knowledge Base:

Page 7: Examine Small Memory Dump Files

253066  (http://support.microsoft.com/kb/253066/ ) OEM Support Tools Phase 3 Service Release 2 availability

Alternatively, you can use the latest debuggers to manually debug the process. For more information about the latest debuggers, visit the following Microsoft Web site:

http://www.microsoft.com/whdc/devtools/debugging/default.mspx (http://www.microsoft.com/whdc/devtools/debugging/default.mspx)

If there is a noticeable effect on performance when you use ADPlus in crash mode. Typically, this is caused by dynamic-link libraries (DLLs) or programs that throw many Microsoft Visual C++ EH exceptions. (These exceptions occur when you use the C++ throw statement or when you use try/catch blocks.) Programs that write a lot of information to the debug output stream can also cause performance to decrease. In the vast majority of cases, ADPlus does not affect performance noticeably when it is running in crash mode.

If you are running in a clustering environment certain precautions should be taken when you use ADPlus. For more information, click the following article number to view the article in the Microsoft Knowledge Base:

841673  (http://support.microsoft.com/kb/841673/ ) A server in a cluster may fail over when you try to create a dump file of the information store using ADPlus or Userdump in Exchange 2000 Server or Exchange Server 2003

Where do you obtain ADPlus?

ADPlus is included with the latest Microsoft Debugging Tools for Windows. To obtain the latest Microsoft Debugging Tools for Windows, visit the following Microsoft Web site:

How does ADPlus work?

ADPlus has two modes of operation: "Hang" mode is used to troubleshoot process hangs, 100 percent CPU utilization,

and other problems that do not involve a crash. When you use ADPlus in hang mode, you must wait until the process or processes stop responding before you run the script (unlike crash mode, hang mode is not persistent).

"Crash" mode is used to troubleshoot crashes that result in Dr. Watson errors, or any other type of error that causes a program or service to quit unexpectedly. When you use ADPlus in crash mode, you must start ADPlus before the crash occurs. You can configure ADPlus to notify an administrator or a computer of a crash through the -notify switch.

Hang mode

Page 8: Examine Small Memory Dump Files

In this mode, ADPlus immediately produces full memory dumps for all the processes that are specified on the command line after the script has completed. Each .dmp file that is created is put in a folder that contains the date/time stamp when ADPlus was run. Each file name contains the process name, the process ID, and the date/time stamp when ADPlus was run. While the process memory is being dumped to a file, the process is frozen. After the memory dump file has been created, the process is resumed by using a noninvasive attach/detach with the CDB debugger.

Usage Tip You can use ADPlus in hang mode instead of Userdump.exe to dump the memory for one or more processes. Additionally, hang mode works inside a Terminal Server session.

Crash mode

In this mode, ADPlus attaches the CDB debugger to all processes that are specified on the command line. ADPlus automatically configures the debugger to monitor for the following types of exceptions:

Invalid Handle Illegal Instruction Integer Divide by Zero Floating Point Divide by Zero Integer Overflow Invalid Lock Sequence Access Violation Stack Overflow C++ EH Exception Unknown Exception

You can use ADPlus in crash mode instead of the IIS Exception Monitor or Userdump.exe when you are troubleshooting these types of exceptions. Because crash mode uses an "invasive" attach through the CDB debugger, it does not work inside a Microsoft Windows NT 4.0 or Windows 2000 Terminal Server session. Only hang mode works inside a Terminal Server session on these operating systems because they require the use of a noninvasive attach. For more information about how to invasively and noninvasive attach to a process with the latest debuggers, see the "Using Debugging Tools for Windows: Attaching to a Running Process (User Mode)" section in the debuggers help.

Note Crash mode is supported in a Terminal Server session on Windows XP and Microsoft Windows Server 2003 operating systems.

When ADPlus is running in crash mode, a debugger remains attached to each process that is specified on the command line for the lifetime of that process until a fatal exception is trapped and the process quits unexpectedly, or until a user presses the CTRL+C key

Page 9: Examine Small Memory Dump Files

combination to detach the debugger from that process. To manually detach the debugger from the process, you must maximize the debugger window, and then press CTRL+C to break into the debugger.

When you press CTRL+C, ADPlus traps this command, starts to list the stacks for all threads to a log file, and then produces a mini memory dump record of the process before it detaches from the debugger. Because crash mode performs an invasive attach, the process is stopped when the debugger is detached. You must restart the process. If it is an MTS or COM+ process, the process is restarted automatically the next time that a call is made to a component in that package.

First chance exceptions

Each type of exception (such as an access violation or a stack overflow) can be raised to a debugger as either a first chance exception or a second chance exception. By definition, a first chance exception is non-fatal unless it is not handled correctly by using an error handler. If this problem occurs, the exception is raised again as a second chance exception (only a debugger can handle these). If no debugger handles a second chance exception, the application quits.

For more information about first and second chance exceptions and the Windows NT SEH (structured exception handling), click the following article number to view the article in the Microsoft Knowledge Base: 105675  (http://support.microsoft.com/kb/105675/ ) First and second chance exception handling By default, when ADPlus detects a first chance (non-fatal) exception for all types of exceptions except unknown and EH exceptions, it takes the following actions:

1. Pauses the process to log the date and time that the exception occurred in the log file for the process that is being monitored.

2. Logs the thread ID and call stack for the thread that raised the exception in the log file for the process that is being monitored.

3. Produces a uniquely named mini memory dump record (.dump -u /m) of the process at the time that the exception occurred, and then resumes the process.

Note By default, ADPlus does not produce a unique mini memory dump record for first chance EH and unknown exceptions because these exceptions occur frequently. Typically, such exceptions are handled by error handling code in a process or DLL. Because these are handled exceptions, they do not become second chance (unhandled) exceptions and they do not end the process.

However, you can configure ADPlus to produce unique mini memory dumps for first chance EH and unknown exceptions. To do this, you must use a configuration file to customize ADPlus.

Second chance exceptions

Page 10: Examine Small Memory Dump Files

When ADPlus detects a second chance (fatal) exception for all types of exceptions (including EH and unknown exceptions), it takes the following actions:

1. Pauses the process to log the date and time that the exception occurred in the log file for the process that is being monitored.

2. Logs the thread ID and call stack for the thread that raised the exception in the log file for the process that is being monitored.

3. Produces a full memory dump of the process at the time that the fatal exception occurred, and then exits the debugger. This action destroys the process.

Note For Microsoft PSS support professionals to analyze memory dumps, they may have to obtain copies of any custom components or DLLs and their corresponding symbol files. For more information about how to create symbol files for your DLLs, click the following article numbers to view the articles in the Microsoft Knowledge Base: 121366  (http://support.microsoft.com/kb/121366/ ) PDB and DBG files - what they are and how they work 291585  (http://support.microsoft.com/kb/291585/ ) How to create debug symbols for a Visual C++ application For more information about how to obtain symbols for Microsoft products (necessary for analyzing memory dumps with the debuggers), visit the following Microsoft Web site:http://www.microsoft.com/whdc/DevTools/Debugging/symbolpkg.mspx (http://www.microsoft.com/whdc/DevTools/Debugging/symbolpkg.mspx)

ADPlus command line switches

To use ADPlus, you must specify a series of command line switches or arguments to the script. At a minimum, ADPlus requires two switches: one that specifies the mode of operation, and one that specifies a target process to operate against.

The following are the most frequently used switches. You can also view the complete list of switches by running ADPlus –help, or by viewing the debuggers help file (Debugger.chm).

-hangThis switch configures ADPlus to run in hang mode. You must use this switch with the -iis, -pn, or -p switches. You cannot use -hang with the -crash switch.

Note When ADPlus is running in hang mode, you must start ADPlus after the process stops responding or is consuming a high percentage of the CPU.

-crashThis switch configures ADPlus to run in crash mode. You must use this switch with the -iis, -pn, or -p switches. You cannot use -crash with the -hang switch.

Note When ADPlus is running in crash mode, you must start ADPlus before the process quits unexpectedlys or becomes unstable.

-pn process nameThe -pn switch is used to specify a process name that you want ADPlus to

Page 11: Examine Small Memory Dump Files

analyze. To specify more than one process, use multiple -pn process name switches. For example:

-pn process1.exe -pn process2.exe

-p process IDThe -p switch is used to specify the process ID (PID) of a process that you want ADPlus to analyze. To specify more than one process, use multiple -p PID switches. For example:

-p 1896 -p 1702

-scspawning commandUnlike the -pn and -p switches, which specifiy processes that are already running to attach the debugger to, the -sc switch defines the application and parameters to be started (or spawned) in the debugger. For example:

-sc "c:\windows\system32\notepad.exe

-iisThe -iis switch is used for debugging server computers that are running Internet Information Server (IIS) 4.0 or later. When you use ADPlus with the -iis switch, ADPlus monitors all the IIS in-process (Inetinfo.exe) and out-of-process (Mtx.exe/Dllhost.exe) applications. You can use the -iis switch with the -pn switch or the -p switch, or you can use it alone to analyze IIS and all running MTS/COM+ applications in either crash mode or hang mode.

If you are trying to analyze a server computer running IIS 3.0 or earlier, use the -pn switch and specify Inetinfo.exe as the process to monitor.

-notify computer name or user nameThis switch is only valid when ADPlus is running in crash mode. This switch instructs ADPlus to alert the specified user name or computer name of a crash. When the debugger detaches from the process because of a second chance exception, or when a user presses CTRL+C to stop debugging, a notification is sent to the remote user or computer through the local messenger service. This notification occurs only if the local messenger service is started on the computer that is being debugged.

-quietThis switch instructs ADPlus to suppress all modal dialog boxes. This switch is useful if you are running ADPlus from a remote command shell where modal dialog boxes can cause ADPlus to wait indefinitely for a user to click OK. For best results, make sure that this is the first switch that is passed to ADPlus.vbs.

-o output directoryThis switch instructs ADPlus where to put the debug output files. If you use long file names, you must enclose them in double quotation marks. Additionally, you can use a UNC path (\\server\share). If you use a UNC path, ADPlus creates a

Page 12: Examine Small Memory Dump Files

new folder immediately below the UNC path that you specified. The folder is named for the server where ADPlus is running (for example, \\server\share\Web1 or \\server\share\Web2). This switch is useful if ADPlus is running on multiple computers in a Web farm that are all putting their output on the same network share.

Run ADPlus for the first time

By default, debuggers install to the C:\Program Files\Debugging Tools for Windows folder. To change the installation folder, do a custom install when you install the debuggers, and specify a different folder. Alternatively, if a typical installation was performed, copy the contents of the Program Files\Debugging Tools for Windows folder to a different folder.

To run ADPlus, open a command shell, switch to the folder where the debuggers were installed or copied, and then type ADPlus.vbs.

You may be prompted to change your default script interpreter from Wscript.exe to Cscript.exe. Microsoft strongly recommends that you allow ADPlus to configure CSCript as the default script interpreter.

Syntax

ADPlus uses the following syntax: ADPlus.vbs mode of operation processes to monitor optional switches where mode of operation is -hang, or -crashwhere processes to monitor is -iis, -pn process.exe, or -p PIDwhere optional switches is -notify, -o, or -quiet.

Prepare the server for crash mode debugging

Before you run ADPlus in crash mode, you must prepare the server to obtain the most information from the ADPlus crash mode debugging sessions.

Steps to prepare a Windows 2000-based server for debugging in crash mode

1. Install the Windows 2000 SP1 or SP2 symbols to the C:\WINNT\Symbols folder on your servers. You can download the symbols from the following Microsoft Web sites:

Windows 2000 SP1http://download.microsoft.com/download/win2000platform/SP/SP1/NT5/EN-US/SP1SYM.exe (http://download.microsoft.com/download/win2000platform/sp/sp1/nt5/en-us/sp1sym.exe)

Windows 2000 SP2

Page 13: Examine Small Memory Dump Files

http://download.microsoft.com/download/win2000platform/SP/SP2/NT5/EN-US/SP2SYM.exe (http://download.microsoft.com/download/win2000platform/sp/sp2/nt5/en-us/sp2sym.exe)

After you download Sp1sym.exe or Sp2sym.exe, run the file from the designated folder.

2. When you are prompted, extract the files to a new temporary folder, such as C:\Sp1sym or C:\Sp2sym, or to a drive or folder that has sufficient disk space.

3. Run C:\Sp1sym\Support\Debug\Symbols\i386\Symbols_spexe or C:\Sp2sym\Support\Debug\Symbols\i386\Symbols_spexe (where C:\Sp1sym or C:\Sp2sym is the folder where you extracted the files in the previous step).

4. When you prompted with the EULA, click Yes.5. When you are prompted for a folder where you can extract the files, click C:\

WINNT\Symbols, and then click OK. Notice that a new C:\WINNT\Symbols folder appears. This folder contains various subfolders that have names such as DLL and EXE.

6. Copy the symbols for your custom DLLs and any post SP1 or SP2 hotfixes to the C:\WINNT\Symbols\Dll folder.

7. Copy the symbols for your custom .exe files to the C:\WINNT\Symbols\Exe folder. Additionally, you must obtain any .pdb or .dbg files from your developers, and then put these files in the C:\WINNT\Symbols\Dll folder.

8. Overwrite any .dbg or .pdb files that already exist in the C:\WINNT\Symbols\Dll folder with versions from your hotfixes.

Note You can use the latest version of Winzip to open hotfix packages. You can extract the symbols from the \Debug subfolder. The \Debug subfolder is contained in each hotfix self-installer.

9. Create an _NT_SYMBOL_PATH environment variable, and then set it equal to C:\WINNT\Symbols. This variable can be either a system variable or a user environment variable.

Steps to prepare a Windows NT 4.0-based server for debugging in crash mode

1. Assume that you are running Windows NT 4.0 Service Pack 6a. Install the Windows NT 4.0 SP6a symbols to the C:\WINNT\Symbols folder on your servers. For more information about Windows NT 4.0 Service Pack 6/6a, click the following article number to view the article in the Microsoft Knowledge Base:

241211  (http://support.microsoft.com/kb/241211/ ) List of bugs fixed in Windows NT 4.0 Service Pack 6/6a (Part 1)

After you download Sp6symi.exe, run it from the designated folder.

Page 14: Examine Small Memory Dump Files

2. When you are prompted, extract the files to the C:\WINNT folder (or substitute the appropriate \WINNT folder if the symbols were not installed to C:\WINNT). Notice that a new C:\WINNT\Symbols folder appears that has various subfolders named DLL, EXE, and others.

3. Copy the subfolders from the C:\WINNT\Symbols\IIS4 folder to C:\WINNT\Symbols. When you are prompted to overwrite all the files, click Yes.

4. Copy the symbols for your custom DLLs and any post SP6a hotfixes to the C:\WINNT\Symbols\Dll folder.

5. Copy the symbols for your custom .exe files to the C:\WINNT\Symbols\Exe folder. Additionally, you must obtain any .pdb or .dbg files from your developers, and then put these files in the C:\WINNT\Symbols\Dll folder.

6. Overwrite any .dbg or .pdb files that already exist in the C:\WINNT\Symbols\Dll folder with the versions from your hotfixes.

Note You can use the latest version of Winzip to open hotfix packages. You can extract the symbols from the \Debug subfolder. This subfolder is included in each hotfix self-installer.

7. Create an _NT_SYMBOL_PATH environment variable, and then set it equal to C:\WINNT\Symbols. This variable can be either a system variable or a user environment variable.

Although you do not have to download and install symbols on the servers that you are debugging, it is highly recommended. When you download and install symbols on the server, the output that the log files capture is much more useful to Microsoft PSS.

For more information about how to obtain Microsoft Debug Symbols, click the following article number to view the article in the Microsoft Knowledge Base: 268343  (http://support.microsoft.com/kb/268343/ ) Umdhtools.exe: How to use Umdh.exe to find memory leaks After you configure your servers, you can run ADPlus in crash mode. This mode is described in the "Typical ADPlus Usage Scenarios" section.

Typical ADPlus usage scenarios

This section describes some of the typical scenarios where you may have to run ADPlus.

Process stops responding or consumes 100 percent CPU utilization

In this scenario, a process may randomly consume 100 percent CPU for sustained periods or indefinitely. Run ADPlus in hang mode to obtain a memory dump of the process or processes that are consuming the CPU after the problem occurs. For example, use one of the following command syntaxes: ADPlus -hang -p 1896This command runs ADPlus in hang mode and produces a full memory dump file of a

Page 15: Examine Small Memory Dump Files

process that has the PID 1896.

ADPlus -hang -pn myapp.exeThis command runs ADPlus in hang mode and produces full memory dump files of all processes that are named Myapp.exe.

ADPlus -hang -iis -pn myapp.exe -o c:\tempThis command runs ADPlus in hang mode and produces full memory dump files of IIS, all instances of Mtx.exe or Dllhost.exe, and all processes that are named Myapp.exe. It then puts the memory dump files in the C:\Temp folder. When you run ADPlus in hang mode during the 100 percent CPU condition, the tool produces memory dump files of the process or processes that you specify on the command line.

Note In certain rare situations, the debugger may not be able to attach to the process after the 100 percent CPU condition or hang has occurred. If you run ADPlus in hang mode after the problem has occurred, the tool may not produce memory dump files. In these scenarios, it may be best to attach the debugger before the problem has occurred. To do this, use one of the following command syntaxes to run ADPlus in crash mode: ADPlus -crash -p 1896This command runs ADPlus in crash mode for a process that has the PID 1896. ADPlus waits for an exception to occur, or for a user to press CTRL+C in the minimized debugger window, to generate a memory dump file and to detach the debugger.

ADPlus -crash -pn myapp.exeThis command runs ADPlus in crash mode for the process named Myapp.exe. ADPlus waits for an exception to occur, or for a user to press CTRL+C in the minimized debugger window, to generate a memory dump file and to detach the debugger.

ADPlus -crash -iis -pn myapp.exe -o c:\tempThis command runs ADPlus in crash mode for all instances of the processes named Myapp.exe and Inetinfo.exe, and for all instances of Mtx.exe or Dllhost.exe. ADPlus waits for an exception to occur, or for a user to press CTRL+C in one or more of the minimized debugger windows, to generate the memory dump file (or files) and to detach the debugger (or debuggers). ADPlus puts the memory dump files and the log files in the C:\Temp folder.Then, after the process hangs or is consuming 100 percent CPU utilization, the user can press CTRL+C in the minimized debugger window (or windows) that ADPlus generates so that the debugger can generate a memory dump file for the process (or processes).

Note By default, ADPlus only produces mini memory dump records when the user presses CTRL+C. This setting conserves disk space. In this scenario, it may be useful to configure ADPlus to generate a full memory dump file when the user presses CTRL+C. To do this, use the –CTCF switch. Additionally, it is frequently helpful to capture a performance log file or a system monitor log file for the time period up to and including

Page 16: Examine Small Memory Dump Files

the 100 percent CPU utilization condition. At a minimum, this log file should capture the following objects at 1 to 5 second intervals:

Memory Process Processor System Thread

Process quits unexpectedly

In this scenario, a process may randomly quit (or crash) unexpectedly. Run ADPlus in crash mode to obtain a memory dump file of the process or processes that quit before the problem occurs. For example, use one of the following command syntaxes: ADPlus -crash -iisThis command runs ADPlus in crash mode and causes it to attach the CDB debugger to Inetinfo.exe and to all Mtx.exe or Dllhost.exe processes that are running on the computer. ADPlus then waits for any first chance and second chance exceptions to occur. By default, ADPlus puts all files in a subfolder of the installation folder because the -o switch is omitted.

ADPlus -quiet -crash -iis -notify remote computer -o c:\tempThis command runs ADPlus quietly (no dialog boxes, log all output to the event log) in crash mode and causes it to attach the CDB debugger to Inetinfo.exe and to all Mtx.exe or Dllhost.exe processes that are running on the computer. Because the -notify switch is used, the debuggers notify all users who are logged on to the computer named remote computer whenever a crash is detected or when the process that is being monitored quits. Because the -o switch is used, ADPlus puts all output in the C:\Temp folder. If the folder does not exist, ADPlus creates it.

ADPlus -crash -iis -o \\server\shareThis command is the same as the previous command except that it logs all output to a network server. ADPlus creates a new subfolder in \\server\share and names the subfolder for the local computer. Therefore, if you are running ADPlus in a Web farm, each server in the farm that has ADPlus running logs its own unique folder under \\server\share. (You do not have to create unique folders for each server. ADPlus does this automatically.) Note If you are running ADPlus in crash mode from the local console (instead of from a remote command shell as described in the next section), you must remain logged on to the console for the duration of the debug session.

For example, assume that you start ADPlus in crash mode and you use the -iis switch to monitor IIS. When you log out of the console, the copies of Cdb.exe that are running on the console (and all other running applications) quit . As a result, debugging stops, and the process that is being monitored is ended.

To avoid this issue, you can lock the console session (press the CTRL+ALT+DEL key

Page 17: Examine Small Memory Dump Files

combination, and then click Lock Computer) or run ADPlus from a remote command shell that you have scheduled to run non-interactively (that is, it does not require an interactive logon).

For more information about how to schedule a remote command shell to run non-interactively, see the "Typical ADPlus Usage Scenarios: Run in Crash Mode Remotely" section.

MTS or COM+ server application quits unexpectedly

Custom Component Object Model (COM) components that run in an MTS or COM+ server application actually run in a surrogate process (Mtx.exe or Dllhost.exe). These surrogate processes have properties and settings that you can configure through the MTS Explorer (for Windows NT 4.0) or through the Component Services Microsoft Management Console (MMC) snap-in (for Windows 2000, Windows XP, and Windows Server 2003).

By default, MTS or COM+ server applications are configured to quit after three minutes of idle time. To make sure that these processes remain running while the debugger is attached and monitoring for exceptions, you must configure them to Leave running when idle.

Additionally, MTS and COM+ implement a failfast. A failfast is a safeguard that is designed to fail (or quit) MTS/COM+ processes that generate unhandled access violations.

By default, the failfast is enabled in MTS or COM+ applications that raise unhandled access violation exceptions. As a result, a failing MTS/COM+ server application cannot raise a second chance access violation exception (that is, it quits after the first chance acess violation). By default, ADPlus is configured to produce only a mini memory dump record when first chance exceptions occur.

To successfully debug MTS/COM+ server applications, followthese steps: 1. Configure the MTS/COM+ server application to Leave running when idle.2. Use the FullOnFirst switch to create full dump files on first chance exceptions.3. Run ADPlus in crash mode, and then wait for the application to fail.

Note Because MTS and COM+ shut down a server application, and because the failfast policy prevents the process from raising a second chance exception, you may only be able to obtain a first chance access violation memory dump file.

Run in crash mode remotely

Page 18: Examine Small Memory Dump Files

There are many occasions when you must initiate ADPlus in crash mode from a local client computer to monitor a process that quits unexpectedly on one or more remote servers in a server farm. Typically, on Windows 2000, you do this through Windows Terminal Services. However, you cannot debug applications that are running in different window stations on Windows NT 4.0 and Windows 2000. Therefore, ADPlus disables crash mode functionality when it detects that it is running in a Terminal Services session. To resolve this issue, share the remote server by using the Remote.exe utility, create a batch file that starts a command shell on the remote server, and then schedule this batch file to run on the target server by using the AT command. (The AT command causes the command shell to run non-interactively, similiar to a service.) The remote command shell is then connected to a local workstation or client computer that uses the same Remote.exe utility that you used to start the command shell.

To start a remote command shell on a server by using the AT command, follow these steps:

On the remote serverAssume that the debuggers are installed to C:\Debuggers. Follow these steps:

1. In the C:\Debuggers folder, create a new batch file named Remoteshell.cmd.

2. Add the following line to this batch file:

c:\debuggers\remote.exe /s "cmd.exe" remoteshell

3. At the console on the server, or in a Terminal Services session, open a new command shell, and then type the following command:

AT 15:00 c:\debuggers\remoteshell.cmd

where 15:00 is one minute later than the current time. For example, if the current time is 14:59, type 15:00.

4. Wait for the AT command to run.5. At the command prompt, type AT with no parameters to verify that the

task has run with no errors. On the local client:

Install the debuggers on the local client computer or (at a minimum) copy the Remote.exe utility locally. (By default, the utility is installed with the debuggers in the root installation folder.)

Assume that the debuggers and the Remote.exe utility are installed to C:\Debuggers. Follow these steps:

1. At a command prompt, switch to the C:\Debuggers folder.2. Type the following command:

remote.exe /c remote server remoteshell

Page 19: Examine Small Memory Dump Files

where remote server is the name of the remote server.

3. Your local command shell is now connected to the remote command shell that is running on the server, and all commands that you type locally will be carried out on the remote server (the DIR c:\ command lists the contents of drive C on the remote server).

4. In the remote command shell, you can now run ADPlus in crash mode as if you were running it locally from the console. However, you must use the -quiet switch to supress all dialog boxes that ADPlus generates by default. If you do not use the -quiet switch, the remote command shell will stop responding after you run ADPlus, and will not return to a prompt. If this problem occurs, you must quit the remote command shell (Cmd.exe) on the server, and then start a new instance.

5. To send a debug break (CTRL+C) to a process that ADPlus is currently debugging remotely through crash mode, you must use the Breakin.exe utility. By default, Breakin.exe is installed with the debuggers in the root of the debuggers folder. For example, to stop debugging IIS (Inetinfo.exe) that is running with a process ID of 1975, type the following command in the remote command shell:

breakin.exe 1975

Alternately, you can use the Kill.exe command (located in the root debuggers folder) to quit any processes that are being debugged

Additional information and known issues

How can you determine if ADPlus has captured information about a crash or if a process that is being monitored in crash mode has quit?

There are several ways to determine this:o Use the -notify switch, and verify that the messenger service is started on

the server that is being debugged and on the client computer that will receive the notifications.

o In a text editor, open the .log file that appears in the output folder for each process, and then scroll to the end of the file. Locate the following text:

o 0:070> * -------- AutodumpPlus 4.01 finished running at: --------

o 0:070> .timeo Debug session time: Mon Aug 06 15:25:15 2001o System Uptime: 3 days 17:00:34 o Process Uptime: 1 days 3:10:38

0:070> * -------------------------------------------------------

o In the output folder, locate any .dmp files that contain the phrase "__2nd_chance". If this phrase appears in the label of a memory dump record, a process has quit unexpectedly.

Page 20: Examine Small Memory Dump Files

o In the output folder, locate any .dmp files that contain the phrase "__Process_was_shutdown". If this phrase appears in the label of a memory dump record, an administrator quit the process or, if it is an MTS/COM+ application, the process quit because it reached the idle limit.

o In the output folder, locate any .dmp files that contain the phrase "__CTRL-C". If this phrase appears in the label of a memory dump record, either a debug break exception was thrown from a DLL that was running in the process or someone pressed CTRL+C from the console (or used Breakin.exe if ADPlus is running remotely) to stop the current debugging session.

You must install Windows Scripting Host components on the system for ADPlus to run. To download the Windows Scripting Host, visit the following Microsoft Web site:

Note Windows Scripting Host components may already be installed if you have any of the following Microsoft products installed:

o Microsoft Internet Explorer 5o Microsoft Office 2000o Microsoft Visual InterDev 6.0o Microsoft Visual Studio 6.0o Microsoft Windows NT Option Packo Microsoft Windows 2000o Microsoft Windows XP o Microsoft Windows Server 2003 o Microsoft Windows Vista

The -iis switch works only if Internet Information Server (IIS) 4.0 or Internet Information Services (IIS) 5.0.x is installed.

When you run ADPlus in quiet mode, the tool logs information to the event log. If you use the -o switch, the specified path must not contain more than one

nonexistent folder. For example:

1. You specify -o c:\temp1\temp2. However, the C:\Temp1 and \Temp2 folders do not exist.

2. You receive an error message from ADPlus that states that the folders do not exist, and ADPlus will not create them.

If you specify only -o c:\temp1, ADPlus creates the C:\Temp1 folder if does not exist, and then puts all output files in that folder. If you want to specify multiple subfolders, and you use the -o switch, verify that all the subfolders exist before you run ADPlus.

In COM+, you can configure a server package to start in the debugger on the Advanced tab in the Properties dialog box of the package. If you enable the Launch in Debugger option, ADPlus cannot attach the CDB debugger to a process. By default, only one debugger can be attached to a process at a time.

Page 21: Examine Small Memory Dump Files

When a remote procedure call (RPC) is made from a process that ADPlus is analyzing in crash mode to another process that has quit (intentionally or unexpectedly), the log file that ADPlus creates for the process that it is analyzing may contain one or more of the following exceptions:

Unknown exception - code 80010105 (first chance)Unknown exception - code 800706be (first chance)Unknown exception - code 800706ba (first chance)

These exceptions are typical. RPC raises these exceptions when a call is made from a process that is being monitored to a nonexistent or failed process.

Additionally, if ADPlus is monitoring Inetinfo.exe in the ADPlus debug log for that process, the following exception may appear in the log:

Unknown exception - code 800706bf (first chance)

This exception typically appears after IIS makes a call to an out-of-process (high-isolation) Web site that has failed. It may be followed by two instances of the following exception:

Unknown exception - code 800706ba (first chance)