Discussion:
DebugDiag Dump Analysis Help
(too old to reply)
RB
2006-03-09 17:11:33 UTC
Permalink
Hello, we are having issues with web applications hanging frequently. We
eventually setup pools and split up the applications assuming they were
conflicting with each other. We are still experiencing hangs. Below are
excerpts from the dump of one of the hangs. Can someone please help in
analyzing what the cause may be. We are running IIS 6.0 on a Windows 2003
server.

Error In w3wp__PID__8496__Date__03_09_2006__Time_11_33_02AM__506__First
chance exception 0XC0000005.dmp the assembly instruction at 2d944ba which
does not correspond to any known native module in the process has caused an
access violation exception (0xC0000005) when trying to read from memory
location 0x00000000 on thread 16
If no further information can be obtained from the detailed stack analysis
below, please contact Microsoft Corporation for troubleshooting steps on
stack corruption

Error In w3wp__PID__8496__Date__03_09_2006__Time_11_33_02AM__959__First
chance exception 0XC0000005.dmp the assembly instruction at
kernel32!RaiseException+53 in C:\WINDOWS\system32\kernel32.dll from Microsoft
Corporation has caused an access violation exception (0xC0000005) when trying
to read from memory location 0x00000000 on thread 16
Please follow up with the vendor Microsoft Corporation for
C:\WINDOWS\system32\kernel32.dll


Error In w3wp__PID__4788__Date__03_09_2006__Time_11_01_24AM__640__First
chance exception 0XE0434F4D.dmp the assembly instruction at
kernel32!RaiseException+53 in C:\WINDOWS\system32\kernel32.dll from Microsoft
Corporation has caused an unknown exception (0xe0434f4d) on thread 19

An exception thrown by Kernel32!RaiseException usually indicates a problem
with another module. Please review the stack for the faulting thread (19)
further to determine which module actually threw the exception raised by
Kernel32.dll.


Detailed stack corruption analysis for thread 16
Call stack with StackWalk

Index Return Address
1 2d944ba
2 2d943ef
3 143f9f88





Call stack - Heuristic

Index Stack Address Child EBP Return Address Destination
1 0x00000000 0x0199f864 2d944ba 0
2 0x0199f868 0x143f9fdc 2d943ef 2d94480
3 0x0199f878 0x1c3b0614 2d93ac4 2d943e0
4 0x0199f888 0x1c3b0614 2d93a04 2d93a60
5 0x0199f8a8 0x40590000 2d907d0 2d93928
6 0x0199f8b8 0x143f9f88 2d90666 2d906a0
7 0x0199f8d0 0x143f9f7c 2d9054c 2d905a8
8 0x0199f8e0 0x143f9c30 2d9040b 2d904f0
9 0x0199f8f4 0x1c3b0614 2d903a0 2d903b8
10 0x0199f900 0x182d300c 2d902af 2d90328
11 0x0199f90c 0x182d300c 2db9e26 0
12 0x0199f948 0x0199f9a0 2db98ab 2db9dd0
13 0x0199f950 0x14390804 2db9884 0
14 0x0199f960 0x14390aec 2d8a4c8 0
15 0x0199f9a4 0x0199f9ec 2d89f32 2d8a470
16 0x0199f9f0 0x0199fa44 2d89e03 2d89e38
17 0x0199fa04 0x18363f78 1ec68a7 0
18 0x0199fa48 0x0199fad4 1ec6458 1ec66c0
19 0x0199fa54 0x1c2d3a28 1ec2fd5 1ec63a8
20 0x0199fa64 0x01a881f3 mscorsvr!ComCallMLStubCache::CompileMLStub+429 0
21 0x0199fbc8 0x0199fca8 1a5a0bd
mscorsvr!ComCallMLStubCache::CompileMLStub+255
22 0x0199fc08 0xffffffff 19c3a04 1a5a06c
23 0x0199fc0c 0x019c3a04
aspnet_isapi!HttpCompletion::ProcessRequestInManagedCode+17e 0
24 0x0199fcbc 0x7c81b26f ntdll!ZwTestAlert+c 0
25 0x0199fd18 0x7c81b23f ntdll!ZwContinue+c 0
26 0x0199fd1c 0x7c8211b4 ntdll!KiUserApcDispatcher+3a ntdll!NtContinue
27 0x0199fdfc 0x00000000 ntdll!NtWaitForSingleObject+c 0
28 0x0199fe00 0x7c822124 kernel32!WaitForSingleObjectEx+ac
ntdll!ZwWaitForSingleObject
29 0x0199fe0c 0x00000000 kernel32!WaitForSingleObjectEx+dc
kernel32!_SEH_epilog
30 0x0199fe3c 0x00000000 aspnet_isapi!PerfDecrementCounter+1c
kernel32!InterlockedDecrement
31 0x0199fe44 0x01862810 aspnet_isapi!PerfDecrementGlobalCounter+f
aspnet_isapi!PerfDecrementCounter
32 0x0199fe60 0x79e8e231 aspnet_isapi!CorThreadPoolWorkitemCallback+13 0
33 0x0199fe74 0x00000000 mscorsvr!ThreadpoolMgr::ExecuteWorkRequest+19 0
34 0x0199fe88 0x0199fea4 mscorsvr!ThreadpoolMgr::WorkerThreadStart+129
mscorsvr!ThreadpoolMgr::ExecuteWorkRequest
35 0x0199fea8 0x0199ffb8 mscorsvr!ThreadpoolMgr::intermediateThreadProc+44 0
36 0x0199ffbc 0x0199ffec kernel32!BaseThreadStart+34 0




^ Memory access error in 'u 0x0000000c l1'


In w3wp__PID__8496__Date__03_09_2006__Time_11_33_02AM__506__First chance
exception 0XC0000005.dmp the assembly instruction at 2d944ba which does not
correspond to any known native module in the process has caused an access
violation exception (0xC0000005) when trying to read from memory location
0x00000000 on thread 16

Report for w3wp__PID__4788__Date__03_09_2006__Time_11_01_24AM__233__First
chance exception 0XE0434F4D.dmp
Type of Analysis Performed Crash Analysis
Machine Name GPISV205
Operating System Windows Server 2003 Service Pack 1
Number Of Processors 4
Process ID 4788
Process Image c:\WINDOWS\system32\inetsrv\w3wp.exe
System Up-Time 2 day(s) 20:38:19
Process Up-Time 0 day(s) 01:19:04


Thread 19 - System ID 8644
Entry point mscorsvr!ThreadpoolMgr::intermediateThreadProc
Create time 3/9/2006 9:42:21 AM
Time spent in user mode 0 Days 0:0:0.0
Time spent in kernel mode 0 Days 0:0:0.46






Function Source
kernel32!RaiseException+53
mscorsvr!RaiseTheException+a0
mscorsvr!RealCOMPlusThrow+48
mscorsvr!RealCOMPlusThrow+d
mscorsvr!CEEInfo::findMethod+123
mscorjit!Compiler::eeFindMethod+23
mscorjit!Compiler::impImportBlockCode+1e80
mscorjit!Compiler::impImportBlock+222
mscorjit!Compiler::impImport+e8
mscorjit!Compiler::fgImport+41
mscorjit!Compiler::compCompile+b
mscorjit!Compiler::compCompile+1e8
mscorjit!jitNativeCode+95
mscorjit!CILJit::compileMethod+a2
mscorsvr!CallCompileMethodWithSEHWrapper+52
mscorsvr!JITFunction+2c7
mscorsvr!MakeJitWorker+2c0
mscorsvr!MethodDesc::DoPrestub+4d3
mscorsvr!PreStubWorker+42
19c2f76
mscorsvr!MethodDesc::CallDescr+155
mscorsvr!MethodDesc::Call+8e
mscorsvr!COMMember::CreateInstance+2fe
1fd12e9
2bf3dc5
2bf3c74
2bbdb53
2bbda60
2bb990a
2bb97db
1ec6458
mscorsvr!ComCallMLStubCache::CompileMLStub+1af
mscorsvr!Thread::DoADCallBack+5c
mscorsvr!ComCallMLStubCache::CompileMLStub+2ba
1a5a0bd
f3c28




In w3wp__PID__4788__Date__03_09_2006__Time_11_01_24AM__233__First chance
exception 0XE0434F4D.dmp the assembly instruction at
kernel32!RaiseException+53 in C:\WINDOWS\system32\kernel32.dll from Microsoft
Corporation has caused an unknown exception (0xe0434f4d) on thread 19


Thanks,
RB
Pat [MSFT]
2006-03-09 22:21:22 UTC
Permalink
The crash occurred in managed code, apparently on thread 16. If a dump file
was created, open it with WinDBG and:
1) Set the symbol path:
.sympath srv*c:\debug*http://msdl.microsoft.com/download/symbols <enter>

2) Reset the debug environment
.reload <enter>

3) Switch to thread 16
~16s <enter>

4) Dump the system stack:
kb <enter>

5) Dump the managed code stack
!clrstack <enter>

You can post the results and we may be able to identify the issue.

Since you are _also_ experiencing hangs, it is likely that you are having
multiple problems. To diagnose that you need a log/dump while the problem
is actually occurring.


Pat
Post by RB
Hello, we are having issues with web applications hanging frequently. We
eventually setup pools and split up the applications assuming they were
conflicting with each other. We are still experiencing hangs. Below are
excerpts from the dump of one of the hangs. Can someone please help in
analyzing what the cause may be. We are running IIS 6.0 on a Windows 2003
server.
Error In w3wp__PID__8496__Date__03_09_2006__Time_11_33_02AM__506__First
chance exception 0XC0000005.dmp the assembly instruction at 2d944ba which
does not correspond to any known native module in the process has caused an
access violation exception (0xC0000005) when trying to read from memory
location 0x00000000 on thread 16
If no further information can be obtained from the detailed stack analysis
below, please contact Microsoft Corporation for troubleshooting steps on
stack corruption
Error In w3wp__PID__8496__Date__03_09_2006__Time_11_33_02AM__959__First
chance exception 0XC0000005.dmp the assembly instruction at
kernel32!RaiseException+53 in C:\WINDOWS\system32\kernel32.dll from Microsoft
Corporation has caused an access violation exception (0xC0000005) when trying
to read from memory location 0x00000000 on thread 16
Please follow up with the vendor Microsoft Corporation for
C:\WINDOWS\system32\kernel32.dll
Error In w3wp__PID__4788__Date__03_09_2006__Time_11_01_24AM__640__First
chance exception 0XE0434F4D.dmp the assembly instruction at
kernel32!RaiseException+53 in C:\WINDOWS\system32\kernel32.dll from Microsoft
Corporation has caused an unknown exception (0xe0434f4d) on thread 19
An exception thrown by Kernel32!RaiseException usually indicates a problem
with another module. Please review the stack for the faulting thread (19)
further to determine which module actually threw the exception raised by
Kernel32.dll.
Detailed stack corruption analysis for thread 16
Call stack with StackWalk
Index Return Address
1 2d944ba
2 2d943ef
3 143f9f88
Call stack - Heuristic
Index Stack Address Child EBP Return Address Destination
1 0x00000000 0x0199f864 2d944ba 0
2 0x0199f868 0x143f9fdc 2d943ef 2d94480
3 0x0199f878 0x1c3b0614 2d93ac4 2d943e0
4 0x0199f888 0x1c3b0614 2d93a04 2d93a60
5 0x0199f8a8 0x40590000 2d907d0 2d93928
6 0x0199f8b8 0x143f9f88 2d90666 2d906a0
7 0x0199f8d0 0x143f9f7c 2d9054c 2d905a8
8 0x0199f8e0 0x143f9c30 2d9040b 2d904f0
9 0x0199f8f4 0x1c3b0614 2d903a0 2d903b8
10 0x0199f900 0x182d300c 2d902af 2d90328
11 0x0199f90c 0x182d300c 2db9e26 0
12 0x0199f948 0x0199f9a0 2db98ab 2db9dd0
13 0x0199f950 0x14390804 2db9884 0
14 0x0199f960 0x14390aec 2d8a4c8 0
15 0x0199f9a4 0x0199f9ec 2d89f32 2d8a470
16 0x0199f9f0 0x0199fa44 2d89e03 2d89e38
17 0x0199fa04 0x18363f78 1ec68a7 0
18 0x0199fa48 0x0199fad4 1ec6458 1ec66c0
19 0x0199fa54 0x1c2d3a28 1ec2fd5 1ec63a8
20 0x0199fa64 0x01a881f3 mscorsvr!ComCallMLStubCache::CompileMLStub+429 0
21 0x0199fbc8 0x0199fca8 1a5a0bd
mscorsvr!ComCallMLStubCache::CompileMLStub+255
22 0x0199fc08 0xffffffff 19c3a04 1a5a06c
23 0x0199fc0c 0x019c3a04
aspnet_isapi!HttpCompletion::ProcessRequestInManagedCode+17e 0
24 0x0199fcbc 0x7c81b26f ntdll!ZwTestAlert+c 0
25 0x0199fd18 0x7c81b23f ntdll!ZwContinue+c 0
26 0x0199fd1c 0x7c8211b4 ntdll!KiUserApcDispatcher+3a ntdll!NtContinue
27 0x0199fdfc 0x00000000 ntdll!NtWaitForSingleObject+c 0
28 0x0199fe00 0x7c822124 kernel32!WaitForSingleObjectEx+ac
ntdll!ZwWaitForSingleObject
29 0x0199fe0c 0x00000000 kernel32!WaitForSingleObjectEx+dc
kernel32!_SEH_epilog
30 0x0199fe3c 0x00000000 aspnet_isapi!PerfDecrementCounter+1c
kernel32!InterlockedDecrement
31 0x0199fe44 0x01862810 aspnet_isapi!PerfDecrementGlobalCounter+f
aspnet_isapi!PerfDecrementCounter
32 0x0199fe60 0x79e8e231 aspnet_isapi!CorThreadPoolWorkitemCallback+13 0
33 0x0199fe74 0x00000000 mscorsvr!ThreadpoolMgr::ExecuteWorkRequest+19 0
34 0x0199fe88 0x0199fea4 mscorsvr!ThreadpoolMgr::WorkerThreadStart+129
mscorsvr!ThreadpoolMgr::ExecuteWorkRequest
35 0x0199fea8 0x0199ffb8 mscorsvr!ThreadpoolMgr::intermediateThreadProc+44 0
36 0x0199ffbc 0x0199ffec kernel32!BaseThreadStart+34 0
^ Memory access error in 'u 0x0000000c l1'
In w3wp__PID__8496__Date__03_09_2006__Time_11_33_02AM__506__First chance
exception 0XC0000005.dmp the assembly instruction at 2d944ba which does not
correspond to any known native module in the process has caused an access
violation exception (0xC0000005) when trying to read from memory location
0x00000000 on thread 16
Report for w3wp__PID__4788__Date__03_09_2006__Time_11_01_24AM__233__First
chance exception 0XE0434F4D.dmp
Type of Analysis Performed Crash Analysis
Machine Name GPISV205
Operating System Windows Server 2003 Service Pack 1
Number Of Processors 4
Process ID 4788
Process Image c:\WINDOWS\system32\inetsrv\w3wp.exe
System Up-Time 2 day(s) 20:38:19
Process Up-Time 0 day(s) 01:19:04
Thread 19 - System ID 8644
Entry point mscorsvr!ThreadpoolMgr::intermediateThreadProc
Create time 3/9/2006 9:42:21 AM
Time spent in user mode 0 Days 0:0:0.0
Time spent in kernel mode 0 Days 0:0:0.46
Function Source
kernel32!RaiseException+53
mscorsvr!RaiseTheException+a0
mscorsvr!RealCOMPlusThrow+48
mscorsvr!RealCOMPlusThrow+d
mscorsvr!CEEInfo::findMethod+123
mscorjit!Compiler::eeFindMethod+23
mscorjit!Compiler::impImportBlockCode+1e80
mscorjit!Compiler::impImportBlock+222
mscorjit!Compiler::impImport+e8
mscorjit!Compiler::fgImport+41
mscorjit!Compiler::compCompile+b
mscorjit!Compiler::compCompile+1e8
mscorjit!jitNativeCode+95
mscorjit!CILJit::compileMethod+a2
mscorsvr!CallCompileMethodWithSEHWrapper+52
mscorsvr!JITFunction+2c7
mscorsvr!MakeJitWorker+2c0
mscorsvr!MethodDesc::DoPrestub+4d3
mscorsvr!PreStubWorker+42
19c2f76
mscorsvr!MethodDesc::CallDescr+155
mscorsvr!MethodDesc::Call+8e
mscorsvr!COMMember::CreateInstance+2fe
1fd12e9
2bf3dc5
2bf3c74
2bbdb53
2bbda60
2bb990a
2bb97db
1ec6458
mscorsvr!ComCallMLStubCache::CompileMLStub+1af
mscorsvr!Thread::DoADCallBack+5c
mscorsvr!ComCallMLStubCache::CompileMLStub+2ba
1a5a0bd
f3c28
In w3wp__PID__4788__Date__03_09_2006__Time_11_01_24AM__233__First chance
exception 0XE0434F4D.dmp the assembly instruction at
kernel32!RaiseException+53 in C:\WINDOWS\system32\kernel32.dll from Microsoft
Corporation has caused an unknown exception (0xe0434f4d) on thread 19
Thanks,
RB
RB
2006-03-09 23:31:28 UTC
Permalink
Here's the windbg trace:

0:016> kb
ChildEBP RetAddr Args to Child
0199f580 79216aed e0434f4d 00000001 00000000 kernel32!RaiseException+0x53
0199f5d8 7924c8d0 183f0fe4 00000000 183f0fd0 mscorsvr!RaiseTheException+0xa0
0199f644 026357e2 183f05ec 183f0754 183f0730 mscorsvr!JIT_Throw+0x4d
WARNING: Frame IP not in any known module. Following frames may be wrong.
0199f670 0263d8f8 182d4034 183f05ec 183f05ec 0x26357e2
0199f6a4 0263d5d6 183e7d78 183f05ec 183f0730 0x263d8f8
0199f6d8 02697e52 183f0cfc 183e7d78 183f0730 0x263d5d6
0199f700 026982b2 183f0638 183f01cc 183649a4 0x2697e52
0199f704 183f0638 183f01cc 183649a4 183647ac 0x26982b2
0199f708 183f01cc 183649a4 183647ac 183f03e4 0x183f0638
0199f70c 183649a4 183647ac 183f03e4 00000000 0x183f01cc
0199f710 183647ac 183f03e4 00000000 02698278 0x183649a4
0199f714 183f03e4 00000000 02698278 02698100 0x183647ac
0199f718 00000000 02698278 02698100 0199f76c 0x183f03e4
0:016> !clrstack
No export clrstack found

I did execute a manual hang dump during the incident, aside from missing
symbols, the analysis for that app does not show any errors.

Thanks,
Rani Baki
Post by Pat [MSFT]
The crash occurred in managed code, apparently on thread 16. If a dump file
..sympath srv*c:\debug*http://msdl.microsoft.com/download/symbols <enter>
2) Reset the debug environment
..reload <enter>
3) Switch to thread 16
~16s <enter>
kb <enter>
5) Dump the managed code stack
!clrstack <enter>
You can post the results and we may be able to identify the issue.
Since you are _also_ experiencing hangs, it is likely that you are having
multiple problems. To diagnose that you need a log/dump while the problem
is actually occurring.
Pat
Post by RB
Hello, we are having issues with web applications hanging frequently. We
eventually setup pools and split up the applications assuming they were
conflicting with each other. We are still experiencing hangs. Below are
excerpts from the dump of one of the hangs. Can someone please help in
analyzing what the cause may be. We are running IIS 6.0 on a Windows 2003
server.
Error In w3wp__PID__8496__Date__03_09_2006__Time_11_33_02AM__506__First
chance exception 0XC0000005.dmp the assembly instruction at 2d944ba which
does not correspond to any known native module in the process has caused an
access violation exception (0xC0000005) when trying to read from memory
location 0x00000000 on thread 16
If no further information can be obtained from the detailed stack analysis
below, please contact Microsoft Corporation for troubleshooting steps on
stack corruption
Error In w3wp__PID__8496__Date__03_09_2006__Time_11_33_02AM__959__First
chance exception 0XC0000005.dmp the assembly instruction at
kernel32!RaiseException+53 in C:\WINDOWS\system32\kernel32.dll from Microsoft
Corporation has caused an access violation exception (0xC0000005) when trying
to read from memory location 0x00000000 on thread 16
Please follow up with the vendor Microsoft Corporation for
C:\WINDOWS\system32\kernel32.dll
Error In w3wp__PID__4788__Date__03_09_2006__Time_11_01_24AM__640__First
chance exception 0XE0434F4D.dmp the assembly instruction at
kernel32!RaiseException+53 in C:\WINDOWS\system32\kernel32.dll from Microsoft
Corporation has caused an unknown exception (0xe0434f4d) on thread 19
An exception thrown by Kernel32!RaiseException usually indicates a problem
with another module. Please review the stack for the faulting thread (19)
further to determine which module actually threw the exception raised by
Kernel32.dll.
Detailed stack corruption analysis for thread 16
Call stack with StackWalk
Index Return Address
1 2d944ba
2 2d943ef
3 143f9f88
Call stack - Heuristic
Index Stack Address Child EBP Return Address Destination
1 0x00000000 0x0199f864 2d944ba 0
2 0x0199f868 0x143f9fdc 2d943ef 2d94480
3 0x0199f878 0x1c3b0614 2d93ac4 2d943e0
4 0x0199f888 0x1c3b0614 2d93a04 2d93a60
5 0x0199f8a8 0x40590000 2d907d0 2d93928
6 0x0199f8b8 0x143f9f88 2d90666 2d906a0
7 0x0199f8d0 0x143f9f7c 2d9054c 2d905a8
8 0x0199f8e0 0x143f9c30 2d9040b 2d904f0
9 0x0199f8f4 0x1c3b0614 2d903a0 2d903b8
10 0x0199f900 0x182d300c 2d902af 2d90328
11 0x0199f90c 0x182d300c 2db9e26 0
12 0x0199f948 0x0199f9a0 2db98ab 2db9dd0
13 0x0199f950 0x14390804 2db9884 0
14 0x0199f960 0x14390aec 2d8a4c8 0
15 0x0199f9a4 0x0199f9ec 2d89f32 2d8a470
16 0x0199f9f0 0x0199fa44 2d89e03 2d89e38
17 0x0199fa04 0x18363f78 1ec68a7 0
18 0x0199fa48 0x0199fad4 1ec6458 1ec66c0
19 0x0199fa54 0x1c2d3a28 1ec2fd5 1ec63a8
20 0x0199fa64 0x01a881f3 mscorsvr!ComCallMLStubCache::CompileMLStub+429 0
21 0x0199fbc8 0x0199fca8 1a5a0bd
mscorsvr!ComCallMLStubCache::CompileMLStub+255
22 0x0199fc08 0xffffffff 19c3a04 1a5a06c
23 0x0199fc0c 0x019c3a04
aspnet_isapi!HttpCompletion::ProcessRequestInManagedCode+17e 0
24 0x0199fcbc 0x7c81b26f ntdll!ZwTestAlert+c 0
25 0x0199fd18 0x7c81b23f ntdll!ZwContinue+c 0
26 0x0199fd1c 0x7c8211b4 ntdll!KiUserApcDispatcher+3a ntdll!NtContinue
27 0x0199fdfc 0x00000000 ntdll!NtWaitForSingleObject+c 0
28 0x0199fe00 0x7c822124 kernel32!WaitForSingleObjectEx+ac
ntdll!ZwWaitForSingleObject
29 0x0199fe0c 0x00000000 kernel32!WaitForSingleObjectEx+dc
kernel32!_SEH_epilog
30 0x0199fe3c 0x00000000 aspnet_isapi!PerfDecrementCounter+1c
kernel32!InterlockedDecrement
31 0x0199fe44 0x01862810 aspnet_isapi!PerfDecrementGlobalCounter+f
aspnet_isapi!PerfDecrementCounter
32 0x0199fe60 0x79e8e231 aspnet_isapi!CorThreadPoolWorkitemCallback+13 0
33 0x0199fe74 0x00000000 mscorsvr!ThreadpoolMgr::ExecuteWorkRequest+19 0
34 0x0199fe88 0x0199fea4 mscorsvr!ThreadpoolMgr::WorkerThreadStart+129
mscorsvr!ThreadpoolMgr::ExecuteWorkRequest
35 0x0199fea8 0x0199ffb8 mscorsvr!ThreadpoolMgr::intermediateThreadProc+44 0
36 0x0199ffbc 0x0199ffec kernel32!BaseThreadStart+34 0
^ Memory access error in 'u 0x0000000c l1'
In w3wp__PID__8496__Date__03_09_2006__Time_11_33_02AM__506__First chance
exception 0XC0000005.dmp the assembly instruction at 2d944ba which does not
correspond to any known native module in the process has caused an access
violation exception (0xC0000005) when trying to read from memory location
0x00000000 on thread 16
Report for w3wp__PID__4788__Date__03_09_2006__Time_11_01_24AM__233__First
chance exception 0XE0434F4D.dmp
Type of Analysis Performed Crash Analysis
Machine Name GPISV205
Operating System Windows Server 2003 Service Pack 1
Number Of Processors 4
Process ID 4788
Process Image c:\WINDOWS\system32\inetsrv\w3wp.exe
System Up-Time 2 day(s) 20:38:19
Process Up-Time 0 day(s) 01:19:04
Thread 19 - System ID 8644
Entry point mscorsvr!ThreadpoolMgr::intermediateThreadProc
Create time 3/9/2006 9:42:21 AM
Time spent in user mode 0 Days 0:0:0.0
Time spent in kernel mode 0 Days 0:0:0.46
Function Source
kernel32!RaiseException+53
mscorsvr!RaiseTheException+a0
mscorsvr!RealCOMPlusThrow+48
mscorsvr!RealCOMPlusThrow+d
mscorsvr!CEEInfo::findMethod+123
mscorjit!Compiler::eeFindMethod+23
mscorjit!Compiler::impImportBlockCode+1e80
mscorjit!Compiler::impImportBlock+222
mscorjit!Compiler::impImport+e8
mscorjit!Compiler::fgImport+41
mscorjit!Compiler::compCompile+b
mscorjit!Compiler::compCompile+1e8
mscorjit!jitNativeCode+95
mscorjit!CILJit::compileMethod+a2
mscorsvr!CallCompileMethodWithSEHWrapper+52
mscorsvr!JITFunction+2c7
mscorsvr!MakeJitWorker+2c0
mscorsvr!MethodDesc::DoPrestub+4d3
mscorsvr!PreStubWorker+42
19c2f76
mscorsvr!MethodDesc::CallDescr+155
mscorsvr!MethodDesc::Call+8e
mscorsvr!COMMember::CreateInstance+2fe
1fd12e9
2bf3dc5
2bf3c74
2bbdb53
2bbda60
2bb990a
2bb97db
1ec6458
mscorsvr!ComCallMLStubCache::CompileMLStub+1af
mscorsvr!Thread::DoADCallBack+5c
mscorsvr!ComCallMLStubCache::CompileMLStub+2ba
1a5a0bd
f3c28
In w3wp__PID__4788__Date__03_09_2006__Time_11_01_24AM__233__First chance
exception 0XE0434F4D.dmp the assembly instruction at
kernel32!RaiseException+53 in C:\WINDOWS\system32\kernel32.dll from Microsoft
Corporation has caused an unknown exception (0xe0434f4d) on thread 19
Thanks,
RB
Pat [MSFT]
2006-03-10 19:19:11 UTC
Permalink
Try:

!.\clr10\sos.ClrStack <enter>

Pat
Post by RB
0:016> kb
ChildEBP RetAddr Args to Child
0199f580 79216aed e0434f4d 00000001 00000000 kernel32!RaiseException+0x53
0199f5d8 7924c8d0 183f0fe4 00000000 183f0fd0
mscorsvr!RaiseTheException+0xa0
0199f644 026357e2 183f05ec 183f0754 183f0730 mscorsvr!JIT_Throw+0x4d
WARNING: Frame IP not in any known module. Following frames may be wrong.
0199f670 0263d8f8 182d4034 183f05ec 183f05ec 0x26357e2
0199f6a4 0263d5d6 183e7d78 183f05ec 183f0730 0x263d8f8
0199f6d8 02697e52 183f0cfc 183e7d78 183f0730 0x263d5d6
0199f700 026982b2 183f0638 183f01cc 183649a4 0x2697e52
0199f704 183f0638 183f01cc 183649a4 183647ac 0x26982b2
0199f708 183f01cc 183649a4 183647ac 183f03e4 0x183f0638
0199f70c 183649a4 183647ac 183f03e4 00000000 0x183f01cc
0199f710 183647ac 183f03e4 00000000 02698278 0x183649a4
0199f714 183f03e4 00000000 02698278 02698100 0x183647ac
0199f718 00000000 02698278 02698100 0199f76c 0x183f03e4
0:016> !clrstack
No export clrstack found
I did execute a manual hang dump during the incident, aside from missing
symbols, the analysis for that app does not show any errors.
Thanks,
Rani Baki
Post by Pat [MSFT]
The crash occurred in managed code, apparently on thread 16. If a dump file
..sympath srv*c:\debug*http://msdl.microsoft.com/download/symbols <enter>
2) Reset the debug environment
..reload <enter>
3) Switch to thread 16
~16s <enter>
kb <enter>
5) Dump the managed code stack
!clrstack <enter>
You can post the results and we may be able to identify the issue.
Since you are _also_ experiencing hangs, it is likely that you are having
multiple problems. To diagnose that you need a log/dump while the problem
is actually occurring.
Pat
Post by RB
Hello, we are having issues with web applications hanging frequently. We
eventually setup pools and split up the applications assuming they were
conflicting with each other. We are still experiencing hangs. Below are
excerpts from the dump of one of the hangs. Can someone please help in
analyzing what the cause may be. We are running IIS 6.0 on a Windows 2003
server.
Error In
w3wp__PID__8496__Date__03_09_2006__Time_11_33_02AM__506__First
chance exception 0XC0000005.dmp the assembly instruction at 2d944ba which
does not correspond to any known native module in the process has
caused
an
access violation exception (0xC0000005) when trying to read from memory
location 0x00000000 on thread 16
If no further information can be obtained from the detailed stack analysis
below, please contact Microsoft Corporation for troubleshooting steps on
stack corruption
Error In
w3wp__PID__8496__Date__03_09_2006__Time_11_33_02AM__959__First
chance exception 0XC0000005.dmp the assembly instruction at
kernel32!RaiseException+53 in C:\WINDOWS\system32\kernel32.dll from Microsoft
Corporation has caused an access violation exception (0xC0000005) when trying
to read from memory location 0x00000000 on thread 16
Please follow up with the vendor Microsoft Corporation for
C:\WINDOWS\system32\kernel32.dll
Error In
w3wp__PID__4788__Date__03_09_2006__Time_11_01_24AM__640__First
chance exception 0XE0434F4D.dmp the assembly instruction at
kernel32!RaiseException+53 in C:\WINDOWS\system32\kernel32.dll from Microsoft
Corporation has caused an unknown exception (0xe0434f4d) on thread 19
An exception thrown by Kernel32!RaiseException usually indicates a problem
with another module. Please review the stack for the faulting thread (19)
further to determine which module actually threw the exception raised by
Kernel32.dll.
Detailed stack corruption analysis for thread 16
Call stack with StackWalk
Index Return Address
1 2d944ba
2 2d943ef
3 143f9f88
Call stack - Heuristic
Index Stack Address Child EBP Return Address Destination
1 0x00000000 0x0199f864 2d944ba 0
2 0x0199f868 0x143f9fdc 2d943ef 2d94480
3 0x0199f878 0x1c3b0614 2d93ac4 2d943e0
4 0x0199f888 0x1c3b0614 2d93a04 2d93a60
5 0x0199f8a8 0x40590000 2d907d0 2d93928
6 0x0199f8b8 0x143f9f88 2d90666 2d906a0
7 0x0199f8d0 0x143f9f7c 2d9054c 2d905a8
8 0x0199f8e0 0x143f9c30 2d9040b 2d904f0
9 0x0199f8f4 0x1c3b0614 2d903a0 2d903b8
10 0x0199f900 0x182d300c 2d902af 2d90328
11 0x0199f90c 0x182d300c 2db9e26 0
12 0x0199f948 0x0199f9a0 2db98ab 2db9dd0
13 0x0199f950 0x14390804 2db9884 0
14 0x0199f960 0x14390aec 2d8a4c8 0
15 0x0199f9a4 0x0199f9ec 2d89f32 2d8a470
16 0x0199f9f0 0x0199fa44 2d89e03 2d89e38
17 0x0199fa04 0x18363f78 1ec68a7 0
18 0x0199fa48 0x0199fad4 1ec6458 1ec66c0
19 0x0199fa54 0x1c2d3a28 1ec2fd5 1ec63a8
20 0x0199fa64 0x01a881f3 mscorsvr!ComCallMLStubCache::CompileMLStub+429 0
21 0x0199fbc8 0x0199fca8 1a5a0bd
mscorsvr!ComCallMLStubCache::CompileMLStub+255
22 0x0199fc08 0xffffffff 19c3a04 1a5a06c
23 0x0199fc0c 0x019c3a04
aspnet_isapi!HttpCompletion::ProcessRequestInManagedCode+17e 0
24 0x0199fcbc 0x7c81b26f ntdll!ZwTestAlert+c 0
25 0x0199fd18 0x7c81b23f ntdll!ZwContinue+c 0
26 0x0199fd1c 0x7c8211b4 ntdll!KiUserApcDispatcher+3a ntdll!NtContinue
27 0x0199fdfc 0x00000000 ntdll!NtWaitForSingleObject+c 0
28 0x0199fe00 0x7c822124 kernel32!WaitForSingleObjectEx+ac
ntdll!ZwWaitForSingleObject
29 0x0199fe0c 0x00000000 kernel32!WaitForSingleObjectEx+dc
kernel32!_SEH_epilog
30 0x0199fe3c 0x00000000 aspnet_isapi!PerfDecrementCounter+1c
kernel32!InterlockedDecrement
31 0x0199fe44 0x01862810 aspnet_isapi!PerfDecrementGlobalCounter+f
aspnet_isapi!PerfDecrementCounter
32 0x0199fe60 0x79e8e231 aspnet_isapi!CorThreadPoolWorkitemCallback+13 0
33 0x0199fe74 0x00000000 mscorsvr!ThreadpoolMgr::ExecuteWorkRequest+19 0
34 0x0199fe88 0x0199fea4 mscorsvr!ThreadpoolMgr::WorkerThreadStart+129
mscorsvr!ThreadpoolMgr::ExecuteWorkRequest
35 0x0199fea8 0x0199ffb8
mscorsvr!ThreadpoolMgr::intermediateThreadProc+44
0
36 0x0199ffbc 0x0199ffec kernel32!BaseThreadStart+34 0
^ Memory access error in 'u 0x0000000c l1'
In w3wp__PID__8496__Date__03_09_2006__Time_11_33_02AM__506__First chance
exception 0XC0000005.dmp the assembly instruction at 2d944ba which does not
correspond to any known native module in the process has caused an access
violation exception (0xC0000005) when trying to read from memory location
0x00000000 on thread 16
Report for
w3wp__PID__4788__Date__03_09_2006__Time_11_01_24AM__233__First
chance exception 0XE0434F4D.dmp
Type of Analysis Performed Crash Analysis
Machine Name GPISV205
Operating System Windows Server 2003 Service Pack 1
Number Of Processors 4
Process ID 4788
Process Image c:\WINDOWS\system32\inetsrv\w3wp.exe
System Up-Time 2 day(s) 20:38:19
Process Up-Time 0 day(s) 01:19:04
Thread 19 - System ID 8644
Entry point mscorsvr!ThreadpoolMgr::intermediateThreadProc
Create time 3/9/2006 9:42:21 AM
Time spent in user mode 0 Days 0:0:0.0
Time spent in kernel mode 0 Days 0:0:0.46
Function Source
kernel32!RaiseException+53
mscorsvr!RaiseTheException+a0
mscorsvr!RealCOMPlusThrow+48
mscorsvr!RealCOMPlusThrow+d
mscorsvr!CEEInfo::findMethod+123
mscorjit!Compiler::eeFindMethod+23
mscorjit!Compiler::impImportBlockCode+1e80
mscorjit!Compiler::impImportBlock+222
mscorjit!Compiler::impImport+e8
mscorjit!Compiler::fgImport+41
mscorjit!Compiler::compCompile+b
mscorjit!Compiler::compCompile+1e8
mscorjit!jitNativeCode+95
mscorjit!CILJit::compileMethod+a2
mscorsvr!CallCompileMethodWithSEHWrapper+52
mscorsvr!JITFunction+2c7
mscorsvr!MakeJitWorker+2c0
mscorsvr!MethodDesc::DoPrestub+4d3
mscorsvr!PreStubWorker+42
19c2f76
mscorsvr!MethodDesc::CallDescr+155
mscorsvr!MethodDesc::Call+8e
mscorsvr!COMMember::CreateInstance+2fe
1fd12e9
2bf3dc5
2bf3c74
2bbdb53
2bbda60
2bb990a
2bb97db
1ec6458
mscorsvr!ComCallMLStubCache::CompileMLStub+1af
mscorsvr!Thread::DoADCallBack+5c
mscorsvr!ComCallMLStubCache::CompileMLStub+2ba
1a5a0bd
f3c28
In w3wp__PID__4788__Date__03_09_2006__Time_11_01_24AM__233__First chance
exception 0XE0434F4D.dmp the assembly instruction at
kernel32!RaiseException+53 in C:\WINDOWS\system32\kernel32.dll from Microsoft
Corporation has caused an unknown exception (0xe0434f4d) on thread 19
Thanks,
RB
RB
2006-03-10 19:53:41 UTC
Permalink
I accidently deleted the original dump file, but do have another one for a
different app. This one is faulting on thread 25. Below is the output. Keep
in mind that we have multiple apps hanging. Is it typical for DebugDiag to
generate a first exception almost immediately after activating a rule? That's
what is happening on those troubled apps.

Thanks,
Rani Baki

0:025> kb
ChildEBP RetAddr Args to Child
009aea9c 79216aed e0434f4d 00000001 00000000 kernel32!RaiseException+0x53
009aeaf4 79216a70 142fbfcc 00000000 009aeb38 mscorsvr!RaiseTheException+0xa0
009aeb1c 79216a24 142fbfcc 00000000 009aeb80 mscorsvr!RealCOMPlusThrow+0x48
009aeb2c 7927f019 142fbfcc 009af154 792ba800 mscorsvr!RealCOMPlusThrow+0xd
009aeb80 7943e9ef 009af39c 021e31e8 0a000129
mscorsvr!CEEInfo::findMethod+0x123
009aeb9c 79445a7f 0a000129 021e31e8 02642a28
mscorjit!Compiler::eeFindMethod+0x23
009af0bc 7943fcc4 01eb1e18 01eb1ec4 01eb0010
mscorjit!Compiler::impImportBlockCode+0x1e80
009af164 79440490 01eb1e18 009af3cc 01eb0010
mscorjit!Compiler::impImportBlock+0x222
009af17c 794428b5 01eb1e18 01eb0010 794310e5 mscorjit!Compiler::impImport+0xe8
009af188 794310e5 01eb0010 009af1e0 79431477 mscorjit!Compiler::fgImport+0x41
009af194 79431477 009af2a8 009af424 009af29c
mscorjit!Compiler::compCompile+0xb
009af1e0 7943151a 021e31e8 009af380 009af3cc
mscorjit!Compiler::compCompile+0x1e8
009af268 794315a4 009af380 009af3cc 009af2a8 mscorjit!jitNativeCode+0x95
009af2ac 791bc3f8 79479124 009af380 009af3cc
mscorjit!CILJit::compileMethod+0xa2
009af2f4 791bb52a 0012a100 009af380 009af3cc
mscorsvr!CallCompileMethodWithSEHWrapper+0x52
009af45c 791bb9c0 02642a28 009af624 009af560 mscorsvr!JITFunction+0x2c7
009af590 791b706a 02642a28 009af624 00000000 mscorsvr!MakeJitWorker+0x2c0
009af67c 791b77e4 02642cd4 009afaa0 009af6b8
mscorsvr!MethodDesc::DoPrestub+0x4d3
009af694 019c2f76 009af6b8 14300f5c 1c34b308 mscorsvr!PreStubWorker+0x42
WARNING: Frame IP not in any known module. Following frames may be wrong.
009af6f4 02659e32 00000008 009af734 1c34b114 0x19c2f76

0:025> !.\clr10\sos.ClrStack
Loaded Son of Strike data table version 5 from
"C:\WINDOWS\Microsoft.NET\Framework\v1.1.4322\mscorsvr.dll"
Thread 25
ESP EIP
0x009aeb50 0x77e55dea [FRAME: GCFrame]
0x009af60c 0x77e55dea [FRAME: GCFrame]
0x009af6b8 0x77e55dea [FRAME: PrestubMethodFrame] [DEFAULT] [hasThis] Void
GCM.Framework.Web.UI.GCMClipboard.OnInit(Class System.EventArgs)
0x009af6c8 0x0265db8a [DEFAULT] [hasThis] Void
System.Web.UI.Control.InitRecursive(Class System.Web.UI.Control)
0x009af6e4 0x02659f5a [DEFAULT] [hasThis] Void
System.Web.UI.Control.AddedControl(Class System.Web.UI.Control,I4)
0x009af700 0x02659e32 [DEFAULT] [hasThis] Void
System.Web.UI.ControlCollection.Add(Class System.Web.UI.Control)
0x009af710 0x026b748a [DEFAULT] [hasThis] Void
GCM.Framework.Web.UI.GCMWebPage.AddActionButtons()
0x009af7dc 0x0265e481 [DEFAULT] [hasThis] Void
GCM.Framework.Web.UI.GCMWebPage.CreateChildControls()
0x009af7e8 0x0265e27d [DEFAULT] [hasThis] Void
System.Web.UI.Control.EnsureChildControls()
0x009af810 0x0265dd56 [DEFAULT] [hasThis] Void
GCM.Framework.Web.UI.GCMWebPage.OnInit(Class System.EventArgs)
0x009af824 0x02621572 [DEFAULT] [hasThis] Void
GCM.Framework.Web.SubscriptionPopup.OnInit(Class System.EventArgs)
0x009af830 0x0265db8a [DEFAULT] [hasThis] Void
System.Web.UI.Control.InitRecursive(Class System.Web.UI.Control)
0x009af84c 0x0265c293 [DEFAULT] [hasThis] Void
System.Web.UI.Page.ProcessRequestMain()
0x009af890 0x02659ab7 [DEFAULT] [hasThis] Void
System.Web.UI.Page.ProcessRequest()
0x009af8cc 0x0265951b [DEFAULT] [hasThis] Void
System.Web.UI.Page.ProcessRequest(Class System.Web.HttpContext)
0x009af8d4 0x026594f4 [DEFAULT] [hasThis] Void
System.Web.HttpApplication/CallHandlerExecutionStep.System.Web.HttpApplication+IExecutionStep.Execute()
0x009af8e4 0x02619ea0 [DEFAULT] [hasThis] Class System.Exception
System.Web.HttpApplication.ExecuteStep(Class IExecutionStep,ByRef Boolean)
0x009af92c 0x0261990a [DEFAULT] [hasThis] Void
System.Web.HttpApplication.ResumeSteps(Class System.Exception)
0x009af974 0x026197db [DEFAULT] [hasThis] Class System.IAsyncResult
System.Web.HttpApplication.System.Web.IHttpAsyncHandler.BeginProcessRequest(Class System.Web.HttpContext,Class System.AsyncCallback,Object)
0x009af990 0x01ec68a7 [DEFAULT] [hasThis] Void
System.Web.HttpRuntime.ProcessRequestInternal(Class
System.Web.HttpWorkerRequest)
0x009af9cc 0x01ec6458 [DEFAULT] Void
System.Web.HttpRuntime.ProcessRequest(Class System.Web.HttpWorkerRequest)
0x009af9d8 0x01ec2fd5 [DEFAULT] [hasThis] I4
System.Web.Hosting.ISAPIRuntime.ProcessRequest(I,I4)
0x009afaa0 0x79217188 [FRAME: ContextTransitionFrame]
0x009afb80 0x79217188 [FRAME: ComMethodFrame]
Post by Pat [MSFT]
!.\clr10\sos.ClrStack <enter>
Pat
Post by RB
0:016> kb
ChildEBP RetAddr Args to Child
0199f580 79216aed e0434f4d 00000001 00000000 kernel32!RaiseException+0x53
0199f5d8 7924c8d0 183f0fe4 00000000 183f0fd0
mscorsvr!RaiseTheException+0xa0
0199f644 026357e2 183f05ec 183f0754 183f0730 mscorsvr!JIT_Throw+0x4d
WARNING: Frame IP not in any known module. Following frames may be wrong.
0199f670 0263d8f8 182d4034 183f05ec 183f05ec 0x26357e2
0199f6a4 0263d5d6 183e7d78 183f05ec 183f0730 0x263d8f8
0199f6d8 02697e52 183f0cfc 183e7d78 183f0730 0x263d5d6
0199f700 026982b2 183f0638 183f01cc 183649a4 0x2697e52
0199f704 183f0638 183f01cc 183649a4 183647ac 0x26982b2
0199f708 183f01cc 183649a4 183647ac 183f03e4 0x183f0638
0199f70c 183649a4 183647ac 183f03e4 00000000 0x183f01cc
0199f710 183647ac 183f03e4 00000000 02698278 0x183649a4
0199f714 183f03e4 00000000 02698278 02698100 0x183647ac
0199f718 00000000 02698278 02698100 0199f76c 0x183f03e4
0:016> !clrstack
No export clrstack found
I did execute a manual hang dump during the incident, aside from missing
symbols, the analysis for that app does not show any errors.
Thanks,
Rani Baki
Post by Pat [MSFT]
The crash occurred in managed code, apparently on thread 16. If a dump file
..sympath srv*c:\debug*http://msdl.microsoft.com/download/symbols <enter>
2) Reset the debug environment
..reload <enter>
3) Switch to thread 16
~16s <enter>
kb <enter>
5) Dump the managed code stack
!clrstack <enter>
You can post the results and we may be able to identify the issue.
Since you are _also_ experiencing hangs, it is likely that you are having
multiple problems. To diagnose that you need a log/dump while the problem
is actually occurring.
Pat
Post by RB
Hello, we are having issues with web applications hanging frequently. We
eventually setup pools and split up the applications assuming they were
conflicting with each other. We are still experiencing hangs. Below are
excerpts from the dump of one of the hangs. Can someone please help in
analyzing what the cause may be. We are running IIS 6.0 on a Windows 2003
server.
Error In
w3wp__PID__8496__Date__03_09_2006__Time_11_33_02AM__506__First
chance exception 0XC0000005.dmp the assembly instruction at 2d944ba which
does not correspond to any known native module in the process has
caused
an
access violation exception (0xC0000005) when trying to read from memory
location 0x00000000 on thread 16
If no further information can be obtained from the detailed stack analysis
below, please contact Microsoft Corporation for troubleshooting steps on
stack corruption
Error In
w3wp__PID__8496__Date__03_09_2006__Time_11_33_02AM__959__First
chance exception 0XC0000005.dmp the assembly instruction at
kernel32!RaiseException+53 in C:\WINDOWS\system32\kernel32.dll from Microsoft
Corporation has caused an access violation exception (0xC0000005) when trying
to read from memory location 0x00000000 on thread 16
Please follow up with the vendor Microsoft Corporation for
C:\WINDOWS\system32\kernel32.dll
Error In
w3wp__PID__4788__Date__03_09_2006__Time_11_01_24AM__640__First
chance exception 0XE0434F4D.dmp the assembly instruction at
kernel32!RaiseException+53 in C:\WINDOWS\system32\kernel32.dll from Microsoft
Corporation has caused an unknown exception (0xe0434f4d) on thread 19
An exception thrown by Kernel32!RaiseException usually indicates a problem
with another module. Please review the stack for the faulting thread (19)
further to determine which module actually threw the exception raised by
Kernel32.dll.
Detailed stack corruption analysis for thread 16
Call stack with StackWalk
Index Return Address
1 2d944ba
2 2d943ef
3 143f9f88
Call stack - Heuristic
Index Stack Address Child EBP Return Address Destination
1 0x00000000 0x0199f864 2d944ba 0
2 0x0199f868 0x143f9fdc 2d943ef 2d94480
3 0x0199f878 0x1c3b0614 2d93ac4 2d943e0
4 0x0199f888 0x1c3b0614 2d93a04 2d93a60
5 0x0199f8a8 0x40590000 2d907d0 2d93928
6 0x0199f8b8 0x143f9f88 2d90666 2d906a0
7 0x0199f8d0 0x143f9f7c 2d9054c 2d905a8
8 0x0199f8e0 0x143f9c30 2d9040b 2d904f0
9 0x0199f8f4 0x1c3b0614 2d903a0 2d903b8
10 0x0199f900 0x182d300c 2d902af 2d90328
11 0x0199f90c 0x182d300c 2db9e26 0
12 0x0199f948 0x0199f9a0 2db98ab 2db9dd0
13 0x0199f950 0x14390804 2db9884 0
14 0x0199f960 0x14390aec 2d8a4c8 0
15 0x0199f9a4 0x0199f9ec 2d89f32 2d8a470
16 0x0199f9f0 0x0199fa44 2d89e03 2d89e38
17 0x0199fa04 0x18363f78 1ec68a7 0
18 0x0199fa48 0x0199fad4 1ec6458 1ec66c0
19 0x0199fa54 0x1c2d3a28 1ec2fd5 1ec63a8
20 0x0199fa64 0x01a881f3 mscorsvr!ComCallMLStubCache::CompileMLStub+429 0
21 0x0199fbc8 0x0199fca8 1a5a0bd
mscorsvr!ComCallMLStubCache::CompileMLStub+255
22 0x0199fc08 0xffffffff 19c3a04 1a5a06c
23 0x0199fc0c 0x019c3a04
aspnet_isapi!HttpCompletion::ProcessRequestInManagedCode+17e 0
24 0x0199fcbc 0x7c81b26f ntdll!ZwTestAlert+c 0
25 0x0199fd18 0x7c81b23f ntdll!ZwContinue+c 0
26 0x0199fd1c 0x7c8211b4 ntdll!KiUserApcDispatcher+3a ntdll!NtContinue
27 0x0199fdfc 0x00000000 ntdll!NtWaitForSingleObject+c 0
28 0x0199fe00 0x7c822124 kernel32!WaitForSingleObjectEx+ac
ntdll!ZwWaitForSingleObject
29 0x0199fe0c 0x00000000 kernel32!WaitForSingleObjectEx+dc
kernel32!_SEH_epilog
30 0x0199fe3c 0x00000000 aspnet_isapi!PerfDecrementCounter+1c
kernel32!InterlockedDecrement
31 0x0199fe44 0x01862810 aspnet_isapi!PerfDecrementGlobalCounter+f
aspnet_isapi!PerfDecrementCounter
32 0x0199fe60 0x79e8e231 aspnet_isapi!CorThreadPoolWorkitemCallback+13 0
33 0x0199fe74 0x00000000 mscorsvr!ThreadpoolMgr::ExecuteWorkRequest+19 0
34 0x0199fe88 0x0199fea4 mscorsvr!ThreadpoolMgr::WorkerThreadStart+129
mscorsvr!ThreadpoolMgr::ExecuteWorkRequest
35 0x0199fea8 0x0199ffb8
mscorsvr!ThreadpoolMgr::intermediateThreadProc+44
0
36 0x0199ffbc 0x0199ffec kernel32!BaseThreadStart+34 0
^ Memory access error in 'u 0x0000000c l1'
In w3wp__PID__8496__Date__03_09_2006__Time_11_33_02AM__506__First chance
exception 0XC0000005.dmp the assembly instruction at 2d944ba which does not
correspond to any known native module in the process has caused an access
violation exception (0xC0000005) when trying to read from memory location
0x00000000 on thread 16
Report for
w3wp__PID__4788__Date__03_09_2006__Time_11_01_24AM__233__First
chance exception 0XE0434F4D.dmp
Type of Analysis Performed Crash Analysis
Machine Name GPISV205
Operating System Windows Server 2003 Service Pack 1
Number Of Processors 4
Process ID 4788
Process Image c:\WINDOWS\system32\inetsrv\w3wp.exe
System Up-Time 2 day(s) 20:38:19
Process Up-Time 0 day(s) 01:19:04
Thread 19 - System ID 8644
Entry point mscorsvr!ThreadpoolMgr::intermediateThreadProc
Create time 3/9/2006 9:42:21 AM
Time spent in user mode 0 Days 0:0:0.0
Time spent in kernel mode 0 Days 0:0:0.46
Function Source
kernel32!RaiseException+53
mscorsvr!RaiseTheException+a0
mscorsvr!RealCOMPlusThrow+48
mscorsvr!RealCOMPlusThrow+d
mscorsvr!CEEInfo::findMethod+123
mscorjit!Compiler::eeFindMethod+23
mscorjit!Compiler::impImportBlockCode+1e80
mscorjit!Compiler::impImportBlock+222
mscorjit!Compiler::impImport+e8
mscorjit!Compiler::fgImport+41
mscorjit!Compiler::compCompile+b
mscorjit!Compiler::compCompile+1e8
mscorjit!jitNativeCode+95
mscorjit!CILJit::compileMethod+a2
mscorsvr!CallCompileMethodWithSEHWrapper+52
mscorsvr!JITFunction+2c7
mscorsvr!MakeJitWorker+2c0
mscorsvr!MethodDesc::DoPrestub+4d3
mscorsvr!PreStubWorker+42
19c2f76
mscorsvr!MethodDesc::CallDescr+155
mscorsvr!MethodDesc::Call+8e
mscorsvr!COMMember::CreateInstance+2fe
1fd12e9
2bf3dc5
2bf3c74
2bbdb53
2bbda60
2bb990a
2bb97db
1ec6458
mscorsvr!ComCallMLStubCache::CompileMLStub+1af
mscorsvr!Thread::DoADCallBack+5c
mscorsvr!ComCallMLStubCache::CompileMLStub+2ba
1a5a0bd
f3c28
In w3wp__PID__4788__Date__03_09_2006__Time_11_01_24AM__233__First chance
exception 0XE0434F4D.dmp the assembly instruction at
kernel32!RaiseException+53 in C:\WINDOWS\system32\kernel32.dll from Microsoft
Corporation has caused an unknown exception (0xe0434f4d) on thread 19
Thanks,
RB
Pat [MSFT]
2006-03-13 06:31:49 UTC
Permalink
What is _probably_ happening is you have a lot of exceptions happening. So,
as soon as you enable DebugDiag - it trips. Exceptions are not inherently
unstable (i.e. leading to a crash or hang) - though they are excessively
expensive. Code that relies on them for normal operations runs much, much
slower than code that only occasionally hits one.

In thics case, the GCM.Framework.Web.UI.GCMClipboard.OnInit() function
appears to be leading to an exception being thrown. Again, this is not
inherently a problem - just a possible issue.


Pat
Post by RB
I accidently deleted the original dump file, but do have another one for a
different app. This one is faulting on thread 25. Below is the output. Keep
in mind that we have multiple apps hanging. Is it typical for DebugDiag to
generate a first exception almost immediately after activating a rule? That's
what is happening on those troubled apps.
Thanks,
Rani Baki
0:025> kb
ChildEBP RetAddr Args to Child
009aea9c 79216aed e0434f4d 00000001 00000000 kernel32!RaiseException+0x53
009aeaf4 79216a70 142fbfcc 00000000 009aeb38
mscorsvr!RaiseTheException+0xa0
009aeb1c 79216a24 142fbfcc 00000000 009aeb80
mscorsvr!RealCOMPlusThrow+0x48
009aeb2c 7927f019 142fbfcc 009af154 792ba800 mscorsvr!RealCOMPlusThrow+0xd
009aeb80 7943e9ef 009af39c 021e31e8 0a000129
mscorsvr!CEEInfo::findMethod+0x123
009aeb9c 79445a7f 0a000129 021e31e8 02642a28
mscorjit!Compiler::eeFindMethod+0x23
009af0bc 7943fcc4 01eb1e18 01eb1ec4 01eb0010
mscorjit!Compiler::impImportBlockCode+0x1e80
009af164 79440490 01eb1e18 009af3cc 01eb0010
mscorjit!Compiler::impImportBlock+0x222
009af17c 794428b5 01eb1e18 01eb0010 794310e5
mscorjit!Compiler::impImport+0xe8
009af188 794310e5 01eb0010 009af1e0 79431477
mscorjit!Compiler::fgImport+0x41
009af194 79431477 009af2a8 009af424 009af29c
mscorjit!Compiler::compCompile+0xb
009af1e0 7943151a 021e31e8 009af380 009af3cc
mscorjit!Compiler::compCompile+0x1e8
009af268 794315a4 009af380 009af3cc 009af2a8 mscorjit!jitNativeCode+0x95
009af2ac 791bc3f8 79479124 009af380 009af3cc
mscorjit!CILJit::compileMethod+0xa2
009af2f4 791bb52a 0012a100 009af380 009af3cc
mscorsvr!CallCompileMethodWithSEHWrapper+0x52
009af45c 791bb9c0 02642a28 009af624 009af560 mscorsvr!JITFunction+0x2c7
009af590 791b706a 02642a28 009af624 00000000 mscorsvr!MakeJitWorker+0x2c0
009af67c 791b77e4 02642cd4 009afaa0 009af6b8
mscorsvr!MethodDesc::DoPrestub+0x4d3
009af694 019c2f76 009af6b8 14300f5c 1c34b308 mscorsvr!PreStubWorker+0x42
WARNING: Frame IP not in any known module. Following frames may be wrong.
009af6f4 02659e32 00000008 009af734 1c34b114 0x19c2f76
0:025> !.\clr10\sos.ClrStack
Loaded Son of Strike data table version 5 from
"C:\WINDOWS\Microsoft.NET\Framework\v1.1.4322\mscorsvr.dll"
Thread 25
ESP EIP
0x009aeb50 0x77e55dea [FRAME: GCFrame]
0x009af60c 0x77e55dea [FRAME: GCFrame]
0x009af6b8 0x77e55dea [FRAME: PrestubMethodFrame] [DEFAULT] [hasThis] Void
GCM.Framework.Web.UI.GCMClipboard.OnInit(Class System.EventArgs)
0x009af6c8 0x0265db8a [DEFAULT] [hasThis] Void
System.Web.UI.Control.InitRecursive(Class System.Web.UI.Control)
0x009af6e4 0x02659f5a [DEFAULT] [hasThis] Void
System.Web.UI.Control.AddedControl(Class System.Web.UI.Control,I4)
0x009af700 0x02659e32 [DEFAULT] [hasThis] Void
System.Web.UI.ControlCollection.Add(Class System.Web.UI.Control)
0x009af710 0x026b748a [DEFAULT] [hasThis] Void
GCM.Framework.Web.UI.GCMWebPage.AddActionButtons()
0x009af7dc 0x0265e481 [DEFAULT] [hasThis] Void
GCM.Framework.Web.UI.GCMWebPage.CreateChildControls()
0x009af7e8 0x0265e27d [DEFAULT] [hasThis] Void
System.Web.UI.Control.EnsureChildControls()
0x009af810 0x0265dd56 [DEFAULT] [hasThis] Void
GCM.Framework.Web.UI.GCMWebPage.OnInit(Class System.EventArgs)
0x009af824 0x02621572 [DEFAULT] [hasThis] Void
GCM.Framework.Web.SubscriptionPopup.OnInit(Class System.EventArgs)
0x009af830 0x0265db8a [DEFAULT] [hasThis] Void
System.Web.UI.Control.InitRecursive(Class System.Web.UI.Control)
0x009af84c 0x0265c293 [DEFAULT] [hasThis] Void
System.Web.UI.Page.ProcessRequestMain()
0x009af890 0x02659ab7 [DEFAULT] [hasThis] Void
System.Web.UI.Page.ProcessRequest()
0x009af8cc 0x0265951b [DEFAULT] [hasThis] Void
System.Web.UI.Page.ProcessRequest(Class System.Web.HttpContext)
0x009af8d4 0x026594f4 [DEFAULT] [hasThis] Void
System.Web.HttpApplication/CallHandlerExecutionStep.System.Web.HttpApplication+IExecutionStep.Execute()
0x009af8e4 0x02619ea0 [DEFAULT] [hasThis] Class System.Exception
System.Web.HttpApplication.ExecuteStep(Class IExecutionStep,ByRef Boolean)
0x009af92c 0x0261990a [DEFAULT] [hasThis] Void
System.Web.HttpApplication.ResumeSteps(Class System.Exception)
0x009af974 0x026197db [DEFAULT] [hasThis] Class System.IAsyncResult
System.Web.HttpApplication.System.Web.IHttpAsyncHandler.BeginProcessRequest(Class
System.Web.HttpContext,Class System.AsyncCallback,Object)
0x009af990 0x01ec68a7 [DEFAULT] [hasThis] Void
System.Web.HttpRuntime.ProcessRequestInternal(Class
System.Web.HttpWorkerRequest)
0x009af9cc 0x01ec6458 [DEFAULT] Void
System.Web.HttpRuntime.ProcessRequest(Class System.Web.HttpWorkerRequest)
0x009af9d8 0x01ec2fd5 [DEFAULT] [hasThis] I4
System.Web.Hosting.ISAPIRuntime.ProcessRequest(I,I4)
0x009afaa0 0x79217188 [FRAME: ContextTransitionFrame]
0x009afb80 0x79217188 [FRAME: ComMethodFrame]
Post by Pat [MSFT]
!.\clr10\sos.ClrStack <enter>
Pat
Post by RB
0:016> kb
ChildEBP RetAddr Args to Child
0199f580 79216aed e0434f4d 00000001 00000000
kernel32!RaiseException+0x53
0199f5d8 7924c8d0 183f0fe4 00000000 183f0fd0
mscorsvr!RaiseTheException+0xa0
0199f644 026357e2 183f05ec 183f0754 183f0730 mscorsvr!JIT_Throw+0x4d
WARNING: Frame IP not in any known module. Following frames may be wrong.
0199f670 0263d8f8 182d4034 183f05ec 183f05ec 0x26357e2
0199f6a4 0263d5d6 183e7d78 183f05ec 183f0730 0x263d8f8
0199f6d8 02697e52 183f0cfc 183e7d78 183f0730 0x263d5d6
0199f700 026982b2 183f0638 183f01cc 183649a4 0x2697e52
0199f704 183f0638 183f01cc 183649a4 183647ac 0x26982b2
0199f708 183f01cc 183649a4 183647ac 183f03e4 0x183f0638
0199f70c 183649a4 183647ac 183f03e4 00000000 0x183f01cc
0199f710 183647ac 183f03e4 00000000 02698278 0x183649a4
0199f714 183f03e4 00000000 02698278 02698100 0x183647ac
0199f718 00000000 02698278 02698100 0199f76c 0x183f03e4
0:016> !clrstack
No export clrstack found
I did execute a manual hang dump during the incident, aside from missing
symbols, the analysis for that app does not show any errors.
Thanks,
Rani Baki
Post by Pat [MSFT]
The crash occurred in managed code, apparently on thread 16. If a
dump
file
..sympath srv*c:\debug*http://msdl.microsoft.com/download/symbols <enter>
2) Reset the debug environment
..reload <enter>
3) Switch to thread 16
~16s <enter>
kb <enter>
5) Dump the managed code stack
!clrstack <enter>
You can post the results and we may be able to identify the issue.
Since you are _also_ experiencing hangs, it is likely that you are having
multiple problems. To diagnose that you need a log/dump while the problem
is actually occurring.
Pat
Post by RB
Hello, we are having issues with web applications hanging
frequently.
We
eventually setup pools and split up the applications assuming they were
conflicting with each other. We are still experiencing hangs. Below are
excerpts from the dump of one of the hangs. Can someone please help in
analyzing what the cause may be. We are running IIS 6.0 on a Windows 2003
server.
Error In
w3wp__PID__8496__Date__03_09_2006__Time_11_33_02AM__506__First
chance exception 0XC0000005.dmp the assembly instruction at 2d944ba which
does not correspond to any known native module in the process has
caused
an
access violation exception (0xC0000005) when trying to read from memory
location 0x00000000 on thread 16
If no further information can be obtained from the detailed stack analysis
below, please contact Microsoft Corporation for troubleshooting
steps
on
stack corruption
Error In
w3wp__PID__8496__Date__03_09_2006__Time_11_33_02AM__959__First
chance exception 0XC0000005.dmp the assembly instruction at
kernel32!RaiseException+53 in C:\WINDOWS\system32\kernel32.dll from Microsoft
Corporation has caused an access violation exception (0xC0000005)
when
trying
to read from memory location 0x00000000 on thread 16
Please follow up with the vendor Microsoft Corporation for
C:\WINDOWS\system32\kernel32.dll
Error In
w3wp__PID__4788__Date__03_09_2006__Time_11_01_24AM__640__First
chance exception 0XE0434F4D.dmp the assembly instruction at
kernel32!RaiseException+53 in C:\WINDOWS\system32\kernel32.dll from Microsoft
Corporation has caused an unknown exception (0xe0434f4d) on thread 19
An exception thrown by Kernel32!RaiseException usually indicates a problem
with another module. Please review the stack for the faulting thread (19)
further to determine which module actually threw the exception
raised
by
Kernel32.dll.
Detailed stack corruption analysis for thread 16
Call stack with StackWalk
Index Return Address
1 2d944ba
2 2d943ef
3 143f9f88
Call stack - Heuristic
Index Stack Address Child EBP Return Address Destination
1 0x00000000 0x0199f864 2d944ba 0
2 0x0199f868 0x143f9fdc 2d943ef 2d94480
3 0x0199f878 0x1c3b0614 2d93ac4 2d943e0
4 0x0199f888 0x1c3b0614 2d93a04 2d93a60
5 0x0199f8a8 0x40590000 2d907d0 2d93928
6 0x0199f8b8 0x143f9f88 2d90666 2d906a0
7 0x0199f8d0 0x143f9f7c 2d9054c 2d905a8
8 0x0199f8e0 0x143f9c30 2d9040b 2d904f0
9 0x0199f8f4 0x1c3b0614 2d903a0 2d903b8
10 0x0199f900 0x182d300c 2d902af 2d90328
11 0x0199f90c 0x182d300c 2db9e26 0
12 0x0199f948 0x0199f9a0 2db98ab 2db9dd0
13 0x0199f950 0x14390804 2db9884 0
14 0x0199f960 0x14390aec 2d8a4c8 0
15 0x0199f9a4 0x0199f9ec 2d89f32 2d8a470
16 0x0199f9f0 0x0199fa44 2d89e03 2d89e38
17 0x0199fa04 0x18363f78 1ec68a7 0
18 0x0199fa48 0x0199fad4 1ec6458 1ec66c0
19 0x0199fa54 0x1c2d3a28 1ec2fd5 1ec63a8
20 0x0199fa64 0x01a881f3
mscorsvr!ComCallMLStubCache::CompileMLStub+429
0
21 0x0199fbc8 0x0199fca8 1a5a0bd
mscorsvr!ComCallMLStubCache::CompileMLStub+255
22 0x0199fc08 0xffffffff 19c3a04 1a5a06c
23 0x0199fc0c 0x019c3a04
aspnet_isapi!HttpCompletion::ProcessRequestInManagedCode+17e 0
24 0x0199fcbc 0x7c81b26f ntdll!ZwTestAlert+c 0
25 0x0199fd18 0x7c81b23f ntdll!ZwContinue+c 0
26 0x0199fd1c 0x7c8211b4 ntdll!KiUserApcDispatcher+3a
ntdll!NtContinue
27 0x0199fdfc 0x00000000 ntdll!NtWaitForSingleObject+c 0
28 0x0199fe00 0x7c822124 kernel32!WaitForSingleObjectEx+ac
ntdll!ZwWaitForSingleObject
29 0x0199fe0c 0x00000000 kernel32!WaitForSingleObjectEx+dc
kernel32!_SEH_epilog
30 0x0199fe3c 0x00000000 aspnet_isapi!PerfDecrementCounter+1c
kernel32!InterlockedDecrement
31 0x0199fe44 0x01862810 aspnet_isapi!PerfDecrementGlobalCounter+f
aspnet_isapi!PerfDecrementCounter
32 0x0199fe60 0x79e8e231
aspnet_isapi!CorThreadPoolWorkitemCallback+13
0
33 0x0199fe74 0x00000000
mscorsvr!ThreadpoolMgr::ExecuteWorkRequest+19
0
34 0x0199fe88 0x0199fea4
mscorsvr!ThreadpoolMgr::WorkerThreadStart+129
mscorsvr!ThreadpoolMgr::ExecuteWorkRequest
35 0x0199fea8 0x0199ffb8
mscorsvr!ThreadpoolMgr::intermediateThreadProc+44
0
36 0x0199ffbc 0x0199ffec kernel32!BaseThreadStart+34 0
^ Memory access error in 'u 0x0000000c l1'
In w3wp__PID__8496__Date__03_09_2006__Time_11_33_02AM__506__First chance
exception 0XC0000005.dmp the assembly instruction at 2d944ba which
does
not
correspond to any known native module in the process has caused an access
violation exception (0xC0000005) when trying to read from memory location
0x00000000 on thread 16
Report for
w3wp__PID__4788__Date__03_09_2006__Time_11_01_24AM__233__First
chance exception 0XE0434F4D.dmp
Type of Analysis Performed Crash Analysis
Machine Name GPISV205
Operating System Windows Server 2003 Service Pack 1
Number Of Processors 4
Process ID 4788
Process Image c:\WINDOWS\system32\inetsrv\w3wp.exe
System Up-Time 2 day(s) 20:38:19
Process Up-Time 0 day(s) 01:19:04
Thread 19 - System ID 8644
Entry point mscorsvr!ThreadpoolMgr::intermediateThreadProc
Create time 3/9/2006 9:42:21 AM
Time spent in user mode 0 Days 0:0:0.0
Time spent in kernel mode 0 Days 0:0:0.46
Function Source
kernel32!RaiseException+53
mscorsvr!RaiseTheException+a0
mscorsvr!RealCOMPlusThrow+48
mscorsvr!RealCOMPlusThrow+d
mscorsvr!CEEInfo::findMethod+123
mscorjit!Compiler::eeFindMethod+23
mscorjit!Compiler::impImportBlockCode+1e80
mscorjit!Compiler::impImportBlock+222
mscorjit!Compiler::impImport+e8
mscorjit!Compiler::fgImport+41
mscorjit!Compiler::compCompile+b
mscorjit!Compiler::compCompile+1e8
mscorjit!jitNativeCode+95
mscorjit!CILJit::compileMethod+a2
mscorsvr!CallCompileMethodWithSEHWrapper+52
mscorsvr!JITFunction+2c7
mscorsvr!MakeJitWorker+2c0
mscorsvr!MethodDesc::DoPrestub+4d3
mscorsvr!PreStubWorker+42
19c2f76
mscorsvr!MethodDesc::CallDescr+155
mscorsvr!MethodDesc::Call+8e
mscorsvr!COMMember::CreateInstance+2fe
1fd12e9
2bf3dc5
2bf3c74
2bbdb53
2bbda60
2bb990a
2bb97db
1ec6458
mscorsvr!ComCallMLStubCache::CompileMLStub+1af
mscorsvr!Thread::DoADCallBack+5c
mscorsvr!ComCallMLStubCache::CompileMLStub+2ba
1a5a0bd
f3c28
In w3wp__PID__4788__Date__03_09_2006__Time_11_01_24AM__233__First chance
exception 0XE0434F4D.dmp the assembly instruction at
kernel32!RaiseException+53 in C:\WINDOWS\system32\kernel32.dll from Microsoft
Corporation has caused an unknown exception (0xe0434f4d) on thread 19
Thanks,
RB
RB
2006-03-13 14:53:57 UTC
Permalink
If this may not be the problem, how can we pin point what the issue is. Can I
use DebugDiag differently to get that data? Should I use a different tool?

Thanks for your help.

Rani Baki
Post by Pat [MSFT]
What is _probably_ happening is you have a lot of exceptions happening. So,
as soon as you enable DebugDiag - it trips. Exceptions are not inherently
unstable (i.e. leading to a crash or hang) - though they are excessively
expensive. Code that relies on them for normal operations runs much, much
slower than code that only occasionally hits one.
In thics case, the GCM.Framework.Web.UI.GCMClipboard.OnInit() function
appears to be leading to an exception being thrown. Again, this is not
inherently a problem - just a possible issue.
Pat
Post by RB
I accidently deleted the original dump file, but do have another one for a
different app. This one is faulting on thread 25. Below is the output. Keep
in mind that we have multiple apps hanging. Is it typical for DebugDiag to
generate a first exception almost immediately after activating a rule? That's
what is happening on those troubled apps.
Thanks,
Rani Baki
0:025> kb
ChildEBP RetAddr Args to Child
009aea9c 79216aed e0434f4d 00000001 00000000 kernel32!RaiseException+0x53
009aeaf4 79216a70 142fbfcc 00000000 009aeb38
mscorsvr!RaiseTheException+0xa0
009aeb1c 79216a24 142fbfcc 00000000 009aeb80
mscorsvr!RealCOMPlusThrow+0x48
009aeb2c 7927f019 142fbfcc 009af154 792ba800 mscorsvr!RealCOMPlusThrow+0xd
009aeb80 7943e9ef 009af39c 021e31e8 0a000129
mscorsvr!CEEInfo::findMethod+0x123
009aeb9c 79445a7f 0a000129 021e31e8 02642a28
mscorjit!Compiler::eeFindMethod+0x23
009af0bc 7943fcc4 01eb1e18 01eb1ec4 01eb0010
mscorjit!Compiler::impImportBlockCode+0x1e80
009af164 79440490 01eb1e18 009af3cc 01eb0010
mscorjit!Compiler::impImportBlock+0x222
009af17c 794428b5 01eb1e18 01eb0010 794310e5
mscorjit!Compiler::impImport+0xe8
009af188 794310e5 01eb0010 009af1e0 79431477
mscorjit!Compiler::fgImport+0x41
009af194 79431477 009af2a8 009af424 009af29c
mscorjit!Compiler::compCompile+0xb
009af1e0 7943151a 021e31e8 009af380 009af3cc
mscorjit!Compiler::compCompile+0x1e8
009af268 794315a4 009af380 009af3cc 009af2a8 mscorjit!jitNativeCode+0x95
009af2ac 791bc3f8 79479124 009af380 009af3cc
mscorjit!CILJit::compileMethod+0xa2
009af2f4 791bb52a 0012a100 009af380 009af3cc
mscorsvr!CallCompileMethodWithSEHWrapper+0x52
009af45c 791bb9c0 02642a28 009af624 009af560 mscorsvr!JITFunction+0x2c7
009af590 791b706a 02642a28 009af624 00000000 mscorsvr!MakeJitWorker+0x2c0
009af67c 791b77e4 02642cd4 009afaa0 009af6b8
mscorsvr!MethodDesc::DoPrestub+0x4d3
009af694 019c2f76 009af6b8 14300f5c 1c34b308 mscorsvr!PreStubWorker+0x42
WARNING: Frame IP not in any known module. Following frames may be wrong.
009af6f4 02659e32 00000008 009af734 1c34b114 0x19c2f76
0:025> !.\clr10\sos.ClrStack
Loaded Son of Strike data table version 5 from
"C:\WINDOWS\Microsoft.NET\Framework\v1.1.4322\mscorsvr.dll"
Thread 25
ESP EIP
0x009aeb50 0x77e55dea [FRAME: GCFrame]
0x009af60c 0x77e55dea [FRAME: GCFrame]
0x009af6b8 0x77e55dea [FRAME: PrestubMethodFrame] [DEFAULT] [hasThis] Void
GCM.Framework.Web.UI.GCMClipboard.OnInit(Class System.EventArgs)
0x009af6c8 0x0265db8a [DEFAULT] [hasThis] Void
System.Web.UI.Control.InitRecursive(Class System.Web.UI.Control)
0x009af6e4 0x02659f5a [DEFAULT] [hasThis] Void
System.Web.UI.Control.AddedControl(Class System.Web.UI.Control,I4)
0x009af700 0x02659e32 [DEFAULT] [hasThis] Void
System.Web.UI.ControlCollection.Add(Class System.Web.UI.Control)
0x009af710 0x026b748a [DEFAULT] [hasThis] Void
GCM.Framework.Web.UI.GCMWebPage.AddActionButtons()
0x009af7dc 0x0265e481 [DEFAULT] [hasThis] Void
GCM.Framework.Web.UI.GCMWebPage.CreateChildControls()
0x009af7e8 0x0265e27d [DEFAULT] [hasThis] Void
System.Web.UI.Control.EnsureChildControls()
0x009af810 0x0265dd56 [DEFAULT] [hasThis] Void
GCM.Framework.Web.UI.GCMWebPage.OnInit(Class System.EventArgs)
0x009af824 0x02621572 [DEFAULT] [hasThis] Void
GCM.Framework.Web.SubscriptionPopup.OnInit(Class System.EventArgs)
0x009af830 0x0265db8a [DEFAULT] [hasThis] Void
System.Web.UI.Control.InitRecursive(Class System.Web.UI.Control)
0x009af84c 0x0265c293 [DEFAULT] [hasThis] Void
System.Web.UI.Page.ProcessRequestMain()
0x009af890 0x02659ab7 [DEFAULT] [hasThis] Void
System.Web.UI.Page.ProcessRequest()
0x009af8cc 0x0265951b [DEFAULT] [hasThis] Void
System.Web.UI.Page.ProcessRequest(Class System.Web.HttpContext)
0x009af8d4 0x026594f4 [DEFAULT] [hasThis] Void
System.Web.HttpApplication/CallHandlerExecutionStep.System.Web.HttpApplication+IExecutionStep.Execute()
0x009af8e4 0x02619ea0 [DEFAULT] [hasThis] Class System.Exception
System.Web.HttpApplication.ExecuteStep(Class IExecutionStep,ByRef Boolean)
0x009af92c 0x0261990a [DEFAULT] [hasThis] Void
System.Web.HttpApplication.ResumeSteps(Class System.Exception)
0x009af974 0x026197db [DEFAULT] [hasThis] Class System.IAsyncResult
System.Web.HttpApplication.System.Web.IHttpAsyncHandler.BeginProcessRequest(Class
System.Web.HttpContext,Class System.AsyncCallback,Object)
0x009af990 0x01ec68a7 [DEFAULT] [hasThis] Void
System.Web.HttpRuntime.ProcessRequestInternal(Class
System.Web.HttpWorkerRequest)
0x009af9cc 0x01ec6458 [DEFAULT] Void
System.Web.HttpRuntime.ProcessRequest(Class System.Web.HttpWorkerRequest)
0x009af9d8 0x01ec2fd5 [DEFAULT] [hasThis] I4
System.Web.Hosting.ISAPIRuntime.ProcessRequest(I,I4)
0x009afaa0 0x79217188 [FRAME: ContextTransitionFrame]
0x009afb80 0x79217188 [FRAME: ComMethodFrame]
Post by Pat [MSFT]
!.\clr10\sos.ClrStack <enter>
Pat
Post by RB
0:016> kb
ChildEBP RetAddr Args to Child
0199f580 79216aed e0434f4d 00000001 00000000
kernel32!RaiseException+0x53
0199f5d8 7924c8d0 183f0fe4 00000000 183f0fd0
mscorsvr!RaiseTheException+0xa0
0199f644 026357e2 183f05ec 183f0754 183f0730 mscorsvr!JIT_Throw+0x4d
WARNING: Frame IP not in any known module. Following frames may be wrong.
0199f670 0263d8f8 182d4034 183f05ec 183f05ec 0x26357e2
0199f6a4 0263d5d6 183e7d78 183f05ec 183f0730 0x263d8f8
0199f6d8 02697e52 183f0cfc 183e7d78 183f0730 0x263d5d6
0199f700 026982b2 183f0638 183f01cc 183649a4 0x2697e52
0199f704 183f0638 183f01cc 183649a4 183647ac 0x26982b2
0199f708 183f01cc 183649a4 183647ac 183f03e4 0x183f0638
0199f70c 183649a4 183647ac 183f03e4 00000000 0x183f01cc
0199f710 183647ac 183f03e4 00000000 02698278 0x183649a4
0199f714 183f03e4 00000000 02698278 02698100 0x183647ac
0199f718 00000000 02698278 02698100 0199f76c 0x183f03e4
0:016> !clrstack
No export clrstack found
I did execute a manual hang dump during the incident, aside from missing
symbols, the analysis for that app does not show any errors.
Thanks,
Rani Baki
Post by Pat [MSFT]
The crash occurred in managed code, apparently on thread 16. If a
dump
file
..sympath srv*c:\debug*http://msdl.microsoft.com/download/symbols <enter>
2) Reset the debug environment
..reload <enter>
3) Switch to thread 16
~16s <enter>
kb <enter>
5) Dump the managed code stack
!clrstack <enter>
You can post the results and we may be able to identify the issue.
Since you are _also_ experiencing hangs, it is likely that you are having
multiple problems. To diagnose that you need a log/dump while the problem
is actually occurring.
Pat
Post by RB
Hello, we are having issues with web applications hanging
frequently.
We
eventually setup pools and split up the applications assuming they were
conflicting with each other. We are still experiencing hangs. Below are
excerpts from the dump of one of the hangs. Can someone please help in
analyzing what the cause may be. We are running IIS 6.0 on a Windows 2003
server.
Error In
w3wp__PID__8496__Date__03_09_2006__Time_11_33_02AM__506__First
chance exception 0XC0000005.dmp the assembly instruction at 2d944ba which
does not correspond to any known native module in the process has
caused
an
access violation exception (0xC0000005) when trying to read from memory
location 0x00000000 on thread 16
If no further information can be obtained from the detailed stack analysis
below, please contact Microsoft Corporation for troubleshooting
steps
on
stack corruption
Error In
w3wp__PID__8496__Date__03_09_2006__Time_11_33_02AM__959__First
chance exception 0XC0000005.dmp the assembly instruction at
kernel32!RaiseException+53 in C:\WINDOWS\system32\kernel32.dll from
Microsoft
Corporation has caused an access violation exception (0xC0000005)
when
trying
to read from memory location 0x00000000 on thread 16
Please follow up with the vendor Microsoft Corporation for
C:\WINDOWS\system32\kernel32.dll
Error In
w3wp__PID__4788__Date__03_09_2006__Time_11_01_24AM__640__First
chance exception 0XE0434F4D.dmp the assembly instruction at
kernel32!RaiseException+53 in C:\WINDOWS\system32\kernel32.dll from
Microsoft
Corporation has caused an unknown exception (0xe0434f4d) on thread 19
An exception thrown by Kernel32!RaiseException usually indicates a problem
with another module. Please review the stack for the faulting thread (19)
further to determine which module actually threw the exception
raised
by
Kernel32.dll.
Detailed stack corruption analysis for thread 16
Call stack with StackWalk
Index Return Address
1 2d944ba
2 2d943ef
3 143f9f88
Call stack - Heuristic
Index Stack Address Child EBP Return Address Destination
1 0x00000000 0x0199f864 2d944ba 0
2 0x0199f868 0x143f9fdc 2d943ef 2d94480
3 0x0199f878 0x1c3b0614 2d93ac4 2d943e0
4 0x0199f888 0x1c3b0614 2d93a04 2d93a60
5 0x0199f8a8 0x40590000 2d907d0 2d93928
6 0x0199f8b8 0x143f9f88 2d90666 2d906a0
7 0x0199f8d0 0x143f9f7c 2d9054c 2d905a8
8 0x0199f8e0 0x143f9c30 2d9040b 2d904f0
9 0x0199f8f4 0x1c3b0614 2d903a0 2d903b8
10 0x0199f900 0x182d300c 2d902af 2d90328
11 0x0199f90c 0x182d300c 2db9e26 0
12 0x0199f948 0x0199f9a0 2db98ab 2db9dd0
13 0x0199f950 0x14390804 2db9884 0
14 0x0199f960 0x14390aec 2d8a4c8 0
15 0x0199f9a4 0x0199f9ec 2d89f32 2d8a470
16 0x0199f9f0 0x0199fa44 2d89e03 2d89e38
17 0x0199fa04 0x18363f78 1ec68a7 0
18 0x0199fa48 0x0199fad4 1ec6458 1ec66c0
19 0x0199fa54 0x1c2d3a28 1ec2fd5 1ec63a8
20 0x0199fa64 0x01a881f3
mscorsvr!ComCallMLStubCache::CompileMLStub+429
0
21 0x0199fbc8 0x0199fca8 1a5a0bd
mscorsvr!ComCallMLStubCache::CompileMLStub+255
22 0x0199fc08 0xffffffff 19c3a04 1a5a06c
23 0x0199fc0c 0x019c3a04
aspnet_isapi!HttpCompletion::ProcessRequestInManagedCode+17e 0
24 0x0199fcbc 0x7c81b26f ntdll!ZwTestAlert+c 0
Pat [MSFT]
2006-03-13 19:40:17 UTC
Permalink
You can try IISState. It doesn't fault on every exception.

Just run it when the problem is actually happening (hang):

iisstate -p <pid of w3wp.exe> -d <enter>


pat
Post by RB
If this may not be the problem, how can we pin point what the issue is. Can I
use DebugDiag differently to get that data? Should I use a different tool?
Thanks for your help.
Rani Baki
Post by Pat [MSFT]
What is _probably_ happening is you have a lot of exceptions happening.
So,
as soon as you enable DebugDiag - it trips. Exceptions are not inherently
unstable (i.e. leading to a crash or hang) - though they are excessively
expensive. Code that relies on them for normal operations runs much, much
slower than code that only occasionally hits one.
In thics case, the GCM.Framework.Web.UI.GCMClipboard.OnInit() function
appears to be leading to an exception being thrown. Again, this is not
inherently a problem - just a possible issue.
Pat
Post by RB
I accidently deleted the original dump file, but do have another one for a
different app. This one is faulting on thread 25. Below is the output. Keep
in mind that we have multiple apps hanging. Is it typical for DebugDiag to
generate a first exception almost immediately after activating a rule? That's
what is happening on those troubled apps.
Thanks,
Rani Baki
0:025> kb
ChildEBP RetAddr Args to Child
009aea9c 79216aed e0434f4d 00000001 00000000
kernel32!RaiseException+0x53
009aeaf4 79216a70 142fbfcc 00000000 009aeb38
mscorsvr!RaiseTheException+0xa0
009aeb1c 79216a24 142fbfcc 00000000 009aeb80
mscorsvr!RealCOMPlusThrow+0x48
009aeb2c 7927f019 142fbfcc 009af154 792ba800
mscorsvr!RealCOMPlusThrow+0xd
009aeb80 7943e9ef 009af39c 021e31e8 0a000129
mscorsvr!CEEInfo::findMethod+0x123
009aeb9c 79445a7f 0a000129 021e31e8 02642a28
mscorjit!Compiler::eeFindMethod+0x23
009af0bc 7943fcc4 01eb1e18 01eb1ec4 01eb0010
mscorjit!Compiler::impImportBlockCode+0x1e80
009af164 79440490 01eb1e18 009af3cc 01eb0010
mscorjit!Compiler::impImportBlock+0x222
009af17c 794428b5 01eb1e18 01eb0010 794310e5
mscorjit!Compiler::impImport+0xe8
009af188 794310e5 01eb0010 009af1e0 79431477
mscorjit!Compiler::fgImport+0x41
009af194 79431477 009af2a8 009af424 009af29c
mscorjit!Compiler::compCompile+0xb
009af1e0 7943151a 021e31e8 009af380 009af3cc
mscorjit!Compiler::compCompile+0x1e8
009af268 794315a4 009af380 009af3cc 009af2a8
mscorjit!jitNativeCode+0x95
009af2ac 791bc3f8 79479124 009af380 009af3cc
mscorjit!CILJit::compileMethod+0xa2
009af2f4 791bb52a 0012a100 009af380 009af3cc
mscorsvr!CallCompileMethodWithSEHWrapper+0x52
009af45c 791bb9c0 02642a28 009af624 009af560 mscorsvr!JITFunction+0x2c7
009af590 791b706a 02642a28 009af624 00000000
mscorsvr!MakeJitWorker+0x2c0
009af67c 791b77e4 02642cd4 009afaa0 009af6b8
mscorsvr!MethodDesc::DoPrestub+0x4d3
009af694 019c2f76 009af6b8 14300f5c 1c34b308
mscorsvr!PreStubWorker+0x42
WARNING: Frame IP not in any known module. Following frames may be wrong.
009af6f4 02659e32 00000008 009af734 1c34b114 0x19c2f76
0:025> !.\clr10\sos.ClrStack
Loaded Son of Strike data table version 5 from
"C:\WINDOWS\Microsoft.NET\Framework\v1.1.4322\mscorsvr.dll"
Thread 25
ESP EIP
0x009aeb50 0x77e55dea [FRAME: GCFrame]
0x009af60c 0x77e55dea [FRAME: GCFrame]
0x009af6b8 0x77e55dea [FRAME: PrestubMethodFrame] [DEFAULT] [hasThis] Void
GCM.Framework.Web.UI.GCMClipboard.OnInit(Class System.EventArgs)
0x009af6c8 0x0265db8a [DEFAULT] [hasThis] Void
System.Web.UI.Control.InitRecursive(Class System.Web.UI.Control)
0x009af6e4 0x02659f5a [DEFAULT] [hasThis] Void
System.Web.UI.Control.AddedControl(Class System.Web.UI.Control,I4)
0x009af700 0x02659e32 [DEFAULT] [hasThis] Void
System.Web.UI.ControlCollection.Add(Class System.Web.UI.Control)
0x009af710 0x026b748a [DEFAULT] [hasThis] Void
GCM.Framework.Web.UI.GCMWebPage.AddActionButtons()
0x009af7dc 0x0265e481 [DEFAULT] [hasThis] Void
GCM.Framework.Web.UI.GCMWebPage.CreateChildControls()
0x009af7e8 0x0265e27d [DEFAULT] [hasThis] Void
System.Web.UI.Control.EnsureChildControls()
0x009af810 0x0265dd56 [DEFAULT] [hasThis] Void
GCM.Framework.Web.UI.GCMWebPage.OnInit(Class System.EventArgs)
0x009af824 0x02621572 [DEFAULT] [hasThis] Void
GCM.Framework.Web.SubscriptionPopup.OnInit(Class System.EventArgs)
0x009af830 0x0265db8a [DEFAULT] [hasThis] Void
System.Web.UI.Control.InitRecursive(Class System.Web.UI.Control)
0x009af84c 0x0265c293 [DEFAULT] [hasThis] Void
System.Web.UI.Page.ProcessRequestMain()
0x009af890 0x02659ab7 [DEFAULT] [hasThis] Void
System.Web.UI.Page.ProcessRequest()
0x009af8cc 0x0265951b [DEFAULT] [hasThis] Void
System.Web.UI.Page.ProcessRequest(Class System.Web.HttpContext)
0x009af8d4 0x026594f4 [DEFAULT] [hasThis] Void
System.Web.HttpApplication/CallHandlerExecutionStep.System.Web.HttpApplication+IExecutionStep.Execute()
0x009af8e4 0x02619ea0 [DEFAULT] [hasThis] Class System.Exception
System.Web.HttpApplication.ExecuteStep(Class IExecutionStep,ByRef Boolean)
0x009af92c 0x0261990a [DEFAULT] [hasThis] Void
System.Web.HttpApplication.ResumeSteps(Class System.Exception)
0x009af974 0x026197db [DEFAULT] [hasThis] Class System.IAsyncResult
System.Web.HttpApplication.System.Web.IHttpAsyncHandler.BeginProcessRequest(Class
System.Web.HttpContext,Class System.AsyncCallback,Object)
0x009af990 0x01ec68a7 [DEFAULT] [hasThis] Void
System.Web.HttpRuntime.ProcessRequestInternal(Class
System.Web.HttpWorkerRequest)
0x009af9cc 0x01ec6458 [DEFAULT] Void
System.Web.HttpRuntime.ProcessRequest(Class
System.Web.HttpWorkerRequest)
0x009af9d8 0x01ec2fd5 [DEFAULT] [hasThis] I4
System.Web.Hosting.ISAPIRuntime.ProcessRequest(I,I4)
0x009afaa0 0x79217188 [FRAME: ContextTransitionFrame]
0x009afb80 0x79217188 [FRAME: ComMethodFrame]
Post by Pat [MSFT]
!.\clr10\sos.ClrStack <enter>
Pat
Post by RB
0:016> kb
ChildEBP RetAddr Args to Child
0199f580 79216aed e0434f4d 00000001 00000000
kernel32!RaiseException+0x53
0199f5d8 7924c8d0 183f0fe4 00000000 183f0fd0
mscorsvr!RaiseTheException+0xa0
0199f644 026357e2 183f05ec 183f0754 183f0730 mscorsvr!JIT_Throw+0x4d
WARNING: Frame IP not in any known module. Following frames may be wrong.
0199f670 0263d8f8 182d4034 183f05ec 183f05ec 0x26357e2
0199f6a4 0263d5d6 183e7d78 183f05ec 183f0730 0x263d8f8
0199f6d8 02697e52 183f0cfc 183e7d78 183f0730 0x263d5d6
0199f700 026982b2 183f0638 183f01cc 183649a4 0x2697e52
0199f704 183f0638 183f01cc 183649a4 183647ac 0x26982b2
0199f708 183f01cc 183649a4 183647ac 183f03e4 0x183f0638
0199f70c 183649a4 183647ac 183f03e4 00000000 0x183f01cc
0199f710 183647ac 183f03e4 00000000 02698278 0x183649a4
0199f714 183f03e4 00000000 02698278 02698100 0x183647ac
0199f718 00000000 02698278 02698100 0199f76c 0x183f03e4
0:016> !clrstack
No export clrstack found
I did execute a manual hang dump during the incident, aside from missing
symbols, the analysis for that app does not show any errors.
Thanks,
Rani Baki
Post by Pat [MSFT]
The crash occurred in managed code, apparently on thread 16. If a
dump
file
..sympath srv*c:\debug*http://msdl.microsoft.com/download/symbols <enter>
2) Reset the debug environment
..reload <enter>
3) Switch to thread 16
~16s <enter>
kb <enter>
5) Dump the managed code stack
!clrstack <enter>
You can post the results and we may be able to identify the issue.
Since you are _also_ experiencing hangs, it is likely that you are having
multiple problems. To diagnose that you need a log/dump while the problem
is actually occurring.
Pat
Post by RB
Hello, we are having issues with web applications hanging
frequently.
We
eventually setup pools and split up the applications assuming
they
were
conflicting with each other. We are still experiencing hangs.
Below
are
excerpts from the dump of one of the hangs. Can someone please
help
in
analyzing what the cause may be. We are running IIS 6.0 on a
Windows
2003
server.
Error In
w3wp__PID__8496__Date__03_09_2006__Time_11_33_02AM__506__First
chance exception 0XC0000005.dmp the assembly instruction at
2d944ba
which
does not correspond to any known native module in the process has
caused
an
access violation exception (0xC0000005) when trying to read from memory
location 0x00000000 on thread 16
If no further information can be obtained from the detailed stack
analysis
below, please contact Microsoft Corporation for troubleshooting
steps
on
stack corruption
Error In
w3wp__PID__8496__Date__03_09_2006__Time_11_33_02AM__959__First
chance exception 0XC0000005.dmp the assembly instruction at
kernel32!RaiseException+53 in C:\WINDOWS\system32\kernel32.dll from
Microsoft
Corporation has caused an access violation exception (0xC0000005)
when
trying
to read from memory location 0x00000000 on thread 16
Please follow up with the vendor Microsoft Corporation for
C:\WINDOWS\system32\kernel32.dll
Error In
w3wp__PID__4788__Date__03_09_2006__Time_11_01_24AM__640__First
chance exception 0XE0434F4D.dmp the assembly instruction at
kernel32!RaiseException+53 in C:\WINDOWS\system32\kernel32.dll from
Microsoft
Corporation has caused an unknown exception (0xe0434f4d) on
thread
19
An exception thrown by Kernel32!RaiseException usually indicates
a
problem
with another module. Please review the stack for the faulting
thread
(19)
further to determine which module actually threw the exception
raised
by
Kernel32.dll.
Detailed stack corruption analysis for thread 16
Call stack with StackWalk
Index Return Address
1 2d944ba
2 2d943ef
3 143f9f88
Call stack - Heuristic
Index Stack Address Child EBP Return Address Destination
1 0x00000000 0x0199f864 2d944ba 0
2 0x0199f868 0x143f9fdc 2d943ef 2d94480
3 0x0199f878 0x1c3b0614 2d93ac4 2d943e0
4 0x0199f888 0x1c3b0614 2d93a04 2d93a60
5 0x0199f8a8 0x40590000 2d907d0 2d93928
6 0x0199f8b8 0x143f9f88 2d90666 2d906a0
7 0x0199f8d0 0x143f9f7c 2d9054c 2d905a8
8 0x0199f8e0 0x143f9c30 2d9040b 2d904f0
9 0x0199f8f4 0x1c3b0614 2d903a0 2d903b8
10 0x0199f900 0x182d300c 2d902af 2d90328
11 0x0199f90c 0x182d300c 2db9e26 0
12 0x0199f948 0x0199f9a0 2db98ab 2db9dd0
13 0x0199f950 0x14390804 2db9884 0
14 0x0199f960 0x14390aec 2d8a4c8 0
15 0x0199f9a4 0x0199f9ec 2d89f32 2d8a470
16 0x0199f9f0 0x0199fa44 2d89e03 2d89e38
17 0x0199fa04 0x18363f78 1ec68a7 0
18 0x0199fa48 0x0199fad4 1ec6458 1ec66c0
19 0x0199fa54 0x1c2d3a28 1ec2fd5 1ec63a8
20 0x0199fa64 0x01a881f3
mscorsvr!ComCallMLStubCache::CompileMLStub+429
0
21 0x0199fbc8 0x0199fca8 1a5a0bd
mscorsvr!ComCallMLStubCache::CompileMLStub+255
22 0x0199fc08 0xffffffff 19c3a04 1a5a06c
23 0x0199fc0c 0x019c3a04
aspnet_isapi!HttpCompletion::ProcessRequestInManagedCode+17e 0
24 0x0199fcbc 0x7c81b26f ntdll!ZwTestAlert+c 0
Rani Baki
2006-03-14 16:00:27 UTC
Permalink
I ran iisstate during an hang. It froze on thread 9, but eventually continued
and then it looks like it failed during thread 16/17. Maybe because the
worker process gave up. I also ran it using the -hc option and it hung on a
dll, could this dll be the culprit? I am including one output below and the
other on the next post.

Thanks,
Rani Baki


C:\iisstate>iisstate -p 5592 -d
Symbol search path is:
SRV*C:\iisstate\symbols*http://msdl.microsoft.com/download/symbols

Microsoft (R) Windows Debugger Version 6.2.0013.1
Copyright (c) Microsoft Corporation. All rights reserved.

*** wait with pending attach
Symbol search path is:
SRV*C:\iisstate\symbols*http://msdl.microsoft.com/download/symbols
Executable search path is:
WARNING: Process 5592 is not attached as a debuggee
The process can be examined but debug events will not be received
..................................................................................................................................................
The call to LoadLibrary(exts) failed, Win32 error 2
"The system cannot find the file specified."
Please check your debugger configuration and/or network access.
(15d8.2068): Wake debugger - code 80000007 (first chance)
Opened log file 'C:\iisstate\output\IISState-5592.log'

***********************
Starting new log output
IISState version 3.3.1

Tue Mar 14 10:06:51 2006

OS = Windows 2003 Server
Executable: w3wp.exe
PID = 5592

Note: Thread times are formatted as HH:MM:SS.ms

***********************




Thread ID: 0
System Thread ID: 2068
Kernel Time: 0:0:0.31
User Time: 0:0:0.31
Thread Status: Thread is in a WAIT state.
Thread Type: HTTP Compression Thread
# ChildEBP RetAddr
00 0006fc08 7c822124 ntdll!KiFastSystemCallRet
01 0006fc0c 77e6baa8 ntdll!NtWaitForSingleObject+0xc
02 0006fc7c 77e6ba12 kernel32!WaitForSingleObjectEx+0xac
03 0006fc90 5a36467a kernel32!WaitForSingleObject+0x12
04 0006fca0 5a366e63 w3dt!WP_CONTEXT::RunMainThreadLoop+0x10
05 0006fca8 5a3af41d w3dt!UlAtqStartListen+0x2d
06 0006fcb8 5a3bc259 w3core!W3_SERVER::StartListen+0xbd
07 0006ff0c 0100187c w3core!UlW3Start+0x26e
08 0006ff44 01001a23 w3wp!wmain+0x22a
09 0006ffc0 77e523cd w3wp!wmainCRTStartup+0x12b
0a 0006fff0 00000000 kernel32!BaseProcessStart+0x23




Thread ID: 1
System Thread ID: 16c0
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Type: Other
# ChildEBP RetAddr
00 009eff9c 7c821364 ntdll!KiFastSystemCallRet
01 009effa0 7c81fe26 ntdll!NtDelayExecution+0xc
02 009effb8 77e66063 ntdll!RtlpTimerThread+0x47
03 009effec 00000000 kernel32!BaseThreadStart+0x34




Thread ID: 2
System Thread ID: b10
Kernel Time: 0:0:0.15
User Time: 0:0:0.0
Thread Type: HTTP Listener
# ChildEBP RetAddr
00 00c8ff24 7c821bf4 ntdll!KiFastSystemCallRet
01 00c8ff28 77e6611a ntdll!NtRemoveIoCompletion+0xc
02 00c8ff54 5a30249e kernel32!GetQueuedCompletionStatus+0x29
03 00c8ff8c 5a3026bc W3TP!THREAD_POOL_DATA::ThreadPoolThread+0x33
04 00c8ffa0 5a301db9 W3TP!THREAD_POOL_DATA::ThreadPoolThread+0x24
05 00c8ffb8 77e66063 W3TP!THREAD_MANAGER::ThreadManagerThread+0x39
06 00c8ffec 00000000 kernel32!BaseThreadStart+0x34




Thread ID: 3
System Thread ID: 11cc
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Type: HTTP Listener
# ChildEBP RetAddr
00 00ccff24 7c821bf4 ntdll!KiFastSystemCallRet
01 00ccff28 77e6611a ntdll!NtRemoveIoCompletion+0xc
02 00ccff54 5a30249e kernel32!GetQueuedCompletionStatus+0x29
03 00ccff8c 5a3026bc W3TP!THREAD_POOL_DATA::ThreadPoolThread+0x33
04 00ccffa0 5a301db9 W3TP!THREAD_POOL_DATA::ThreadPoolThread+0x24
05 00ccffb8 77e66063 W3TP!THREAD_MANAGER::ThreadManagerThread+0x39
06 00ccffec 00000000 kernel32!BaseThreadStart+0x34




Thread ID: 4
System Thread ID: 2110
Kernel Time: 0:0:0.15
User Time: 0:0:0.0
Thread Type: HTTP Listener
# ChildEBP RetAddr
00 00d0ff24 7c821bf4 ntdll!KiFastSystemCallRet
01 00d0ff28 77e6611a ntdll!NtRemoveIoCompletion+0xc
02 00d0ff54 5a30249e kernel32!GetQueuedCompletionStatus+0x29
03 00d0ff8c 5a3026bc W3TP!THREAD_POOL_DATA::ThreadPoolThread+0x33
04 00d0ffa0 5a301db9 W3TP!THREAD_POOL_DATA::ThreadPoolThread+0x24
05 00d0ffb8 77e66063 W3TP!THREAD_MANAGER::ThreadManagerThread+0x39
06 00d0ffec 00000000 kernel32!BaseThreadStart+0x34




Thread ID: 5
System Thread ID: 19bc
Kernel Time: 0:0:0.15
User Time: 0:0:0.0
Thread Type: HTTP Listener
# ChildEBP RetAddr
00 00d4ff24 7c821bf4 ntdll!KiFastSystemCallRet
01 00d4ff28 77e6611a ntdll!NtRemoveIoCompletion+0xc
02 00d4ff54 5a30249e kernel32!GetQueuedCompletionStatus+0x29
03 00d4ff8c 5a3026bc W3TP!THREAD_POOL_DATA::ThreadPoolThread+0x33
04 00d4ffa0 5a301db9 W3TP!THREAD_POOL_DATA::ThreadPoolThread+0x24
05 00d4ffb8 77e66063 W3TP!THREAD_MANAGER::ThreadManagerThread+0x39
06 00d4ffec 00000000 kernel32!BaseThreadStart+0x34




Thread ID: 6
System Thread ID: 12e8
Kernel Time: 0:0:0.15
User Time: 0:0:0.0
Thread Status: Thread is in a WAIT state.
Thread Type: Other
# ChildEBP RetAddr
00 00e0fcec 7c822114 ntdll!KiFastSystemCallRet
01 00e0fcf0 7c83acfd ntdll!NtWaitForMultipleObjects+0xc
02 00e0ffb8 77e66063 ntdll!RtlpWaitThread+0x161
03 00e0ffec 00000000 kernel32!BaseThreadStart+0x34




Thread ID: 7
System Thread ID: 10e0
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Status: Thread is in a WAIT state.
Thread Type: HTTP Compression Thread
# ChildEBP RetAddr
00 017dfa84 7c822124 ntdll!KiFastSystemCallRet
01 017dfa88 77e6baa8 ntdll!NtWaitForSingleObject+0xc
02 017dfaf8 77e6ba12 kernel32!WaitForSingleObjectEx+0xac
03 017dfb0c 5a3b8147 kernel32!WaitForSingleObject+0x12
04 017dffb8 77e66063 w3core!HTTP_COMPRESSION::CompressionThread+0x126
05 017dffec 00000000 kernel32!BaseThreadStart+0x34




Thread ID: 8
System Thread ID: 10a4
Kernel Time: 0:0:0.15
User Time: 0:0:0.0
Thread Type: Other
# ChildEBP RetAddr
00 0185ff8c 7c821364 ntdll!KiFastSystemCallRet
01 0185ff90 7c815267 ntdll!NtDelayExecution+0xc
02 0185ffb8 77e66063 ntdll!RtlpIOWorkerThread+0x3f
03 0185ffec 00000000 kernel32!BaseThreadStart+0x34




Thread ID: 9
System Thread ID: 288
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Type: Other
# ChildEBP RetAddr
00 0191fec0 7c821bf4 ntdll!KiFastSystemCallRet
01 0191fec4 77e6611a ntdll!NtRemoveIoCompletion+0xc
02 0191fef0 791fdb22 kernel32!GetQueuedCompletionStatus+0x29
03 0191ff24 791fdacc mscorsvr+0x4db22
04 0191ffb8 77e66063 mscorsvr+0x4dacc
05 0191ffec 00000000 kernel32!BaseThreadStart+0x34




Thread ID: 10
System Thread ID: 1a4
Kernel Time: 0:0:0.15
User Time: 0:0:0.0
Thread Type: Other
# ChildEBP RetAddr
00 0195fe50 7c821364 ntdll!KiFastSystemCallRet
01 0195fe54 77e42439 ntdll!NtDelayExecution+0xc
02 0195febc 77e424b7 kernel32!SleepEx+0x68
03 0195fecc 791bf4f9 kernel32!Sleep+0xf
04 0195ffb8 77e66063 mscorsvr+0xf4f9
05 0195ffec 00000000 kernel32!BaseThreadStart+0x34




Thread ID: 11
System Thread ID: 178
Kernel Time: 0:0:0.765
User Time: 0:0:5.453
Thread Status: Thread is in a WAIT state.
Thread Type: Other
# ChildEBP RetAddr
00 0199fdf8 7c822124 ntdll!KiFastSystemCallRet
01 0199fdfc 77e6baa8 ntdll!NtWaitForSingleObject+0xc
02 0199fe6c 77e6ba12 kernel32!WaitForSingleObjectEx+0xac
03 0199fe80 791d401f kernel32!WaitForSingleObject+0x12
04 0199fea4 791fdacc mscorsvr+0x2401f
05 0199ffb8 77e66063 mscorsvr+0x4dacc
06 0199ffec 00000000 kernel32!BaseThreadStart+0x34




Thread ID: 12
System Thread ID: 265c
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Status: Thread is in a WAIT state.
Thread Type: Other
# ChildEBP RetAddr
00 01acfe70 7c822114 ntdll!KiFastSystemCallRet
01 01acfe74 77e6711b ntdll!NtWaitForMultipleObjects+0xc
02 01acff1c 77e61075 kernel32!WaitForMultipleObjectsEx+0x11a
03 01acff38 79238fd6 kernel32!WaitForMultipleObjects+0x18
04 01acffa0 79238f4d mscorsvr+0x88fd6
05 01acffb0 7923a714 mscorsvr+0x88f4d
06 01acffb8 77e66063 mscorsvr+0x8a714
07 01acffec 00000000 kernel32!BaseThreadStart+0x34




Thread ID: 13
System Thread ID: 1fe4
Kernel Time: 0:0:0.46
User Time: 0:0:0.906
Thread Status: Thread is in a WAIT state.
Thread Type: Other
# ChildEBP RetAddr
00 01d5fefc 7c822124 ntdll!KiFastSystemCallRet
01 01d5ff00 77e6baa8 ntdll!NtWaitForSingleObject+0xc
02 01d5ff70 77e6ba12 kernel32!WaitForSingleObjectEx+0xac
03 01d5ff84 791fe6b0 kernel32!WaitForSingleObject+0x12
04 01d5ffac 792356bf mscorsvr+0x4e6b0
05 01d5ffb8 77e66063 mscorsvr+0x856bf
06 01d5ffec 00000000 kernel32!BaseThreadStart+0x34




Thread ID: 14
System Thread ID: 239c
Kernel Time: 0:0:0.31
User Time: 0:0:1.93
Thread Status: Thread is in a WAIT state.
Thread Type: Other
# ChildEBP RetAddr
00 01d9fdfc 7c822124 ntdll!KiFastSystemCallRet
01 01d9fe00 77e6baa8 ntdll!NtWaitForSingleObject+0xc
02 01d9fe70 77e6ba12 kernel32!WaitForSingleObjectEx+0xac
03 01d9fe84 791fe6b0 kernel32!WaitForSingleObject+0x12
04 01d9feac 792356bf mscorsvr+0x4e6b0
05 01d9ffb8 77e66063 mscorsvr+0x856bf
06 01d9ffec 00000000 kernel32!BaseThreadStart+0x34




Thread ID: 15
System Thread ID: 11f4
Kernel Time: 0:0:0.46
User Time: 0:0:0.843
Thread Status: Thread is in a WAIT state.
Thread Type: Other
# ChildEBP RetAddr
00 01ddfcfc 7c822124 ntdll!KiFastSystemCallRet
01 01ddfd00 77e6baa8 ntdll!NtWaitForSingleObject+0xc
02 01ddfd70 77e6ba12 kernel32!WaitForSingleObjectEx+0xac
03 01ddfd84 791fe6b0 kernel32!WaitForSingleObject+0x12
04 01ddfdac 792356bf mscorsvr+0x4e6b0
05 01ddffb8 77e66063 mscorsvr+0x856bf
06 01ddffec 00000000 kernel32!BaseThreadStart+0x34




Thread ID: 16
System Thread ID: 1b2c
Kernel Time: 0:0:0.0
User Time: 0:0:1.140
Thread Status: Thread is in a WAIT state.
Thread Type: Other
# ChildEBP RetAddr
00 01e1fbfc 7c822124 ntdll!KiFastSystemCallRet
01 01e1fc00 77e6baa8 ntdll!NtWaitForSingleObject+0xc
02 01e1fc70 77e6ba12 kernel32!WaitForSingleObjectEx+0xac
03 01e1fc84 791fe6b0 kernel32!WaitForSingleObject+0x12
04 01e1fcac 792356bf mscorsvr+0x4e6b0
05 01e1ffb8 77e66063 mscorsvr+0x856bf
06 01e1ffec 00000000 kernel32!BaseThreadStart+0x34
GetContextState failed, 0x8007001F
GetContextState failed, 0x8007001F
GetContextState failed, 0x8007001F




Thread ID: 17
System Thread ID: 0
Kernel Time: 0:0:0.15
User Time: 0:0:0.15
GetContextState failed, 0x8007001F
Post by Pat [MSFT]
You can try IISState. It doesn't fault on every exception.
iisstate -p <pid of w3wp.exe> -d <enter>
pat
Post by RB
If this may not be the problem, how can we pin point what the issue is. Can I
use DebugDiag differently to get that data? Should I use a different tool?
Thanks for your help.
Rani Baki
Post by Pat [MSFT]
What is _probably_ happening is you have a lot of exceptions happening.
So,
as soon as you enable DebugDiag - it trips. Exceptions are not inherently
unstable (i.e. leading to a crash or hang) - though they are excessively
expensive. Code that relies on them for normal operations runs much, much
slower than code that only occasionally hits one.
In thics case, the GCM.Framework.Web.UI.GCMClipboard.OnInit() function
appears to be leading to an exception being thrown. Again, this is not
inherently a problem - just a possible issue.
Pat
Post by RB
I accidently deleted the original dump file, but do have another one for a
different app. This one is faulting on thread 25. Below is the output. Keep
in mind that we have multiple apps hanging. Is it typical for DebugDiag to
generate a first exception almost immediately after activating a rule? That's
what is happening on those troubled apps.
Thanks,
Rani Baki
0:025> kb
ChildEBP RetAddr Args to Child
009aea9c 79216aed e0434f4d 00000001 00000000
kernel32!RaiseException+0x53
009aeaf4 79216a70 142fbfcc 00000000 009aeb38
mscorsvr!RaiseTheException+0xa0
009aeb1c 79216a24 142fbfcc 00000000 009aeb80
mscorsvr!RealCOMPlusThrow+0x48
009aeb2c 7927f019 142fbfcc 009af154 792ba800
mscorsvr!RealCOMPlusThrow+0xd
009aeb80 7943e9ef 009af39c 021e31e8 0a000129
mscorsvr!CEEInfo::findMethod+0x123
009aeb9c 79445a7f 0a000129 021e31e8 02642a28
mscorjit!Compiler::eeFindMethod+0x23
009af0bc 7943fcc4 01eb1e18 01eb1ec4 01eb0010
mscorjit!Compiler::impImportBlockCode+0x1e80
009af164 79440490 01eb1e18 009af3cc 01eb0010
mscorjit!Compiler::impImportBlock+0x222
009af17c 794428b5 01eb1e18 01eb0010 794310e5
mscorjit!Compiler::impImport+0xe8
009af188 794310e5 01eb0010 009af1e0 79431477
mscorjit!Compiler::fgImport+0x41
009af194 79431477 009af2a8 009af424 009af29c
mscorjit!Compiler::compCompile+0xb
009af1e0 7943151a 021e31e8 009af380 009af3cc
mscorjit!Compiler::compCompile+0x1e8
009af268 794315a4 009af380 009af3cc 009af2a8
mscorjit!jitNativeCode+0x95
009af2ac 791bc3f8 79479124 009af380 009af3cc
mscorjit!CILJit::compileMethod+0xa2
009af2f4 791bb52a 0012a100 009af380 009af3cc
mscorsvr!CallCompileMethodWithSEHWrapper+0x52
009af45c 791bb9c0 02642a28 009af624 009af560 mscorsvr!JITFunction+0x2c7
009af590 791b706a 02642a28 009af624 00000000
mscorsvr!MakeJitWorker+0x2c0
009af67c 791b77e4 02642cd4 009afaa0 009af6b8
mscorsvr!MethodDesc::DoPrestub+0x4d3
009af694 019c2f76 009af6b8 14300f5c 1c34b308
mscorsvr!PreStubWorker+0x42
WARNING: Frame IP not in any known module. Following frames may be wrong.
009af6f4 02659e32 00000008 009af734 1c34b114 0x19c2f76
0:025> !.\clr10\sos.ClrStack
Loaded Son of Strike data table version 5 from
"C:\WINDOWS\Microsoft.NET\Framework\v1.1.4322\mscorsvr.dll"
Thread 25
ESP EIP
0x009aeb50 0x77e55dea [FRAME: GCFrame]
0x009af60c 0x77e55dea [FRAME: GCFrame]
0x009af6b8 0x77e55dea [FRAME: PrestubMethodFrame] [DEFAULT] [hasThis] Void
GCM.Framework.Web.UI.GCMClipboard.OnInit(Class System.EventArgs)
0x009af6c8 0x0265db8a [DEFAULT] [hasThis] Void
System.Web.UI.Control.InitRecursive(Class System.Web.UI.Control)
0x009af6e4 0x02659f5a [DEFAULT] [hasThis] Void
System.Web.UI.Control.AddedControl(Class System.Web.UI.Control,I4)
0x009af700 0x02659e32 [DEFAULT] [hasThis] Void
System.Web.UI.ControlCollection.Add(Class System.Web.UI.Control)
0x009af710 0x026b748a [DEFAULT] [hasThis] Void
GCM.Framework.Web.UI.GCMWebPage.AddActionButtons()
0x009af7dc 0x0265e481 [DEFAULT] [hasThis] Void
GCM.Framework.Web.UI.GCMWebPage.CreateChildControls()
0x009af7e8 0x0265e27d [DEFAULT] [hasThis] Void
System.Web.UI.Control.EnsureChildControls()
0x009af810 0x0265dd56 [DEFAULT] [hasThis] Void
GCM.Framework.Web.UI.GCMWebPage.OnInit(Class System.EventArgs)
0x009af824 0x02621572 [DEFAULT] [hasThis] Void
GCM.Framework.Web.SubscriptionPopup.OnInit(Class System.EventArgs)
0x009af830 0x0265db8a [DEFAULT] [hasThis] Void
System.Web.UI.Control.InitRecursive(Class System.Web.UI.Control)
0x009af84c 0x0265c293 [DEFAULT] [hasThis] Void
System.Web.UI.Page.ProcessRequestMain()
0x009af890 0x02659ab7 [DEFAULT] [hasThis] Void
System.Web.UI.Page.ProcessRequest()
0x009af8cc 0x0265951b [DEFAULT] [hasThis] Void
System.Web.UI.Page.ProcessRequest(Class System.Web.HttpContext)
0x009af8d4 0x026594f4 [DEFAULT] [hasThis] Void
System.Web.HttpApplication/CallHandlerExecutionStep.System.Web.HttpApplication+IExecutionStep.Execute()
0x009af8e4 0x02619ea0 [DEFAULT] [hasThis] Class System.Exception
System.Web.HttpApplication.ExecuteStep(Class IExecutionStep,ByRef Boolean)
0x009af92c 0x0261990a [DEFAULT] [hasThis] Void
System.Web.HttpApplication.ResumeSteps(Class System.Exception)
0x009af974 0x026197db [DEFAULT] [hasThis] Class System.IAsyncResult
System.Web.HttpApplication.System.Web.IHttpAsyncHandler.BeginProcessRequest(Class
System.Web.HttpContext,Class System.AsyncCallback,Object)
0x009af990 0x01ec68a7 [DEFAULT] [hasThis] Void
System.Web.HttpRuntime.ProcessRequestInternal(Class
System.Web.HttpWorkerRequest)
0x009af9cc 0x01ec6458 [DEFAULT] Void
System.Web.HttpRuntime.ProcessRequest(Class
System.Web.HttpWorkerRequest)
0x009af9d8 0x01ec2fd5 [DEFAULT] [hasThis] I4
System.Web.Hosting.ISAPIRuntime.ProcessRequest(I,I4)
0x009afaa0 0x79217188 [FRAME: ContextTransitionFrame]
0x009afb80 0x79217188 [FRAME: ComMethodFrame]
Post by Pat [MSFT]
!.\clr10\sos.ClrStack <enter>
Pat
Post by RB
0:016> kb
ChildEBP RetAddr Args to Child
0199f580 79216aed e0434f4d 00000001 00000000
kernel32!RaiseException+0x53
0199f5d8 7924c8d0 183f0fe4 00000000 183f0fd0
mscorsvr!RaiseTheException+0xa0
0199f644 026357e2 183f05ec 183f0754 183f0730 mscorsvr!JIT_Throw+0x4d
WARNING: Frame IP not in any known module. Following frames may be wrong.
0199f670 0263d8f8 182d4034 183f05ec 183f05ec 0x26357e2
0199f6a4 0263d5d6 183e7d78 183f05ec 183f0730 0x263d8f8
0199f6d8 02697e52 183f0cfc 183e7d78 183f0730 0x263d5d6
0199f700 026982b2 183f0638 183f01cc 183649a4 0x2697e52
0199f704 183f0638 183f01cc 183649a4 183647ac 0x26982b2
0199f708 183f01cc 183649a4 183647ac 183f03e4 0x183f0638
0199f70c 183649a4 183647ac 183f03e4 00000000 0x183f01cc
0199f710 183647ac 183f03e4 00000000 02698278 0x183649a4
0199f714 183f03e4 00000000 02698278 02698100 0x183647ac
0199f718 00000000 02698278 02698100 0199f76c 0x183f03e4
0:016> !clrstack
No export clrstack found
I did execute a manual hang dump during the incident, aside from missing
symbols, the analysis for that app does not show any errors.
Thanks,
Rani Baki
Post by Pat [MSFT]
The crash occurred in managed code, apparently on thread 16. If a
dump
file
..sympath srv*c:\debug*http://msdl.microsoft.com/download/symbols <enter>
2) Reset the debug environment
..reload <enter>
3) Switch to thread 16
~16s <enter>
kb <enter>
5) Dump the managed code stack
!clrstack <enter>
You can post the results and we may be able to identify the issue.
Since you are _also_ experiencing hangs, it is likely that you are having
multiple problems. To diagnose that you need a log/dump while the
problem
is actually occurring.
Pat
Post by RB
Hello, we are having issues with web applications hanging
frequently.
We
eventually setup pools and split up the applications assuming
they
were
conflicting with each other. We are still experiencing hangs.
Below
are
excerpts from the dump of one of the hangs. Can someone please
help
in
analyzing what the cause may be. We are running IIS 6.0 on a
Windows
2003
server.
Error In
w3wp__PID__8496__Date__03_09_2006__Time_11_33_02AM__506__First
chance exception 0XC0000005.dmp the assembly instruction at
2d944ba
which
does not correspond to any known native module in the process has
caused
an
access violation exception (0xC0000005) when trying to read from
memory
location 0x00000000 on thread 16
If no further information can be obtained from the detailed stack
analysis
below, please contact Microsoft Corporation for troubleshooting
steps
on
stack corruption
Error In
w3wp__PID__8496__Date__03_09_2006__Time_11_33_02AM__959__First
chance exception 0XC0000005.dmp the assembly instruction at
kernel32!RaiseException+53 in C:\WINDOWS\system32\kernel32.dll from
Microsoft
Corporation has caused an access violation exception (0xC0000005)
when
trying
to read from memory location 0x00000000 on thread 16
Please follow up with the vendor Microsoft Corporation for
C:\WINDOWS\system32\kernel32.dll
Error In
w3wp__PID__4788__Date__03_09_2006__Time_11_01_24AM__640__First
chance exception 0XE0434F4D.dmp the assembly instruction at
kernel32!RaiseException+53 in C:\WINDOWS\system32\kernel32.dll from
Microsoft
Corporation has caused an unknown exception (0xe0434f4d) on
thread
19
An exception thrown by Kernel32!RaiseException usually indicates
a
problem
with another module. Please review the stack for the faulting
thread
(19)
further to determine which module actually threw the exception
raised
by
Kernel32.dll.
Rani Baki
2006-03-14 16:10:28 UTC
Permalink
Here's the iisstate dump with the -hc option:

C:\iisstate>iisstate -p 5592 -hc -d
Symbol search path is:
SRV*C:\iisstate\symbols*http://msdl.microsoft.com/downloa
d/symbols

Microsoft (R) Windows Debugger Version 6.2.0013.1
Copyright (c) Microsoft Corporation. All rights reserved.

sxd av
WARNING: The debugger does not have a current process or thread
WARNING: Many commands will not work
*** wait with pending attach
The call to LoadLibrary(exts) failed, Win32 error 2
"The system cannot find the file specified."
Please check your debugger configuration and/or network access.
Symbol search path is:
SRV*C:\iisstate\symbols*http://msdl.microsoft.com/downloa
d/symbols
Executable search path is:
ModLoad: 01000000 01005000 c:\windows\system32\inetsrv\w3wp.exe
ModLoad: 7c800000 7c8c0000 C:\WINDOWS\system32\ntdll.dll
ModLoad: 77e40000 77f42000 C:\WINDOWS\system32\kernel32.dll
ModLoad: 77ba0000 77bfa000 C:\WINDOWS\system32\msvcrt.dll
ModLoad: 77f50000 77fec000 C:\WINDOWS\system32\ADVAPI32.dll
ModLoad: 77c50000 77cef000 C:\WINDOWS\system32\RPCRT4.dll
ModLoad: 77380000 77412000 C:\WINDOWS\system32\USER32.dll
ModLoad: 77c00000 77c48000 C:\WINDOWS\system32\GDI32.dll
ModLoad: 77670000 777a4000 C:\WINDOWS\system32\ole32.dll
ModLoad: 64700000 6472d000 c:\windows\system32\inetsrv\IISUTIL.dll
ModLoad: 761b0000 76243000 C:\WINDOWS\system32\CRYPT32.dll
ModLoad: 76190000 761a2000 C:\WINDOWS\system32\MSASN1.dll
ModLoad: 10000000 102c5000 C:\WINDOWS\system32\xpsp2res.dll
ModLoad: 5a390000 5a3e8000 c:\windows\system32\inetsrv\w3core.dll
ModLoad: 5a420000 5a431000 C:\WINDOWS\system32\IISMAP.dll
ModLoad: 647b0000 647d7000 C:\WINDOWS\system32\IisRTL.DLL
ModLoad: 71c00000 71c17000 C:\WINDOWS\system32\WS2_32.dll
ModLoad: 71bf0000 71bf8000 C:\WINDOWS\system32\WS2HELP.dll
ModLoad: 62da0000 62da7000 c:\windows\system32\inetsrv\LONSINT.dll
ModLoad: 76f50000 76f63000 C:\WINDOWS\system32\Secur32.dll
ModLoad: 71c40000 71c98000 C:\WINDOWS\system32\NETAPI32.dll
ModLoad: 77d00000 77d8c000 C:\WINDOWS\system32\OLEAUT32.dll
ModLoad: 5b640000 5b658000 C:\WINDOWS\system32\strmfilt.dll
ModLoad: 67150000 67159000 C:\WINDOWS\system32\HTTPAPI.dll
ModLoad: 77da0000 77df2000 C:\WINDOWS\system32\SHLWAPI.dll
ModLoad: 5a400000 5a408000 c:\windows\system32\inetsrv\W3CACHE.dll
ModLoad: 5a300000 5a307000 c:\windows\system32\inetsrv\W3TP.dll
ModLoad: 5a3f0000 5a3f6000 c:\windows\system32\inetsrv\W3COMLOG.dll
ModLoad: 5a360000 5a36d000 c:\windows\system32\inetsrv\w3dt.dll
ModLoad: 60ba0000 60bb1000 c:\windows\system32\inetsrv\wamreg.DLL
ModLoad: 71bb0000 71bb9000 C:\WINDOWS\system32\WSOCK32.dll
ModLoad: 6d0f0000 6d110000 c:\windows\system32\inetsrv\iisres.dll
ModLoad: 777b0000 77833000 C:\WINDOWS\system32\CLBCatQ.DLL
ModLoad: 77010000 770d6000 C:\WINDOWS\system32\COMRes.dll
ModLoad: 77b90000 77b98000 C:\WINDOWS\system32\VERSION.dll
ModLoad: 608f0000 60901000 C:\WINDOWS\system32\ADMWPROX.DLL
ModLoad: 68000000 6802f000 C:\WINDOWS\system32\rsaenh.dll
ModLoad: 76b70000 76b7b000 C:\WINDOWS\system32\PSAPI.DLL
ModLoad: 79e50000 79e56000
C:\WINDOWS\Microsoft.NET\Framework\v1.1.4322\aspnet
_filter.dll
ModLoad: 7c340000 7c396000
C:\WINDOWS\Microsoft.NET\Framework\v1.1.4322\MSVCR7
1.dll
ModLoad: 5a320000 5a332000 c:\windows\system32\inetsrv\w3isapi.dll
ModLoad: 71b20000 71b61000 C:\WINDOWS\System32\mswsock.dll
ModLoad: 76ed0000 76ef9000 C:\WINDOWS\system32\DNSAPI.dll
ModLoad: 76f70000 76f77000 C:\WINDOWS\System32\winrnr.dll
ModLoad: 76f10000 76f3e000 C:\WINDOWS\system32\WLDAP32.dll
ModLoad: 76f80000 76f85000 C:\WINDOWS\system32\rasadhlp.dll
ModLoad: 685b0000 685ba000 C:\WINDOWS\system32\inetsrv\gzip.dll
ModLoad: 71bd0000 71be1000 C:\WINDOWS\system32\mpr.dll
ModLoad: 79e60000 79ea2000
\\?\C:\WINDOWS\Microsoft.NET\Framework\v1.1.4322\as
pnet_isapi.dll
ModLoad: 76920000 769e4000 C:\WINDOWS\system32\USERENV.dll
ModLoad: 79170000 79198000 C:\WINDOWS\system32\mscoree.dll
ModLoad: 791b0000 79419000
C:\WINDOWS\Microsoft.NET\Framework\v1.1.4322\mscors
vr.dll
ModLoad: 79040000 79085000
C:\WINDOWS\Microsoft.NET\Framework\v1.1.4322\fusion
.dll
ModLoad: 7c8d0000 7d0d3000 C:\WINDOWS\system32\SHELL32.dll
ModLoad: 77420000 77523000
C:\WINDOWS\WinSxS\x86_Microsoft.Windows.Common-Cont
rols_6595b64144ccf1df_6.0.3790.1830_x-ww_7AE38CCF\comctl32.dll
ModLoad: 79780000 7998c000
c:\windows\microsoft.net\framework\v1.1.4322\mscorl
ib.dll
ModLoad: 79990000 79cce000
c:\windows\assembly\nativeimages1_v1.1.4322\mscorli
b\1.0.5000.0__b77a5c561934e089_9ab75ea9\mscorlib.dll
ModLoad: 7a090000 7a1c8000
c:\windows\assembly\gac\system.web\1.0.5000.0__b03f
5f7f11d50a3a\system.web.dll
ModLoad: 7b0a0000 7b1ce000
c:\windows\assembly\gac\system\1.0.5000.0__b77a5c56
1934e089\system.dll
ModLoad: 7b490000 7b506000
c:\windows\assembly\gac\system.drawing\1.0.5000.0__
b03f5f7f11d50a3a\system.drawing.dll
ModLoad: 7bc10000 7bd5e000
c:\windows\assembly\gac\system.xml\1.0.5000.0__b77a
5c561934e089\system.xml.dll
ModLoad: 7a6e0000 7a820000
c:\windows\assembly\gac\system.data\1.0.5000.0__b77
a5c561934e089\system.data.dll
ModLoad: 79000000 7903e000
c:\windows\assembly\gac\system.enterpriseservices\1
.0.5000.0__b03f5f7f11d50a3a\system.enterpriseservices.dll
ModLoad: 7a2a0000 7a2b2000
c:\windows\assembly\gac\system.web.regularexpressio
ns\1.0.5000.0__b03f5f7f11d50a3a\system.web.regularexpressions.dll
ModLoad: 01e70000 01e75000
c:\windows\assembly\gac\microsoft.visualc\7.0.5000.
0__b03f5f7f11d50a3a\microsoft.visualc.dll
ModLoad: 7a000000 7a01a000
c:\windows\assembly\gac\system.directoryservices\1.
0.5000.0__b03f5f7f11d50a3a\system.directoryservices.dll
ModLoad: 79640000 79692000
c:\windows\assembly\gac\system.runtime.remoting\1.0
.5000.0__b77a5c561934e089\system.runtime.remoting.dll
ModLoad: 79550000 79574000
c:\windows\assembly\gac\system.runtime.serializatio
n.formatters.soap\1.0.5000.0__b03f5f7f11d50a3a\system.runtime.serialization.form
atters.soap.dll
ModLoad: 53760000 5376a000
c:\windows\assembly\gac\cscompmgd\7.0.5000.0__b03f5
f7f11d50a3a\cscompmgd.dll
ModLoad: 79430000 7947d000
C:\WINDOWS\Microsoft.NET\Framework\v1.1.4322\MSCORJ
IT.DLL
ModLoad: 02260000 02270000
c:\windows\microsoft.net\framework\v1.1.4322\tempor
ary asp.net
files\products\bfa52050\27b5eb55\assembly\dl2\dec2d5fd\f0587714_1116
c601\chartdataengine.dll
ModLoad: 02280000 02296000
c:\windows\microsoft.net\framework\v1.1.4322\tempor
ary asp.net
files\products\bfa52050\27b5eb55\assembly\dl2\f42c429f\c636da4a_6f15
c601\chartformatter.dll
ModLoad: 79510000 79523000
C:\WINDOWS\Microsoft.NET\Framework\v1.1.4322\mscors
n.dll
ModLoad: 022a0000 0231c000
c:\windows\microsoft.net\framework\v1.1.4322\tempor
ary asp.net
files\products\bfa52050\27b5eb55\assembly\dl2\5be8c14d\7afb1759_f0c9
c501\componentart.web.ui.dll
ModLoad: 02320000 02454000
c:\windows\microsoft.net\framework\v1.1.4322\tempor
ary asp.net
files\products\bfa52050\27b5eb55\assembly\dl2\db34d8fe\005124c2_529b
c401\dundaswebchart.dll
ModLoad: 02470000 02478000
c:\windows\microsoft.net\framework\v1.1.4322\tempor
ary asp.net
files\products\bfa52050\27b5eb55\assembly\dl2\37d9293d\b47a4b3f_abcc
c401\gcmchartfunctionbuilder.dll
ModLoad: 02480000 02492000
c:\windows\microsoft.net\framework\v1.1.4322\tempor
ary asp.net
files\products\bfa52050\27b5eb55\assembly\dl2\a2551b9e\7d4a099b_54f4
c501\gcmdocument.dll
ModLoad: 024a0000 0258a000
c:\windows\microsoft.net\framework\v1.1.4322\tempor
ary asp.net
files\products\bfa52050\27b5eb55\assembly\dl2\f45dabdf\1ba29f5e_ba6d
c501\gcmframeworknet.dll
ModLoad: 02590000 025a1000 C:\WINDOWS\system32\pwrpc32.dll
ModLoad: 7c3a0000 7c41b000 C:\WINDOWS\system32\MSVCP71.dll
ModLoad: 025b0000 02612000 C:\Progra~1\Sybase\12.5\OCS-12_5\dll\libsybdb.dll
ModLoad: 02620000 0264b000 C:\Progra~1\Sybase\12.5\OCS-12_5\dll\libtcl.dll
ModLoad: 02650000 0265b000 C:\Progra~1\Sybase\12.5\OCS-12_5\dll\libintl.dll
ModLoad: 02660000 026c7000 C:\Progra~1\Sybase\12.5\OCS-12_5\dll\libcomn.dll
ModLoad: 026d0000 026de000 C:\Progra~1\Sybase\12.5\OCS-12_5\dll\libcs.dll
ModLoad: 026e0000 02745000 C:\Progra~1\Sybase\12.5\OCS-12_5\dll\libct.dll
ModLoad: 02790000 027c6000
c:\windows\microsoft.net\framework\v1.1.4322\tempor
ary asp.net
files\products\bfa52050\27b5eb55\assembly\dl2\4dc4428b\70566cf5_7d02
c601\gcmwebui.dll
ModLoad: 027d0000 027da000
c:\windows\microsoft.net\framework\v1.1.4322\tempor
ary asp.net
files\products\bfa52050\27b5eb55\assembly\dl2\eb6832c8\00d05520_4fa5
c301\math_expression_evaluator.dll
ModLoad: 027e0000 02822000
c:\windows\microsoft.net\framework\v1.1.4322\tempor
ary asp.net
files\products\bfa52050\27b5eb55\assembly\dl2\b9de8e72\baf8efb2_2341
c601\products.dll
ModLoad: 02840000 0284c000
c:\windows\microsoft.net\framework\v1.1.4322\tempor
ary asp.net
files\products\bfa52050\27b5eb55\assembly\dl2\53c0176d\f83d494c_94e7
c401\timeseriesengine.dll
ModLoad: 02860000 02868000
c:\windows\microsoft.net\framework\v1.1.4322\tempor
ary asp.net files\products\bfa52050\27b5eb55\tgbddzph.dll
ModLoad: 02890000 028c6000
c:\windows\assembly\gac\gcmwebui\1001.1000.1000.106
5__80d799f04f11805c\gcmwebui.dll
ModLoad: 7ab90000 7ad34000
c:\windows\assembly\gac\system.design\1.0.5000.0__b
03f5f7f11d50a3a\system.design.dll
ModLoad: 028d0000 028e2000
c:\windows\assembly\gac\gcmdocument\1000.1000.1001.
1012__80d799f04f11805c\gcmdocument.dll
ModLoad: 028f0000 02908000
c:\windows\assembly\gac\gcmframeworkcs\1.1000.1006.
1001__80d799f04f11805c\gcmframeworkcs.dll
ModLoad: 02910000 02918000
c:\windows\assembly\gac\gcmdistributedcache\1000.10
00.1001.1000__80d799f04f11805c\gcmdistributedcache.dll
ModLoad: 02920000 0299c000
c:\windows\assembly\gac\componentart.web.ui\2.0.198
3.0__9bc9f846553156bb\componentart.web.ui.dll
ModLoad: 029a0000 029a8000
c:\windows\assembly\gac\gcmadlogonclient\1000.1000.
1000.1000__80d799f04f11805c\gcmadlogonclient.dll
ModLoad: 029b0000 029c2000
c:\windows\assembly\gac\gcmwebutils\1000.1000.1001.
1023__80d799f04f11805c\gcmwebutils.dll
ModLoad: 029d0000 029da000
c:\windows\assembly\gac\gcmmailposter\1000.1000.100
0.1004__80d799f04f11805c\gcmmailposter.dll
ModLoad: 7a020000 7a05e000
c:\windows\assembly\gac\system.messaging\1.0.5000.0
__b03f5f7f11d50a3a\system.messaging.dll
ModLoad: 7b610000 7b80a000
c:\windows\assembly\gac\system.windows.forms\1.0.50
00.0__b77a5c561934e089\system.windows.forms.dll
ModLoad: 79e40000 79e48000
c:\windows\assembly\gac\accessibility\1.0.5000.0__b
03f5f7f11d50a3a\accessibility.dll
ModLoad: 029e0000 029f6000
c:\windows\assembly\gac\websitecomet\1000.1000.1000
.1034__80d799f04f11805c\websitecomet.dll
ModLoad: 02a00000 02b26000
c:\windows\assembly\gac\componentart.web.ui\3.0.152
5.0__9bc9f846553156bb\componentart.web.ui.dll
ModLoad: 02b30000 02b46000
c:\windows\assembly\gac\gcmframeworkcs\1.2.0.15__80
d799f04f11805c\gcmframeworkcs.dll
ModLoad: 79fe0000 79ff6000
c:\windows\assembly\gac\system.configuration.instal
l\1.0.5000.0__b03f5f7f11d50a3a\system.configuration.install.dll
ModLoad: 02b50000 02b5a000
c:\windows\assembly\gac\gcmbasecomet\1000.1000.1000
.1003__80d799f04f11805c\gcmbasecomet.dll
ModLoad: 7a2c0000 7a350000
c:\windows\assembly\gac\system.web.services\1.0.500
0.0__b03f5f7f11d50a3a\system.web.services.dll
ModLoad: 02b80000 02b88000
c:\windows\assembly\gac\gcmchartfunctionbuilder\100
0.1000.1000.1001__80d799f04f11805c\gcmchartfunctionbuilder.dll
ModLoad: 7a1d0000 7a29c000
c:\windows\assembly\gac\system.web.mobile\1.0.5000.
0__b03f5f7f11d50a3a\system.web.mobile.dll
ModLoad: 02bc0000 02bcc000
c:\windows\assembly\gac\timeseriesengine\1000.1000.
1000.1004__80d799f04f11805c\timeseriesengine.dll
ModLoad: 02bd0000 02d04000
c:\windows\assembly\gac\dundaswebchart\4.1.0.1671__
90d06b0c62d592d0\dundaswebchart.dll
ModLoad: 02d70000 02d7c000
c:\windows\microsoft.net\framework\v1.1.4322\tempor
ary asp.net files\products\bfa52050\27b5eb55\7zryf4yi.dll
ModLoad: 02da0000 02dc2000
c:\windows\microsoft.net\framework\v1.1.4322\tempor
ary asp.net files\products\bfa52050\27b5eb55\ixozfmrv.dll
ModLoad: 4dd60000 4df07000
C:\WINDOWS\WinSxS\x86_Microsoft.Windows.GdiPlus_659
5b64144ccf1df_1.0.3790.1830_x-ww_24C40C58\gdiplus.dll
ModLoad: 5f270000 5f2c9000 C:\WINDOWS\system32\hnetcfg.dll
ModLoad: 71ae0000 71ae8000 C:\WINDOWS\System32\wshtcpip.dll
ModLoad: 79620000 79636000
c:\windows\assembly\gac\system.enterpriseservices\1
.0.5000.0__b03f5f7f11d50a3a\system.enterpriseservices.thunk.dll
ModLoad: 6e060000 6e093000 C:\WINDOWS\system32\Com\comadmin.dll
ModLoad: 61e50000 61e59000 C:\WINDOWS\system32\MfcSubs.dll
ModLoad: 6d560000 6d57c000 C:\WINDOWS\system32\dbnetlib.dll
ModLoad: 71f60000 71f64000 C:\WINDOWS\system32\security.dll
ModLoad: 76c90000 76cb7000 C:\WINDOWS\system32\msv1_0.dll
ModLoad: 76cf0000 76d0a000 C:\WINDOWS\system32\iphlpapi.dll
ModLoad: 766f0000 76705000 C:\WINDOWS\system32\ntdsapi.dll
ModLoad: 61530000 61560000 C:\WINDOWS\system32\mqrt.dll
ModLoad: 780c0000 78121000 C:\WINDOWS\system32\msvcp60.dll
ModLoad: 614d0000 614f7000 C:\WINDOWS\system32\mqsec.dll
ModLoad: 5ccf0000 5ccff000 C:\WINDOWS\system32\SAMLIB.dll
ModLoad: 034d0000 034dc000
c:\windows\microsoft.net\framework\v1.1.4322\tempor
ary asp.net files\products\bfa52050\27b5eb55\ase3ietw.dll
ModLoad: 03660000 03745000
c:\windows\assembly\gac\gcmframeworknet\1.0.15.2__8
0d799f04f11805c\gcmframeworknet.dll
ModLoad: 00a90000 00a9a000
c:\windows\microsoft.net\framework\v1.1.4322\tempor
ary asp.net files\products\bfa52050\27b5eb55\9tbvalmp.dll
ModLoad: 00aa0000 00aa8000
c:\windows\microsoft.net\framework\v1.1.4322\tempor
ary asp.net files\products\bfa52050\27b5eb55\ostv9xwb.dll
ModLoad: 00ac0000 00ac8000
c:\windows\microsoft.net\framework\v1.1.4322\tempor
ary asp.net files\products\bfa52050\27b5eb55\vbrvyihn.dll
ModLoad: 03ac0000 03ac8000
c:\windows\microsoft.net\framework\v1.1.4322\tempor
ary asp.net files\products\bfa52050\27b5eb55\4ddzadvu.dll
ModLoad: 02040000 02050000
c:\windows\microsoft.net\framework\v1.1.4322\tempor
ary asp.net files\products\bfa52050\27b5eb55\b55suuwg.dll
ModLoad: 4c850000 4c8c9000 C:\Program Files\Common Files\System\Ole
DB\oledb32
.dll
ModLoad: 74a50000 74a6a000 C:\WINDOWS\system32\MSDART.DLL
ModLoad: 4dbb0000 4dbc1000 C:\Program Files\Common Files\System\Ole
DB\OLEDB32
R.DLL
ModLoad: 041c0000 0438f000 C:\Progra~1\Sybase\12.5\OLEDB\sydaase.dll
ModLoad: 77530000 775c7000
C:\WINDOWS\WinSxS\x86_Microsoft.Windows.Common-Cont
rols_6595b64144ccf1df_5.82.3790.1830_x-ww_1B6F474A\COMCTL32.dll
ModLoad: 03540000 03556000 C:\Progra~1\Sybase\12.5\OLEDB\sydaaseR.DLL
ModLoad: 75b70000 75ca9000 C:\WINDOWS\system32\comsvcs.dll
ModLoad: 03910000 0391c000
c:\windows\microsoft.net\framework\v1.1.4322\tempor
ary asp.net files\products\bfa52050\27b5eb55\fyln4sbe.dll
Post by Rani Baki
I ran iisstate during an hang. It froze on thread 9, but eventually continued
and then it looks like it failed during thread 16/17. Maybe because the
worker process gave up. I also ran it using the -hc option and it hung on a
dll, could this dll be the culprit? I am including one output below and the
other on the next post.
Thanks,
Rani Baki
C:\iisstate>iisstate -p 5592 -d
SRV*C:\iisstate\symbols*http://msdl.microsoft.com/download/symbols
Microsoft (R) Windows Debugger Version 6.2.0013.1
Copyright (c) Microsoft Corporation. All rights reserved.
*** wait with pending attach
SRV*C:\iisstate\symbols*http://msdl.microsoft.com/download/symbols
WARNING: Process 5592 is not attached as a debuggee
The process can be examined but debug events will not be received
.................................................................................................................................................
The call to LoadLibrary(exts) failed, Win32 error 2
"The system cannot find the file specified."
Please check your debugger configuration and/or network access.
(15d8.2068): Wake debugger - code 80000007 (first chance)
Opened log file 'C:\iisstate\output\IISState-5592.log'
***********************
Starting new log output
IISState version 3.3.1
Tue Mar 14 10:06:51 2006
OS = Windows 2003 Server
Executable: w3wp.exe
PID = 5592
Note: Thread times are formatted as HH:MM:SS.ms
***********************
Thread ID: 0
System Thread ID: 2068
Kernel Time: 0:0:0.31
User Time: 0:0:0.31
Thread Status: Thread is in a WAIT state.
Thread Type: HTTP Compression Thread
# ChildEBP RetAddr
00 0006fc08 7c822124 ntdll!KiFastSystemCallRet
01 0006fc0c 77e6baa8 ntdll!NtWaitForSingleObject+0xc
02 0006fc7c 77e6ba12 kernel32!WaitForSingleObjectEx+0xac
03 0006fc90 5a36467a kernel32!WaitForSingleObject+0x12
04 0006fca0 5a366e63 w3dt!WP_CONTEXT::RunMainThreadLoop+0x10
05 0006fca8 5a3af41d w3dt!UlAtqStartListen+0x2d
06 0006fcb8 5a3bc259 w3core!W3_SERVER::StartListen+0xbd
07 0006ff0c 0100187c w3core!UlW3Start+0x26e
08 0006ff44 01001a23 w3wp!wmain+0x22a
09 0006ffc0 77e523cd w3wp!wmainCRTStartup+0x12b
0a 0006fff0 00000000 kernel32!BaseProcessStart+0x23
Thread ID: 1
System Thread ID: 16c0
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Type: Other
# ChildEBP RetAddr
00 009eff9c 7c821364 ntdll!KiFastSystemCallRet
01 009effa0 7c81fe26 ntdll!NtDelayExecution+0xc
02 009effb8 77e66063 ntdll!RtlpTimerThread+0x47
03 009effec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 2
System Thread ID: b10
Kernel Time: 0:0:0.15
User Time: 0:0:0.0
Thread Type: HTTP Listener
# ChildEBP RetAddr
00 00c8ff24 7c821bf4 ntdll!KiFastSystemCallRet
01 00c8ff28 77e6611a ntdll!NtRemoveIoCompletion+0xc
02 00c8ff54 5a30249e kernel32!GetQueuedCompletionStatus+0x29
03 00c8ff8c 5a3026bc W3TP!THREAD_POOL_DATA::ThreadPoolThread+0x33
04 00c8ffa0 5a301db9 W3TP!THREAD_POOL_DATA::ThreadPoolThread+0x24
05 00c8ffb8 77e66063 W3TP!THREAD_MANAGER::ThreadManagerThread+0x39
06 00c8ffec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 3
System Thread ID: 11cc
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Type: HTTP Listener
# ChildEBP RetAddr
00 00ccff24 7c821bf4 ntdll!KiFastSystemCallRet
01 00ccff28 77e6611a ntdll!NtRemoveIoCompletion+0xc
02 00ccff54 5a30249e kernel32!GetQueuedCompletionStatus+0x29
03 00ccff8c 5a3026bc W3TP!THREAD_POOL_DATA::ThreadPoolThread+0x33
04 00ccffa0 5a301db9 W3TP!THREAD_POOL_DATA::ThreadPoolThread+0x24
05 00ccffb8 77e66063 W3TP!THREAD_MANAGER::ThreadManagerThread+0x39
06 00ccffec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 4
System Thread ID: 2110
Kernel Time: 0:0:0.15
User Time: 0:0:0.0
Thread Type: HTTP Listener
# ChildEBP RetAddr
00 00d0ff24 7c821bf4 ntdll!KiFastSystemCallRet
01 00d0ff28 77e6611a ntdll!NtRemoveIoCompletion+0xc
02 00d0ff54 5a30249e kernel32!GetQueuedCompletionStatus+0x29
03 00d0ff8c 5a3026bc W3TP!THREAD_POOL_DATA::ThreadPoolThread+0x33
04 00d0ffa0 5a301db9 W3TP!THREAD_POOL_DATA::ThreadPoolThread+0x24
05 00d0ffb8 77e66063 W3TP!THREAD_MANAGER::ThreadManagerThread+0x39
06 00d0ffec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 5
System Thread ID: 19bc
Kernel Time: 0:0:0.15
User Time: 0:0:0.0
Thread Type: HTTP Listener
# ChildEBP RetAddr
00 00d4ff24 7c821bf4 ntdll!KiFastSystemCallRet
01 00d4ff28 77e6611a ntdll!NtRemoveIoCompletion+0xc
02 00d4ff54 5a30249e kernel32!GetQueuedCompletionStatus+0x29
03 00d4ff8c 5a3026bc W3TP!THREAD_POOL_DATA::ThreadPoolThread+0x33
04 00d4ffa0 5a301db9 W3TP!THREAD_POOL_DATA::ThreadPoolThread+0x24
05 00d4ffb8 77e66063 W3TP!THREAD_MANAGER::ThreadManagerThread+0x39
06 00d4ffec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 6
System Thread ID: 12e8
Kernel Time: 0:0:0.15
User Time: 0:0:0.0
Thread Status: Thread is in a WAIT state.
Thread Type: Other
# ChildEBP RetAddr
00 00e0fcec 7c822114 ntdll!KiFastSystemCallRet
01 00e0fcf0 7c83acfd ntdll!NtWaitForMultipleObjects+0xc
02 00e0ffb8 77e66063 ntdll!RtlpWaitThread+0x161
03 00e0ffec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 7
System Thread ID: 10e0
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Status: Thread is in a WAIT state.
Thread Type: HTTP Compression Thread
# ChildEBP RetAddr
00 017dfa84 7c822124 ntdll!KiFastSystemCallRet
01 017dfa88 77e6baa8 ntdll!NtWaitForSingleObject+0xc
02 017dfaf8 77e6ba12 kernel32!WaitForSingleObjectEx+0xac
03 017dfb0c 5a3b8147 kernel32!WaitForSingleObject+0x12
04 017dffb8 77e66063 w3core!HTTP_COMPRESSION::CompressionThread+0x126
05 017dffec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 8
System Thread ID: 10a4
Kernel Time: 0:0:0.15
User Time: 0:0:0.0
Thread Type: Other
# ChildEBP RetAddr
00 0185ff8c 7c821364 ntdll!KiFastSystemCallRet
01 0185ff90 7c815267 ntdll!NtDelayExecution+0xc
02 0185ffb8 77e66063 ntdll!RtlpIOWorkerThread+0x3f
03 0185ffec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 9
System Thread ID: 288
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Type: Other
# ChildEBP RetAddr
00 0191fec0 7c821bf4 ntdll!KiFastSystemCallRet
01 0191fec4 77e6611a ntdll!NtRemoveIoCompletion+0xc
02 0191fef0 791fdb22 kernel32!GetQueuedCompletionStatus+0x29
03 0191ff24 791fdacc mscorsvr+0x4db22
04 0191ffb8 77e66063 mscorsvr+0x4dacc
05 0191ffec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 10
System Thread ID: 1a4
Kernel Time: 0:0:0.15
User Time: 0:0:0.0
Thread Type: Other
# ChildEBP RetAddr
00 0195fe50 7c821364 ntdll!KiFastSystemCallRet
01 0195fe54 77e42439 ntdll!NtDelayExecution+0xc
02 0195febc 77e424b7 kernel32!SleepEx+0x68
03 0195fecc 791bf4f9 kernel32!Sleep+0xf
04 0195ffb8 77e66063 mscorsvr+0xf4f9
05 0195ffec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 11
System Thread ID: 178
Kernel Time: 0:0:0.765
User Time: 0:0:5.453
Thread Status: Thread is in a WAIT state.
Thread Type: Other
# ChildEBP RetAddr
00 0199fdf8 7c822124 ntdll!KiFastSystemCallRet
01 0199fdfc 77e6baa8 ntdll!NtWaitForSingleObject+0xc
02 0199fe6c 77e6ba12 kernel32!WaitForSingleObjectEx+0xac
03 0199fe80 791d401f kernel32!WaitForSingleObject+0x12
04 0199fea4 791fdacc mscorsvr+0x2401f
05 0199ffb8 77e66063 mscorsvr+0x4dacc
06 0199ffec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 12
System Thread ID: 265c
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Status: Thread is in a WAIT state.
Thread Type: Other
# ChildEBP RetAddr
00 01acfe70 7c822114 ntdll!KiFastSystemCallRet
01 01acfe74 77e6711b ntdll!NtWaitForMultipleObjects+0xc
02 01acff1c 77e61075 kernel32!WaitForMultipleObjectsEx+0x11a
03 01acff38 79238fd6 kernel32!WaitForMultipleObjects+0x18
04 01acffa0 79238f4d mscorsvr+0x88fd6
05 01acffb0 7923a714 mscorsvr+0x88f4d
06 01acffb8 77e66063 mscorsvr+0x8a714
07 01acffec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 13
System Thread ID: 1fe4
Kernel Time: 0:0:0.46
User Time: 0:0:0.906
Thread Status: Thread is in a WAIT state.
Thread Type: Other
# ChildEBP RetAddr
00 01d5fefc 7c822124 ntdll!KiFastSystemCallRet
01 01d5ff00 77e6baa8 ntdll!NtWaitForSingleObject+0xc
02 01d5ff70 77e6ba12 kernel32!WaitForSingleObjectEx+0xac
03 01d5ff84 791fe6b0 kernel32!WaitForSingleObject+0x12
04 01d5ffac 792356bf mscorsvr+0x4e6b0
05 01d5ffb8 77e66063 mscorsvr+0x856bf
06 01d5ffec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 14
System Thread ID: 239c
Kernel Time: 0:0:0.31
User Time: 0:0:1.93
Thread Status: Thread is in a WAIT state.
Thread Type: Other
# ChildEBP RetAddr
00 01d9fdfc 7c822124 ntdll!KiFastSystemCallRet
01 01d9fe00 77e6baa8 ntdll!NtWaitForSingleObject+0xc
02 01d9fe70 77e6ba12 kernel32!WaitForSingleObjectEx+0xac
03 01d9fe84 791fe6b0 kernel32!WaitForSingleObject+0x12
04 01d9feac 792356bf mscorsvr+0x4e6b0
05 01d9ffb8 77e66063 mscorsvr+0x856bf
06 01d9ffec 00000000 kernel32!BaseThreadStart+0x34
Pat [MSFT]
2006-03-14 18:18:56 UTC
Permalink
Looks like things were still running normally. A dll was in the process of
loading - looks like you terminated IISState. Using the -sc/-hc options
tells IISState to attach and then wait for a crash to occur. This can take
minutes-->days.


Pat
Post by Rani Baki
C:\iisstate>iisstate -p 5592 -hc -d
SRV*C:\iisstate\symbols*http://msdl.microsoft.com/downloa
d/symbols
Microsoft (R) Windows Debugger Version 6.2.0013.1
Copyright (c) Microsoft Corporation. All rights reserved.
sxd av
WARNING: The debugger does not have a current process or thread
WARNING: Many commands will not work
*** wait with pending attach
The call to LoadLibrary(exts) failed, Win32 error 2
"The system cannot find the file specified."
Please check your debugger configuration and/or network access.
SRV*C:\iisstate\symbols*http://msdl.microsoft.com/downloa
d/symbols
ModLoad: 01000000 01005000 c:\windows\system32\inetsrv\w3wp.exe
ModLoad: 7c800000 7c8c0000 C:\WINDOWS\system32\ntdll.dll
ModLoad: 77e40000 77f42000 C:\WINDOWS\system32\kernel32.dll
ModLoad: 77ba0000 77bfa000 C:\WINDOWS\system32\msvcrt.dll
ModLoad: 77f50000 77fec000 C:\WINDOWS\system32\ADVAPI32.dll
ModLoad: 77c50000 77cef000 C:\WINDOWS\system32\RPCRT4.dll
ModLoad: 77380000 77412000 C:\WINDOWS\system32\USER32.dll
ModLoad: 77c00000 77c48000 C:\WINDOWS\system32\GDI32.dll
ModLoad: 77670000 777a4000 C:\WINDOWS\system32\ole32.dll
ModLoad: 64700000 6472d000 c:\windows\system32\inetsrv\IISUTIL.dll
ModLoad: 761b0000 76243000 C:\WINDOWS\system32\CRYPT32.dll
ModLoad: 76190000 761a2000 C:\WINDOWS\system32\MSASN1.dll
ModLoad: 10000000 102c5000 C:\WINDOWS\system32\xpsp2res.dll
ModLoad: 5a390000 5a3e8000 c:\windows\system32\inetsrv\w3core.dll
ModLoad: 5a420000 5a431000 C:\WINDOWS\system32\IISMAP.dll
ModLoad: 647b0000 647d7000 C:\WINDOWS\system32\IisRTL.DLL
ModLoad: 71c00000 71c17000 C:\WINDOWS\system32\WS2_32.dll
ModLoad: 71bf0000 71bf8000 C:\WINDOWS\system32\WS2HELP.dll
ModLoad: 62da0000 62da7000 c:\windows\system32\inetsrv\LONSINT.dll
ModLoad: 76f50000 76f63000 C:\WINDOWS\system32\Secur32.dll
ModLoad: 71c40000 71c98000 C:\WINDOWS\system32\NETAPI32.dll
ModLoad: 77d00000 77d8c000 C:\WINDOWS\system32\OLEAUT32.dll
ModLoad: 5b640000 5b658000 C:\WINDOWS\system32\strmfilt.dll
ModLoad: 67150000 67159000 C:\WINDOWS\system32\HTTPAPI.dll
ModLoad: 77da0000 77df2000 C:\WINDOWS\system32\SHLWAPI.dll
ModLoad: 5a400000 5a408000 c:\windows\system32\inetsrv\W3CACHE.dll
ModLoad: 5a300000 5a307000 c:\windows\system32\inetsrv\W3TP.dll
ModLoad: 5a3f0000 5a3f6000 c:\windows\system32\inetsrv\W3COMLOG.dll
ModLoad: 5a360000 5a36d000 c:\windows\system32\inetsrv\w3dt.dll
ModLoad: 60ba0000 60bb1000 c:\windows\system32\inetsrv\wamreg.DLL
ModLoad: 71bb0000 71bb9000 C:\WINDOWS\system32\WSOCK32.dll
ModLoad: 6d0f0000 6d110000 c:\windows\system32\inetsrv\iisres.dll
ModLoad: 777b0000 77833000 C:\WINDOWS\system32\CLBCatQ.DLL
ModLoad: 77010000 770d6000 C:\WINDOWS\system32\COMRes.dll
ModLoad: 77b90000 77b98000 C:\WINDOWS\system32\VERSION.dll
ModLoad: 608f0000 60901000 C:\WINDOWS\system32\ADMWPROX.DLL
ModLoad: 68000000 6802f000 C:\WINDOWS\system32\rsaenh.dll
ModLoad: 76b70000 76b7b000 C:\WINDOWS\system32\PSAPI.DLL
ModLoad: 79e50000 79e56000
C:\WINDOWS\Microsoft.NET\Framework\v1.1.4322\aspnet
_filter.dll
ModLoad: 7c340000 7c396000
C:\WINDOWS\Microsoft.NET\Framework\v1.1.4322\MSVCR7
1.dll
ModLoad: 5a320000 5a332000 c:\windows\system32\inetsrv\w3isapi.dll
ModLoad: 71b20000 71b61000 C:\WINDOWS\System32\mswsock.dll
ModLoad: 76ed0000 76ef9000 C:\WINDOWS\system32\DNSAPI.dll
ModLoad: 76f70000 76f77000 C:\WINDOWS\System32\winrnr.dll
ModLoad: 76f10000 76f3e000 C:\WINDOWS\system32\WLDAP32.dll
ModLoad: 76f80000 76f85000 C:\WINDOWS\system32\rasadhlp.dll
ModLoad: 685b0000 685ba000 C:\WINDOWS\system32\inetsrv\gzip.dll
ModLoad: 71bd0000 71be1000 C:\WINDOWS\system32\mpr.dll
ModLoad: 79e60000 79ea2000
\\?\C:\WINDOWS\Microsoft.NET\Framework\v1.1.4322\as
pnet_isapi.dll
ModLoad: 76920000 769e4000 C:\WINDOWS\system32\USERENV.dll
ModLoad: 79170000 79198000 C:\WINDOWS\system32\mscoree.dll
ModLoad: 791b0000 79419000
C:\WINDOWS\Microsoft.NET\Framework\v1.1.4322\mscors
vr.dll
ModLoad: 79040000 79085000
C:\WINDOWS\Microsoft.NET\Framework\v1.1.4322\fusion
.dll
ModLoad: 7c8d0000 7d0d3000 C:\WINDOWS\system32\SHELL32.dll
ModLoad: 77420000 77523000
C:\WINDOWS\WinSxS\x86_Microsoft.Windows.Common-Cont
rols_6595b64144ccf1df_6.0.3790.1830_x-ww_7AE38CCF\comctl32.dll
ModLoad: 79780000 7998c000
c:\windows\microsoft.net\framework\v1.1.4322\mscorl
ib.dll
ModLoad: 79990000 79cce000
c:\windows\assembly\nativeimages1_v1.1.4322\mscorli
b\1.0.5000.0__b77a5c561934e089_9ab75ea9\mscorlib.dll
ModLoad: 7a090000 7a1c8000
c:\windows\assembly\gac\system.web\1.0.5000.0__b03f
5f7f11d50a3a\system.web.dll
ModLoad: 7b0a0000 7b1ce000
c:\windows\assembly\gac\system\1.0.5000.0__b77a5c56
1934e089\system.dll
ModLoad: 7b490000 7b506000
c:\windows\assembly\gac\system.drawing\1.0.5000.0__
b03f5f7f11d50a3a\system.drawing.dll
ModLoad: 7bc10000 7bd5e000
c:\windows\assembly\gac\system.xml\1.0.5000.0__b77a
5c561934e089\system.xml.dll
ModLoad: 7a6e0000 7a820000
c:\windows\assembly\gac\system.data\1.0.5000.0__b77
a5c561934e089\system.data.dll
ModLoad: 79000000 7903e000
c:\windows\assembly\gac\system.enterpriseservices\1
.0.5000.0__b03f5f7f11d50a3a\system.enterpriseservices.dll
ModLoad: 7a2a0000 7a2b2000
c:\windows\assembly\gac\system.web.regularexpressio
ns\1.0.5000.0__b03f5f7f11d50a3a\system.web.regularexpressions.dll
ModLoad: 01e70000 01e75000
c:\windows\assembly\gac\microsoft.visualc\7.0.5000.
0__b03f5f7f11d50a3a\microsoft.visualc.dll
ModLoad: 7a000000 7a01a000
c:\windows\assembly\gac\system.directoryservices\1.
0.5000.0__b03f5f7f11d50a3a\system.directoryservices.dll
ModLoad: 79640000 79692000
c:\windows\assembly\gac\system.runtime.remoting\1.0
.5000.0__b77a5c561934e089\system.runtime.remoting.dll
ModLoad: 79550000 79574000
c:\windows\assembly\gac\system.runtime.serializatio
n.formatters.soap\1.0.5000.0__b03f5f7f11d50a3a\system.runtime.serialization.form
atters.soap.dll
ModLoad: 53760000 5376a000
c:\windows\assembly\gac\cscompmgd\7.0.5000.0__b03f5
f7f11d50a3a\cscompmgd.dll
ModLoad: 79430000 7947d000
C:\WINDOWS\Microsoft.NET\Framework\v1.1.4322\MSCORJ
IT.DLL
ModLoad: 02260000 02270000
c:\windows\microsoft.net\framework\v1.1.4322\tempor
ary asp.net
files\products\bfa52050\27b5eb55\assembly\dl2\dec2d5fd\f0587714_1116
c601\chartdataengine.dll
ModLoad: 02280000 02296000
c:\windows\microsoft.net\framework\v1.1.4322\tempor
ary asp.net
files\products\bfa52050\27b5eb55\assembly\dl2\f42c429f\c636da4a_6f15
c601\chartformatter.dll
ModLoad: 79510000 79523000
C:\WINDOWS\Microsoft.NET\Framework\v1.1.4322\mscors
n.dll
ModLoad: 022a0000 0231c000
c:\windows\microsoft.net\framework\v1.1.4322\tempor
ary asp.net
files\products\bfa52050\27b5eb55\assembly\dl2\5be8c14d\7afb1759_f0c9
c501\componentart.web.ui.dll
ModLoad: 02320000 02454000
c:\windows\microsoft.net\framework\v1.1.4322\tempor
ary asp.net
files\products\bfa52050\27b5eb55\assembly\dl2\db34d8fe\005124c2_529b
c401\dundaswebchart.dll
ModLoad: 02470000 02478000
c:\windows\microsoft.net\framework\v1.1.4322\tempor
ary asp.net
files\products\bfa52050\27b5eb55\assembly\dl2\37d9293d\b47a4b3f_abcc
c401\gcmchartfunctionbuilder.dll
ModLoad: 02480000 02492000
c:\windows\microsoft.net\framework\v1.1.4322\tempor
ary asp.net
files\products\bfa52050\27b5eb55\assembly\dl2\a2551b9e\7d4a099b_54f4
c501\gcmdocument.dll
ModLoad: 024a0000 0258a000
c:\windows\microsoft.net\framework\v1.1.4322\tempor
ary asp.net
files\products\bfa52050\27b5eb55\assembly\dl2\f45dabdf\1ba29f5e_ba6d
c501\gcmframeworknet.dll
ModLoad: 02590000 025a1000 C:\WINDOWS\system32\pwrpc32.dll
ModLoad: 7c3a0000 7c41b000 C:\WINDOWS\system32\MSVCP71.dll
ModLoad: 025b0000 02612000
C:\Progra~1\Sybase\12.5\OCS-12_5\dll\libsybdb.dll
ModLoad: 02620000 0264b000
C:\Progra~1\Sybase\12.5\OCS-12_5\dll\libtcl.dll
ModLoad: 02650000 0265b000
C:\Progra~1\Sybase\12.5\OCS-12_5\dll\libintl.dll
ModLoad: 02660000 026c7000
C:\Progra~1\Sybase\12.5\OCS-12_5\dll\libcomn.dll
ModLoad: 026d0000 026de000
C:\Progra~1\Sybase\12.5\OCS-12_5\dll\libcs.dll
ModLoad: 026e0000 02745000
C:\Progra~1\Sybase\12.5\OCS-12_5\dll\libct.dll
ModLoad: 02790000 027c6000
c:\windows\microsoft.net\framework\v1.1.4322\tempor
ary asp.net
files\products\bfa52050\27b5eb55\assembly\dl2\4dc4428b\70566cf5_7d02
c601\gcmwebui.dll
ModLoad: 027d0000 027da000
c:\windows\microsoft.net\framework\v1.1.4322\tempor
ary asp.net
files\products\bfa52050\27b5eb55\assembly\dl2\eb6832c8\00d05520_4fa5
c301\math_expression_evaluator.dll
ModLoad: 027e0000 02822000
c:\windows\microsoft.net\framework\v1.1.4322\tempor
ary asp.net
files\products\bfa52050\27b5eb55\assembly\dl2\b9de8e72\baf8efb2_2341
c601\products.dll
ModLoad: 02840000 0284c000
c:\windows\microsoft.net\framework\v1.1.4322\tempor
ary asp.net
files\products\bfa52050\27b5eb55\assembly\dl2\53c0176d\f83d494c_94e7
c401\timeseriesengine.dll
ModLoad: 02860000 02868000
c:\windows\microsoft.net\framework\v1.1.4322\tempor
ary asp.net files\products\bfa52050\27b5eb55\tgbddzph.dll
ModLoad: 02890000 028c6000
c:\windows\assembly\gac\gcmwebui\1001.1000.1000.106
5__80d799f04f11805c\gcmwebui.dll
ModLoad: 7ab90000 7ad34000
c:\windows\assembly\gac\system.design\1.0.5000.0__b
03f5f7f11d50a3a\system.design.dll
ModLoad: 028d0000 028e2000
c:\windows\assembly\gac\gcmdocument\1000.1000.1001.
1012__80d799f04f11805c\gcmdocument.dll
ModLoad: 028f0000 02908000
c:\windows\assembly\gac\gcmframeworkcs\1.1000.1006.
1001__80d799f04f11805c\gcmframeworkcs.dll
ModLoad: 02910000 02918000
c:\windows\assembly\gac\gcmdistributedcache\1000.10
00.1001.1000__80d799f04f11805c\gcmdistributedcache.dll
ModLoad: 02920000 0299c000
c:\windows\assembly\gac\componentart.web.ui\2.0.198
3.0__9bc9f846553156bb\componentart.web.ui.dll
ModLoad: 029a0000 029a8000
c:\windows\assembly\gac\gcmadlogonclient\1000.1000.
1000.1000__80d799f04f11805c\gcmadlogonclient.dll
ModLoad: 029b0000 029c2000
c:\windows\assembly\gac\gcmwebutils\1000.1000.1001.
1023__80d799f04f11805c\gcmwebutils.dll
ModLoad: 029d0000 029da000
c:\windows\assembly\gac\gcmmailposter\1000.1000.100
0.1004__80d799f04f11805c\gcmmailposter.dll
ModLoad: 7a020000 7a05e000
c:\windows\assembly\gac\system.messaging\1.0.5000.0
__b03f5f7f11d50a3a\system.messaging.dll
ModLoad: 7b610000 7b80a000
c:\windows\assembly\gac\system.windows.forms\1.0.50
00.0__b77a5c561934e089\system.windows.forms.dll
ModLoad: 79e40000 79e48000
c:\windows\assembly\gac\accessibility\1.0.5000.0__b
03f5f7f11d50a3a\accessibility.dll
ModLoad: 029e0000 029f6000
c:\windows\assembly\gac\websitecomet\1000.1000.1000
.1034__80d799f04f11805c\websitecomet.dll
ModLoad: 02a00000 02b26000
c:\windows\assembly\gac\componentart.web.ui\3.0.152
5.0__9bc9f846553156bb\componentart.web.ui.dll
ModLoad: 02b30000 02b46000
c:\windows\assembly\gac\gcmframeworkcs\1.2.0.15__80
d799f04f11805c\gcmframeworkcs.dll
ModLoad: 79fe0000 79ff6000
c:\windows\assembly\gac\system.configuration.instal
l\1.0.5000.0__b03f5f7f11d50a3a\system.configuration.install.dll
ModLoad: 02b50000 02b5a000
c:\windows\assembly\gac\gcmbasecomet\1000.1000.1000
.1003__80d799f04f11805c\gcmbasecomet.dll
ModLoad: 7a2c0000 7a350000
c:\windows\assembly\gac\system.web.services\1.0.500
0.0__b03f5f7f11d50a3a\system.web.services.dll
ModLoad: 02b80000 02b88000
c:\windows\assembly\gac\gcmchartfunctionbuilder\100
0.1000.1000.1001__80d799f04f11805c\gcmchartfunctionbuilder.dll
ModLoad: 7a1d0000 7a29c000
c:\windows\assembly\gac\system.web.mobile\1.0.5000.
0__b03f5f7f11d50a3a\system.web.mobile.dll
ModLoad: 02bc0000 02bcc000
c:\windows\assembly\gac\timeseriesengine\1000.1000.
1000.1004__80d799f04f11805c\timeseriesengine.dll
ModLoad: 02bd0000 02d04000
c:\windows\assembly\gac\dundaswebchart\4.1.0.1671__
90d06b0c62d592d0\dundaswebchart.dll
ModLoad: 02d70000 02d7c000
c:\windows\microsoft.net\framework\v1.1.4322\tempor
ary asp.net files\products\bfa52050\27b5eb55\7zryf4yi.dll
ModLoad: 02da0000 02dc2000
c:\windows\microsoft.net\framework\v1.1.4322\tempor
ary asp.net files\products\bfa52050\27b5eb55\ixozfmrv.dll
ModLoad: 4dd60000 4df07000
C:\WINDOWS\WinSxS\x86_Microsoft.Windows.GdiPlus_659
5b64144ccf1df_1.0.3790.1830_x-ww_24C40C58\gdiplus.dll
ModLoad: 5f270000 5f2c9000 C:\WINDOWS\system32\hnetcfg.dll
ModLoad: 71ae0000 71ae8000 C:\WINDOWS\System32\wshtcpip.dll
ModLoad: 79620000 79636000
c:\windows\assembly\gac\system.enterpriseservices\1
.0.5000.0__b03f5f7f11d50a3a\system.enterpriseservices.thunk.dll
ModLoad: 6e060000 6e093000 C:\WINDOWS\system32\Com\comadmin.dll
ModLoad: 61e50000 61e59000 C:\WINDOWS\system32\MfcSubs.dll
ModLoad: 6d560000 6d57c000 C:\WINDOWS\system32\dbnetlib.dll
ModLoad: 71f60000 71f64000 C:\WINDOWS\system32\security.dll
ModLoad: 76c90000 76cb7000 C:\WINDOWS\system32\msv1_0.dll
ModLoad: 76cf0000 76d0a000 C:\WINDOWS\system32\iphlpapi.dll
ModLoad: 766f0000 76705000 C:\WINDOWS\system32\ntdsapi.dll
ModLoad: 61530000 61560000 C:\WINDOWS\system32\mqrt.dll
ModLoad: 780c0000 78121000 C:\WINDOWS\system32\msvcp60.dll
ModLoad: 614d0000 614f7000 C:\WINDOWS\system32\mqsec.dll
ModLoad: 5ccf0000 5ccff000 C:\WINDOWS\system32\SAMLIB.dll
ModLoad: 034d0000 034dc000
c:\windows\microsoft.net\framework\v1.1.4322\tempor
ary asp.net files\products\bfa52050\27b5eb55\ase3ietw.dll
ModLoad: 03660000 03745000
c:\windows\assembly\gac\gcmframeworknet\1.0.15.2__8
0d799f04f11805c\gcmframeworknet.dll
ModLoad: 00a90000 00a9a000
c:\windows\microsoft.net\framework\v1.1.4322\tempor
ary asp.net files\products\bfa52050\27b5eb55\9tbvalmp.dll
ModLoad: 00aa0000 00aa8000
c:\windows\microsoft.net\framework\v1.1.4322\tempor
ary asp.net files\products\bfa52050\27b5eb55\ostv9xwb.dll
ModLoad: 00ac0000 00ac8000
c:\windows\microsoft.net\framework\v1.1.4322\tempor
ary asp.net files\products\bfa52050\27b5eb55\vbrvyihn.dll
ModLoad: 03ac0000 03ac8000
c:\windows\microsoft.net\framework\v1.1.4322\tempor
ary asp.net files\products\bfa52050\27b5eb55\4ddzadvu.dll
ModLoad: 02040000 02050000
c:\windows\microsoft.net\framework\v1.1.4322\tempor
ary asp.net files\products\bfa52050\27b5eb55\b55suuwg.dll
ModLoad: 4c850000 4c8c9000 C:\Program Files\Common Files\System\Ole
DB\oledb32
.dll
ModLoad: 74a50000 74a6a000 C:\WINDOWS\system32\MSDART.DLL
ModLoad: 4dbb0000 4dbc1000 C:\Program Files\Common Files\System\Ole
DB\OLEDB32
R.DLL
ModLoad: 041c0000 0438f000 C:\Progra~1\Sybase\12.5\OLEDB\sydaase.dll
ModLoad: 77530000 775c7000
C:\WINDOWS\WinSxS\x86_Microsoft.Windows.Common-Cont
rols_6595b64144ccf1df_5.82.3790.1830_x-ww_1B6F474A\COMCTL32.dll
ModLoad: 03540000 03556000 C:\Progra~1\Sybase\12.5\OLEDB\sydaaseR.DLL
ModLoad: 75b70000 75ca9000 C:\WINDOWS\system32\comsvcs.dll
ModLoad: 03910000 0391c000
c:\windows\microsoft.net\framework\v1.1.4322\tempor
ary asp.net files\products\bfa52050\27b5eb55\fyln4sbe.dll
Post by Rani Baki
I ran iisstate during an hang. It froze on thread 9, but eventually continued
and then it looks like it failed during thread 16/17. Maybe because the
worker process gave up. I also ran it using the -hc option and it hung on a
dll, could this dll be the culprit? I am including one output below and the
other on the next post.
Thanks,
Rani Baki
C:\iisstate>iisstate -p 5592 -d
SRV*C:\iisstate\symbols*http://msdl.microsoft.com/download/symbols
Microsoft (R) Windows Debugger Version 6.2.0013.1
Copyright (c) Microsoft Corporation. All rights reserved.
*** wait with pending attach
SRV*C:\iisstate\symbols*http://msdl.microsoft.com/download/symbols
WARNING: Process 5592 is not attached as a debuggee
The process can be examined but debug events will not be received
.................................................................................................................................................
The call to LoadLibrary(exts) failed, Win32 error 2
"The system cannot find the file specified."
Please check your debugger configuration and/or network access.
(15d8.2068): Wake debugger - code 80000007 (first chance)
Opened log file 'C:\iisstate\output\IISState-5592.log'
***********************
Starting new log output
IISState version 3.3.1
Tue Mar 14 10:06:51 2006
OS = Windows 2003 Server
Executable: w3wp.exe
PID = 5592
Note: Thread times are formatted as HH:MM:SS.ms
***********************
Thread ID: 0
System Thread ID: 2068
Kernel Time: 0:0:0.31
User Time: 0:0:0.31
Thread Status: Thread is in a WAIT state.
Thread Type: HTTP Compression Thread
# ChildEBP RetAddr
00 0006fc08 7c822124 ntdll!KiFastSystemCallRet
01 0006fc0c 77e6baa8 ntdll!NtWaitForSingleObject+0xc
02 0006fc7c 77e6ba12 kernel32!WaitForSingleObjectEx+0xac
03 0006fc90 5a36467a kernel32!WaitForSingleObject+0x12
04 0006fca0 5a366e63 w3dt!WP_CONTEXT::RunMainThreadLoop+0x10
05 0006fca8 5a3af41d w3dt!UlAtqStartListen+0x2d
06 0006fcb8 5a3bc259 w3core!W3_SERVER::StartListen+0xbd
07 0006ff0c 0100187c w3core!UlW3Start+0x26e
08 0006ff44 01001a23 w3wp!wmain+0x22a
09 0006ffc0 77e523cd w3wp!wmainCRTStartup+0x12b
0a 0006fff0 00000000 kernel32!BaseProcessStart+0x23
Thread ID: 1
System Thread ID: 16c0
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Type: Other
# ChildEBP RetAddr
00 009eff9c 7c821364 ntdll!KiFastSystemCallRet
01 009effa0 7c81fe26 ntdll!NtDelayExecution+0xc
02 009effb8 77e66063 ntdll!RtlpTimerThread+0x47
03 009effec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 2
System Thread ID: b10
Kernel Time: 0:0:0.15
User Time: 0:0:0.0
Thread Type: HTTP Listener
# ChildEBP RetAddr
00 00c8ff24 7c821bf4 ntdll!KiFastSystemCallRet
01 00c8ff28 77e6611a ntdll!NtRemoveIoCompletion+0xc
02 00c8ff54 5a30249e kernel32!GetQueuedCompletionStatus+0x29
03 00c8ff8c 5a3026bc W3TP!THREAD_POOL_DATA::ThreadPoolThread+0x33
04 00c8ffa0 5a301db9 W3TP!THREAD_POOL_DATA::ThreadPoolThread+0x24
05 00c8ffb8 77e66063 W3TP!THREAD_MANAGER::ThreadManagerThread+0x39
06 00c8ffec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 3
System Thread ID: 11cc
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Type: HTTP Listener
# ChildEBP RetAddr
00 00ccff24 7c821bf4 ntdll!KiFastSystemCallRet
01 00ccff28 77e6611a ntdll!NtRemoveIoCompletion+0xc
02 00ccff54 5a30249e kernel32!GetQueuedCompletionStatus+0x29
03 00ccff8c 5a3026bc W3TP!THREAD_POOL_DATA::ThreadPoolThread+0x33
04 00ccffa0 5a301db9 W3TP!THREAD_POOL_DATA::ThreadPoolThread+0x24
05 00ccffb8 77e66063 W3TP!THREAD_MANAGER::ThreadManagerThread+0x39
06 00ccffec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 4
System Thread ID: 2110
Kernel Time: 0:0:0.15
User Time: 0:0:0.0
Thread Type: HTTP Listener
# ChildEBP RetAddr
00 00d0ff24 7c821bf4 ntdll!KiFastSystemCallRet
01 00d0ff28 77e6611a ntdll!NtRemoveIoCompletion+0xc
02 00d0ff54 5a30249e kernel32!GetQueuedCompletionStatus+0x29
03 00d0ff8c 5a3026bc W3TP!THREAD_POOL_DATA::ThreadPoolThread+0x33
04 00d0ffa0 5a301db9 W3TP!THREAD_POOL_DATA::ThreadPoolThread+0x24
05 00d0ffb8 77e66063 W3TP!THREAD_MANAGER::ThreadManagerThread+0x39
06 00d0ffec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 5
System Thread ID: 19bc
Kernel Time: 0:0:0.15
User Time: 0:0:0.0
Thread Type: HTTP Listener
# ChildEBP RetAddr
00 00d4ff24 7c821bf4 ntdll!KiFastSystemCallRet
01 00d4ff28 77e6611a ntdll!NtRemoveIoCompletion+0xc
02 00d4ff54 5a30249e kernel32!GetQueuedCompletionStatus+0x29
03 00d4ff8c 5a3026bc W3TP!THREAD_POOL_DATA::ThreadPoolThread+0x33
04 00d4ffa0 5a301db9 W3TP!THREAD_POOL_DATA::ThreadPoolThread+0x24
05 00d4ffb8 77e66063 W3TP!THREAD_MANAGER::ThreadManagerThread+0x39
06 00d4ffec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 6
System Thread ID: 12e8
Kernel Time: 0:0:0.15
User Time: 0:0:0.0
Thread Status: Thread is in a WAIT state.
Thread Type: Other
# ChildEBP RetAddr
00 00e0fcec 7c822114 ntdll!KiFastSystemCallRet
01 00e0fcf0 7c83acfd ntdll!NtWaitForMultipleObjects+0xc
02 00e0ffb8 77e66063 ntdll!RtlpWaitThread+0x161
03 00e0ffec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 7
System Thread ID: 10e0
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Status: Thread is in a WAIT state.
Thread Type: HTTP Compression Thread
# ChildEBP RetAddr
00 017dfa84 7c822124 ntdll!KiFastSystemCallRet
01 017dfa88 77e6baa8 ntdll!NtWaitForSingleObject+0xc
02 017dfaf8 77e6ba12 kernel32!WaitForSingleObjectEx+0xac
03 017dfb0c 5a3b8147 kernel32!WaitForSingleObject+0x12
04 017dffb8 77e66063 w3core!HTTP_COMPRESSION::CompressionThread+0x126
05 017dffec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 8
System Thread ID: 10a4
Kernel Time: 0:0:0.15
User Time: 0:0:0.0
Thread Type: Other
# ChildEBP RetAddr
00 0185ff8c 7c821364 ntdll!KiFastSystemCallRet
01 0185ff90 7c815267 ntdll!NtDelayExecution+0xc
02 0185ffb8 77e66063 ntdll!RtlpIOWorkerThread+0x3f
03 0185ffec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 9
System Thread ID: 288
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Type: Other
# ChildEBP RetAddr
00 0191fec0 7c821bf4 ntdll!KiFastSystemCallRet
01 0191fec4 77e6611a ntdll!NtRemoveIoCompletion+0xc
02 0191fef0 791fdb22 kernel32!GetQueuedCompletionStatus+0x29
03 0191ff24 791fdacc mscorsvr+0x4db22
04 0191ffb8 77e66063 mscorsvr+0x4dacc
05 0191ffec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 10
System Thread ID: 1a4
Kernel Time: 0:0:0.15
User Time: 0:0:0.0
Thread Type: Other
# ChildEBP RetAddr
00 0195fe50 7c821364 ntdll!KiFastSystemCallRet
01 0195fe54 77e42439 ntdll!NtDelayExecution+0xc
02 0195febc 77e424b7 kernel32!SleepEx+0x68
03 0195fecc 791bf4f9 kernel32!Sleep+0xf
04 0195ffb8 77e66063 mscorsvr+0xf4f9
05 0195ffec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 11
System Thread ID: 178
Kernel Time: 0:0:0.765
User Time: 0:0:5.453
Thread Status: Thread is in a WAIT state.
Thread Type: Other
# ChildEBP RetAddr
00 0199fdf8 7c822124 ntdll!KiFastSystemCallRet
01 0199fdfc 77e6baa8 ntdll!NtWaitForSingleObject+0xc
02 0199fe6c 77e6ba12 kernel32!WaitForSingleObjectEx+0xac
03 0199fe80 791d401f kernel32!WaitForSingleObject+0x12
04 0199fea4 791fdacc mscorsvr+0x2401f
05 0199ffb8 77e66063 mscorsvr+0x4dacc
06 0199ffec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 12
System Thread ID: 265c
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Status: Thread is in a WAIT state.
Thread Type: Other
# ChildEBP RetAddr
00 01acfe70 7c822114 ntdll!KiFastSystemCallRet
01 01acfe74 77e6711b ntdll!NtWaitForMultipleObjects+0xc
02 01acff1c 77e61075 kernel32!WaitForMultipleObjectsEx+0x11a
03 01acff38 79238fd6 kernel32!WaitForMultipleObjects+0x18
04 01acffa0 79238f4d mscorsvr+0x88fd6
05 01acffb0 7923a714 mscorsvr+0x88f4d
06 01acffb8 77e66063 mscorsvr+0x8a714
07 01acffec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 13
System Thread ID: 1fe4
Kernel Time: 0:0:0.46
User Time: 0:0:0.906
Thread Status: Thread is in a WAIT state.
Thread Type: Other
# ChildEBP RetAddr
00 01d5fefc 7c822124 ntdll!KiFastSystemCallRet
01 01d5ff00 77e6baa8 ntdll!NtWaitForSingleObject+0xc
02 01d5ff70 77e6ba12 kernel32!WaitForSingleObjectEx+0xac
03 01d5ff84 791fe6b0 kernel32!WaitForSingleObject+0x12
04 01d5ffac 792356bf mscorsvr+0x4e6b0
05 01d5ffb8 77e66063 mscorsvr+0x856bf
06 01d5ffec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 14
System Thread ID: 239c
Kernel Time: 0:0:0.31
User Time: 0:0:1.93
Thread Status: Thread is in a WAIT state.
Thread Type: Other
# ChildEBP RetAddr
00 01d9fdfc 7c822124 ntdll!KiFastSystemCallRet
01 01d9fe00 77e6baa8 ntdll!NtWaitForSingleObject+0xc
02 01d9fe70 77e6ba12 kernel32!WaitForSingleObjectEx+0xac
03 01d9fe84 791fe6b0 kernel32!WaitForSingleObject+0x12
04 01d9feac 792356bf mscorsvr+0x4e6b0
05 01d9ffb8 77e66063 mscorsvr+0x856bf
06 01d9ffec 00000000 kernel32!BaseThreadStart+0x34
Pat [MSFT]
2006-03-14 18:17:38 UTC
Permalink
Thread 16 shows that IIS reset the process out from under you. The delay is
usually caused by the downloading of symbols - which are cached so the
problem goes away after the initial run.

-hc or -sc will only help you if you are seeing unexpected process
terminations (i.e. a crash). Hangs won't be caught by them.


Pat
Post by Rani Baki
I ran iisstate during an hang. It froze on thread 9, but eventually continued
and then it looks like it failed during thread 16/17. Maybe because the
worker process gave up. I also ran it using the -hc option and it hung on a
dll, could this dll be the culprit? I am including one output below and the
other on the next post.
Thanks,
Rani Baki
C:\iisstate>iisstate -p 5592 -d
SRV*C:\iisstate\symbols*http://msdl.microsoft.com/download/symbols
Microsoft (R) Windows Debugger Version 6.2.0013.1
Copyright (c) Microsoft Corporation. All rights reserved.
*** wait with pending attach
SRV*C:\iisstate\symbols*http://msdl.microsoft.com/download/symbols
WARNING: Process 5592 is not attached as a debuggee
The process can be examined but debug events will not be received
.................................................................................................................................................
The call to LoadLibrary(exts) failed, Win32 error 2
"The system cannot find the file specified."
Please check your debugger configuration and/or network access.
(15d8.2068): Wake debugger - code 80000007 (first chance)
Opened log file 'C:\iisstate\output\IISState-5592.log'
***********************
Starting new log output
IISState version 3.3.1
Tue Mar 14 10:06:51 2006
OS = Windows 2003 Server
Executable: w3wp.exe
PID = 5592
Note: Thread times are formatted as HH:MM:SS.ms
***********************
Thread ID: 0
System Thread ID: 2068
Kernel Time: 0:0:0.31
User Time: 0:0:0.31
Thread Status: Thread is in a WAIT state.
Thread Type: HTTP Compression Thread
# ChildEBP RetAddr
00 0006fc08 7c822124 ntdll!KiFastSystemCallRet
01 0006fc0c 77e6baa8 ntdll!NtWaitForSingleObject+0xc
02 0006fc7c 77e6ba12 kernel32!WaitForSingleObjectEx+0xac
03 0006fc90 5a36467a kernel32!WaitForSingleObject+0x12
04 0006fca0 5a366e63 w3dt!WP_CONTEXT::RunMainThreadLoop+0x10
05 0006fca8 5a3af41d w3dt!UlAtqStartListen+0x2d
06 0006fcb8 5a3bc259 w3core!W3_SERVER::StartListen+0xbd
07 0006ff0c 0100187c w3core!UlW3Start+0x26e
08 0006ff44 01001a23 w3wp!wmain+0x22a
09 0006ffc0 77e523cd w3wp!wmainCRTStartup+0x12b
0a 0006fff0 00000000 kernel32!BaseProcessStart+0x23
Thread ID: 1
System Thread ID: 16c0
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Type: Other
# ChildEBP RetAddr
00 009eff9c 7c821364 ntdll!KiFastSystemCallRet
01 009effa0 7c81fe26 ntdll!NtDelayExecution+0xc
02 009effb8 77e66063 ntdll!RtlpTimerThread+0x47
03 009effec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 2
System Thread ID: b10
Kernel Time: 0:0:0.15
User Time: 0:0:0.0
Thread Type: HTTP Listener
# ChildEBP RetAddr
00 00c8ff24 7c821bf4 ntdll!KiFastSystemCallRet
01 00c8ff28 77e6611a ntdll!NtRemoveIoCompletion+0xc
02 00c8ff54 5a30249e kernel32!GetQueuedCompletionStatus+0x29
03 00c8ff8c 5a3026bc W3TP!THREAD_POOL_DATA::ThreadPoolThread+0x33
04 00c8ffa0 5a301db9 W3TP!THREAD_POOL_DATA::ThreadPoolThread+0x24
05 00c8ffb8 77e66063 W3TP!THREAD_MANAGER::ThreadManagerThread+0x39
06 00c8ffec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 3
System Thread ID: 11cc
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Type: HTTP Listener
# ChildEBP RetAddr
00 00ccff24 7c821bf4 ntdll!KiFastSystemCallRet
01 00ccff28 77e6611a ntdll!NtRemoveIoCompletion+0xc
02 00ccff54 5a30249e kernel32!GetQueuedCompletionStatus+0x29
03 00ccff8c 5a3026bc W3TP!THREAD_POOL_DATA::ThreadPoolThread+0x33
04 00ccffa0 5a301db9 W3TP!THREAD_POOL_DATA::ThreadPoolThread+0x24
05 00ccffb8 77e66063 W3TP!THREAD_MANAGER::ThreadManagerThread+0x39
06 00ccffec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 4
System Thread ID: 2110
Kernel Time: 0:0:0.15
User Time: 0:0:0.0
Thread Type: HTTP Listener
# ChildEBP RetAddr
00 00d0ff24 7c821bf4 ntdll!KiFastSystemCallRet
01 00d0ff28 77e6611a ntdll!NtRemoveIoCompletion+0xc
02 00d0ff54 5a30249e kernel32!GetQueuedCompletionStatus+0x29
03 00d0ff8c 5a3026bc W3TP!THREAD_POOL_DATA::ThreadPoolThread+0x33
04 00d0ffa0 5a301db9 W3TP!THREAD_POOL_DATA::ThreadPoolThread+0x24
05 00d0ffb8 77e66063 W3TP!THREAD_MANAGER::ThreadManagerThread+0x39
06 00d0ffec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 5
System Thread ID: 19bc
Kernel Time: 0:0:0.15
User Time: 0:0:0.0
Thread Type: HTTP Listener
# ChildEBP RetAddr
00 00d4ff24 7c821bf4 ntdll!KiFastSystemCallRet
01 00d4ff28 77e6611a ntdll!NtRemoveIoCompletion+0xc
02 00d4ff54 5a30249e kernel32!GetQueuedCompletionStatus+0x29
03 00d4ff8c 5a3026bc W3TP!THREAD_POOL_DATA::ThreadPoolThread+0x33
04 00d4ffa0 5a301db9 W3TP!THREAD_POOL_DATA::ThreadPoolThread+0x24
05 00d4ffb8 77e66063 W3TP!THREAD_MANAGER::ThreadManagerThread+0x39
06 00d4ffec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 6
System Thread ID: 12e8
Kernel Time: 0:0:0.15
User Time: 0:0:0.0
Thread Status: Thread is in a WAIT state.
Thread Type: Other
# ChildEBP RetAddr
00 00e0fcec 7c822114 ntdll!KiFastSystemCallRet
01 00e0fcf0 7c83acfd ntdll!NtWaitForMultipleObjects+0xc
02 00e0ffb8 77e66063 ntdll!RtlpWaitThread+0x161
03 00e0ffec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 7
System Thread ID: 10e0
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Status: Thread is in a WAIT state.
Thread Type: HTTP Compression Thread
# ChildEBP RetAddr
00 017dfa84 7c822124 ntdll!KiFastSystemCallRet
01 017dfa88 77e6baa8 ntdll!NtWaitForSingleObject+0xc
02 017dfaf8 77e6ba12 kernel32!WaitForSingleObjectEx+0xac
03 017dfb0c 5a3b8147 kernel32!WaitForSingleObject+0x12
04 017dffb8 77e66063 w3core!HTTP_COMPRESSION::CompressionThread+0x126
05 017dffec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 8
System Thread ID: 10a4
Kernel Time: 0:0:0.15
User Time: 0:0:0.0
Thread Type: Other
# ChildEBP RetAddr
00 0185ff8c 7c821364 ntdll!KiFastSystemCallRet
01 0185ff90 7c815267 ntdll!NtDelayExecution+0xc
02 0185ffb8 77e66063 ntdll!RtlpIOWorkerThread+0x3f
03 0185ffec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 9
System Thread ID: 288
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Type: Other
# ChildEBP RetAddr
00 0191fec0 7c821bf4 ntdll!KiFastSystemCallRet
01 0191fec4 77e6611a ntdll!NtRemoveIoCompletion+0xc
02 0191fef0 791fdb22 kernel32!GetQueuedCompletionStatus+0x29
03 0191ff24 791fdacc mscorsvr+0x4db22
04 0191ffb8 77e66063 mscorsvr+0x4dacc
05 0191ffec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 10
System Thread ID: 1a4
Kernel Time: 0:0:0.15
User Time: 0:0:0.0
Thread Type: Other
# ChildEBP RetAddr
00 0195fe50 7c821364 ntdll!KiFastSystemCallRet
01 0195fe54 77e42439 ntdll!NtDelayExecution+0xc
02 0195febc 77e424b7 kernel32!SleepEx+0x68
03 0195fecc 791bf4f9 kernel32!Sleep+0xf
04 0195ffb8 77e66063 mscorsvr+0xf4f9
05 0195ffec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 11
System Thread ID: 178
Kernel Time: 0:0:0.765
User Time: 0:0:5.453
Thread Status: Thread is in a WAIT state.
Thread Type: Other
# ChildEBP RetAddr
00 0199fdf8 7c822124 ntdll!KiFastSystemCallRet
01 0199fdfc 77e6baa8 ntdll!NtWaitForSingleObject+0xc
02 0199fe6c 77e6ba12 kernel32!WaitForSingleObjectEx+0xac
03 0199fe80 791d401f kernel32!WaitForSingleObject+0x12
04 0199fea4 791fdacc mscorsvr+0x2401f
05 0199ffb8 77e66063 mscorsvr+0x4dacc
06 0199ffec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 12
System Thread ID: 265c
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Status: Thread is in a WAIT state.
Thread Type: Other
# ChildEBP RetAddr
00 01acfe70 7c822114 ntdll!KiFastSystemCallRet
01 01acfe74 77e6711b ntdll!NtWaitForMultipleObjects+0xc
02 01acff1c 77e61075 kernel32!WaitForMultipleObjectsEx+0x11a
03 01acff38 79238fd6 kernel32!WaitForMultipleObjects+0x18
04 01acffa0 79238f4d mscorsvr+0x88fd6
05 01acffb0 7923a714 mscorsvr+0x88f4d
06 01acffb8 77e66063 mscorsvr+0x8a714
07 01acffec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 13
System Thread ID: 1fe4
Kernel Time: 0:0:0.46
User Time: 0:0:0.906
Thread Status: Thread is in a WAIT state.
Thread Type: Other
# ChildEBP RetAddr
00 01d5fefc 7c822124 ntdll!KiFastSystemCallRet
01 01d5ff00 77e6baa8 ntdll!NtWaitForSingleObject+0xc
02 01d5ff70 77e6ba12 kernel32!WaitForSingleObjectEx+0xac
03 01d5ff84 791fe6b0 kernel32!WaitForSingleObject+0x12
04 01d5ffac 792356bf mscorsvr+0x4e6b0
05 01d5ffb8 77e66063 mscorsvr+0x856bf
06 01d5ffec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 14
System Thread ID: 239c
Kernel Time: 0:0:0.31
User Time: 0:0:1.93
Thread Status: Thread is in a WAIT state.
Thread Type: Other
# ChildEBP RetAddr
00 01d9fdfc 7c822124 ntdll!KiFastSystemCallRet
01 01d9fe00 77e6baa8 ntdll!NtWaitForSingleObject+0xc
02 01d9fe70 77e6ba12 kernel32!WaitForSingleObjectEx+0xac
03 01d9fe84 791fe6b0 kernel32!WaitForSingleObject+0x12
04 01d9feac 792356bf mscorsvr+0x4e6b0
05 01d9ffb8 77e66063 mscorsvr+0x856bf
06 01d9ffec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 15
System Thread ID: 11f4
Kernel Time: 0:0:0.46
User Time: 0:0:0.843
Thread Status: Thread is in a WAIT state.
Thread Type: Other
# ChildEBP RetAddr
00 01ddfcfc 7c822124 ntdll!KiFastSystemCallRet
01 01ddfd00 77e6baa8 ntdll!NtWaitForSingleObject+0xc
02 01ddfd70 77e6ba12 kernel32!WaitForSingleObjectEx+0xac
03 01ddfd84 791fe6b0 kernel32!WaitForSingleObject+0x12
04 01ddfdac 792356bf mscorsvr+0x4e6b0
05 01ddffb8 77e66063 mscorsvr+0x856bf
06 01ddffec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 16
System Thread ID: 1b2c
Kernel Time: 0:0:0.0
User Time: 0:0:1.140
Thread Status: Thread is in a WAIT state.
Thread Type: Other
# ChildEBP RetAddr
00 01e1fbfc 7c822124 ntdll!KiFastSystemCallRet
01 01e1fc00 77e6baa8 ntdll!NtWaitForSingleObject+0xc
02 01e1fc70 77e6ba12 kernel32!WaitForSingleObjectEx+0xac
03 01e1fc84 791fe6b0 kernel32!WaitForSingleObject+0x12
04 01e1fcac 792356bf mscorsvr+0x4e6b0
05 01e1ffb8 77e66063 mscorsvr+0x856bf
06 01e1ffec 00000000 kernel32!BaseThreadStart+0x34
GetContextState failed, 0x8007001F
GetContextState failed, 0x8007001F
GetContextState failed, 0x8007001F
Thread ID: 17
System Thread ID: 0
Kernel Time: 0:0:0.15
User Time: 0:0:0.15
GetContextState failed, 0x8007001F
Post by Pat [MSFT]
You can try IISState. It doesn't fault on every exception.
iisstate -p <pid of w3wp.exe> -d <enter>
pat
Post by RB
If this may not be the problem, how can we pin point what the issue is. Can I
use DebugDiag differently to get that data? Should I use a different tool?
Thanks for your help.
Rani Baki
Post by Pat [MSFT]
What is _probably_ happening is you have a lot of exceptions happening.
So,
as soon as you enable DebugDiag - it trips. Exceptions are not inherently
unstable (i.e. leading to a crash or hang) - though they are excessively
expensive. Code that relies on them for normal operations runs much, much
slower than code that only occasionally hits one.
In thics case, the GCM.Framework.Web.UI.GCMClipboard.OnInit() function
appears to be leading to an exception being thrown. Again, this is not
inherently a problem - just a possible issue.
Pat
Post by RB
I accidently deleted the original dump file, but do have another one
for
a
different app. This one is faulting on thread 25. Below is the
output.
Keep
in mind that we have multiple apps hanging. Is it typical for
DebugDiag
to
generate a first exception almost immediately after activating a
rule?
That's
what is happening on those troubled apps.
Thanks,
Rani Baki
0:025> kb
ChildEBP RetAddr Args to Child
009aea9c 79216aed e0434f4d 00000001 00000000
kernel32!RaiseException+0x53
009aeaf4 79216a70 142fbfcc 00000000 009aeb38
mscorsvr!RaiseTheException+0xa0
009aeb1c 79216a24 142fbfcc 00000000 009aeb80
mscorsvr!RealCOMPlusThrow+0x48
009aeb2c 7927f019 142fbfcc 009af154 792ba800
mscorsvr!RealCOMPlusThrow+0xd
009aeb80 7943e9ef 009af39c 021e31e8 0a000129
mscorsvr!CEEInfo::findMethod+0x123
009aeb9c 79445a7f 0a000129 021e31e8 02642a28
mscorjit!Compiler::eeFindMethod+0x23
009af0bc 7943fcc4 01eb1e18 01eb1ec4 01eb0010
mscorjit!Compiler::impImportBlockCode+0x1e80
009af164 79440490 01eb1e18 009af3cc 01eb0010
mscorjit!Compiler::impImportBlock+0x222
009af17c 794428b5 01eb1e18 01eb0010 794310e5
mscorjit!Compiler::impImport+0xe8
009af188 794310e5 01eb0010 009af1e0 79431477
mscorjit!Compiler::fgImport+0x41
009af194 79431477 009af2a8 009af424 009af29c
mscorjit!Compiler::compCompile+0xb
009af1e0 7943151a 021e31e8 009af380 009af3cc
mscorjit!Compiler::compCompile+0x1e8
009af268 794315a4 009af380 009af3cc 009af2a8
mscorjit!jitNativeCode+0x95
009af2ac 791bc3f8 79479124 009af380 009af3cc
mscorjit!CILJit::compileMethod+0xa2
009af2f4 791bb52a 0012a100 009af380 009af3cc
mscorsvr!CallCompileMethodWithSEHWrapper+0x52
009af45c 791bb9c0 02642a28 009af624 009af560
mscorsvr!JITFunction+0x2c7
009af590 791b706a 02642a28 009af624 00000000
mscorsvr!MakeJitWorker+0x2c0
009af67c 791b77e4 02642cd4 009afaa0 009af6b8
mscorsvr!MethodDesc::DoPrestub+0x4d3
009af694 019c2f76 009af6b8 14300f5c 1c34b308
mscorsvr!PreStubWorker+0x42
WARNING: Frame IP not in any known module. Following frames may be wrong.
009af6f4 02659e32 00000008 009af734 1c34b114 0x19c2f76
0:025> !.\clr10\sos.ClrStack
Loaded Son of Strike data table version 5 from
"C:\WINDOWS\Microsoft.NET\Framework\v1.1.4322\mscorsvr.dll"
Thread 25
ESP EIP
0x009aeb50 0x77e55dea [FRAME: GCFrame]
0x009af60c 0x77e55dea [FRAME: GCFrame]
0x009af6b8 0x77e55dea [FRAME: PrestubMethodFrame] [DEFAULT]
[hasThis]
Void
GCM.Framework.Web.UI.GCMClipboard.OnInit(Class System.EventArgs)
0x009af6c8 0x0265db8a [DEFAULT] [hasThis] Void
System.Web.UI.Control.InitRecursive(Class System.Web.UI.Control)
0x009af6e4 0x02659f5a [DEFAULT] [hasThis] Void
System.Web.UI.Control.AddedControl(Class System.Web.UI.Control,I4)
0x009af700 0x02659e32 [DEFAULT] [hasThis] Void
System.Web.UI.ControlCollection.Add(Class System.Web.UI.Control)
0x009af710 0x026b748a [DEFAULT] [hasThis] Void
GCM.Framework.Web.UI.GCMWebPage.AddActionButtons()
0x009af7dc 0x0265e481 [DEFAULT] [hasThis] Void
GCM.Framework.Web.UI.GCMWebPage.CreateChildControls()
0x009af7e8 0x0265e27d [DEFAULT] [hasThis] Void
System.Web.UI.Control.EnsureChildControls()
0x009af810 0x0265dd56 [DEFAULT] [hasThis] Void
GCM.Framework.Web.UI.GCMWebPage.OnInit(Class System.EventArgs)
0x009af824 0x02621572 [DEFAULT] [hasThis] Void
GCM.Framework.Web.SubscriptionPopup.OnInit(Class System.EventArgs)
0x009af830 0x0265db8a [DEFAULT] [hasThis] Void
System.Web.UI.Control.InitRecursive(Class System.Web.UI.Control)
0x009af84c 0x0265c293 [DEFAULT] [hasThis] Void
System.Web.UI.Page.ProcessRequestMain()
0x009af890 0x02659ab7 [DEFAULT] [hasThis] Void
System.Web.UI.Page.ProcessRequest()
0x009af8cc 0x0265951b [DEFAULT] [hasThis] Void
System.Web.UI.Page.ProcessRequest(Class System.Web.HttpContext)
0x009af8d4 0x026594f4 [DEFAULT] [hasThis] Void
System.Web.HttpApplication/CallHandlerExecutionStep.System.Web.HttpApplication+IExecutionStep.Execute()
0x009af8e4 0x02619ea0 [DEFAULT] [hasThis] Class System.Exception
System.Web.HttpApplication.ExecuteStep(Class IExecutionStep,ByRef Boolean)
0x009af92c 0x0261990a [DEFAULT] [hasThis] Void
System.Web.HttpApplication.ResumeSteps(Class System.Exception)
0x009af974 0x026197db [DEFAULT] [hasThis] Class System.IAsyncResult
System.Web.HttpApplication.System.Web.IHttpAsyncHandler.BeginProcessRequest(Class
System.Web.HttpContext,Class System.AsyncCallback,Object)
0x009af990 0x01ec68a7 [DEFAULT] [hasThis] Void
System.Web.HttpRuntime.ProcessRequestInternal(Class
System.Web.HttpWorkerRequest)
0x009af9cc 0x01ec6458 [DEFAULT] Void
System.Web.HttpRuntime.ProcessRequest(Class
System.Web.HttpWorkerRequest)
0x009af9d8 0x01ec2fd5 [DEFAULT] [hasThis] I4
System.Web.Hosting.ISAPIRuntime.ProcessRequest(I,I4)
0x009afaa0 0x79217188 [FRAME: ContextTransitionFrame]
0x009afb80 0x79217188 [FRAME: ComMethodFrame]
Post by Pat [MSFT]
!.\clr10\sos.ClrStack <enter>
Pat
Post by RB
0:016> kb
ChildEBP RetAddr Args to Child
0199f580 79216aed e0434f4d 00000001 00000000
kernel32!RaiseException+0x53
0199f5d8 7924c8d0 183f0fe4 00000000 183f0fd0
mscorsvr!RaiseTheException+0xa0
0199f644 026357e2 183f05ec 183f0754 183f0730
mscorsvr!JIT_Throw+0x4d
WARNING: Frame IP not in any known module. Following frames may
be
wrong.
0199f670 0263d8f8 182d4034 183f05ec 183f05ec 0x26357e2
0199f6a4 0263d5d6 183e7d78 183f05ec 183f0730 0x263d8f8
0199f6d8 02697e52 183f0cfc 183e7d78 183f0730 0x263d5d6
0199f700 026982b2 183f0638 183f01cc 183649a4 0x2697e52
0199f704 183f0638 183f01cc 183649a4 183647ac 0x26982b2
0199f708 183f01cc 183649a4 183647ac 183f03e4 0x183f0638
0199f70c 183649a4 183647ac 183f03e4 00000000 0x183f01cc
0199f710 183647ac 183f03e4 00000000 02698278 0x183649a4
0199f714 183f03e4 00000000 02698278 02698100 0x183647ac
0199f718 00000000 02698278 02698100 0199f76c 0x183f03e4
0:016> !clrstack
No export clrstack found
I did execute a manual hang dump during the incident, aside from missing
symbols, the analysis for that app does not show any errors.
Thanks,
Rani Baki
Post by Pat [MSFT]
The crash occurred in managed code, apparently on thread 16. If a
dump
file
..sympath
srv*c:\debug*http://msdl.microsoft.com/download/symbols
<enter>
2) Reset the debug environment
..reload <enter>
3) Switch to thread 16
~16s <enter>
kb <enter>
5) Dump the managed code stack
!clrstack <enter>
You can post the results and we may be able to identify the issue.
Since you are _also_ experiencing hangs, it is likely that you
are
having
multiple problems. To diagnose that you need a log/dump while the
problem
is actually occurring.
Pat
Post by RB
Hello, we are having issues with web applications hanging
frequently.
We
eventually setup pools and split up the applications assuming
they
were
conflicting with each other. We are still experiencing hangs.
Below
are
excerpts from the dump of one of the hangs. Can someone please
help
in
analyzing what the cause may be. We are running IIS 6.0 on a
Windows
2003
server.
Error In
w3wp__PID__8496__Date__03_09_2006__Time_11_33_02AM__506__First
chance exception 0XC0000005.dmp the assembly instruction at
2d944ba
which
does not correspond to any known native module in the process has
caused
an
access violation exception (0xC0000005) when trying to read from
memory
location 0x00000000 on thread 16
If no further information can be obtained from the detailed stack
analysis
below, please contact Microsoft Corporation for
troubleshooting
steps
on
stack corruption
Error In
w3wp__PID__8496__Date__03_09_2006__Time_11_33_02AM__959__First
chance exception 0XC0000005.dmp the assembly instruction at
kernel32!RaiseException+53 in C:\WINDOWS\system32\kernel32.dll from
Microsoft
Corporation has caused an access violation exception (0xC0000005)
when
trying
to read from memory location 0x00000000 on thread 16
Please follow up with the vendor Microsoft Corporation for
C:\WINDOWS\system32\kernel32.dll
Error In
w3wp__PID__4788__Date__03_09_2006__Time_11_01_24AM__640__First
chance exception 0XE0434F4D.dmp the assembly instruction at
kernel32!RaiseException+53 in C:\WINDOWS\system32\kernel32.dll from
Microsoft
Corporation has caused an unknown exception (0xe0434f4d) on
thread
19
An exception thrown by Kernel32!RaiseException usually indicates
a
problem
with another module. Please review the stack for the faulting
thread
(19)
further to determine which module actually threw the exception
raised
by
Kernel32.dll.
Rani Baki
2006-03-14 23:18:28 UTC
Permalink
I had a feeling that's what happened. Another hang took place and I ran
iisstate and waited it to finish. Hopefully this will show what's going on:

Opened log file 'C:\iisstate\output\IISState-2752.log'

***********************
Starting new log output
IISState version 3.3.1

Tue Mar 14 13:48:42 2006

OS = Windows 2003 Server
Executable: w3wp.exe
PID = 2752

Note: Thread times are formatted as HH:MM:SS.ms

***********************




Thread ID: 0
System Thread ID: 1dcc
Kernel Time: 0:0:0.15
User Time: 0:0:0.46
Thread Status: Thread is in a WAIT state.
Thread Type: HTTP Compression Thread
# ChildEBP RetAddr
00 0006fc08 7c822124 ntdll!KiFastSystemCallRet
01 0006fc0c 77e6baa8 ntdll!NtWaitForSingleObject+0xc
02 0006fc7c 77e6ba12 kernel32!WaitForSingleObjectEx+0xac
03 0006fc90 5a36467a kernel32!WaitForSingleObject+0x12
04 0006fca0 5a366e63 w3dt!WP_CONTEXT::RunMainThreadLoop+0x10
05 0006fca8 5a3af41d w3dt!UlAtqStartListen+0x2d
06 0006fcb8 5a3bc259 w3core!W3_SERVER::StartListen+0xbd
07 0006ff0c 0100187c w3core!UlW3Start+0x26e
08 0006ff44 01001a23 w3wp!wmain+0x22a
09 0006ffc0 77e523cd w3wp!wmainCRTStartup+0x12b
0a 0006fff0 00000000 kernel32!BaseProcessStart+0x23




Thread ID: 1
System Thread ID: 221c
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Type: Other
# ChildEBP RetAddr
00 009eff9c 7c821364 ntdll!KiFastSystemCallRet
01 009effa0 7c81fe26 ntdll!NtDelayExecution+0xc
02 009effb8 77e66063 ntdll!RtlpTimerThread+0x47
03 009effec 00000000 kernel32!BaseThreadStart+0x34




Thread ID: 2
System Thread ID: 2640
Kernel Time: 0:0:0.0
User Time: 0:0:0.15
Thread Type: Possible ASP page. Possible DCOM activity
Executing Page: ASP.dll symbols not found. Unable to locate ASP page.
Continuing with other analysis.

No remote call being made

# ChildEBP RetAddr
00 00a8fe18 7c821c54 ntdll!KiFastSystemCallRet
01 00a8fe1c 77c7538c ntdll!ZwReplyWaitReceivePortEx+0xc
02 00a8ff84 77c5778f RPCRT4!LRPC_ADDRESS::ReceiveLotsaCalls+0x198
03 00a8ff8c 77c5f7dd RPCRT4!RecvLotsaCallsWrapper+0xd
04 00a8ffac 77c5de88 RPCRT4!BaseCachedThreadRoutine+0x9d
05 00a8ffb8 77e66063 RPCRT4!ThreadStartRoutine+0x1b
06 00a8ffec 00000000 kernel32!BaseThreadStart+0x34




Thread ID: 3
System Thread ID: 11e0
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Type: HTTP Listener
# ChildEBP RetAddr
00 00c8ff24 7c821bf4 ntdll!KiFastSystemCallRet
01 00c8ff28 77e6611a ntdll!NtRemoveIoCompletion+0xc
02 00c8ff54 5a30249e kernel32!GetQueuedCompletionStatus+0x29
03 00c8ff8c 5a3026bc W3TP!THREAD_POOL_DATA::ThreadPoolThread+0x33
04 00c8ffa0 5a301db9 W3TP!THREAD_POOL_DATA::ThreadPoolThread+0x24
05 00c8ffb8 77e66063 W3TP!THREAD_MANAGER::ThreadManagerThread+0x39
06 00c8ffec 00000000 kernel32!BaseThreadStart+0x34




Thread ID: 4
System Thread ID: 92c
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Type: HTTP Listener
# ChildEBP RetAddr
00 00ccff24 7c821bf4 ntdll!KiFastSystemCallRet
01 00ccff28 77e6611a ntdll!NtRemoveIoCompletion+0xc
02 00ccff54 5a30249e kernel32!GetQueuedCompletionStatus+0x29
03 00ccff8c 5a3026bc W3TP!THREAD_POOL_DATA::ThreadPoolThread+0x33
04 00ccffa0 5a301db9 W3TP!THREAD_POOL_DATA::ThreadPoolThread+0x24
05 00ccffb8 77e66063 W3TP!THREAD_MANAGER::ThreadManagerThread+0x39
06 00ccffec 00000000 kernel32!BaseThreadStart+0x34




Thread ID: 5
System Thread ID: 16c0
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Type: HTTP Listener
# ChildEBP RetAddr
00 00d0ff24 7c821bf4 ntdll!KiFastSystemCallRet
01 00d0ff28 77e6611a ntdll!NtRemoveIoCompletion+0xc
02 00d0ff54 5a30249e kernel32!GetQueuedCompletionStatus+0x29
03 00d0ff8c 5a3026bc W3TP!THREAD_POOL_DATA::ThreadPoolThread+0x33
04 00d0ffa0 5a301db9 W3TP!THREAD_POOL_DATA::ThreadPoolThread+0x24
05 00d0ffb8 77e66063 W3TP!THREAD_MANAGER::ThreadManagerThread+0x39
06 00d0ffec 00000000 kernel32!BaseThreadStart+0x34




Thread ID: 6
System Thread ID: 1dfc
Kernel Time: 0:0:0.15
User Time: 0:0:0.0
Thread Type: HTTP Listener
# ChildEBP RetAddr
00 00d4ff24 7c821bf4 ntdll!KiFastSystemCallRet
01 00d4ff28 77e6611a ntdll!NtRemoveIoCompletion+0xc
02 00d4ff54 5a30249e kernel32!GetQueuedCompletionStatus+0x29
03 00d4ff8c 5a3026bc W3TP!THREAD_POOL_DATA::ThreadPoolThread+0x33
04 00d4ffa0 5a301db9 W3TP!THREAD_POOL_DATA::ThreadPoolThread+0x24
05 00d4ffb8 77e66063 W3TP!THREAD_MANAGER::ThreadManagerThread+0x39
06 00d4ffec 00000000 kernel32!BaseThreadStart+0x34




Thread ID: 7
System Thread ID: 25ec
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Status: Thread is in a WAIT state.
Thread Type: Other
# ChildEBP RetAddr
00 00e0fcec 7c822114 ntdll!KiFastSystemCallRet
01 00e0fcf0 7c83acfd ntdll!NtWaitForMultipleObjects+0xc
02 00e0ffb8 77e66063 ntdll!RtlpWaitThread+0x161
03 00e0ffec 00000000 kernel32!BaseThreadStart+0x34




Thread ID: 8
System Thread ID: 196c
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Status: Thread is in a WAIT state.
Thread Type: HTTP Compression Thread
# ChildEBP RetAddr
00 017dfa84 7c822124 ntdll!KiFastSystemCallRet
01 017dfa88 77e6baa8 ntdll!NtWaitForSingleObject+0xc
02 017dfaf8 77e6ba12 kernel32!WaitForSingleObjectEx+0xac
03 017dfb0c 5a3b8147 kernel32!WaitForSingleObject+0x12
04 017dffb8 77e66063 w3core!HTTP_COMPRESSION::CompressionThread+0x126
05 017dffec 00000000 kernel32!BaseThreadStart+0x34




Thread ID: 9
System Thread ID: 1bc4
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Type: Other
# ChildEBP RetAddr
00 0185ff8c 7c821364 ntdll!KiFastSystemCallRet
01 0185ff90 7c815267 ntdll!NtDelayExecution+0xc
02 0185ffb8 77e66063 ntdll!RtlpIOWorkerThread+0x3f
03 0185ffec 00000000 kernel32!BaseThreadStart+0x34




Thread ID: 10
System Thread ID: 1618
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Type: Other
# ChildEBP RetAddr
00 0191fec0 7c821bf4 ntdll!KiFastSystemCallRet
01 0191fec4 77e6611a ntdll!NtRemoveIoCompletion+0xc
02 0191fef0 791fdb22 kernel32!GetQueuedCompletionStatus+0x29
03 0191ff24 791fdacc mscorsvr+0x4db22
04 0191ffb8 77e66063 mscorsvr+0x4dacc
05 0191ffec 00000000 kernel32!BaseThreadStart+0x34




Thread ID: 11
System Thread ID: 190c
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Type: Other
# ChildEBP RetAddr
00 0195fe50 7c821364 ntdll!KiFastSystemCallRet
01 0195fe54 77e42439 ntdll!NtDelayExecution+0xc
02 0195febc 77e424b7 kernel32!SleepEx+0x68
03 0195fecc 791bf4f9 kernel32!Sleep+0xf
04 0195ffb8 77e66063 mscorsvr+0xf4f9
05 0195ffec 00000000 kernel32!BaseThreadStart+0x34




Thread ID: 12
System Thread ID: 1e2c
Kernel Time: 0:0:0.171
User Time: 0:0:1.156
Thread Status: Thread is in a WAIT state.
Thread Type: Other
# ChildEBP RetAddr
00 0199fdf8 7c822124 ntdll!KiFastSystemCallRet
01 0199fdfc 77e6baa8 ntdll!NtWaitForSingleObject+0xc
02 0199fe6c 77e6ba12 kernel32!WaitForSingleObjectEx+0xac
03 0199fe80 791d401f kernel32!WaitForSingleObject+0x12
04 0199fea4 791fdacc mscorsvr+0x2401f
05 0199ffb8 77e66063 mscorsvr+0x4dacc
06 0199ffec 00000000 kernel32!BaseThreadStart+0x34




Thread ID: 13
System Thread ID: 1ec8
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Status: Thread is in a WAIT state.
Thread Type: Other
# ChildEBP RetAddr
00 01acfe70 7c822114 ntdll!KiFastSystemCallRet
01 01acfe74 77e6711b ntdll!NtWaitForMultipleObjects+0xc
02 01acff1c 77e61075 kernel32!WaitForMultipleObjectsEx+0x11a
03 01acff38 79238fd6 kernel32!WaitForMultipleObjects+0x18
04 01acffa0 79238f4d mscorsvr+0x88fd6
05 01acffb0 7923a714 mscorsvr+0x88f4d
06 01acffb8 77e66063 mscorsvr+0x8a714
07 01acffec 00000000 kernel32!BaseThreadStart+0x34




Thread ID: 14
System Thread ID: 1520
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Status: Thread is in a WAIT state.
Thread Type: Other
# ChildEBP RetAddr
00 01d5fefc 7c822124 ntdll!KiFastSystemCallRet
01 01d5ff00 77e6baa8 ntdll!NtWaitForSingleObject+0xc
02 01d5ff70 77e6ba12 kernel32!WaitForSingleObjectEx+0xac
03 01d5ff84 791fe6b0 kernel32!WaitForSingleObject+0x12
04 01d5ffac 792356bf mscorsvr+0x4e6b0
05 01d5ffb8 77e66063 mscorsvr+0x856bf
06 01d5ffec 00000000 kernel32!BaseThreadStart+0x34




Thread ID: 15
System Thread ID: 1f18
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Status: Thread is in a WAIT state.
Thread Type: Other
# ChildEBP RetAddr
00 01d9fdfc 7c822124 ntdll!KiFastSystemCallRet
01 01d9fe00 77e6baa8 ntdll!NtWaitForSingleObject+0xc
02 01d9fe70 77e6ba12 kernel32!WaitForSingleObjectEx+0xac
03 01d9fe84 791fe6b0 kernel32!WaitForSingleObject+0x12
04 01d9feac 792356bf mscorsvr+0x4e6b0
05 01d9ffb8 77e66063 mscorsvr+0x856bf
06 01d9ffec 00000000 kernel32!BaseThreadStart+0x34




Thread ID: 16
System Thread ID: 11c0
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Status: Thread is in a WAIT state.
Thread Type: Other
# ChildEBP RetAddr
00 01ddfcfc 7c822124 ntdll!KiFastSystemCallRet
01 01ddfd00 77e6baa8 ntdll!NtWaitForSingleObject+0xc
02 01ddfd70 77e6ba12 kernel32!WaitForSingleObjectEx+0xac
03 01ddfd84 791fe6b0 kernel32!WaitForSingleObject+0x12
04 01ddfdac 792356bf mscorsvr+0x4e6b0
05 01ddffb8 77e66063 mscorsvr+0x856bf
06 01ddffec 00000000 kernel32!BaseThreadStart+0x34




Thread ID: 17
System Thread ID: 258c
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Status: Thread is in a WAIT state.
Thread Type: Other
# ChildEBP RetAddr
00 01e1fbfc 7c822124 ntdll!KiFastSystemCallRet
01 01e1fc00 77e6baa8 ntdll!NtWaitForSingleObject+0xc
02 01e1fc70 77e6ba12 kernel32!WaitForSingleObjectEx+0xac
03 01e1fc84 791fe6b0 kernel32!WaitForSingleObject+0x12
04 01e1fcac 792356bf mscorsvr+0x4e6b0
05 01e1ffb8 77e66063 mscorsvr+0x856bf
06 01e1ffec 00000000 kernel32!BaseThreadStart+0x34




Thread ID: 18
System Thread ID: 226c
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Status: Thread is in a WAIT state.
Thread Type: Other
# ChildEBP RetAddr
00 01e5fdf8 7c822114 ntdll!KiFastSystemCallRet
01 01e5fdfc 77e6711b ntdll!NtWaitForMultipleObjects+0xc
02 01e5fea4 77e61075 kernel32!WaitForMultipleObjectsEx+0x11a
03 01e5fec0 7927826b kernel32!WaitForMultipleObjects+0x18
04 01e5fee0 791fecf4 mscorsvr+0xc826b
05 01e5ff24 79245681 mscorsvr+0x4ecf4
06 01e5ffb8 77e66063 mscorsvr+0x95681
07 01e5ffec 00000000 kernel32!BaseThreadStart+0x34




Thread ID: 19
System Thread ID: 1390
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Type: Other
# ChildEBP RetAddr
00 0201ff38 7c821364 ntdll!KiFastSystemCallRet
01 0201ff3c 77e42439 ntdll!NtDelayExecution+0xc
02 0201ffa4 791cc578 kernel32!SleepEx+0x68
03 0201ffb8 77e66063 mscorsvr+0x1c578
04 0201ffec 00000000 kernel32!BaseThreadStart+0x34




Thread ID: 20
System Thread ID: b04
Kernel Time: 0:0:0.0
User Time: 0:0:0.31
Thread Status: Thread is in a WAIT state.
Thread Type: Other
# ChildEBP RetAddr
00 0296fa78 7c822124 ntdll!KiFastSystemCallRet
01 0296fa7c 77e6baa8 ntdll!NtWaitForSingleObject+0xc
02 0296faec 77e6ba12 kernel32!WaitForSingleObjectEx+0xac
03 0296fb00 791d401f kernel32!WaitForSingleObject+0x12
04 0296fb24 791fdacc mscorsvr+0x2401f
05 0296ffb8 77e66063 mscorsvr+0x4dacc
06 0296ffec 00000000 kernel32!BaseThreadStart+0x34




Thread ID: 21
System Thread ID: 840
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Type: Other
# ChildEBP RetAddr
00 029aff70 7c821bf4 ntdll!KiFastSystemCallRet
01 029aff74 7c83ad75 ntdll!NtRemoveIoCompletion+0xc
02 029affb8 77e66063 ntdll!RtlpWorkerThread+0x3d
03 029affec 00000000 kernel32!BaseThreadStart+0x34




Thread ID: 22
System Thread ID: 2178
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Type: Other
# ChildEBP RetAddr
00 0214f7c0 7c821bf4 ntdll!KiFastSystemCallRet
01 0214f7c4 77e6611a ntdll!NtRemoveIoCompletion+0xc
02 0214f7f0 791fdb22 kernel32!GetQueuedCompletionStatus+0x29
03 0214f824 791fdacc mscorsvr+0x4db22
04 0214ffb8 77e66063 mscorsvr+0x4dacc
05 0214ffec 00000000 kernel32!BaseThreadStart+0x34




Thread ID: 23
System Thread ID: 2118
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Type: Other
# ChildEBP RetAddr
00 009aff70 7c821bf4 ntdll!KiFastSystemCallRet
01 009aff74 7c83ad75 ntdll!NtRemoveIoCompletion+0xc
02 009affb8 77e66063 ntdll!RtlpWorkerThread+0x3d
03 009affec 00000000 kernel32!BaseThreadStart+0x34




Thread ID: 24
System Thread ID: 1380
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Type: Other
# ChildEBP RetAddr
00 00a2ff70 7c821bf4 ntdll!KiFastSystemCallRet
01 00a2ff74 7c83ad75 ntdll!NtRemoveIoCompletion+0xc
02 00a2ffb8 77e66063 ntdll!RtlpWorkerThread+0x3d
03 00a2ffec 00000000 kernel32!BaseThreadStart+0x34




Thread ID: 25
System Thread ID: 1524
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Status: Thread is in a WAIT state.
Thread Type: Other
# ChildEBP RetAddr
00 00acf4f8 7c822124 ntdll!KiFastSystemCallRet
01 00acf4fc 77e6baa8 ntdll!NtWaitForSingleObject+0xc
02 00acf56c 77e6ba12 kernel32!WaitForSingleObjectEx+0xac
03 00acf580 791d401f kernel32!WaitForSingleObject+0x12
04 00acf5a4 791fdacc mscorsvr+0x2401f
05 00acffb8 77e66063 mscorsvr+0x4dacc
06 00acffec 00000000 kernel32!BaseThreadStart+0x34




Thread ID: 26
System Thread ID: 1aa8
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Type: Other
# ChildEBP RetAddr
00 0181ff70 7c821bf4 ntdll!KiFastSystemCallRet
01 0181ff74 7c83ad75 ntdll!NtRemoveIoCompletion+0xc
02 0181ffb8 77e66063 ntdll!RtlpWorkerThread+0x3d
03 0181ffec 00000000 kernel32!BaseThreadStart+0x34




Thread ID: 27
System Thread ID: 1168
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Type: Other
# ChildEBP RetAddr
00 0209ff70 7c821bf4 ntdll!KiFastSystemCallRet
01 0209ff74 7c83ad75 ntdll!NtRemoveIoCompletion+0xc
02 0209ffb8 77e66063 ntdll!RtlpWorkerThread+0x3d
03 0209ffec 00000000 kernel32!BaseThreadStart+0x34




Thread ID: 28
System Thread ID: 1088
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Type: Other
# ChildEBP RetAddr
00 0210ff70 7c821bf4 ntdll!KiFastSystemCallRet
01 0210ff74 7c83ad75 ntdll!NtRemoveIoCompletion+0xc
02 0210ffb8 77e66063 ntdll!RtlpWorkerThread+0x3d
03 0210ffec 00000000 kernel32!BaseThreadStart+0x34

*****

Dump name is formatted as: PID-Timestamp.dmp

Creating C:\iisstate\output\2752-1142363561.dmp - mini user dump

*****

Closing open log file C:\iisstate\output\IISState-2752.log
Post by Pat [MSFT]
Thread 16 shows that IIS reset the process out from under you. The delay is
usually caused by the downloading of symbols - which are cached so the
problem goes away after the initial run.
-hc or -sc will only help you if you are seeing unexpected process
terminations (i.e. a crash). Hangs won't be caught by them.
Pat
Post by Rani Baki
I ran iisstate during an hang. It froze on thread 9, but eventually continued
and then it looks like it failed during thread 16/17. Maybe because the
worker process gave up. I also ran it using the -hc option and it hung on a
dll, could this dll be the culprit? I am including one output below and the
other on the next post.
Thanks,
Rani Baki
C:\iisstate>iisstate -p 5592 -d
SRV*C:\iisstate\symbols*http://msdl.microsoft.com/download/symbols
Microsoft (R) Windows Debugger Version 6.2.0013.1
Copyright (c) Microsoft Corporation. All rights reserved.
*** wait with pending attach
SRV*C:\iisstate\symbols*http://msdl.microsoft.com/download/symbols
WARNING: Process 5592 is not attached as a debuggee
The process can be examined but debug events will not be received
.................................................................................................................................................
The call to LoadLibrary(exts) failed, Win32 error 2
"The system cannot find the file specified."
Please check your debugger configuration and/or network access.
(15d8.2068): Wake debugger - code 80000007 (first chance)
Opened log file 'C:\iisstate\output\IISState-5592.log'
***********************
Starting new log output
IISState version 3.3.1
Tue Mar 14 10:06:51 2006
OS = Windows 2003 Server
Executable: w3wp.exe
PID = 5592
Note: Thread times are formatted as HH:MM:SS.ms
***********************
Thread ID: 0
System Thread ID: 2068
Kernel Time: 0:0:0.31
User Time: 0:0:0.31
Thread Status: Thread is in a WAIT state.
Thread Type: HTTP Compression Thread
# ChildEBP RetAddr
00 0006fc08 7c822124 ntdll!KiFastSystemCallRet
01 0006fc0c 77e6baa8 ntdll!NtWaitForSingleObject+0xc
02 0006fc7c 77e6ba12 kernel32!WaitForSingleObjectEx+0xac
03 0006fc90 5a36467a kernel32!WaitForSingleObject+0x12
04 0006fca0 5a366e63 w3dt!WP_CONTEXT::RunMainThreadLoop+0x10
05 0006fca8 5a3af41d w3dt!UlAtqStartListen+0x2d
06 0006fcb8 5a3bc259 w3core!W3_SERVER::StartListen+0xbd
07 0006ff0c 0100187c w3core!UlW3Start+0x26e
08 0006ff44 01001a23 w3wp!wmain+0x22a
09 0006ffc0 77e523cd w3wp!wmainCRTStartup+0x12b
0a 0006fff0 00000000 kernel32!BaseProcessStart+0x23
Thread ID: 1
System Thread ID: 16c0
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Type: Other
# ChildEBP RetAddr
00 009eff9c 7c821364 ntdll!KiFastSystemCallRet
01 009effa0 7c81fe26 ntdll!NtDelayExecution+0xc
02 009effb8 77e66063 ntdll!RtlpTimerThread+0x47
03 009effec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 2
System Thread ID: b10
Kernel Time: 0:0:0.15
User Time: 0:0:0.0
Thread Type: HTTP Listener
# ChildEBP RetAddr
00 00c8ff24 7c821bf4 ntdll!KiFastSystemCallRet
01 00c8ff28 77e6611a ntdll!NtRemoveIoCompletion+0xc
02 00c8ff54 5a30249e kernel32!GetQueuedCompletionStatus+0x29
03 00c8ff8c 5a3026bc W3TP!THREAD_POOL_DATA::ThreadPoolThread+0x33
04 00c8ffa0 5a301db9 W3TP!THREAD_POOL_DATA::ThreadPoolThread+0x24
05 00c8ffb8 77e66063 W3TP!THREAD_MANAGER::ThreadManagerThread+0x39
06 00c8ffec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 3
System Thread ID: 11cc
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Type: HTTP Listener
# ChildEBP RetAddr
00 00ccff24 7c821bf4 ntdll!KiFastSystemCallRet
01 00ccff28 77e6611a ntdll!NtRemoveIoCompletion+0xc
02 00ccff54 5a30249e kernel32!GetQueuedCompletionStatus+0x29
03 00ccff8c 5a3026bc W3TP!THREAD_POOL_DATA::ThreadPoolThread+0x33
04 00ccffa0 5a301db9 W3TP!THREAD_POOL_DATA::ThreadPoolThread+0x24
05 00ccffb8 77e66063 W3TP!THREAD_MANAGER::ThreadManagerThread+0x39
06 00ccffec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 4
System Thread ID: 2110
Kernel Time: 0:0:0.15
User Time: 0:0:0.0
Thread Type: HTTP Listener
# ChildEBP RetAddr
00 00d0ff24 7c821bf4 ntdll!KiFastSystemCallRet
01 00d0ff28 77e6611a ntdll!NtRemoveIoCompletion+0xc
02 00d0ff54 5a30249e kernel32!GetQueuedCompletionStatus+0x29
03 00d0ff8c 5a3026bc W3TP!THREAD_POOL_DATA::ThreadPoolThread+0x33
04 00d0ffa0 5a301db9 W3TP!THREAD_POOL_DATA::ThreadPoolThread+0x24
05 00d0ffb8 77e66063 W3TP!THREAD_MANAGER::ThreadManagerThread+0x39
06 00d0ffec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 5
System Thread ID: 19bc
Kernel Time: 0:0:0.15
User Time: 0:0:0.0
Thread Type: HTTP Listener
# ChildEBP RetAddr
00 00d4ff24 7c821bf4 ntdll!KiFastSystemCallRet
01 00d4ff28 77e6611a ntdll!NtRemoveIoCompletion+0xc
02 00d4ff54 5a30249e kernel32!GetQueuedCompletionStatus+0x29
03 00d4ff8c 5a3026bc W3TP!THREAD_POOL_DATA::ThreadPoolThread+0x33
04 00d4ffa0 5a301db9 W3TP!THREAD_POOL_DATA::ThreadPoolThread+0x24
05 00d4ffb8 77e66063 W3TP!THREAD_MANAGER::ThreadManagerThread+0x39
06 00d4ffec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 6
System Thread ID: 12e8
Kernel Time: 0:0:0.15
User Time: 0:0:0.0
Thread Status: Thread is in a WAIT state.
Thread Type: Other
# ChildEBP RetAddr
00 00e0fcec 7c822114 ntdll!KiFastSystemCallRet
01 00e0fcf0 7c83acfd ntdll!NtWaitForMultipleObjects+0xc
02 00e0ffb8 77e66063 ntdll!RtlpWaitThread+0x161
03 00e0ffec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 7
System Thread ID: 10e0
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Status: Thread is in a WAIT state.
Thread Type: HTTP Compression Thread
# ChildEBP RetAddr
00 017dfa84 7c822124 ntdll!KiFastSystemCallRet
01 017dfa88 77e6baa8 ntdll!NtWaitForSingleObject+0xc
02 017dfaf8 77e6ba12 kernel32!WaitForSingleObjectEx+0xac
03 017dfb0c 5a3b8147 kernel32!WaitForSingleObject+0x12
04 017dffb8 77e66063 w3core!HTTP_COMPRESSION::CompressionThread+0x126
05 017dffec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 8
System Thread ID: 10a4
Kernel Time: 0:0:0.15
User Time: 0:0:0.0
Thread Type: Other
# ChildEBP RetAddr
00 0185ff8c 7c821364 ntdll!KiFastSystemCallRet
01 0185ff90 7c815267 ntdll!NtDelayExecution+0xc
02 0185ffb8 77e66063 ntdll!RtlpIOWorkerThread+0x3f
03 0185ffec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 9
System Thread ID: 288
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Type: Other
# ChildEBP RetAddr
00 0191fec0 7c821bf4 ntdll!KiFastSystemCallRet
01 0191fec4 77e6611a ntdll!NtRemoveIoCompletion+0xc
02 0191fef0 791fdb22 kernel32!GetQueuedCompletionStatus+0x29
03 0191ff24 791fdacc mscorsvr+0x4db22
04 0191ffb8 77e66063 mscorsvr+0x4dacc
05 0191ffec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 10
System Thread ID: 1a4
Kernel Time: 0:0:0.15
User Time: 0:0:0.0
Thread Type: Other
# ChildEBP RetAddr
00 0195fe50 7c821364 ntdll!KiFastSystemCallRet
01 0195fe54 77e42439 ntdll!NtDelayExecution+0xc
02 0195febc 77e424b7 kernel32!SleepEx+0x68
03 0195fecc 791bf4f9 kernel32!Sleep+0xf
04 0195ffb8 77e66063 mscorsvr+0xf4f9
05 0195ffec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 11
System Thread ID: 178
Kernel Time: 0:0:0.765
User Time: 0:0:5.453
Thread Status: Thread is in a WAIT state.
Thread Type: Other
# ChildEBP RetAddr
00 0199fdf8 7c822124 ntdll!KiFastSystemCallRet
01 0199fdfc 77e6baa8 ntdll!NtWaitForSingleObject+0xc
02 0199fe6c 77e6ba12 kernel32!WaitForSingleObjectEx+0xac
03 0199fe80 791d401f kernel32!WaitForSingleObject+0x12
04 0199fea4 791fdacc mscorsvr+0x2401f
05 0199ffb8 77e66063 mscorsvr+0x4dacc
06 0199ffec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 12
System Thread ID: 265c
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Status: Thread is in a WAIT state.
Thread Type: Other
# ChildEBP RetAddr
00 01acfe70 7c822114 ntdll!KiFastSystemCallRet
01 01acfe74 77e6711b ntdll!NtWaitForMultipleObjects+0xc
02 01acff1c 77e61075 kernel32!WaitForMultipleObjectsEx+0x11a
03 01acff38 79238fd6 kernel32!WaitForMultipleObjects+0x18
04 01acffa0 79238f4d mscorsvr+0x88fd6
05 01acffb0 7923a714 mscorsvr+0x88f4d
06 01acffb8 77e66063 mscorsvr+0x8a714
07 01acffec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 13
System Thread ID: 1fe4
Kernel Time: 0:0:0.46
User Time: 0:0:0.906
Thread Status: Thread is in a WAIT state.
Thread Type: Other
# ChildEBP RetAddr
00 01d5fefc 7c822124 ntdll!KiFastSystemCallRet
01 01d5ff00 77e6baa8 ntdll!NtWaitForSingleObject+0xc
02 01d5ff70 77e6ba12 kernel32!WaitForSingleObjectEx+0xac
03 01d5ff84 791fe6b0 kernel32!WaitForSingleObject+0x12
04 01d5ffac 792356bf mscorsvr+0x4e6b0
05 01d5ffb8 77e66063 mscorsvr+0x856bf
06 01d5ffec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 14
Pat [MSFT]
2006-03-15 15:11:21 UTC
Permalink
Well...

The threads below show that the server had been serving up pages, but there
was no activity (pages executing, etc.) when the log was actually taken. Do
you have multiple w3wp.exe processes running?


Pat
Post by Rani Baki
I had a feeling that's what happened. Another hang took place and I ran
Opened log file 'C:\iisstate\output\IISState-2752.log'
***********************
Starting new log output
IISState version 3.3.1
Tue Mar 14 13:48:42 2006
OS = Windows 2003 Server
Executable: w3wp.exe
PID = 2752
Note: Thread times are formatted as HH:MM:SS.ms
***********************
Thread ID: 0
System Thread ID: 1dcc
Kernel Time: 0:0:0.15
User Time: 0:0:0.46
Thread Status: Thread is in a WAIT state.
Thread Type: HTTP Compression Thread
# ChildEBP RetAddr
00 0006fc08 7c822124 ntdll!KiFastSystemCallRet
01 0006fc0c 77e6baa8 ntdll!NtWaitForSingleObject+0xc
02 0006fc7c 77e6ba12 kernel32!WaitForSingleObjectEx+0xac
03 0006fc90 5a36467a kernel32!WaitForSingleObject+0x12
04 0006fca0 5a366e63 w3dt!WP_CONTEXT::RunMainThreadLoop+0x10
05 0006fca8 5a3af41d w3dt!UlAtqStartListen+0x2d
06 0006fcb8 5a3bc259 w3core!W3_SERVER::StartListen+0xbd
07 0006ff0c 0100187c w3core!UlW3Start+0x26e
08 0006ff44 01001a23 w3wp!wmain+0x22a
09 0006ffc0 77e523cd w3wp!wmainCRTStartup+0x12b
0a 0006fff0 00000000 kernel32!BaseProcessStart+0x23
Thread ID: 1
System Thread ID: 221c
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Type: Other
# ChildEBP RetAddr
00 009eff9c 7c821364 ntdll!KiFastSystemCallRet
01 009effa0 7c81fe26 ntdll!NtDelayExecution+0xc
02 009effb8 77e66063 ntdll!RtlpTimerThread+0x47
03 009effec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 2
System Thread ID: 2640
Kernel Time: 0:0:0.0
User Time: 0:0:0.15
Thread Type: Possible ASP page. Possible DCOM activity
Executing Page: ASP.dll symbols not found. Unable to locate ASP page.
Continuing with other analysis.
No remote call being made
# ChildEBP RetAddr
00 00a8fe18 7c821c54 ntdll!KiFastSystemCallRet
01 00a8fe1c 77c7538c ntdll!ZwReplyWaitReceivePortEx+0xc
02 00a8ff84 77c5778f RPCRT4!LRPC_ADDRESS::ReceiveLotsaCalls+0x198
03 00a8ff8c 77c5f7dd RPCRT4!RecvLotsaCallsWrapper+0xd
04 00a8ffac 77c5de88 RPCRT4!BaseCachedThreadRoutine+0x9d
05 00a8ffb8 77e66063 RPCRT4!ThreadStartRoutine+0x1b
06 00a8ffec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 3
System Thread ID: 11e0
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Type: HTTP Listener
# ChildEBP RetAddr
00 00c8ff24 7c821bf4 ntdll!KiFastSystemCallRet
01 00c8ff28 77e6611a ntdll!NtRemoveIoCompletion+0xc
02 00c8ff54 5a30249e kernel32!GetQueuedCompletionStatus+0x29
03 00c8ff8c 5a3026bc W3TP!THREAD_POOL_DATA::ThreadPoolThread+0x33
04 00c8ffa0 5a301db9 W3TP!THREAD_POOL_DATA::ThreadPoolThread+0x24
05 00c8ffb8 77e66063 W3TP!THREAD_MANAGER::ThreadManagerThread+0x39
06 00c8ffec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 4
System Thread ID: 92c
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Type: HTTP Listener
# ChildEBP RetAddr
00 00ccff24 7c821bf4 ntdll!KiFastSystemCallRet
01 00ccff28 77e6611a ntdll!NtRemoveIoCompletion+0xc
02 00ccff54 5a30249e kernel32!GetQueuedCompletionStatus+0x29
03 00ccff8c 5a3026bc W3TP!THREAD_POOL_DATA::ThreadPoolThread+0x33
04 00ccffa0 5a301db9 W3TP!THREAD_POOL_DATA::ThreadPoolThread+0x24
05 00ccffb8 77e66063 W3TP!THREAD_MANAGER::ThreadManagerThread+0x39
06 00ccffec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 5
System Thread ID: 16c0
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Type: HTTP Listener
# ChildEBP RetAddr
00 00d0ff24 7c821bf4 ntdll!KiFastSystemCallRet
01 00d0ff28 77e6611a ntdll!NtRemoveIoCompletion+0xc
02 00d0ff54 5a30249e kernel32!GetQueuedCompletionStatus+0x29
03 00d0ff8c 5a3026bc W3TP!THREAD_POOL_DATA::ThreadPoolThread+0x33
04 00d0ffa0 5a301db9 W3TP!THREAD_POOL_DATA::ThreadPoolThread+0x24
05 00d0ffb8 77e66063 W3TP!THREAD_MANAGER::ThreadManagerThread+0x39
06 00d0ffec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 6
System Thread ID: 1dfc
Kernel Time: 0:0:0.15
User Time: 0:0:0.0
Thread Type: HTTP Listener
# ChildEBP RetAddr
00 00d4ff24 7c821bf4 ntdll!KiFastSystemCallRet
01 00d4ff28 77e6611a ntdll!NtRemoveIoCompletion+0xc
02 00d4ff54 5a30249e kernel32!GetQueuedCompletionStatus+0x29
03 00d4ff8c 5a3026bc W3TP!THREAD_POOL_DATA::ThreadPoolThread+0x33
04 00d4ffa0 5a301db9 W3TP!THREAD_POOL_DATA::ThreadPoolThread+0x24
05 00d4ffb8 77e66063 W3TP!THREAD_MANAGER::ThreadManagerThread+0x39
06 00d4ffec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 7
System Thread ID: 25ec
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Status: Thread is in a WAIT state.
Thread Type: Other
# ChildEBP RetAddr
00 00e0fcec 7c822114 ntdll!KiFastSystemCallRet
01 00e0fcf0 7c83acfd ntdll!NtWaitForMultipleObjects+0xc
02 00e0ffb8 77e66063 ntdll!RtlpWaitThread+0x161
03 00e0ffec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 8
System Thread ID: 196c
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Status: Thread is in a WAIT state.
Thread Type: HTTP Compression Thread
# ChildEBP RetAddr
00 017dfa84 7c822124 ntdll!KiFastSystemCallRet
01 017dfa88 77e6baa8 ntdll!NtWaitForSingleObject+0xc
02 017dfaf8 77e6ba12 kernel32!WaitForSingleObjectEx+0xac
03 017dfb0c 5a3b8147 kernel32!WaitForSingleObject+0x12
04 017dffb8 77e66063 w3core!HTTP_COMPRESSION::CompressionThread+0x126
05 017dffec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 9
System Thread ID: 1bc4
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Type: Other
# ChildEBP RetAddr
00 0185ff8c 7c821364 ntdll!KiFastSystemCallRet
01 0185ff90 7c815267 ntdll!NtDelayExecution+0xc
02 0185ffb8 77e66063 ntdll!RtlpIOWorkerThread+0x3f
03 0185ffec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 10
System Thread ID: 1618
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Type: Other
# ChildEBP RetAddr
00 0191fec0 7c821bf4 ntdll!KiFastSystemCallRet
01 0191fec4 77e6611a ntdll!NtRemoveIoCompletion+0xc
02 0191fef0 791fdb22 kernel32!GetQueuedCompletionStatus+0x29
03 0191ff24 791fdacc mscorsvr+0x4db22
04 0191ffb8 77e66063 mscorsvr+0x4dacc
05 0191ffec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 11
System Thread ID: 190c
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Type: Other
# ChildEBP RetAddr
00 0195fe50 7c821364 ntdll!KiFastSystemCallRet
01 0195fe54 77e42439 ntdll!NtDelayExecution+0xc
02 0195febc 77e424b7 kernel32!SleepEx+0x68
03 0195fecc 791bf4f9 kernel32!Sleep+0xf
04 0195ffb8 77e66063 mscorsvr+0xf4f9
05 0195ffec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 12
System Thread ID: 1e2c
Kernel Time: 0:0:0.171
User Time: 0:0:1.156
Thread Status: Thread is in a WAIT state.
Thread Type: Other
# ChildEBP RetAddr
00 0199fdf8 7c822124 ntdll!KiFastSystemCallRet
01 0199fdfc 77e6baa8 ntdll!NtWaitForSingleObject+0xc
02 0199fe6c 77e6ba12 kernel32!WaitForSingleObjectEx+0xac
03 0199fe80 791d401f kernel32!WaitForSingleObject+0x12
04 0199fea4 791fdacc mscorsvr+0x2401f
05 0199ffb8 77e66063 mscorsvr+0x4dacc
06 0199ffec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 13
System Thread ID: 1ec8
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Status: Thread is in a WAIT state.
Thread Type: Other
# ChildEBP RetAddr
00 01acfe70 7c822114 ntdll!KiFastSystemCallRet
01 01acfe74 77e6711b ntdll!NtWaitForMultipleObjects+0xc
02 01acff1c 77e61075 kernel32!WaitForMultipleObjectsEx+0x11a
03 01acff38 79238fd6 kernel32!WaitForMultipleObjects+0x18
04 01acffa0 79238f4d mscorsvr+0x88fd6
05 01acffb0 7923a714 mscorsvr+0x88f4d
06 01acffb8 77e66063 mscorsvr+0x8a714
07 01acffec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 14
System Thread ID: 1520
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Status: Thread is in a WAIT state.
Thread Type: Other
# ChildEBP RetAddr
00 01d5fefc 7c822124 ntdll!KiFastSystemCallRet
01 01d5ff00 77e6baa8 ntdll!NtWaitForSingleObject+0xc
02 01d5ff70 77e6ba12 kernel32!WaitForSingleObjectEx+0xac
03 01d5ff84 791fe6b0 kernel32!WaitForSingleObject+0x12
04 01d5ffac 792356bf mscorsvr+0x4e6b0
05 01d5ffb8 77e66063 mscorsvr+0x856bf
06 01d5ffec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 15
System Thread ID: 1f18
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Status: Thread is in a WAIT state.
Thread Type: Other
# ChildEBP RetAddr
00 01d9fdfc 7c822124 ntdll!KiFastSystemCallRet
01 01d9fe00 77e6baa8 ntdll!NtWaitForSingleObject+0xc
02 01d9fe70 77e6ba12 kernel32!WaitForSingleObjectEx+0xac
03 01d9fe84 791fe6b0 kernel32!WaitForSingleObject+0x12
04 01d9feac 792356bf mscorsvr+0x4e6b0
05 01d9ffb8 77e66063 mscorsvr+0x856bf
06 01d9ffec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 16
System Thread ID: 11c0
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Status: Thread is in a WAIT state.
Thread Type: Other
# ChildEBP RetAddr
00 01ddfcfc 7c822124 ntdll!KiFastSystemCallRet
01 01ddfd00 77e6baa8 ntdll!NtWaitForSingleObject+0xc
02 01ddfd70 77e6ba12 kernel32!WaitForSingleObjectEx+0xac
03 01ddfd84 791fe6b0 kernel32!WaitForSingleObject+0x12
04 01ddfdac 792356bf mscorsvr+0x4e6b0
05 01ddffb8 77e66063 mscorsvr+0x856bf
06 01ddffec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 17
System Thread ID: 258c
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Status: Thread is in a WAIT state.
Thread Type: Other
# ChildEBP RetAddr
00 01e1fbfc 7c822124 ntdll!KiFastSystemCallRet
01 01e1fc00 77e6baa8 ntdll!NtWaitForSingleObject+0xc
02 01e1fc70 77e6ba12 kernel32!WaitForSingleObjectEx+0xac
03 01e1fc84 791fe6b0 kernel32!WaitForSingleObject+0x12
04 01e1fcac 792356bf mscorsvr+0x4e6b0
05 01e1ffb8 77e66063 mscorsvr+0x856bf
06 01e1ffec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 18
System Thread ID: 226c
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Status: Thread is in a WAIT state.
Thread Type: Other
# ChildEBP RetAddr
00 01e5fdf8 7c822114 ntdll!KiFastSystemCallRet
01 01e5fdfc 77e6711b ntdll!NtWaitForMultipleObjects+0xc
02 01e5fea4 77e61075 kernel32!WaitForMultipleObjectsEx+0x11a
03 01e5fec0 7927826b kernel32!WaitForMultipleObjects+0x18
04 01e5fee0 791fecf4 mscorsvr+0xc826b
05 01e5ff24 79245681 mscorsvr+0x4ecf4
06 01e5ffb8 77e66063 mscorsvr+0x95681
07 01e5ffec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 19
System Thread ID: 1390
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Type: Other
# ChildEBP RetAddr
00 0201ff38 7c821364 ntdll!KiFastSystemCallRet
01 0201ff3c 77e42439 ntdll!NtDelayExecution+0xc
02 0201ffa4 791cc578 kernel32!SleepEx+0x68
03 0201ffb8 77e66063 mscorsvr+0x1c578
04 0201ffec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 20
System Thread ID: b04
Kernel Time: 0:0:0.0
User Time: 0:0:0.31
Thread Status: Thread is in a WAIT state.
Thread Type: Other
# ChildEBP RetAddr
00 0296fa78 7c822124 ntdll!KiFastSystemCallRet
01 0296fa7c 77e6baa8 ntdll!NtWaitForSingleObject+0xc
02 0296faec 77e6ba12 kernel32!WaitForSingleObjectEx+0xac
03 0296fb00 791d401f kernel32!WaitForSingleObject+0x12
04 0296fb24 791fdacc mscorsvr+0x2401f
05 0296ffb8 77e66063 mscorsvr+0x4dacc
06 0296ffec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 21
System Thread ID: 840
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Type: Other
# ChildEBP RetAddr
00 029aff70 7c821bf4 ntdll!KiFastSystemCallRet
01 029aff74 7c83ad75 ntdll!NtRemoveIoCompletion+0xc
02 029affb8 77e66063 ntdll!RtlpWorkerThread+0x3d
03 029affec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 22
System Thread ID: 2178
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Type: Other
# ChildEBP RetAddr
00 0214f7c0 7c821bf4 ntdll!KiFastSystemCallRet
01 0214f7c4 77e6611a ntdll!NtRemoveIoCompletion+0xc
02 0214f7f0 791fdb22 kernel32!GetQueuedCompletionStatus+0x29
03 0214f824 791fdacc mscorsvr+0x4db22
04 0214ffb8 77e66063 mscorsvr+0x4dacc
05 0214ffec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 23
System Thread ID: 2118
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Type: Other
# ChildEBP RetAddr
00 009aff70 7c821bf4 ntdll!KiFastSystemCallRet
01 009aff74 7c83ad75 ntdll!NtRemoveIoCompletion+0xc
02 009affb8 77e66063 ntdll!RtlpWorkerThread+0x3d
03 009affec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 24
System Thread ID: 1380
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Type: Other
# ChildEBP RetAddr
00 00a2ff70 7c821bf4 ntdll!KiFastSystemCallRet
01 00a2ff74 7c83ad75 ntdll!NtRemoveIoCompletion+0xc
02 00a2ffb8 77e66063 ntdll!RtlpWorkerThread+0x3d
03 00a2ffec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 25
System Thread ID: 1524
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Status: Thread is in a WAIT state.
Thread Type: Other
# ChildEBP RetAddr
00 00acf4f8 7c822124 ntdll!KiFastSystemCallRet
01 00acf4fc 77e6baa8 ntdll!NtWaitForSingleObject+0xc
02 00acf56c 77e6ba12 kernel32!WaitForSingleObjectEx+0xac
03 00acf580 791d401f kernel32!WaitForSingleObject+0x12
04 00acf5a4 791fdacc mscorsvr+0x2401f
05 00acffb8 77e66063 mscorsvr+0x4dacc
06 00acffec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 26
System Thread ID: 1aa8
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Type: Other
# ChildEBP RetAddr
00 0181ff70 7c821bf4 ntdll!KiFastSystemCallRet
01 0181ff74 7c83ad75 ntdll!NtRemoveIoCompletion+0xc
02 0181ffb8 77e66063 ntdll!RtlpWorkerThread+0x3d
03 0181ffec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 27
System Thread ID: 1168
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Type: Other
# ChildEBP RetAddr
00 0209ff70 7c821bf4 ntdll!KiFastSystemCallRet
01 0209ff74 7c83ad75 ntdll!NtRemoveIoCompletion+0xc
02 0209ffb8 77e66063 ntdll!RtlpWorkerThread+0x3d
03 0209ffec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 28
System Thread ID: 1088
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Type: Other
# ChildEBP RetAddr
00 0210ff70 7c821bf4 ntdll!KiFastSystemCallRet
01 0210ff74 7c83ad75 ntdll!NtRemoveIoCompletion+0xc
02 0210ffb8 77e66063 ntdll!RtlpWorkerThread+0x3d
03 0210ffec 00000000 kernel32!BaseThreadStart+0x34
*****
Dump name is formatted as: PID-Timestamp.dmp
Creating C:\iisstate\output\2752-1142363561.dmp - mini user dump
*****
Closing open log file C:\iisstate\output\IISState-2752.log
Post by Pat [MSFT]
Thread 16 shows that IIS reset the process out from under you. The delay is
usually caused by the downloading of symbols - which are cached so the
problem goes away after the initial run.
-hc or -sc will only help you if you are seeing unexpected process
terminations (i.e. a crash). Hangs won't be caught by them.
Pat
Post by Rani Baki
I ran iisstate during an hang. It froze on thread 9, but eventually continued
and then it looks like it failed during thread 16/17. Maybe because the
worker process gave up. I also ran it using the -hc option and it hung
on
a
dll, could this dll be the culprit? I am including one output below and the
other on the next post.
Thanks,
Rani Baki
C:\iisstate>iisstate -p 5592 -d
SRV*C:\iisstate\symbols*http://msdl.microsoft.com/download/symbols
Microsoft (R) Windows Debugger Version 6.2.0013.1
Copyright (c) Microsoft Corporation. All rights reserved.
*** wait with pending attach
SRV*C:\iisstate\symbols*http://msdl.microsoft.com/download/symbols
WARNING: Process 5592 is not attached as a debuggee
The process can be examined but debug events will not be received
.................................................................................................................................................
The call to LoadLibrary(exts) failed, Win32 error 2
"The system cannot find the file specified."
Please check your debugger configuration and/or network access.
(15d8.2068): Wake debugger - code 80000007 (first chance)
Opened log file 'C:\iisstate\output\IISState-5592.log'
***********************
Starting new log output
IISState version 3.3.1
Tue Mar 14 10:06:51 2006
OS = Windows 2003 Server
Executable: w3wp.exe
PID = 5592
Note: Thread times are formatted as HH:MM:SS.ms
***********************
Thread ID: 0
System Thread ID: 2068
Kernel Time: 0:0:0.31
User Time: 0:0:0.31
Thread Status: Thread is in a WAIT state.
Thread Type: HTTP Compression Thread
# ChildEBP RetAddr
00 0006fc08 7c822124 ntdll!KiFastSystemCallRet
01 0006fc0c 77e6baa8 ntdll!NtWaitForSingleObject+0xc
02 0006fc7c 77e6ba12 kernel32!WaitForSingleObjectEx+0xac
03 0006fc90 5a36467a kernel32!WaitForSingleObject+0x12
04 0006fca0 5a366e63 w3dt!WP_CONTEXT::RunMainThreadLoop+0x10
05 0006fca8 5a3af41d w3dt!UlAtqStartListen+0x2d
06 0006fcb8 5a3bc259 w3core!W3_SERVER::StartListen+0xbd
07 0006ff0c 0100187c w3core!UlW3Start+0x26e
08 0006ff44 01001a23 w3wp!wmain+0x22a
09 0006ffc0 77e523cd w3wp!wmainCRTStartup+0x12b
0a 0006fff0 00000000 kernel32!BaseProcessStart+0x23
Thread ID: 1
System Thread ID: 16c0
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Type: Other
# ChildEBP RetAddr
00 009eff9c 7c821364 ntdll!KiFastSystemCallRet
01 009effa0 7c81fe26 ntdll!NtDelayExecution+0xc
02 009effb8 77e66063 ntdll!RtlpTimerThread+0x47
03 009effec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 2
System Thread ID: b10
Kernel Time: 0:0:0.15
User Time: 0:0:0.0
Thread Type: HTTP Listener
# ChildEBP RetAddr
00 00c8ff24 7c821bf4 ntdll!KiFastSystemCallRet
01 00c8ff28 77e6611a ntdll!NtRemoveIoCompletion+0xc
02 00c8ff54 5a30249e kernel32!GetQueuedCompletionStatus+0x29
03 00c8ff8c 5a3026bc W3TP!THREAD_POOL_DATA::ThreadPoolThread+0x33
04 00c8ffa0 5a301db9 W3TP!THREAD_POOL_DATA::ThreadPoolThread+0x24
05 00c8ffb8 77e66063 W3TP!THREAD_MANAGER::ThreadManagerThread+0x39
06 00c8ffec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 3
System Thread ID: 11cc
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Type: HTTP Listener
# ChildEBP RetAddr
00 00ccff24 7c821bf4 ntdll!KiFastSystemCallRet
01 00ccff28 77e6611a ntdll!NtRemoveIoCompletion+0xc
02 00ccff54 5a30249e kernel32!GetQueuedCompletionStatus+0x29
03 00ccff8c 5a3026bc W3TP!THREAD_POOL_DATA::ThreadPoolThread+0x33
04 00ccffa0 5a301db9 W3TP!THREAD_POOL_DATA::ThreadPoolThread+0x24
05 00ccffb8 77e66063 W3TP!THREAD_MANAGER::ThreadManagerThread+0x39
06 00ccffec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 4
System Thread ID: 2110
Kernel Time: 0:0:0.15
User Time: 0:0:0.0
Thread Type: HTTP Listener
# ChildEBP RetAddr
00 00d0ff24 7c821bf4 ntdll!KiFastSystemCallRet
01 00d0ff28 77e6611a ntdll!NtRemoveIoCompletion+0xc
02 00d0ff54 5a30249e kernel32!GetQueuedCompletionStatus+0x29
03 00d0ff8c 5a3026bc W3TP!THREAD_POOL_DATA::ThreadPoolThread+0x33
04 00d0ffa0 5a301db9 W3TP!THREAD_POOL_DATA::ThreadPoolThread+0x24
05 00d0ffb8 77e66063 W3TP!THREAD_MANAGER::ThreadManagerThread+0x39
06 00d0ffec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 5
System Thread ID: 19bc
Kernel Time: 0:0:0.15
User Time: 0:0:0.0
Thread Type: HTTP Listener
# ChildEBP RetAddr
00 00d4ff24 7c821bf4 ntdll!KiFastSystemCallRet
01 00d4ff28 77e6611a ntdll!NtRemoveIoCompletion+0xc
02 00d4ff54 5a30249e kernel32!GetQueuedCompletionStatus+0x29
03 00d4ff8c 5a3026bc W3TP!THREAD_POOL_DATA::ThreadPoolThread+0x33
04 00d4ffa0 5a301db9 W3TP!THREAD_POOL_DATA::ThreadPoolThread+0x24
05 00d4ffb8 77e66063 W3TP!THREAD_MANAGER::ThreadManagerThread+0x39
06 00d4ffec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 6
System Thread ID: 12e8
Kernel Time: 0:0:0.15
User Time: 0:0:0.0
Thread Status: Thread is in a WAIT state.
Thread Type: Other
# ChildEBP RetAddr
00 00e0fcec 7c822114 ntdll!KiFastSystemCallRet
01 00e0fcf0 7c83acfd ntdll!NtWaitForMultipleObjects+0xc
02 00e0ffb8 77e66063 ntdll!RtlpWaitThread+0x161
03 00e0ffec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 7
System Thread ID: 10e0
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Status: Thread is in a WAIT state.
Thread Type: HTTP Compression Thread
# ChildEBP RetAddr
00 017dfa84 7c822124 ntdll!KiFastSystemCallRet
01 017dfa88 77e6baa8 ntdll!NtWaitForSingleObject+0xc
02 017dfaf8 77e6ba12 kernel32!WaitForSingleObjectEx+0xac
03 017dfb0c 5a3b8147 kernel32!WaitForSingleObject+0x12
04 017dffb8 77e66063 w3core!HTTP_COMPRESSION::CompressionThread+0x126
05 017dffec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 8
System Thread ID: 10a4
Kernel Time: 0:0:0.15
User Time: 0:0:0.0
Thread Type: Other
# ChildEBP RetAddr
00 0185ff8c 7c821364 ntdll!KiFastSystemCallRet
01 0185ff90 7c815267 ntdll!NtDelayExecution+0xc
02 0185ffb8 77e66063 ntdll!RtlpIOWorkerThread+0x3f
03 0185ffec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 9
System Thread ID: 288
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Type: Other
# ChildEBP RetAddr
00 0191fec0 7c821bf4 ntdll!KiFastSystemCallRet
01 0191fec4 77e6611a ntdll!NtRemoveIoCompletion+0xc
02 0191fef0 791fdb22 kernel32!GetQueuedCompletionStatus+0x29
03 0191ff24 791fdacc mscorsvr+0x4db22
04 0191ffb8 77e66063 mscorsvr+0x4dacc
05 0191ffec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 10
System Thread ID: 1a4
Kernel Time: 0:0:0.15
User Time: 0:0:0.0
Thread Type: Other
# ChildEBP RetAddr
00 0195fe50 7c821364 ntdll!KiFastSystemCallRet
01 0195fe54 77e42439 ntdll!NtDelayExecution+0xc
02 0195febc 77e424b7 kernel32!SleepEx+0x68
03 0195fecc 791bf4f9 kernel32!Sleep+0xf
04 0195ffb8 77e66063 mscorsvr+0xf4f9
05 0195ffec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 11
System Thread ID: 178
Kernel Time: 0:0:0.765
User Time: 0:0:5.453
Thread Status: Thread is in a WAIT state.
Thread Type: Other
# ChildEBP RetAddr
00 0199fdf8 7c822124 ntdll!KiFastSystemCallRet
01 0199fdfc 77e6baa8 ntdll!NtWaitForSingleObject+0xc
02 0199fe6c 77e6ba12 kernel32!WaitForSingleObjectEx+0xac
03 0199fe80 791d401f kernel32!WaitForSingleObject+0x12
04 0199fea4 791fdacc mscorsvr+0x2401f
05 0199ffb8 77e66063 mscorsvr+0x4dacc
06 0199ffec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 12
System Thread ID: 265c
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Status: Thread is in a WAIT state.
Thread Type: Other
# ChildEBP RetAddr
00 01acfe70 7c822114 ntdll!KiFastSystemCallRet
01 01acfe74 77e6711b ntdll!NtWaitForMultipleObjects+0xc
02 01acff1c 77e61075 kernel32!WaitForMultipleObjectsEx+0x11a
03 01acff38 79238fd6 kernel32!WaitForMultipleObjects+0x18
04 01acffa0 79238f4d mscorsvr+0x88fd6
05 01acffb0 7923a714 mscorsvr+0x88f4d
06 01acffb8 77e66063 mscorsvr+0x8a714
07 01acffec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 13
System Thread ID: 1fe4
Kernel Time: 0:0:0.46
User Time: 0:0:0.906
Thread Status: Thread is in a WAIT state.
Thread Type: Other
# ChildEBP RetAddr
00 01d5fefc 7c822124 ntdll!KiFastSystemCallRet
01 01d5ff00 77e6baa8 ntdll!NtWaitForSingleObject+0xc
02 01d5ff70 77e6ba12 kernel32!WaitForSingleObjectEx+0xac
03 01d5ff84 791fe6b0 kernel32!WaitForSingleObject+0x12
04 01d5ffac 792356bf mscorsvr+0x4e6b0
05 01d5ffb8 77e66063 mscorsvr+0x856bf
06 01d5ffec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 14
Rani Baki
2006-03-15 17:16:08 UTC
Permalink
No, 1 process per app pool. Anything else you can think of? I also have a
mini dump. Would that be helpful?

Thanks,
Rani Baki
Post by Pat [MSFT]
Well...
The threads below show that the server had been serving up pages, but there
was no activity (pages executing, etc.) when the log was actually taken. Do
you have multiple w3wp.exe processes running?
Pat
Post by Rani Baki
I had a feeling that's what happened. Another hang took place and I ran
Opened log file 'C:\iisstate\output\IISState-2752.log'
***********************
Starting new log output
IISState version 3.3.1
Tue Mar 14 13:48:42 2006
OS = Windows 2003 Server
Executable: w3wp.exe
PID = 2752
Note: Thread times are formatted as HH:MM:SS.ms
***********************
Thread ID: 0
System Thread ID: 1dcc
Kernel Time: 0:0:0.15
User Time: 0:0:0.46
Thread Status: Thread is in a WAIT state.
Thread Type: HTTP Compression Thread
# ChildEBP RetAddr
00 0006fc08 7c822124 ntdll!KiFastSystemCallRet
01 0006fc0c 77e6baa8 ntdll!NtWaitForSingleObject+0xc
02 0006fc7c 77e6ba12 kernel32!WaitForSingleObjectEx+0xac
03 0006fc90 5a36467a kernel32!WaitForSingleObject+0x12
04 0006fca0 5a366e63 w3dt!WP_CONTEXT::RunMainThreadLoop+0x10
05 0006fca8 5a3af41d w3dt!UlAtqStartListen+0x2d
06 0006fcb8 5a3bc259 w3core!W3_SERVER::StartListen+0xbd
07 0006ff0c 0100187c w3core!UlW3Start+0x26e
08 0006ff44 01001a23 w3wp!wmain+0x22a
09 0006ffc0 77e523cd w3wp!wmainCRTStartup+0x12b
0a 0006fff0 00000000 kernel32!BaseProcessStart+0x23
Thread ID: 1
System Thread ID: 221c
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Type: Other
# ChildEBP RetAddr
00 009eff9c 7c821364 ntdll!KiFastSystemCallRet
01 009effa0 7c81fe26 ntdll!NtDelayExecution+0xc
02 009effb8 77e66063 ntdll!RtlpTimerThread+0x47
03 009effec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 2
System Thread ID: 2640
Kernel Time: 0:0:0.0
User Time: 0:0:0.15
Thread Type: Possible ASP page. Possible DCOM activity
Executing Page: ASP.dll symbols not found. Unable to locate ASP page.
Continuing with other analysis.
No remote call being made
# ChildEBP RetAddr
00 00a8fe18 7c821c54 ntdll!KiFastSystemCallRet
01 00a8fe1c 77c7538c ntdll!ZwReplyWaitReceivePortEx+0xc
02 00a8ff84 77c5778f RPCRT4!LRPC_ADDRESS::ReceiveLotsaCalls+0x198
03 00a8ff8c 77c5f7dd RPCRT4!RecvLotsaCallsWrapper+0xd
04 00a8ffac 77c5de88 RPCRT4!BaseCachedThreadRoutine+0x9d
05 00a8ffb8 77e66063 RPCRT4!ThreadStartRoutine+0x1b
06 00a8ffec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 3
System Thread ID: 11e0
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Type: HTTP Listener
# ChildEBP RetAddr
00 00c8ff24 7c821bf4 ntdll!KiFastSystemCallRet
01 00c8ff28 77e6611a ntdll!NtRemoveIoCompletion+0xc
02 00c8ff54 5a30249e kernel32!GetQueuedCompletionStatus+0x29
03 00c8ff8c 5a3026bc W3TP!THREAD_POOL_DATA::ThreadPoolThread+0x33
04 00c8ffa0 5a301db9 W3TP!THREAD_POOL_DATA::ThreadPoolThread+0x24
05 00c8ffb8 77e66063 W3TP!THREAD_MANAGER::ThreadManagerThread+0x39
06 00c8ffec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 4
System Thread ID: 92c
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Type: HTTP Listener
# ChildEBP RetAddr
00 00ccff24 7c821bf4 ntdll!KiFastSystemCallRet
01 00ccff28 77e6611a ntdll!NtRemoveIoCompletion+0xc
02 00ccff54 5a30249e kernel32!GetQueuedCompletionStatus+0x29
03 00ccff8c 5a3026bc W3TP!THREAD_POOL_DATA::ThreadPoolThread+0x33
04 00ccffa0 5a301db9 W3TP!THREAD_POOL_DATA::ThreadPoolThread+0x24
05 00ccffb8 77e66063 W3TP!THREAD_MANAGER::ThreadManagerThread+0x39
06 00ccffec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 5
System Thread ID: 16c0
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Type: HTTP Listener
# ChildEBP RetAddr
00 00d0ff24 7c821bf4 ntdll!KiFastSystemCallRet
01 00d0ff28 77e6611a ntdll!NtRemoveIoCompletion+0xc
02 00d0ff54 5a30249e kernel32!GetQueuedCompletionStatus+0x29
03 00d0ff8c 5a3026bc W3TP!THREAD_POOL_DATA::ThreadPoolThread+0x33
04 00d0ffa0 5a301db9 W3TP!THREAD_POOL_DATA::ThreadPoolThread+0x24
05 00d0ffb8 77e66063 W3TP!THREAD_MANAGER::ThreadManagerThread+0x39
06 00d0ffec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 6
System Thread ID: 1dfc
Kernel Time: 0:0:0.15
User Time: 0:0:0.0
Thread Type: HTTP Listener
# ChildEBP RetAddr
00 00d4ff24 7c821bf4 ntdll!KiFastSystemCallRet
01 00d4ff28 77e6611a ntdll!NtRemoveIoCompletion+0xc
02 00d4ff54 5a30249e kernel32!GetQueuedCompletionStatus+0x29
03 00d4ff8c 5a3026bc W3TP!THREAD_POOL_DATA::ThreadPoolThread+0x33
04 00d4ffa0 5a301db9 W3TP!THREAD_POOL_DATA::ThreadPoolThread+0x24
05 00d4ffb8 77e66063 W3TP!THREAD_MANAGER::ThreadManagerThread+0x39
06 00d4ffec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 7
System Thread ID: 25ec
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Status: Thread is in a WAIT state.
Thread Type: Other
# ChildEBP RetAddr
00 00e0fcec 7c822114 ntdll!KiFastSystemCallRet
01 00e0fcf0 7c83acfd ntdll!NtWaitForMultipleObjects+0xc
02 00e0ffb8 77e66063 ntdll!RtlpWaitThread+0x161
03 00e0ffec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 8
System Thread ID: 196c
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Status: Thread is in a WAIT state.
Thread Type: HTTP Compression Thread
# ChildEBP RetAddr
00 017dfa84 7c822124 ntdll!KiFastSystemCallRet
01 017dfa88 77e6baa8 ntdll!NtWaitForSingleObject+0xc
02 017dfaf8 77e6ba12 kernel32!WaitForSingleObjectEx+0xac
03 017dfb0c 5a3b8147 kernel32!WaitForSingleObject+0x12
04 017dffb8 77e66063 w3core!HTTP_COMPRESSION::CompressionThread+0x126
05 017dffec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 9
System Thread ID: 1bc4
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Type: Other
# ChildEBP RetAddr
00 0185ff8c 7c821364 ntdll!KiFastSystemCallRet
01 0185ff90 7c815267 ntdll!NtDelayExecution+0xc
02 0185ffb8 77e66063 ntdll!RtlpIOWorkerThread+0x3f
03 0185ffec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 10
System Thread ID: 1618
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Type: Other
# ChildEBP RetAddr
00 0191fec0 7c821bf4 ntdll!KiFastSystemCallRet
01 0191fec4 77e6611a ntdll!NtRemoveIoCompletion+0xc
02 0191fef0 791fdb22 kernel32!GetQueuedCompletionStatus+0x29
03 0191ff24 791fdacc mscorsvr+0x4db22
04 0191ffb8 77e66063 mscorsvr+0x4dacc
05 0191ffec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 11
System Thread ID: 190c
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Type: Other
# ChildEBP RetAddr
00 0195fe50 7c821364 ntdll!KiFastSystemCallRet
01 0195fe54 77e42439 ntdll!NtDelayExecution+0xc
02 0195febc 77e424b7 kernel32!SleepEx+0x68
03 0195fecc 791bf4f9 kernel32!Sleep+0xf
04 0195ffb8 77e66063 mscorsvr+0xf4f9
05 0195ffec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 12
System Thread ID: 1e2c
Kernel Time: 0:0:0.171
User Time: 0:0:1.156
Thread Status: Thread is in a WAIT state.
Thread Type: Other
# ChildEBP RetAddr
00 0199fdf8 7c822124 ntdll!KiFastSystemCallRet
01 0199fdfc 77e6baa8 ntdll!NtWaitForSingleObject+0xc
02 0199fe6c 77e6ba12 kernel32!WaitForSingleObjectEx+0xac
03 0199fe80 791d401f kernel32!WaitForSingleObject+0x12
04 0199fea4 791fdacc mscorsvr+0x2401f
05 0199ffb8 77e66063 mscorsvr+0x4dacc
06 0199ffec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 13
System Thread ID: 1ec8
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Status: Thread is in a WAIT state.
Thread Type: Other
# ChildEBP RetAddr
00 01acfe70 7c822114 ntdll!KiFastSystemCallRet
01 01acfe74 77e6711b ntdll!NtWaitForMultipleObjects+0xc
02 01acff1c 77e61075 kernel32!WaitForMultipleObjectsEx+0x11a
03 01acff38 79238fd6 kernel32!WaitForMultipleObjects+0x18
04 01acffa0 79238f4d mscorsvr+0x88fd6
05 01acffb0 7923a714 mscorsvr+0x88f4d
06 01acffb8 77e66063 mscorsvr+0x8a714
07 01acffec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 14
System Thread ID: 1520
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Status: Thread is in a WAIT state.
Thread Type: Other
# ChildEBP RetAddr
00 01d5fefc 7c822124 ntdll!KiFastSystemCallRet
01 01d5ff00 77e6baa8 ntdll!NtWaitForSingleObject+0xc
02 01d5ff70 77e6ba12 kernel32!WaitForSingleObjectEx+0xac
03 01d5ff84 791fe6b0 kernel32!WaitForSingleObject+0x12
04 01d5ffac 792356bf mscorsvr+0x4e6b0
05 01d5ffb8 77e66063 mscorsvr+0x856bf
06 01d5ffec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 15
System Thread ID: 1f18
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Status: Thread is in a WAIT state.
Thread Type: Other
# ChildEBP RetAddr
Pat [MSFT]
2006-03-16 19:38:42 UTC
Permalink
Probably not. You should probably open a case w/MS-Support. They can help
you track down the issue.

Pat
Post by Rani Baki
No, 1 process per app pool. Anything else you can think of? I also have a
mini dump. Would that be helpful?
Thanks,
Rani Baki
Post by Pat [MSFT]
Well...
The threads below show that the server had been serving up pages, but there
was no activity (pages executing, etc.) when the log was actually taken.
Do
you have multiple w3wp.exe processes running?
Pat
Post by Rani Baki
I had a feeling that's what happened. Another hang took place and I ran
Opened log file 'C:\iisstate\output\IISState-2752.log'
***********************
Starting new log output
IISState version 3.3.1
Tue Mar 14 13:48:42 2006
OS = Windows 2003 Server
Executable: w3wp.exe
PID = 2752
Note: Thread times are formatted as HH:MM:SS.ms
***********************
Thread ID: 0
System Thread ID: 1dcc
Kernel Time: 0:0:0.15
User Time: 0:0:0.46
Thread Status: Thread is in a WAIT state.
Thread Type: HTTP Compression Thread
# ChildEBP RetAddr
00 0006fc08 7c822124 ntdll!KiFastSystemCallRet
01 0006fc0c 77e6baa8 ntdll!NtWaitForSingleObject+0xc
02 0006fc7c 77e6ba12 kernel32!WaitForSingleObjectEx+0xac
03 0006fc90 5a36467a kernel32!WaitForSingleObject+0x12
04 0006fca0 5a366e63 w3dt!WP_CONTEXT::RunMainThreadLoop+0x10
05 0006fca8 5a3af41d w3dt!UlAtqStartListen+0x2d
06 0006fcb8 5a3bc259 w3core!W3_SERVER::StartListen+0xbd
07 0006ff0c 0100187c w3core!UlW3Start+0x26e
08 0006ff44 01001a23 w3wp!wmain+0x22a
09 0006ffc0 77e523cd w3wp!wmainCRTStartup+0x12b
0a 0006fff0 00000000 kernel32!BaseProcessStart+0x23
Thread ID: 1
System Thread ID: 221c
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Type: Other
# ChildEBP RetAddr
00 009eff9c 7c821364 ntdll!KiFastSystemCallRet
01 009effa0 7c81fe26 ntdll!NtDelayExecution+0xc
02 009effb8 77e66063 ntdll!RtlpTimerThread+0x47
03 009effec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 2
System Thread ID: 2640
Kernel Time: 0:0:0.0
User Time: 0:0:0.15
Thread Type: Possible ASP page. Possible DCOM activity
Executing Page: ASP.dll symbols not found. Unable to locate ASP page.
Continuing with other analysis.
No remote call being made
# ChildEBP RetAddr
00 00a8fe18 7c821c54 ntdll!KiFastSystemCallRet
01 00a8fe1c 77c7538c ntdll!ZwReplyWaitReceivePortEx+0xc
02 00a8ff84 77c5778f RPCRT4!LRPC_ADDRESS::ReceiveLotsaCalls+0x198
03 00a8ff8c 77c5f7dd RPCRT4!RecvLotsaCallsWrapper+0xd
04 00a8ffac 77c5de88 RPCRT4!BaseCachedThreadRoutine+0x9d
05 00a8ffb8 77e66063 RPCRT4!ThreadStartRoutine+0x1b
06 00a8ffec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 3
System Thread ID: 11e0
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Type: HTTP Listener
# ChildEBP RetAddr
00 00c8ff24 7c821bf4 ntdll!KiFastSystemCallRet
01 00c8ff28 77e6611a ntdll!NtRemoveIoCompletion+0xc
02 00c8ff54 5a30249e kernel32!GetQueuedCompletionStatus+0x29
03 00c8ff8c 5a3026bc W3TP!THREAD_POOL_DATA::ThreadPoolThread+0x33
04 00c8ffa0 5a301db9 W3TP!THREAD_POOL_DATA::ThreadPoolThread+0x24
05 00c8ffb8 77e66063 W3TP!THREAD_MANAGER::ThreadManagerThread+0x39
06 00c8ffec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 4
System Thread ID: 92c
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Type: HTTP Listener
# ChildEBP RetAddr
00 00ccff24 7c821bf4 ntdll!KiFastSystemCallRet
01 00ccff28 77e6611a ntdll!NtRemoveIoCompletion+0xc
02 00ccff54 5a30249e kernel32!GetQueuedCompletionStatus+0x29
03 00ccff8c 5a3026bc W3TP!THREAD_POOL_DATA::ThreadPoolThread+0x33
04 00ccffa0 5a301db9 W3TP!THREAD_POOL_DATA::ThreadPoolThread+0x24
05 00ccffb8 77e66063 W3TP!THREAD_MANAGER::ThreadManagerThread+0x39
06 00ccffec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 5
System Thread ID: 16c0
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Type: HTTP Listener
# ChildEBP RetAddr
00 00d0ff24 7c821bf4 ntdll!KiFastSystemCallRet
01 00d0ff28 77e6611a ntdll!NtRemoveIoCompletion+0xc
02 00d0ff54 5a30249e kernel32!GetQueuedCompletionStatus+0x29
03 00d0ff8c 5a3026bc W3TP!THREAD_POOL_DATA::ThreadPoolThread+0x33
04 00d0ffa0 5a301db9 W3TP!THREAD_POOL_DATA::ThreadPoolThread+0x24
05 00d0ffb8 77e66063 W3TP!THREAD_MANAGER::ThreadManagerThread+0x39
06 00d0ffec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 6
System Thread ID: 1dfc
Kernel Time: 0:0:0.15
User Time: 0:0:0.0
Thread Type: HTTP Listener
# ChildEBP RetAddr
00 00d4ff24 7c821bf4 ntdll!KiFastSystemCallRet
01 00d4ff28 77e6611a ntdll!NtRemoveIoCompletion+0xc
02 00d4ff54 5a30249e kernel32!GetQueuedCompletionStatus+0x29
03 00d4ff8c 5a3026bc W3TP!THREAD_POOL_DATA::ThreadPoolThread+0x33
04 00d4ffa0 5a301db9 W3TP!THREAD_POOL_DATA::ThreadPoolThread+0x24
05 00d4ffb8 77e66063 W3TP!THREAD_MANAGER::ThreadManagerThread+0x39
06 00d4ffec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 7
System Thread ID: 25ec
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Status: Thread is in a WAIT state.
Thread Type: Other
# ChildEBP RetAddr
00 00e0fcec 7c822114 ntdll!KiFastSystemCallRet
01 00e0fcf0 7c83acfd ntdll!NtWaitForMultipleObjects+0xc
02 00e0ffb8 77e66063 ntdll!RtlpWaitThread+0x161
03 00e0ffec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 8
System Thread ID: 196c
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Status: Thread is in a WAIT state.
Thread Type: HTTP Compression Thread
# ChildEBP RetAddr
00 017dfa84 7c822124 ntdll!KiFastSystemCallRet
01 017dfa88 77e6baa8 ntdll!NtWaitForSingleObject+0xc
02 017dfaf8 77e6ba12 kernel32!WaitForSingleObjectEx+0xac
03 017dfb0c 5a3b8147 kernel32!WaitForSingleObject+0x12
04 017dffb8 77e66063 w3core!HTTP_COMPRESSION::CompressionThread+0x126
05 017dffec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 9
System Thread ID: 1bc4
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Type: Other
# ChildEBP RetAddr
00 0185ff8c 7c821364 ntdll!KiFastSystemCallRet
01 0185ff90 7c815267 ntdll!NtDelayExecution+0xc
02 0185ffb8 77e66063 ntdll!RtlpIOWorkerThread+0x3f
03 0185ffec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 10
System Thread ID: 1618
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Type: Other
# ChildEBP RetAddr
00 0191fec0 7c821bf4 ntdll!KiFastSystemCallRet
01 0191fec4 77e6611a ntdll!NtRemoveIoCompletion+0xc
02 0191fef0 791fdb22 kernel32!GetQueuedCompletionStatus+0x29
03 0191ff24 791fdacc mscorsvr+0x4db22
04 0191ffb8 77e66063 mscorsvr+0x4dacc
05 0191ffec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 11
System Thread ID: 190c
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Type: Other
# ChildEBP RetAddr
00 0195fe50 7c821364 ntdll!KiFastSystemCallRet
01 0195fe54 77e42439 ntdll!NtDelayExecution+0xc
02 0195febc 77e424b7 kernel32!SleepEx+0x68
03 0195fecc 791bf4f9 kernel32!Sleep+0xf
04 0195ffb8 77e66063 mscorsvr+0xf4f9
05 0195ffec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 12
System Thread ID: 1e2c
Kernel Time: 0:0:0.171
User Time: 0:0:1.156
Thread Status: Thread is in a WAIT state.
Thread Type: Other
# ChildEBP RetAddr
00 0199fdf8 7c822124 ntdll!KiFastSystemCallRet
01 0199fdfc 77e6baa8 ntdll!NtWaitForSingleObject+0xc
02 0199fe6c 77e6ba12 kernel32!WaitForSingleObjectEx+0xac
03 0199fe80 791d401f kernel32!WaitForSingleObject+0x12
04 0199fea4 791fdacc mscorsvr+0x2401f
05 0199ffb8 77e66063 mscorsvr+0x4dacc
06 0199ffec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 13
System Thread ID: 1ec8
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Status: Thread is in a WAIT state.
Thread Type: Other
# ChildEBP RetAddr
00 01acfe70 7c822114 ntdll!KiFastSystemCallRet
01 01acfe74 77e6711b ntdll!NtWaitForMultipleObjects+0xc
02 01acff1c 77e61075 kernel32!WaitForMultipleObjectsEx+0x11a
03 01acff38 79238fd6 kernel32!WaitForMultipleObjects+0x18
04 01acffa0 79238f4d mscorsvr+0x88fd6
05 01acffb0 7923a714 mscorsvr+0x88f4d
06 01acffb8 77e66063 mscorsvr+0x8a714
07 01acffec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 14
System Thread ID: 1520
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Status: Thread is in a WAIT state.
Thread Type: Other
# ChildEBP RetAddr
00 01d5fefc 7c822124 ntdll!KiFastSystemCallRet
01 01d5ff00 77e6baa8 ntdll!NtWaitForSingleObject+0xc
02 01d5ff70 77e6ba12 kernel32!WaitForSingleObjectEx+0xac
03 01d5ff84 791fe6b0 kernel32!WaitForSingleObject+0x12
04 01d5ffac 792356bf mscorsvr+0x4e6b0
05 01d5ffb8 77e66063 mscorsvr+0x856bf
06 01d5ffec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 15
System Thread ID: 1f18
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Status: Thread is in a WAIT state.
Thread Type: Other
# ChildEBP RetAddr
Rani Baki
2006-03-17 14:27:27 UTC
Permalink
Thanks for all your help.

Rani Baki
Post by Pat [MSFT]
Probably not. You should probably open a case w/MS-Support. They can help
you track down the issue.
Pat
Post by Rani Baki
No, 1 process per app pool. Anything else you can think of? I also have a
mini dump. Would that be helpful?
Thanks,
Rani Baki
Post by Pat [MSFT]
Well...
The threads below show that the server had been serving up pages, but there
was no activity (pages executing, etc.) when the log was actually taken.
Do
you have multiple w3wp.exe processes running?
Pat
Post by Rani Baki
I had a feeling that's what happened. Another hang took place and I ran
Opened log file 'C:\iisstate\output\IISState-2752.log'
***********************
Starting new log output
IISState version 3.3.1
Tue Mar 14 13:48:42 2006
OS = Windows 2003 Server
Executable: w3wp.exe
PID = 2752
Note: Thread times are formatted as HH:MM:SS.ms
***********************
Thread ID: 0
System Thread ID: 1dcc
Kernel Time: 0:0:0.15
User Time: 0:0:0.46
Thread Status: Thread is in a WAIT state.
Thread Type: HTTP Compression Thread
# ChildEBP RetAddr
00 0006fc08 7c822124 ntdll!KiFastSystemCallRet
01 0006fc0c 77e6baa8 ntdll!NtWaitForSingleObject+0xc
02 0006fc7c 77e6ba12 kernel32!WaitForSingleObjectEx+0xac
03 0006fc90 5a36467a kernel32!WaitForSingleObject+0x12
04 0006fca0 5a366e63 w3dt!WP_CONTEXT::RunMainThreadLoop+0x10
05 0006fca8 5a3af41d w3dt!UlAtqStartListen+0x2d
06 0006fcb8 5a3bc259 w3core!W3_SERVER::StartListen+0xbd
07 0006ff0c 0100187c w3core!UlW3Start+0x26e
08 0006ff44 01001a23 w3wp!wmain+0x22a
09 0006ffc0 77e523cd w3wp!wmainCRTStartup+0x12b
0a 0006fff0 00000000 kernel32!BaseProcessStart+0x23
Thread ID: 1
System Thread ID: 221c
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Type: Other
# ChildEBP RetAddr
00 009eff9c 7c821364 ntdll!KiFastSystemCallRet
01 009effa0 7c81fe26 ntdll!NtDelayExecution+0xc
02 009effb8 77e66063 ntdll!RtlpTimerThread+0x47
03 009effec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 2
System Thread ID: 2640
Kernel Time: 0:0:0.0
User Time: 0:0:0.15
Thread Type: Possible ASP page. Possible DCOM activity
Executing Page: ASP.dll symbols not found. Unable to locate ASP page.
Continuing with other analysis.
No remote call being made
# ChildEBP RetAddr
00 00a8fe18 7c821c54 ntdll!KiFastSystemCallRet
01 00a8fe1c 77c7538c ntdll!ZwReplyWaitReceivePortEx+0xc
02 00a8ff84 77c5778f RPCRT4!LRPC_ADDRESS::ReceiveLotsaCalls+0x198
03 00a8ff8c 77c5f7dd RPCRT4!RecvLotsaCallsWrapper+0xd
04 00a8ffac 77c5de88 RPCRT4!BaseCachedThreadRoutine+0x9d
05 00a8ffb8 77e66063 RPCRT4!ThreadStartRoutine+0x1b
06 00a8ffec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 3
System Thread ID: 11e0
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Type: HTTP Listener
# ChildEBP RetAddr
00 00c8ff24 7c821bf4 ntdll!KiFastSystemCallRet
01 00c8ff28 77e6611a ntdll!NtRemoveIoCompletion+0xc
02 00c8ff54 5a30249e kernel32!GetQueuedCompletionStatus+0x29
03 00c8ff8c 5a3026bc W3TP!THREAD_POOL_DATA::ThreadPoolThread+0x33
04 00c8ffa0 5a301db9 W3TP!THREAD_POOL_DATA::ThreadPoolThread+0x24
05 00c8ffb8 77e66063 W3TP!THREAD_MANAGER::ThreadManagerThread+0x39
06 00c8ffec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 4
System Thread ID: 92c
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Type: HTTP Listener
# ChildEBP RetAddr
00 00ccff24 7c821bf4 ntdll!KiFastSystemCallRet
01 00ccff28 77e6611a ntdll!NtRemoveIoCompletion+0xc
02 00ccff54 5a30249e kernel32!GetQueuedCompletionStatus+0x29
03 00ccff8c 5a3026bc W3TP!THREAD_POOL_DATA::ThreadPoolThread+0x33
04 00ccffa0 5a301db9 W3TP!THREAD_POOL_DATA::ThreadPoolThread+0x24
05 00ccffb8 77e66063 W3TP!THREAD_MANAGER::ThreadManagerThread+0x39
06 00ccffec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 5
System Thread ID: 16c0
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Type: HTTP Listener
# ChildEBP RetAddr
00 00d0ff24 7c821bf4 ntdll!KiFastSystemCallRet
01 00d0ff28 77e6611a ntdll!NtRemoveIoCompletion+0xc
02 00d0ff54 5a30249e kernel32!GetQueuedCompletionStatus+0x29
03 00d0ff8c 5a3026bc W3TP!THREAD_POOL_DATA::ThreadPoolThread+0x33
04 00d0ffa0 5a301db9 W3TP!THREAD_POOL_DATA::ThreadPoolThread+0x24
05 00d0ffb8 77e66063 W3TP!THREAD_MANAGER::ThreadManagerThread+0x39
06 00d0ffec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 6
System Thread ID: 1dfc
Kernel Time: 0:0:0.15
User Time: 0:0:0.0
Thread Type: HTTP Listener
# ChildEBP RetAddr
00 00d4ff24 7c821bf4 ntdll!KiFastSystemCallRet
01 00d4ff28 77e6611a ntdll!NtRemoveIoCompletion+0xc
02 00d4ff54 5a30249e kernel32!GetQueuedCompletionStatus+0x29
03 00d4ff8c 5a3026bc W3TP!THREAD_POOL_DATA::ThreadPoolThread+0x33
04 00d4ffa0 5a301db9 W3TP!THREAD_POOL_DATA::ThreadPoolThread+0x24
05 00d4ffb8 77e66063 W3TP!THREAD_MANAGER::ThreadManagerThread+0x39
06 00d4ffec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 7
System Thread ID: 25ec
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Status: Thread is in a WAIT state.
Thread Type: Other
# ChildEBP RetAddr
00 00e0fcec 7c822114 ntdll!KiFastSystemCallRet
01 00e0fcf0 7c83acfd ntdll!NtWaitForMultipleObjects+0xc
02 00e0ffb8 77e66063 ntdll!RtlpWaitThread+0x161
03 00e0ffec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 8
System Thread ID: 196c
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Status: Thread is in a WAIT state.
Thread Type: HTTP Compression Thread
# ChildEBP RetAddr
00 017dfa84 7c822124 ntdll!KiFastSystemCallRet
01 017dfa88 77e6baa8 ntdll!NtWaitForSingleObject+0xc
02 017dfaf8 77e6ba12 kernel32!WaitForSingleObjectEx+0xac
03 017dfb0c 5a3b8147 kernel32!WaitForSingleObject+0x12
04 017dffb8 77e66063 w3core!HTTP_COMPRESSION::CompressionThread+0x126
05 017dffec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 9
System Thread ID: 1bc4
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Type: Other
# ChildEBP RetAddr
00 0185ff8c 7c821364 ntdll!KiFastSystemCallRet
01 0185ff90 7c815267 ntdll!NtDelayExecution+0xc
02 0185ffb8 77e66063 ntdll!RtlpIOWorkerThread+0x3f
03 0185ffec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 10
System Thread ID: 1618
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Type: Other
# ChildEBP RetAddr
00 0191fec0 7c821bf4 ntdll!KiFastSystemCallRet
01 0191fec4 77e6611a ntdll!NtRemoveIoCompletion+0xc
02 0191fef0 791fdb22 kernel32!GetQueuedCompletionStatus+0x29
03 0191ff24 791fdacc mscorsvr+0x4db22
04 0191ffb8 77e66063 mscorsvr+0x4dacc
05 0191ffec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 11
System Thread ID: 190c
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Type: Other
# ChildEBP RetAddr
00 0195fe50 7c821364 ntdll!KiFastSystemCallRet
01 0195fe54 77e42439 ntdll!NtDelayExecution+0xc
02 0195febc 77e424b7 kernel32!SleepEx+0x68
03 0195fecc 791bf4f9 kernel32!Sleep+0xf
04 0195ffb8 77e66063 mscorsvr+0xf4f9
05 0195ffec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 12
System Thread ID: 1e2c
Kernel Time: 0:0:0.171
User Time: 0:0:1.156
Thread Status: Thread is in a WAIT state.
Thread Type: Other
# ChildEBP RetAddr
00 0199fdf8 7c822124 ntdll!KiFastSystemCallRet
01 0199fdfc 77e6baa8 ntdll!NtWaitForSingleObject+0xc
02 0199fe6c 77e6ba12 kernel32!WaitForSingleObjectEx+0xac
03 0199fe80 791d401f kernel32!WaitForSingleObject+0x12
04 0199fea4 791fdacc mscorsvr+0x2401f
05 0199ffb8 77e66063 mscorsvr+0x4dacc
06 0199ffec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 13
System Thread ID: 1ec8
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Status: Thread is in a WAIT state.
Thread Type: Other
# ChildEBP RetAddr
00 01acfe70 7c822114 ntdll!KiFastSystemCallRet
01 01acfe74 77e6711b ntdll!NtWaitForMultipleObjects+0xc
02 01acff1c 77e61075 kernel32!WaitForMultipleObjectsEx+0x11a
03 01acff38 79238fd6 kernel32!WaitForMultipleObjects+0x18
04 01acffa0 79238f4d mscorsvr+0x88fd6
05 01acffb0 7923a714 mscorsvr+0x88f4d
06 01acffb8 77e66063 mscorsvr+0x8a714
07 01acffec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 14
System Thread ID: 1520
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Status: Thread is in a WAIT state.
Thread Type: Other
# ChildEBP RetAddr
00 01d5fefc 7c822124 ntdll!KiFastSystemCallRet
Loading...