CAS/Hub Transport Crashing with BSOD
Hello everyone and thanks in advance for any help you can offer.
For starters I will give a bit of information about our setup and some background...
We have two front end exchange servers (CAS and Hub transport on each) running Server 2008 Ent in an NLB cluster. These boxes are vm's running on Hyper-v configured in a failover cluster. The Hyper-v cluster has four nodes with 15+
vms running at any one time. The blue screens have never happened on any machines in the hyper-v environment except the exchange boxes which is why I suspect it has something to do with Exchange. Although I have been collecting the Kernal dumps
from these crashes I haven't been able to decipher the cause. I doubt that the host boxes have anything to do with this since the two Exchange vm's have crashed on various host (moved them to test) with different loads. Also since no other
vm is affected I am thinking that it isn't likely the host...is that a bad assumption? These blue screens have been going on for months (first of the year or even before) but sometimes the servers will go for months themselves without a blue screen.
I am sure I have left out helpful information, but just let me know and I will try to get that for you. Thanks again. Below is the latest Kernal dump for you to take a look at...
Microsoft (R) Windows Debugger Version 6.11.0001.404 AMD64
Copyright (c) Microsoft Corporation. All rights reserved.
Loading Dump File [C:\Windows\MEMORY.DMP]
Kernel Summary Dump File: Only kernel address space is available
Symbol search path is: srv*
Executable search path is:
Windows Server 2008/Windows Vista Kernel Version 6001 (Service Pack 1) MP (4 procs) Free x64
Product: Server, suite: Enterprise TerminalServer SingleUserTS
Built by: 6001.18427.amd64fre.vistasp1_gdr.100218-0019
Machine Name:
Kernel base = 0xfffff800`01815000 PsLoadedModuleList = 0xfffff800`019dadb0
Debug session time: Mon Jun 14 01:20:23.790 2010 (GMT-4)
System Uptime: 5 days 18:46:20.187
Loading Kernel Symbols
...............................................................
..........................................................
Loading User Symbols
Loading unloaded module list
....
*******************************************************************************
*
*
* Bugcheck Analysis
*
*
*
*******************************************************************************
Use !analyze -v to get detailed debugging information.
BugCheck D1, {fffffffff9605763, 2, 1, fffffa8010752629}
Probably caused by : ntkrnlmp.exe ( nt!KiPageFault+20b )
Followup: MachineOwner
---------
0: kd> !analyze -v
*******************************************************************************
*
*
* Bugcheck Analysis
*
*
*
*******************************************************************************
DRIVER_IRQL_NOT_LESS_OR_EQUAL (d1)
An attempt was made to access a pageable (or completely invalid) address at an
interrupt request level (IRQL) that is too high. This is usually
caused by drivers using improper addresses.
If kernel debugger is available get stack backtrace.
Arguments:
Arg1: fffffffff9605763, memory referenced
Arg2: 0000000000000002, IRQL
Arg3: 0000000000000001, value 0 = read operation, 1 = write operation
Arg4: fffffa8010752629, address which referenced memory
Debugging Details:
------------------
WRITE_ADDRESS: fffffffff9605763
CURRENT_IRQL: 2
FAULTING_IP:
+0
Page 203b52 not present in the dump file. Type ".hh dbgerr004" for details
Page 203b52 not present in the dump file. Type ".hh dbgerr004" for details
fffffa80`10752629 ?? ???
DEFAULT_BUCKET_ID: VISTA_DRIVER_FAULT
BUGCHECK_STR: 0xD1
PROCESS_NAME: System
TRAP_FRAME: fffff8000467eb90 -- (.trap 0xfffff8000467eb90)
NOTE: The trap frame does not contain all registers.
Some register values may be zeroed or incorrect.
rax=0000000000000000 rbx=0000000000000000 rcx=0000000000000202
rdx=0000000002000000 rsi=0000000000000000 rdi=0000000000000000
rip=fffffa8010752629 rsp=fffff8000467ed20 rbp=fffff8000467ed80
r8=000000000000082f r9=000000000000000c r10=0000000000000000
r11=fffff80001a3e640 r12=0000000000000000 r13=0000000000000000
r14=0000000000000000 r15=0000000000000000
iopl=0 nv up ei ng nz na pe nc
Page 203b52 not present in the dump file. Type ".hh dbgerr004" for details
Page 203b52 not present in the dump file. Type ".hh dbgerr004" for details
fffffa80`10752629 ?? ???
Resetting default scope
LAST_CONTROL_TRANSFER: from fffff8000186942e to fffff80001869690
STACK_TEXT:
fffff800`0467ea48 fffff800`0186942e : 00000000`0000000a ffffffff`f9605763 00000000`00000002 00000000`00000001 : nt!KeBugCheckEx
fffff800`0467ea50 fffff800`0186830b : 00000000`00000001 00000000`00000000 ffff0260`04945b00 fffffa80`10751023 : nt!KiBugCheckDispatch+0x6e
fffff800`0467eb90 fffffa80`10752629 : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : nt!KiPageFault+0x20b
fffff800`0467ed20 00000000`00000000 : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : 0xfffffa80`10752629
June 14th, 2010 6:44pm
You probably want to post this in the Sysinternals forums as they are better are debugging crash dumps than Exchange admins or the Hyper-V forums. However I'd offer two suggestions. First point your debugger to the MS Symbols server. This might clean up
some of the report and lead you to the cause. Also what drivers are you using? Have you installed the Integration services for Hyper-V?Mark Morowczynski|MCT| MCSE 2003:Messaging, Security|MCITP:EMA 2K7,EDA Win 7,ES,SA,EA|MCTS:Windows Mobile Admin|Security+|http://almostdailytech.com
Free Windows Admin Tool Kit Click here and download it now
June 15th, 2010 6:20am
I agree to what Mark wrote, or if they can't help you and this is critical open a PSS caseJonas Andersson MCTS: Microsoft Exchange Server 2007/2010 | MCITP: EMA 2007/2010 | MCSE/MCSA Blog:
http://www.testlabs.se/blog
June 15th, 2010 9:56am
Hi,
I suggest you contact our PSS at
http://support.microsoft.com/select/Default.aspx?target=assistance .
From Exchange aspect, we could not help you more.
I also suggest you uninstall all third party software and update the system and driver into the latest patch. You could also write a post in our Windows
Server forum to see if someone can help
http://social.technet.microsoft.com/Forums/en/category/windowsserver
Thanks,
Elvis
Free Windows Admin Tool Kit Click here and download it now
June 15th, 2010 1:07pm
Thanks for the suggestions. I will change the symbols and look to another forum.
JD
June 17th, 2010 11:19pm