Failed Exchange 2010 SP1 Update
Hi everyone,
I attempted to upgrade my Exchange 2010 RTM installation to SP1 this evening and it failed miserably. I have four servers at our main site; two HT/CAS and two mailbox servers configured in a DAG. I have two additional servers offsite and different locations
(connected by VPN) that run all three roles. The HT/CAS servers upgraded fine but the mailbox servers failed with lots of errors. At this point I can't get the mailbox database to mount and the status under Database Copies in ESM shows ServiceDown for the
two offsite servers and "Disconnected and Synchronizing" for the two main site mailbox servers. The copy queue length on both main mailbox servers (EXCHMB01 and EXCHMB02) show to be 9223372036854717327 and 9223372036854717261 respectively. I'm certain
that's not accurate and is some by-product of the failed upgrade. All servers were working fine before the upgrade attempt.
The first mailbox server, EXCHMB01 failed with the following errors:
Summary: 13 item(s). 10 succeeded, 1 failed.
Elapsed time: 00:29:35
Organization Preparation
Completed
Elapsed Time: 00:10:02
Preparing Setup
Completed
Elapsed Time: 00:01:11
Stopping Services
Completed
Elapsed Time: 00:02:37
Languages
Completed
Elapsed Time: 00:00:00
Remove Exchange Files
Completed
Elapsed Time: 00:01:05
Preparing Files
Completed
Elapsed Time: 00:00:00
Copy Exchange Files
Completed
Elapsed Time: 00:00:43
Language Files
Completed
Elapsed Time: 00:06:30
Restoring services
Completed
Elapsed Time: 00:00:05
Languages
Completed
Elapsed Time: 00:02:38
Mailbox Role
Failed
Error:
The following error was generated when "$error.Clear();
configure-WSManIISHosting -EnableKerberosModule;
" was run: "Failed to start WINRM service. Error: System.InvalidOperationException: Service winrm was not found on computer '.'. ---> System.ComponentModel.Win32Exception:
The specified service does not exist as an installed service
--- End of inner exception stack trace ---
at System.ServiceProcess.ServiceController.GenerateNames()
at System.ServiceProcess.ServiceController.get_ServiceName()
at System.ServiceProcess.ServiceController.GenerateStatus()
at System.ServiceProcess.ServiceController.get_Status()
at Microsoft.Exchange.Management.Deployment.ConfigureWSManIISHosting.ChangeWinrmServiceSettings()".
Failed to start WINRM service. Error: System.InvalidOperationException: Service winrm was not found on computer '.'. ---> System.ComponentModel.Win32Exception: The specified service does not exist as an installed service
--- End of inner exception stack trace ---
at System.ServiceProcess.ServiceController.GenerateNames()
at System.ServiceProcess.ServiceController.get_ServiceName()
at System.ServiceProcess.ServiceController.GenerateStatus()
at System.ServiceProcess.ServiceController.get_Status()
at Microsoft.Exchange.Management.Deployment.ConfigureWSManIISHosting.ChangeWinrmServiceSettings()
Service winrm was not found on computer '.'.
The specified service does not exist as an installed service
Click here for help... http://technet.microsoft.com/en-US/library/ms.exch.err.default(EXCHG.141).aspx?v=14.1.218.11&e=ms.exch.err.Ex88D115&l=0&cl=cp
Elapsed Time: 00:04:40
Management Tools
Cancelled
Finalizing Setup
Cancelled
A second attempt yielded the following errors:
Summary: 6 item(s). 3 succeeded, 1 failed.
Elapsed time: 00:17:34
Language Files
Completed
Elapsed Time: 00:06:51
Restoring services
Completed
Elapsed Time: 00:00:01
Languages
Completed
Elapsed Time: 00:02:09
Mailbox Role
Failed
Error:
The following error was generated when "$error.Clear();
$name = [Microsoft.Exchange.Management.RecipientTasks.EnableMailbox]::DiscoveryMailboxUniqueName;
$dispname = [Microsoft.Exchange.Management.RecipientTasks.EnableMailbox]::DiscoveryMailboxDisplayName;
$dismbx = get-mailbox -Filter {name -eq $name} -IgnoreDefaultScope -resultSize 1;
if( $dismbx -ne $null)
{
$srvname = $dismbx.ServerName;
if( $dismbx.Database -ne $null -and $RoleFqdnOrName -like "$srvname.*" )
{
Write-ExchangeSetupLog -info "Setup DiscoverySearchMailbox Permission.";
$mountedMdb = get-mailboxdatabase $dismbx.Database -status | where { $_.Mounted -eq $true };
if( $mountedMdb -eq $null )
{
Write-ExchangeSetupLog -info "Mounting database before stamp DiscoverySearchMailbox Permission...";
mount-database $dismbx.Database;
}
$mountedMdb = get-mailboxdatabase $dismbx.Database -status | where { $_.Mounted -eq $true };
if( $mountedMdb -ne $null )
{
$dmRoleGroupGuid = [Microsoft.Exchange.Data.Directory.Management.RoleGroup]::DiscoveryManagementWkGuid;
$dmRoleGroup = Get-RoleGroup -Identity $dmRoleGroupGuid -DomainController $RoleDomainController -ErrorAction:SilentlyContinue;
if( $dmRoleGroup -ne $null )
{
Add-MailboxPermission $dismbx -User $dmRoleGroup.Identity -AccessRights FullAccess -DomainController $RoleDomainController -WarningAction SilentlyContinue;
}
}
}
}
" was run: "Couldn't mount the database that you specified. Specified database: Mailbox Database; Error code: An Active Manager operation failed. Error: The database action failed. Error: Operation failed
with message: MapiExceptionCallFailed: Unable to mount database. (hr=0x80004005, ec=1105)
[Database: Mailbox Database, Server: EXCHMB02.mydomain.com].".
Couldn't mount the database that you specified. Specified database: Mailbox Database; Error code: An Active Manager operation failed. Error: The database action failed. Error: Operation failed with message: MapiExceptionCallFailed: Unable to mount database.
(hr=0x80004005, ec=1105)
[Database: Mailbox Database, Server: EXCHMB02.mydomain.com].
An Active Manager operation failed. Error: The database action failed. Error: Operation failed with message: MapiExceptionCallFailed: Unable to mount database. (hr=0x80004005, ec=1105)
[Database: Mailbox Database, Server: EXCHMB02.mydomain.com]
An Active Manager operation failed. Error: Operation failed with message: MapiExceptionCallFailed: Unable to mount database. (hr=0x80004005, ec=1105)
[Server: DC4.mydomain.com]
MapiExceptionCallFailed: Unable to mount database. (hr=0x80004005, ec=1105)
Click here for help... http://technet.microsoft.com/en-US/library/ms.exch.err.default(EXCHG.141).aspx?v=14.1.218.11&e=ms.exch.err.Ex88D115&l=0&cl=cp
Elapsed Time: 00:08:30
Management Tools
Cancelled
Finalizing Setup
Cancelled
The other mailbox server, EXCHMB02, completed the installation successfully but the Microsoft Exchange Service Host service won't start. It just gives an error stating "Windows could not start the Microsoft Exchange Service Host service on EXCHMB02"
and "Error 1053: The service did not respond to the start or control request in a timely fashion."
I'm not sure what to do at this point and my users are completely down until I can get the mailbox database re-mounted again. Someone please help!
Thanks!
September 1st, 2010 12:00pm
I had something very similar happen on one of my sites and in the end I had to DR the server. That will mean you will have to remove the servers from the DAG, wipe the OS, reinstall it and then do a recover server installation of Exchange. You should then
be able to mount the databases and recreate the DAG. I turned round the one I did in about three hours. I came to the conclusion that it would be better to build fresh rather than trying to repair a broken installation, and with the speed of installation of
Windows 2008 and Exchange, it was the quickest way.
Simon.Simon Butler, Exchange MVP. http://blog.sembee.co.uk , http://exbpa.com/
Free Windows Admin Tool Kit Click here and download it now
September 1st, 2010 2:54pm
After numerous attempts at reinstalling the service pack on various servers mailbox I finally got the database to mount on one of the disaster recovery servers. From there I was able to get the service pack to install successfully on one of the main mailbox
servers (EXCHMB01) and moved the active copy to that. Mailboxes are once again accessible from the Outlook client and from OWA.
I am in the process of completely rebuilding the other main mailbox server (EXCHMB02) from a clean OS install because I could never get all the services on that server to start properly. Once the rebuild is done, the new mailbox server is joined to the existing
DAG, and replication finishes I will do a clean reinstall on the first mailbox server (EXCHMB01). This has been a monumental pain in the a$$ but hopefully the new features and fixes in SP1 will be worth it.
Thanks!
September 1st, 2010 11:30pm