Exchange Backup is failing
Microsoft Windows [Version 6.0.6001]
Copyright (c) 2006 Microsoft Corporation. All rights reserved.
C:\Users\administrator.EAFG>cd\
C:\>vssadmin list writers
vssadmin 1.1 - Volume Shadow Copy Service administrative command-line tool
(C) Copyright 2001-2005 Microsoft Corp.
Writer name: 'System Writer'
Writer Id: {e8132975-6f93-4464-a53e-1050253ae220}
Writer Instance Id: {33265de3-94a9-4bb4-a116-32d7fbb3610f}
State: [1] Stable
Last error: No error
Writer name: 'Microsoft Exchange Writer'
Writer Id: {76fe1ac4-15f7-4bcd-987e-8e1acb462fb7}
Writer Instance Id: {e94067f4-2409-412d-95cc-ee62053f889c}
State: [1] Stable
Last error: Retryable error
Writer name: 'SqlServerWriter'
Writer Id: {a65faa63-5ea8-4ebc-9dbd-a0c4db26912a}
Writer Instance Id: {6cb85e0e-d1d6-4c0a-8e22-2b8820a37823}
State: [1] Stable
Last error: No error
Writer name: 'ASR Writer'
Writer Id: {be000cbe-11fe-4426-9c58-531aa6355fc4}
Writer Instance Id: {a06f0deb-4d41-455b-96fa-8c6204477cb9}
State: [1] Stable
Last error: No error
Writer name: 'Shadow Copy Optimization Writer'
Writer Id: {4dc3bdd4-ab48-4d07-adb0-3bee2926fd7f}
Writer Instance Id: {210def5a-b5bb-400a-9d64-c81856ed8501}
State: [1] Stable
Last error: No error
Writer name: 'Registry Writer'
Writer Id: {afbab4a2-367d-4d15-a586-71dbb18f8485}
Writer Instance Id: {bd4bbc30-a61e-4bec-8c08-adcefb1d598e}
State: [1] Stable
Last error: No error
Writer name: 'COM+ REGDB Writer'
Writer Id: {542da469-d3e1-473c-9f4f-7847f01fc64f}
Writer Instance Id: {d5f9a951-513d-4e3a-91b8-3569c4c12f6f}
State: [1] Stable
Last error: No error
Writer name: 'BITS Writer'
Writer Id: {4969d978-be47-48b0-b100-f328f07ac1e0}
Writer Instance Id: {952735e5-38cf-40c1-b504-3538690971f8}
State: [1] Stable
Last error: No error
Writer name: 'WMI Writer'
Writer Id: {a6ad56c2-b509-4e6c-bb19-49d8f43532f0}
Writer Instance Id: {bcf8d987-b579-45b8-9281-6c561a83b254}
State: [1] Stable
Last error: No error
Writer name: 'IIS Config Writer'
Writer Id: {2a40fd15-dfca-4aa8-a654-1f8c654603f6}
Writer Instance Id: {fdd3ca0d-fcd3-432f-a4b6-2d13ba20b3fe}
State: [1] Stable
Last error: No error
Writer name: 'IIS Metabase Writer'
Writer Id: {59b1f0cf-90ef-465f-9609-6ca8b2938366}
Writer Instance Id: {f70ddeae-ffdd-48f9-8f9b-a8a2197bfc2f}
State: [1] Stable
Last error: No error
Writer name: 'Cluster Database'
Writer Id: {41e12264-35d8-479b-8e5c-9b23d1dad37e}
Writer Instance Id: {fa0cb64e-251d-4d64-8457-5751f22a48ab}
State: [1] Stable
Last error: No error
C:\>vssadmin list writers
vssadmin 1.1 - Volume Shadow Copy Service administrative command-line tool
(C) Copyright 2001-2005 Microsoft Corp.
Writer name: 'System Writer'
Writer Id: {e8132975-6f93-4464-a53e-1050253ae220}
Writer Instance Id: {33265de3-94a9-4bb4-a116-32d7fbb3610f}
State: [1] Stable
Last error: No error
Writer name: 'Microsoft Exchange Writer'
Writer Id: {76fe1ac4-15f7-4bcd-987e-8e1acb462fb7}
Writer Instance Id: {e94067f4-2409-412d-95cc-ee62053f889c}
State: [1] Stable
Last error: Retryable error
Writer name: 'ASR Writer'
Writer Id: {be000cbe-11fe-4426-9c58-531aa6355fc4}
Writer Instance Id: {a06f0deb-4d41-455b-96fa-8c6204477cb9}
State: [1] Stable
Last error: No error
Writer name: 'SqlServerWriter'
Writer Id: {a65faa63-5ea8-4ebc-9dbd-a0c4db26912a}
Writer Instance Id: {6cb85e0e-d1d6-4c0a-8e22-2b8820a37823}
State: [1] Stable
Last error: No error
Writer name: 'Shadow Copy Optimization Writer'
Writer Id: {4dc3bdd4-ab48-4d07-adb0-3bee2926fd7f}
Writer Instance Id: {210def5a-b5bb-400a-9d64-c81856ed8501}
State: [1] Stable
Last error: No error
Writer name: 'Cluster Database'
Writer Id: {41e12264-35d8-479b-8e5c-9b23d1dad37e}
Writer Instance Id: {fa0cb64e-251d-4d64-8457-5751f22a48ab}
State: [1] Stable
Last error: No error
Writer name: 'Registry Writer'
Writer Id: {afbab4a2-367d-4d15-a586-71dbb18f8485}
Writer Instance Id: {bd4bbc30-a61e-4bec-8c08-adcefb1d598e}
State: [1] Stable
Last error: No error
Writer name: 'IIS Metabase Writer'
Writer Id: {59b1f0cf-90ef-465f-9609-6ca8b2938366}
Writer Instance Id: {f70ddeae-ffdd-48f9-8f9b-a8a2197bfc2f}
State: [1] Stable
Last error: No error
Writer name: 'BITS Writer'
Writer Id: {4969d978-be47-48b0-b100-f328f07ac1e0}
Writer Instance Id: {952735e5-38cf-40c1-b504-3538690971f8}
State: [1] Stable
Last error: No error
Writer name: 'IIS Config Writer'
Writer Id: {2a40fd15-dfca-4aa8-a654-1f8c654603f6}
Writer Instance Id: {fdd3ca0d-fcd3-432f-a4b6-2d13ba20b3fe}
State: [1] Stable
Last error: No error
Writer name: 'COM+ REGDB Writer'
Writer Id: {542da469-d3e1-473c-9f4f-7847f01fc64f}
Writer Instance Id: {d5f9a951-513d-4e3a-91b8-3569c4c12f6f}
State: [1] Stable
Last error: No error
Writer name: 'WMI Writer'
Writer Id: {a6ad56c2-b509-4e6c-bb19-49d8f43532f0}
Writer Instance Id: {bcf8d987-b579-45b8-9281-6c561a83b254}
State: [1] Stable
Last error: No error
C:\>
I'm running Exchange 2007 Ent (no SP) on Windows Server 2008 Ent.
I'm using Symantec Backup Exec 2010.
I've restarted the services a couple of time it doens't fix it. Next time I rebooted the server and the servces was running fine. Then few hours laster it stopped. Where oculd be the problem!
October 12th, 2010 5:55pm
has BE 2010 ever worked for you? I would first try to apply Exchange SP and OS SP's since there are many updates to the Exchange Writer
Troy Werelius
www.Lucid8.com
Free Windows Admin Tool Kit Click here and download it now
October 12th, 2010 7:21pm
has BE 2010 ever worked for you? What rights does the account the BE is using have granted to it? To ensure its not a rights based issue you should use an account that has Local machine admin, Domain admin, Exchange Full Admin Rights
I would first try to apply Exchange SP and OS SP's since there are many updates to the Exchange Writer
Troy Werelius
www.Lucid8.com
October 12th, 2010 7:21pm
This accounts permission has full access, I've run the test BE Test to make sure that backup account works fine and it did passed all the test. So no issues with the backup. Backup jobs runs fine for let say an hour after restarting the server. I did a test
backup of temp SG and it ran for 10 minutes without any issues and later after few hours VSS stopped working or as mentioned above.
As far as SP for Exchange and Windows is concerned what crticial things to consider. I've this mailbox setup in a cluster.
thanks in advance.
Free Windows Admin Tool Kit Click here and download it now
October 13th, 2010 9:52am
As far as SP and Clusters you will want to ensure both are installed to the same SP level, and I would suggest that you go to SP3 on 2007
Is the Cluster setup as Active/Active or Active/Passive? Also have a look at these articles since they may be of assistance but again many of these assume you are on later SP levels for Exchange so I would do that first to ensure you are up to date
http://msexchangeteam.com/archive/2008/08/25/449684.aspx
http://technet.microsoft.com/en-us/library/ee221172(EXCHG.80).aspx
http://social.technet.microsoft.com/Forums/en-US/exchangesvravailabilityandisasterrecovery/thread/5bac7d4d-4024-40e7-a0a9-1a1211722476
Troy Werelius
www.Lucid8.com
October 13th, 2010 2:43pm
As far as SP and Clusters you will want to ensure both are installed to the same SP level, and I would suggest that you go to SP3 on 2007
Is the Cluster setup as Active/Active or Active/Passive? Also have a look at these articles since they may be of assistance but again many of these assume you are on later SP levels for Exchange so I would do that first to ensure you are up to date
http://msexchangeteam.com/archive/2008/08/25/449684.aspx
http://technet.microsoft.com/en-us/library/ee221172(EXCHG.80).aspx
http://social.technet.microsoft.com/Forums/en-US/exchangesvravailabilityandisasterrecovery/thread/5bac7d4d-4024-40e7-a0a9-1a1211722476
Troy Werelius
www.Lucid8.com
Free Windows Admin Tool Kit Click here and download it now
October 13th, 2010 2:43pm
As far as SP and Clusters you will want to ensure both are installed to the same SP level, and I would suggest that you go to SP3 on 2007
Is the Cluster setup as Active/Active or Active/Passive? Also have a look at these articles since they may be of assistance but again many of these assume you are on later SP levels for Exchange so I would do that first to ensure you are up to date
http://msexchangeteam.com/archive/2008/08/25/449684.aspx
http://technet.microsoft.com/en-us/library/ee221172(EXCHG.80).aspx
http://social.technet.microsoft.com/Forums/en-US/exchangesvravailabilityandisasterrecovery/thread/5bac7d4d-4024-40e7-a0a9-1a1211722476
Troy Werelius
www.Lucid8.com
October 13th, 2010 2:43pm
Hi Vministrator,
Is there any error event in the Event Viewer?
You can also increase the diagnostic logging on Exchange Writer.
get-eventloglevel |where-object {$_.identity -like "*Exchange Writer*"} | set-eventloglevel -level expert
More information:
Exchange 2007 Data Backup and Volume Shadow Copy Services
http://technet.microsoft.com/en-us/library/dd233256(EXCHG.80).aspxFrank Wang
Free Windows Admin Tool Kit Click here and download it now
October 15th, 2010 4:38am
the VSS Writer for Exchange is stable now. (in the last 24 hr). The test backup runs fine.
i've also updated the accunt credential for Symantec Backup Exec Engine and related services.
Now there is an issue with the hardware and Dell is replaec it but I've noticed this error in the event view, does it relate to it?
Log Name: Application
Source: MSExchangeRepl
Date: 10/15/2010 11:06:36 AM
Event ID: 2147
Task Category: Service
Level: Error
Keywords: Classic
User: N/A
Computer: EAI-MAILCLUS-01.eafg.local
Description:
There was a problem with 'eai-mailclus-02', which is an alternate name for 'EAI-MAILCLUS-02'. The list of aliases is now 'eai-mailclus-02', and the alias 'was' removed from the list. The specific problem is 'CreateFile(\\eai-mailclus-02\94f32e5a-fa3f-498b-9be2-a5d108fe29c2$\E000004BEB9.log)
= 2'.
Event Xml:
<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
<System>
<Provider Name="MSExchangeRepl" />
<EventID Qualifiers="49156">2147</EventID>
<Level>2</Level>
<Task>1</Task>
<Keywords>0x80000000000000</Keywords>
<TimeCreated SystemTime="2010-10-15T15:06:36.000Z" />
<EventRecordID>240935</EventRecordID>
<Channel>Application</Channel>
<Computer>EAI-MAILCLUS-01.eafg.local</Computer>
<Security />
</System>
<EventData>
<Data>eai-mailclus-02</Data>
<Data>EAI-MAILCLUS-02</Data>
<Data>eai-mailclus-02</Data>
<Data>was</Data>
<Data>CreateFile(\\eai-mailclus-02\94f32e5a-fa3f-498b-9be2-a5d108fe29c2$\E000004BEB9.log) = 2</Data>
</EventData>
</Event>
October 15th, 2010 12:30pm
What did you do to resolve the first issue? What changed? Looks like a problem with the replication service, open this log
\\eai-mailclus-02\94f32e5a-fa3f-498b-9be2-a5d108fe29c2$\E000004BEB9.log, review and then if its still a mystery paste the contents of the log in a reply
Troy Werelius
www.Lucid8.com
Free Windows Admin Tool Kit Click here and download it now
October 15th, 2010 12:35pm
I'm using Symantec Backup Excec 2010 to backup Exchange 2007.
For Exchange 2007 SP and OS SPs, I'm reluctant to it becuase
WAHT IF -
Something goes wrong what is the impact on Clusters (Mailbox are clustered as Active/Passive CCR) - how do I recover?
Two CAS/HUB are in standalone mode (no NLB). I'm thinking to make SP updates on one of these (preferably the secondary one )
The above Articles are good from Troy - First two doesn't really relate becuase I'm using Symantec Backup exce.. I'm going through the thrid one but still it is related to Windows update.
October 15th, 2010 12:44pm
I understand your concern, however, IMO its always best to be on the latest SP for multiple reasons including the fact that it may solve your current issue. That said you say its working now, but with an additional error regarding the Exchange replication
so as before;
What did you do to resolve the first issue? What changed? Looks like a problem with the replication service, open this log
\\eai-mailclus-02\94f32e5a-fa3f-498b-9be2-a5d108fe29c2$\E000004BEB9.log, review and then if its still a mystery paste the contents of the log in a reply
Also if you know you have a hardware issue I would suggest you get that fixed as well ASAP since it could be related to the hardware problem.
Troy Werelius
www.Lucid8.com
Free Windows Admin Tool Kit Click here and download it now
October 15th, 2010 12:58pm
the only thing i've done to fix the first issue is
After I spoke to the Syamntec Tech Support , he had me to create a new service account and give it all the related permissions (domain admin, exchange admine etc) and add this account into Symantec Backup Configuration where this account is being used. After
doing that i was able to to do the backup.
on the same support call I've asked them that i need to restore individual items , when we tried we coulnd't. he asked me install MAPI client on the server, so did I. Backup job failed again that night. Next morning I've check the logs and found the backup
job failed. Called again Symantec and figured out that VSS Writer was in Retryable State. Tried to fix it but didn't have any clue..restarted the services , restarted the server etc.
While doing all this I've noticed in the Task Manager that BEENGINE.EXE (backupexe engine) services was using old service account, then I changed the service account for all symantec services to the new account. And after 10-15 mins i've noticed that VSS
Writer was with No Errors (Stable Mode). And i think this was fix.
And since yesterday there is an issue with IBM Tape Drive (from DELL) and it is going to be replaced by DELL today. I hope this will be the final fix. I'm still able to do the backup on Active Node which is identical to this server with the same internal
drive.
I think now I should publish this on my blog :) wait ..let the DELL Tech come and isntall the new tape drive first.
this post will be here http://adnanrafik.com/blog
October 15th, 2010 2:39pm
\\eai-mailclus-02\94f32e5a-fa3f-498b-9be2-a5d108fe29c2$\E000004BEB9.log
i don't find it anymore here. All i cans ee the log files from 2:17 PM to current time...isn't weired?
now this is new file.
CreateFile(\\eai-mailclus-02\54290031-d4d4-451f-ae33-ea1c152f76e5$\E040000E037.log)
how do I review it... is there any tool?
Free Windows Admin Tool Kit Click here and download it now
October 15th, 2010 2:42pm
Ok so 99% of this was a BE configuration issue eh? Thats one of the reasons I asked if it had ever worked. BUE is a finicky little SOB and if you don't have it configured just so it will not work and then give you obtuse errors.
This is also why I posted the links to backup rights since they are fairly common between backup products
Yes I would wait until you have it all figured out and then post the details.
Troy Werelius
www.Lucid8.com
October 15th, 2010 2:59pm
I would bet that the logs expire or get overwritten which is why you don't find the old one
Just use NOTEPAD to open it up and then past it in as a response Check out these articles as well I think they will be of assistance
http://social.technet.microsoft.com/Forums/en-CA/exchangesoftwareupdate/thread/eca3bbf7-ee9f-41bd-89e8-47a81780292b
http://support.microsoft.com/kb/975255
http://blogs.technet.com/b/timmcmic/archive/2010/07/11/msexchangerepl-2147-msexchangerepl-2104-msexchangerepl-2127-occurring-on-windows-2008-or-windows-2008-r2-with-exchange-2007-cluster-continuous-replication-ccr.aspx
Troy Werelius
www.Lucid8.com
Free Windows Admin Tool Kit Click here and download it now
October 15th, 2010 3:07pm
thnx for the additional links - going through it.
DELL Tech came over and replaced it. I did a test backup ans restore of a 50 Megs file and it went fine. So I think this has been solved.
October 15th, 2010 4:11pm
it failed again while the VSSAdmin show that the Exchage Writer is Stable.
FYI- the backup job only fails when the Symantec is confifugre to backup the Passive Node instead Active. When I change backup policy from Passive to Active it does work.
Free Windows Admin Tool Kit Click here and download it now
October 18th, 2010 4:35pm
Have you spoken to Symantec about this? Troy Werelius
www.Lucid8.com
October 18th, 2010 4:46pm
Not yet, I'm going to call them again. Last time I faced the same problem and end up seeing the issue with VSS Exchange Writer. Let me call them and update this thread. thanks for the quikc response,
Free Windows Admin Tool Kit Click here and download it now
October 18th, 2010 4:48pm
Happy to assist, however yeah sounds like a product specific issue now so I would definitely check with them firstTroy Werelius
www.Lucid8.com
October 18th, 2010 4:50pm
BTW how is your page file set and what is it set to? I ran into similar long back with BE failing every couple days with VSS going into retryable state. Ended up being memory related with incorrect page file set being the main culprit.
James Chong (MVP)
MCITP | EA | EMA; MCSE | M+, S+
Security+, Project+, ITIL
msexchangetips.blogspot.comJames Chong (MVP) MCITP | EA | EMA; MCSE | M+, S+ Security+, Project+, ITIL msexchangetips.blogspot.com
Free Windows Admin Tool Kit Click here and download it now
October 18th, 2010 5:09pm
I don't remember if I ever changed it to customized but it is set as
12271 MB on C drive and Server has 8 gigs
October 18th, 2010 5:13pm
http://msdn.microsoft.com/en-us/library/bb530721(VS.85).aspx
BETEST.EXE is same for both windows 2003 and 2008?
The bestest.exe command failed a couple of times, it seems that MSFT Level 1 support engineer didn't have the idea and let me ran the wrong betest.exe file. BETEST.EXE have been failing while saving the output to component file and it generated Error 112
Free Windows Admin Tool Kit Click here and download it now
October 19th, 2010 11:43am
Well I think thats a two part answer
The article says "BETest is included in the Microsoft Windows Software Development Kit (SDK) for Windows Vista and later" so you need to download the SDK for the OS platform you want to run BETest on
The article also says "The VSS 7.2 SDK includes a version of BETest that runs only on Windows Server 2003" so that version only supports 2003
Troy Werelius
www.Lucid8.com
October 19th, 2010 11:55am
http://www.microsoft.com/downloads/en/details.aspx?FamilyID=c17ba869-9671-4330-a63e-1fd44e0e2505&displaylang=en
I've downloaded and installed the SDK for Windows 7/2008 but when I run the VSWriter and Betest these EXCE doesnt run and come up with tha mesaage that it is not a valid 32 bit app , though I've tried running the 32-bit exe too but same error message.
Free Windows Admin Tool Kit Click here and download it now
October 19th, 2010 12:47pm
Hmm I really count say what the actual issue there is. Have you talked to Symantec since they are the product that is failing?Troy Werelius
www.Lucid8.com
October 19th, 2010 1:58pm