Offline Address Book not downloading (from CCR cluster)
Hi there,
I have an issue getting my Outlook 2007 clients to download the offline OAB in the Exchange 2007 dev environment I've setup. The Dev configuration is:
Exchange/Domino 2007 POC environment
1 x Windows 2003 Domain Controller
1 x E2k7 Hub Transport / CAS Server - Windows 2008 RC 1 / Exchange 2007 SP1 / Transporter / MAPI+CDO 1.2.1
2 x E2k7 Mailbox Server (CCR cluster) - Windows 2008 RC 1 / Exchange 2007 SP1
When I attempt a download of the offline OAB, my Outlook client comes back with an error citing 0x80190194 as the code. After much investigation, it seems that error is basically because the client can't find the relevent OAB files on my CAS server. Sure enough, when I look at the CAS virtual OAB directory, there's nothing in there. It hasn't been able to retrieve them from the Mailbox servers. Investigating the event log on the CAS server throws up several errors from the Microsoft Exchange File Distribution service:
Event ID - 1021
Source - MSExchangeFDS
Error message - "Process MSExchangeFDS.exe (PID=1500). Could not find directory \\E2K7EXCS02\ExchangeOAB\13dc5977-0a50-4506-ad04-7081c945f618. This is normal if the directory has never been generated. Otherwise, make sure this directory and share has read permission for the "Exchange Servers" group. "
Now here's where it get's wierd. That is the OAB download path for the backend mailbox server (\\E2K7EXCS02\ExchangeOAB\...) , however it doesn't exist from a cluster perspective, and can't exist because my CCR cluster doesn't have shared storage, and relies on CCR replication to replicate mailbox data from one node to the other. However, the OAB files aren't replicated as part of this process. Thearticle here explains the dilemma http://blogs.msdn.com/dgoldman/archive/2007/02/08/oab-generation-on-a-cluster-server-fails-with-event-id-9395-or-9396.aspx
It basically says to ensure the 'SYSTEM\CurrentControlSet\Services\MSExchangeSA\Parameters\OABGen-Server\EnableOabGenOnThisNode' registry key reflects the active node - which it does. And I've checked the ExchangeOAB share does exist on my primary node, and that the OAB files are there. HOWEVER - this share is not accesible as a cluster resource as it's not on shared disk.
So some questions:
How can the CAS server ever be expected to retrieve these files using the cluster UNC path \E2K7EXCS02\ExchangeOAB\ when it's not accesible (as we have no shared storage on the cluster)? Can we change the UNC path to the active node somehow? eg. \\ActiveNode\ExchangeOAB
How do you achieve high availability of the OAB files in this scenario? - is the only solution to use shared storage and failover the OAB files that way?
Thanks for your help.
Cheers, James.
February 14th, 2008 3:16am
Hi James,
I have exactly the same issue. It looks like CCR-clustered OAB server now works only on the Windows Server 2003...
On W2K3 it was possible to access locally shared foldersusing cluster name. Looks like a bug in the E2K7...
Best regards,
Dmitry
Free Windows Admin Tool Kit Click here and download it now
April 12th, 2008 1:50am
Hi James,
I too am having the exact same problem. The OAB share exists on the primary node, but is not accessible via the cluster name. Have you been able to resolve this at all?
Thanks,
Steven
May 1st, 2008 2:39am
Anyone have ideas on this?
Free Windows Admin Tool Kit Click here and download it now
May 2nd, 2008 3:10am
Hi Together,
We have exactly the same problem at a new customers W2K8 RTM + E2K7 SP1 (2 * MB CCR + 2* CAS, HT) environment. It seems to be a problem with Windows Server 2008.
OAB is generated on the first nodeof the CCR and is correctly placed in the corresponding local folder of the first CCR node.We have an existing ExchangeOAB share on the first node but not on the virtual Cluster.
In my testing environment W2K8 RTM + E2K7 SP1 (2 * MB CCR + 1 * CAS HT + 1 * CAS, HT, MB) I moved theOAB generator to the single Mailbox Role on the CAS, HT Server. This works fine, because the CAS can access ExchangeOAB share on the Mailbox Server. OAB generator back on the CCR - same error like before!!!
Never the less I tried to fix this problem onmany ways - none worked.... It not seems to be anew feature? ;-)
Just as Info: The same environment with W2K3 SP2 E2K7 SP1works fine.
Any Ideas?!
So long
Dominik R.
May 5th, 2008 10:43pm
I think that the "root cause" of the problem is a changed share visibility in W2K8 server.
For W2K3, if you access cluster from the network you see all cluster sharesPLUS sharesof thecurrent NODE. For W2K8 - shares that exists on the current node are NOT visible is accessedvia cluster network name.
Maybe there is a command that revert W2K8 behavior to W2K3? Something like "cluster res ..." (i guess).
Free Windows Admin Tool Kit Click here and download it now
May 6th, 2008 7:47am
I agree with your assessment of the root cause. However, I have not found a way to expose the shares of the node through the cluster/application name for the CCR...
May 6th, 2008 3:31pm
Just thought I would confirm that I've been dealing with this issue also, but dont have a solution as of yet.
If I get an answer or come up with something I'll be sure to share it.
Erik
Free Windows Admin Tool Kit Click here and download it now
May 7th, 2008 2:25am
Hi @ all,
opened a premier support call.
I am sure this is a not implemented feature (or called a bug ;-) )
I will inform you all when a solution is found.
Best wishes
Dominik
May 13th, 2008 12:05pm
I also have a premier support issue open and there are at least several others. The folks @ MSFT are working on the issue & trying to identify a solution that will work for all effected.
If you have the issue I suggest contacting MSFT support so they can work with you. There are a couple of workarounds in the mix but so far nothing solid I can comment on as of yet.
Erik
Free Windows Admin Tool Kit Click here and download it now
May 14th, 2008 3:35am
Thanks forthe info so far. I'm in the same situation: Windows2008 with Exchange2007CCR.
As a temporary workaround, I simply copied the OAB folder from one of the CCR nodes to the Client Access servers' OAB folder. I added Domain Users Read permissions, and all users are happily obtaining the OAB. Our environment is fairly static, so that will work for now.
Edit: Opened case with MS.
Case status as of 20080523-091200EDT:
All cases on this issue are being gathered by the Exchange support team.
It is a bug, and they are goingescalate itto the Exchange development team.
I suspect that they will modifyExchange to pull the OAB from the active CCR cluster node rather than from the cluster name.
May 15th, 2008 8:33pm
I've got a customer that is also copying the OAB out to the CASes (& modifying the NTFS ACL), so far that seems to be working fine.
At the moment there are a couple of issues relating to having the OAB generation take place on a CMS. There is this issue with CCR as well as an outstanding issue I've heard about where the OAB generation fails with SCC.
So far as I'm aware the only sure-fire and supported way to avoid these problems is to have OAB generation occur on a non-clustered mailbox server. If you've got sufficient scale in your environment you may already have one so simply move generation to this server, in smaller environments this is generally less of an option. For what it's worth if you are looking at adding an additional MBX server to your environment for this purpose the requirements are pretty low, it could probably even be virtualized (insert standard virtualized Exchange disclaimer here).
Again, if you are running into these issues be sure to open a case with PSS. The more cases they have on it the easier time the folks there will have pushing this over to sustained engineering for a fix.
Erik
Free Windows Admin Tool Kit Click here and download it now
May 20th, 2008 12:36am
A little bit more information:http://www.spyordie007.com/blog/index.php?mode=viewid&post_id=25
May 29th, 2008 9:48pm
I have the exact same problem. WS2008 CCR cluster, Exchange 2007 SP1, two CAS/HTS combo servers. The OAB is trying to download from CMS1 which is my cluster virtual node and it can't find the share. Very frustrating! Seems like a pretty obvious problem that slipped through testing.
Free Windows Admin Tool Kit Click here and download it now
May 31st, 2008 2:40am
What NTFS ACL changes are needed? Do any IIS permissions need to be changed as well?
June 2nd, 2008 10:16pm
I used a Band-Aid for the time being, I just used Robocop built into 2008 to replicate the OAB files to the CAS servers. Setup a schedule job to update it every 15 minutes. I am guessing you could use DFS to do the same.
From the active CCR OAB Server
Robocopy.exe T:\OAB "\\CAS\c$\Program Files\Microsoft\Exchange Server\ClientAccess\OAB" /B /MIR /TEE /LOG:t:\oab.log
Still waiting for the true fix as this will fail the second the Active CCR node fails over to the Passive node, because the passive node does not create or update the OAB.
Hope that helps some,
Scott
Free Windows Admin Tool Kit Click here and download it now
June 3rd, 2008 10:12pm
This is what i am facing , is it same you guys are facing
I have server 2008 running as domain controller and 3 server 2008 enterprise edition running on 2 active and passive CCR cluster and third is hub transport , i can check email using owa , but mapi client fails shows server unavailable , seems to be issue with OAB is not downloading , i checked all status is good but event viewer is showing this error message event id 1021 indicating \\MBX\EXCHANGE OAB\(OAB GUID) IS NOT AVAILABLE, IF YOU TRY AND PULL THE PATH YOU WILL NOTICE THAT IT DOES NOT EXIST.
THE NET RESULT IS THAT YOUR OUTLOOK CLIENT CANNOT DOWNLOAD A COPY OF OAB and get the error server unavailable. this message comes exchangeFDS , file distribution service
another thing is that when you go to properties of mailbox cluster you get grayed out sign , but on active and passive node it shows fine and shows 1st active node is owner of the quoram.
does anyone face this issue.
i tried everything but no luck
June 18th, 2008 9:17pm
YngDiego777 wrote:
What NTFS ACL changes are needed? Do any IIS permissions need to be changed as well?I've got some details on the file-copy workaround in my blog (link above); however this is unsupported and if you can wait I highly recommend that you do. I'm being cautious giving out much details because if you do this you're on your own.Just to provide an update the folks at Microsoft have confirmed this issue and are working on a fix. How we will see this fix has yet to be announced (hotfix, rollup, etc.)I don't know how soon we'll see a supported fix, but we will see one. If I were to speculate I'd say more likely sooner rather then later.Erik
Free Windows Admin Tool Kit Click here and download it now
June 18th, 2008 10:16pm
Mr henrik this is for your attention since you are having the same issue, look at my blog and please comment on it. i will appreciate your comments
http://forums.microsoft.com/TechNet/ShowPost.aspx?PostID=3519232&SiteID=17&mode=1
June 22nd, 2008 11:13am
Hi James
You can create batch file to copy OAB directory contents form yourCMS to the CAS server.
Configure scheduled task to run the batch file when event id 9107 is logged under Application log by MSExchangeSA.
Free Windows Admin Tool Kit Click here and download it now
September 15th, 2008 8:29am
I have Windows 2003 R2 SP2 x64 CCR cluster. the EXCHANGEOAB file share ONLY exists on the first CCR node. Even if I failover to the passive node, this directory is NOT re-created. I can only use my original CCR Active node.
September 18th, 2008 9:41pm
Manual workarounds are no longer required, please contact PSS for the hotfix.
Erik
Free Windows Admin Tool Kit Click here and download it now
September 25th, 2008 11:26pm
Do you know the hotfix # offhand? It would help when calling MS. Thanks!
September 25th, 2008 11:30pm
Sorry I dont have the hotfix number, I think it's still early distribution(they just finished the localized builds) so you wouldnt be able to download through the regular hotfix request channels anyways.
They should be aware of the issue now, so give them a call and describe the problem.
Erik
Free Windows Admin Tool Kit Click here and download it now
September 25th, 2008 11:38pm
Any idea if this will be included in the re-release of Update Rollup 4 for Exchange Server 2007 Service Pack 1 (KB952580)?
Scott
September 25th, 2008 11:43pm
s33butler wrote:
Any idea if this will be included in the re-release of Update Rollup 4 for Exchange Server 2007 Service Pack 1 (KB952580)?
Scott
No, it is not in RU4 for SP1.
Erik
Free Windows Admin Tool Kit Click here and download it now
September 25th, 2008 11:47pm
Has anyone recieved this hotfix? Does anyone have the hotfix # or MSKB article ID?
Thank you,
Scott
October 9th, 2008 7:56pm
s33butler wrote:
Has anyone recieved this hotfix? Does anyone have the hotfix # or MSKB article ID?
Thank you,
Scott
Pretty sure the KB is 954197. It's not public yet. PSS is checking on the build status of the hotfix. The sustaining engineering team has the hotfix, and seems they will work with customers to install it.
Free Windows Admin Tool Kit Click here and download it now
October 9th, 2008 9:41pm
I received additional information from Microsoft on the hotfix and wrote a blog about it. Please see my blog for more information. The information is not as positive as I would have hoped.
http://blogs.pointbridge.com/Blogs/seaman_derek/Pages/Post.aspx?_ID=32
October 10th, 2008 7:59pm
Iinstalled 2 days ago aWindows 2003 / Exchange 2007 SP1 CCR with update roollup 4.
And I have the same issue.
I don't think it's just a problem with Windows 2008
Free Windows Admin Tool Kit Click here and download it now
October 16th, 2008 11:01pm
Same issue here, under 2003 Enterprise.Running a CCR Cluster, and whenever the "primary" server (the one installed first) is active, \\clusterMB\ExchangeOAB is available. When the "secondary" server is active, that share doesn't exist. According to MS, the share should exist but be stale, however it simply goes away.The c:\program files\microsoft\exchange server\exchangeoab directory doesn't even exist on the secondary node, which I am given to understand is by design...?
December 23rd, 2008 6:20pm
We had the same issue and it is specific to Windows Server 2008 and Exchange Server 2007 in a SCC / CCR configuration. See this http://www.korneel.nl/?blogid=1&archive=2009-01 article. There is a proposed fix for this in the article but it is a 32bit fix and no 64bit fix is listed or available. What we did is move the OAB to one of our Hub Transport/ Client Access machines and it now works fine. My question is, when can we expect a fix for this from Microsoft? Regards Frederick
Free Windows Admin Tool Kit Click here and download it now
January 4th, 2010 12:45pm
I was having a similar problem (2xCCR MBX and 2 x CAS, Server 2008 R1). To recap, what I was seeing in the event logs when updating the offline address book on the Active Mailbox Node (In order of occurance):Event ID: 9343Event ID: 9117Event ID: 9106Event ID: 9377Event ID: 9108Event ID: 9347Event ID: 9377Event ID: 9115Event ID: 9377Event ID: 9354Event ID: 9408 - Error: OALGen failed to load the offline address book version 4 manifest file '\\EXCHANGE\ExchangeOAB\GUID\oab.xml'. This indicates data tampering or disk problems. Restore files in this folder from the recent backup or clean up folder content and force a full OAB generation.Event ID: 9377Event ID: 9331 - Error: OALGen encountered error 8004010f (internal ID 501080c) accessing the public folder store while generating the offline address list for address list '\Global Address List'.
- Default Offline Address ListEvent ID: 9371 - Error: OALGen encountered an error while generating the differential downloads of address list '\Global Address List'. The offline address list has not been updated so clients will not be able to download the current set of changes. Check other logged events to find the cause of this error.
- Default Offline Address ListEvent ID: 9107To make a long story short, this was my eventual solution: When looking at the Shares on the Active Mailbox server, I noticed that there were 2 identical share names (same path, permissions, etc.) pointing to the \\EXCHANGE\EXCHANGEOAB\. I removed one of the shares and re-ran update offline address book and the errors went away and my OAB folder finally populated.Good luck,David
March 15th, 2010 8:46pm