Store.exe high memory and high CPU
One of our mailbox servers has caused some issues for us in the past week. It is currently running on SCC nodes with Win2k3, quad 2.4 GHz processors, 8GB RAM, and a redundant storage array for the DB. Page file is ~15 GB, with 7.5 GB being used. Only 100 or so users on it, with an approx mailbox size of 1-3 GB. Earlier this week, while the server had 4 GB of RAM, it failed over to the second node for some reason. After a reboot of the primary and then a fail back, we took the virtual server offline then increased the RAM to 8 GB, thinking a reboot and low memory might have been the cause. It's now been 3 days and the server has been fine so far, but just a short while ago a check of resources showed store.exe taking up 7.1GB of RAM on the server, and the CPU spiking periodically to about 30%, causing slowness issues for users. Exchange 2007 is running with SP1 on it, and I'm reading plenty of material out there that this is "by design" but if thats the case, why are the other 2 mailbox servers running fine with 4 GB RAM and utilization at about 2 GB for the store.exe process? CPU is also not spiking like it is on this server. I am going to try to reboot the server if it gets bad enough, but as a long term solution it's obviously not ideal.
June 12th, 2009 11:32pm
Hi, As you said this is a default nature of exchange 2007 to use maximum memory as it available and release if other process need it periodically. Are you using scan mail for your exchange 2007 node, If yes then please check whether scan mail SP1 is updated. this could be one of the issue to raising high memory utilization. I believe that 4 GB RAM is not enough as this is holding 100 + users (we need to plan it as per user utilization basis)
Exchange 2007 Processor and Memory Recommendations::
http://msexchangeteam.com/archive/2007/01/16/432222.aspxAnil
Free Windows Admin Tool Kit Click here and download it now
June 14th, 2009 6:12pm
Hi Anil, we are not using scan mail and the RAM has been upgraded to 8 GB already. Logs and the db's are on different disks, as is the page file.
June 14th, 2009 6:44pm
Check info:
1. Based on your description, its an exchange 2007 SP1 SCC environment, right? Do another two nodes also run as active node? Please describe more details about the exchange topology
2. You said the active node failover for some reasons in the past week, could you recall any change on the exchange server?
3. Is there any difference between the problematic active node and other fine active node? Like the number of the mailbox, the configuration and etc
4. If any other application is built on the Microsoft .NET Framework 2.0 in the exchange server, similar symptom will occur, please see HotFix 942027
5. If the automatic failover issue occurs again, please check if theres any error/warning event in the application log on the exchange servers
6. Please run ExBPA against the exchange server for a health check
7. If there is no other memory pressure and when the store is busy (During business peaks, or performing backups/online defragments), we could expect that the store.exe process will grow to use up to 8GB memory
Resources:
Similar case: Huge process called store.exe in active node in Exchange Server 2007 Cluster
Free Windows Admin Tool Kit Click here and download it now
June 15th, 2009 6:08am
1) Each mailbox server is in it's own 2-node cluster, with one active and one passive 2) We were not able to find the cause of the failover 3) One of the other mailbox servers is running fine with half the amount of RAM and just a few less users. 4) There are no other applications on these Exchange servers, just the pre-requisites and Exchange. The system directory, Page file, Exchange Install location, database, and logs directory are all on different disks. 5) Event log is not generating any errors or warnings for the affected mailbox server 6) Will be doing so today 7) I'm not so concerned with the amount of RAM since this is expected behavior, I'm more concerned with the CPU periodically spiking to above 30%. This is when we are getting the bulk of the performance problems. Outlook clients are locking up when the CPU spikes, which is mainly the store.exe process eating up all that CPU.
June 16th, 2009 4:31pm
I just came across this article: http://support.microsoft.com/kb/925252 While the symptoms do not match to our specific issue, the mention of the index not being available causing the store service to create views is interesting. I know that on this mailbox server and one of our other servers, we need to reset the search indexes because they are corrupt, but it has not yet been done because it seemed to only affect OWA search capabilities. I will schedule the process to be done tonight when the server is not being as heavily used to cut down on any additional performance problems.
Free Windows Admin Tool Kit Click here and download it now
June 16th, 2009 4:36pm
Hi AgentChutney,Do you use a 3Gb switch option in your Boot.ini file?Last month we had a similair problem and our Exchange Server used as much memory as the DB Store size.I have removed the 3Gb Switch and the server performed better, also we found Backup mail send to a db store from one of our sites.This copymail was send via our mail server to a DB store in Italy, and some of these mail where between 1-10 Mb and slowed down the Exchange server.Try the tracking options for mail and look for any mail that should not go over your system.Henny,
June 16th, 2009 9:01pm
Hows the issue after rebuilding the Full-Text Index Catalog
Free Windows Admin Tool Kit Click here and download it now
June 22nd, 2009 12:40pm
Rebuilding the catalog didn't seem to help, Outlook clients are still slowing down intermittently and resources are still spiking like they did before.
June 23rd, 2009 10:00pm
Please use Process Explorer to check if theres any other application which also use the Store.exe
Free Windows Admin Tool Kit Click here and download it now
June 24th, 2009 12:00pm