100% CPU Utilization on All Exchange Servers

We have 6 Exchange 2013 CU7 servers (vm's) behind a loadbalancer.There are 7500 mailboxes but the last login shows an average of about 4000 mailboxes that login's daily. Most are 500mb quota accounts. 99% of our users use Outlook 2010.This system has been stable and rock solid. The issue was first noticed on March 11th. CU7 was the last update applied and that was Feb 20th. We do have a handful of Iphone that have updated to 8.2.

We are seeing CPU spikes on all 6 Exchange 2013 servers to 100%. Each day around 10-11:00am. During this time Outlook clients move to "Trying to Connect" to connected. OWA and Iphones appear NOT to be affected. We are seeing an increase of users with mail stuck in their Outbox also. Many users the inbox does not update for hours and they come in the next day to see countless messages not read from the previous day.

The current workaround is running IISReset on one server and then all the servers stabilize to normal levels. Looking at the logs nothing jumps out as the root cause. Somedays the IISReset command is run more than once to fix the 100% spikes. The Exchange servers never spike out of working hours or weekends and each morning they are running at normal levels.

I am guessing there could be RPC issues from googling but have nothing to back that up. Using the performance monitor seeing averages of

  • RPC Clients request at 3,200,000
  • RPC Operations/Sec at 3500

I am not sure if this normal or crazy high.

I have read days of google searches but most are Exchange 2010 not 2013. I don't see any utilities in Exchange 2013 to see if a user or users could cause this behavior and just at loss on what the root cause is.

I don't know what else to include to help, Thanks for any help!

This post is very similar to our situation:

https://social.technet.microsoft.com/Forums/en-US/dc902fff-1208-4726-896f-9f7cfbea7aaa/exchange-2013-cu7-frequent-outlook-2013-disconnections-and-delays?forum=exchangesvrgeneral

April 1st, 2015 11:37am

Hello

tip: try analize top request with Log Parser Studio if you luck see which client  flooding exchange.

Free Windows Admin Tool Kit Click here and download it now
April 1st, 2015 1:50pm

Sounds similar to an issue I saw last year where there were excessive connections from mobile users.  It brought their entire org (multiple servers) to a crawl as well.  With assistance from MS support, they were able to identify the cause and we subsequently disabled the users activesync ability.  I'm trying to remember the utility we used; it was not a native Exchange tool.  I'll post back if I remember it.
April 1st, 2015 2:25pm

We have identified the issue as a problem with the F5 BigIP load balancer; by removing the load balancer from servicing internal requests (using round-robin DNS instead) the issue is minimized.

We are in the process of seeking support from F5.

Free Windows Admin Tool Kit Click here and download it now
April 6th, 2015 2:13pm

Good luck.  Thanks for replying that you've identified the issue.
April 6th, 2015 2:23pm

This topic is archived. No further replies will be accepted.

Other recent topics Other recent topics