Which Cumulative Updates are these Exchange 2013 servers at?
I would concur with Li's recommendation to upgrade the .NET to 4.5.2. While that fixes issues with Garbage Collection and performance issues and alleviate some of your issues, I still think you may have a different underlying cause for requests queuing up.
Also check the TcpKeepAlive and TCP Time Outs on devices between the clients and Exchange CAS server. The default value on Windows is 2 hours. It is always recommended that Exchange is the one with the least value (typically 15 mins) and then devices in front of Exchange going all way to the Client is higher than the value set on Exchange. This way Exchange (Server) is the one responsible for closing or leaving the session open.
Also to answer your question if Recycling the App Pool will be advisable: You can certainly do that which will provide temporary relief but if you don't address the root cause, the issue might still return after a while.
- Edited by Sainath Vijayaraghavan Tuesday, August 11, 2015 4:49 AM