Hello,

We recently moved mailboxes (180 or so) from Exchange 2007 to Exchange 2013.  We are noticing that Outlook navigation seems to be very slow compared to the last server.  Changing folder views, switching from Inbox to Calendar, Deleting items, etc.  Also we used to be able to send 40MB HTML emails very quickly.  This now takes about 5 min just to open a reply. (These are tripwire reports.)  Most users are using Outlook in Online mode on the new server and were using Online mode on the old server

Environment: Exchange 2013 SP1 on Server 2012r2 Standard (6 cores 24 gigs of RAM, VMWare VM.)  Old server is Exchange 2007 8.3.348.2 on Server 2008x64 Standard SP2 (8 cores 12 gigs of RAM. Physical.)

I have a ticket open with MS but didn't really get anywhere after 9 hours yesterday.

Thanks.

The first thing that popped into my head was you changed from physical to a VM. Are you sure your VM Cluster is not over provisioned? I see you have 6 Cores assigned to the VM do you know about CPU ready time? Also RAM over provisioning can cause issues as similar to CPU ready. Perhaps its also disk/SAN(if you have one) latency as well.

RAM Bolloning:  http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1003470

CPU ready: http://www.sqlskills.com/blogs/jonathan/cpu-ready-time-in-vmware-and-how-to-interpret-its-real-meaning/

Hope this helps!

• Edited by Mark Morant Friday, June 20, 2014 3:08 PM

The first thing that popped into my head was you changed from physical to a VM. Are you sure your VM Cluster is not over provisioned? I see you have 6 Cores assigned to the VM do you know about CPU ready time? Also RAM over provisioning can cause issues as similar to CPU ready. Perhaps its also disk/SAN(if you have one) latency as well.

RAM Bolloning:  http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1003470

CPU ready: http://www.sqlskills.com/blogs/jonathan/cpu-ready-time-in-vmware-and-how-to-interpret-its-real-meaning/

Hope this helps!

• Edited by Mark Morant Friday, June 20, 2014 3:08 PM

1. Ensure you have sized the guest correctly:

http://blogs.technet.com/b/exchange/archive/2013/05/14/released-exchange-2013-server-role-requirements-calculator.aspx

2. Verify that the BIOS on the vmhost is set to high performance

http://blogs.vmware.com/vsphere/2014/01/bios-power-policies-affect-performance.html

As Andy said- size your Exchange using Role Req. Calc.

On top of that i will make sure that you have not enabled the hyperthreading , if so then you need to disabled.

and also

guest virtual machine share a fixed number of logical processors in the physical system. Exchange supports a virtual processor-to-logical processor ratio no greater than 2:1, although we recommend a ratio of 1:1.

http://technet.microsoft.com/en-us/library/jj619301%28v=exchg.150%29.aspx#BKMK_Prereq

As Andy said- size your Exchange using Role Req. Calc.

On top of that i will make sure that you have not enabled the hyperthreading , if so then you need to disabled.

and also

guest virtual machine share a fixed number of logical processors in the physical system. Exchange supports a virtual processor-to-logical processor ratio no greater than 2:1, although we recommend a ratio of 1:1.

http://technet.microsoft.com/en-us/library/jj619301%28v=exchg.150%29.aspx#BKMK_Prereq

Still no luck.  And I've been on the phone with MS each day.

The VM has been scaled down to 4 CPU's and 18 gigs of RAM.  It is currently only using 13 of the 18 gigs.  I also changed the CPU's to 4 sockets instead of 2 sockets with 2 cores.  We also set the memory reservation to 18 gigs.

The vCPU allocation is much higher than 2:1, but VMWare handles CPU loads different than Hyper-V.  Currently it is at less than half capacity.  It's using 7,602 MHz out of 8X2.599 GHz

how many mailboxes and what is the avg size of the mailbox?

do you see in the task manager which application is consuming the resources?

Still no luck.  And I've been on the phone with MS each day.

The VM has been scaled down to 4 CPU's and 18 gigs of RAM.  It is currently only using 13 of the 18 gigs.  I also changed the CPU's to 4 sockets instead of 2 sockets with 2 cores.  We also set the memory reservation to 18 gigs.

The vCPU allocation is much higher than 2:1, but VMWare handles CPU loads different than Hyper-V.  Currently it is at less than half capacity.  It's using 7,602 MHz out of 8X2.599 GHz

Microsoft.Exchange.Store.Worker is consuming the most RAM at 1.2 and 1.4 gigs (there are 2 of these processes.)

Mailbox count is 210 and the average size is probably 350 MB.

What is the reason for the vCPU ratio if systems like VMware do not segregate CPU workload?

Its about what is supported and recommended:

http://technet.microsoft.com/en-us/library/jj619301(v=exchg.150).aspx

http://www.vmware.com/files/pdf/Exchange_2013_on_VMware_Best_Practices_Guide.pdf

I would also ensure the host power options are set to "High Performance"

I think disk usage is causing performance issues. Did you check Disk Queue length and Average Disk space usage on Exchange 2007 and Exchange 2013?

What is the storage that you use on VMware? Try with sharing file on Exchange 2007 and Exchange 2013, and see how much times it takes to copy to client machine.

I hope that it leads investigation to right direction.

We are using a Compellent SAN.  We have 1 datastore that houses the C and D drive of our Exchange 2013 server.  These are the only 2 volumes on that data store and it is in tier 1 storage.

Disk queue length 2013: C=2.3 and lower / D=.1 and lower

Disk queue length 2007: C and D are barely above 0

Would setting a CPU reservation equate to correcting the ratio.  Might be easier than flushing the host of most VM's

We are using a Compellent SAN.  We have 1 datastore that houses the C and D drive of our Exchange 2013 server.  These are the only 2 volumes on that data store and it is in tier 1 storage.

Disk queue length 2013: C=2.3 and lower / D=.1 and lower

Disk queue length 2007: C and D are barely above 0

Would setting a CPU reservation equate to correcting the ratio.  Might be easier than flushing the host of most VM's

I Don't think reservations are recommended. I would read that white paper or contact VMware support for the recommended settings.

You may need to run ESXTOP to see where the performance bottleneck is.

It is clear that issue is most likely with environment outside Exchange. It could be either SAN or VMware.

Thanks for all the replies.

I have a few things I can try at this point.  I just need to get the ok to do so and schedule them.

1. Clear the VMhost of other VM's to reduce the vCPU ratio

2. Try setting the power management to High performance

3. Disable offload features on the VMXNet3 NIC. My co-worker found this one.  I also noticed when moving the mailboxes it took about 9 hours for 60 gigs.  And both servers have gig connections between them.  Last night when trying to move everyone back the old server the send/receive stats on the new server (2012r2) were about 3.2 MBs

http://virtualrealization.blogspot.com/2013/11/vmxnet3-and-windows-operating-system.html

The disks in VMware are thin provisioned. 

2013 Server: The SAN volume only the C and D drive for the 2013 server on it.  Both are separate VMDK files

2007 Server: We use a passthough disk from the SAN.  Meaning we map a drive right to the SAN from the BIOS on the 2007 server. No VHD or VMDK files here.

We are experiencing the similar issue with Exchange 2013 SP1 on VMware ESXi4 and Outlook 2010.

OWA runs just fine, but Outlook in online mode is unacceptably slow.

There is no processor, memory or storage bottleneck. Servers were sized for a greater load and the configuration was checked with Jetstress tool.

The disks in VMware are thin provisioned. 

2013 Server: The SAN volume only the C and D drive for the 2013 server on it.  Both are separate VMDK files

2007 Server: We use a passthough disk from the SAN.  Meaning we map a drive right to the SAN from the BIOS on the 2007 server. No VHD or VMDK files here.

BriGi86

Could you check the performance when you are running Outlook in safe mode?

outlook.exe /safe

We have many similarities in our environment and yours (VMWare/Compellent/Echange 2010 Online mode/etc.) and are seeing the exact same issues with performance.  The biggest difference that I see is that our Exchange 2010 environment was virtual and not physical servers.  Outlook 2010 and Exchange 2010 in online mode extremely responsive.  Now that we are running Exchange 2013 the experience is much worse.  OWA is very responsive only Outlook has the issue.  We can't run in cached mode because we have a VDI environment.  We've had a case opened with Microsoft for two months plus and are trying to have it escalated to a US based Exchange team.  Would love to share notes with you and find out if you'd be ok with sharing your case number that you had opened. 

Hi everyone,

Here we have exactly the same thing. We did the same migration from Exchange 2k7 to Exchange 2013. We run on a VMware 5.5 infrastructure. We have a lot of users running on a VM, but I can also tell you that I'm experiencing the problem even on a physical PC. So the problem about the client side infrastructure is maybe not that much related.

 I also found an old thread talking about a known issue with the grouping option in Outlook and online mode, but this is supposed to be only for Exchange and Outlook 2007, so i'm not able to prove this problem is exactly and only related to this issue. however, I tried to disable the grouping option for some folder in my Outlook and I saw an improvement immediately. Here is the thread talking about this issue and the knowledge base associated from Microsoft:

http: //blogs.technet.com/b/mikelag/archive/2008/08/31/slow-outlook-online-mode-performance.aspx

http: //support.microsoft.com/kb/948828

Sorry, I cannot post the real link since my account is not verified yet... simply remove the space.

If anyone finally found an answer please post-it, thanks.

We are having the same issue on two properly sized physical Exchange 2013 servers.  We opened a case with Microsoft over six months ago and are still waiting for resolution.

Hi everyone,

After some search on our environment we found that it seem to come from the connexion type (still not sure at all but...). Exchange 2007 was supporting Tcp/IP connexions with Outlook, but now with Exchange 2013, this isn't possible if i'm not wrong. I did a short test since I still have access to my 2k7 Exchange server. I forced my Outlook connected to our old Exchange server (2k7) to use the HTTP connexion type first to try to reproduce similar condition that we have on our 2013. As we expected we had pretty similar symptoms of slowness in Outlook. Then I turned off the setting to use http first and retry with the same Outlook and I retreived the fast and sweet performance we were having before our migration. I'm not an Exchange expert but this sound for me a good explication (theory). Our next step will be to update our Exchange server with Cumulative update 5 for our Exchange 2013 server. We hope this will help us. If anyone can confirm this theory, it may help Microsoft or other people to find a solution.

Hi there,

Same here, I've got a multisite architecture and all the users who have been moved from 2007 to 2013 are experiencing the same slowness in online mode. 

In owa the problem does not exists.

By proxying the access trough 2013 cas to mailbox on 2007 the issue does not exists.

This behavior is happening just in coexistence, I did some new installation and the online mode is fast.

Any idea?

Any news?

Does anybody tried with the CU6?

CU6 does not resolve this.
Take a look at this post:

http://social.technet.microsoft.com/Forums/office/en-US/0d45b1b0-3047-4666-ad04-217e98ed8823/slow-online-mode-browsing?forum=exchangesvrclients

• Proposed as answer by MasterKic Tuesday, September 16, 2014 5:06 PM
• Unproposed as answer by MasterKic Tuesday, September 16, 2014 5:07 PM

CU6 does not resolve this.
Take a look at this post:

http://social.technet.microsoft.com/Forums/office/en-US/0d45b1b0-3047-4666-ad04-217e98ed8823/slow-online-mode-browsing?forum=exchangesvrclients

• Proposed as answer by MasterKic Tuesday, September 16, 2014 5:06 PM
• Unproposed as answer by MasterKic Tuesday, September 16, 2014 5:07 PM

OK We found something very very interesting about this. We found a solution that worked for us. Have a look at this other thread:

http:// community.spiceworks.com/topic/571571-outlook-slow-after-migrating-to-exchange-2013

Since these changes can affect the entire Workstation, I suggest you try it and read the entire post before doing it.

TcpAckFrequency:

http:// support.microsoft.com/kb/328890/fr

I'm sorry the link are not valid since I'm not able to post it correctly right now but, I hope this will help many other people. Thx.

• Marked as answer by BriGy86 Wednesday, February 18, 2015 5:07 PM

OK We found something very very interesting about this. We found a solution that worked for us. Have a look at this other thread:

http:// community.spiceworks.com/topic/571571-outlook-slow-after-migrating-to-exchange-2013

Since these changes can affect the entire Workstation, I suggest you try it and read the entire post before doing it.

TcpAckFrequency:

http:// support.microsoft.com/kb/328890/fr

I'm sorry the link are not valid since I'm not able to post it correctly right now but, I hope this will help many other people. Thx.

• Marked as answer by BriGy86 Wednesday, February 18, 2015 5:07 PM

OK We found something very very interesting about this. We found a solution that worked for us. Have a look at this other thread:

http:// community.spiceworks.com/topic/571571-outlook-slow-after-migrating-to-exchange-2013

Since these changes can affect the entire Workstation, I suggest you try it and read the entire post before doing it.

TcpAckFrequency:

http:// support.microsoft.com/kb/328890/fr

I'm sorry the link are not valid since I'm not able to post it correctly right now but, I hope this will help many other people. Thx.

We are having the EXACT same issue. Exchange 2013 single server environment running on Server 2012 R2, updated to CU6. Running on VMWare 5.5 (latest updates) with VMXNET3 drivers. Compellent SAN. Also, all users migrated from Exchange 2007. Clients are running in ONLINE MODE due to VDI. OWA works flawlessly. All clients experiencing VERY HIGH RPC latency (4000+ ms when running RPCPING).

Ticket opened with Microsoft over 6 months ago and has been escalated MANY times. Network team points fingers at Exchange, Exchange team points fingers at Outlook team, and Outlook team points fingers back stating it's not their issue.

FYI, Outlook runs GREAT when installed ON the Exchange server (less than 15ms RPC latency). I've even tried installing Outlook on another VM on the same VMWare host and in the same subnet so that all networking stays on the host.

Subscribing to this thread...

This is from the SpiceWorks link posted above. I have not tried it though.

Looks like there is a registry key that can be done on each client or a fix for VMware. https://communities.vmware.com/message/2439033I

I am going to try rolling the vmwaretools back to 5.1 RU1 and move a problem user to the new server.  If this works I will be marking MasterKic's reply as the answer as well as this one.

• Marked as answer by BriGy86 Wednesday, February 18, 2015 5:06 PM

This is from the SpiceWorks link posted above. I have not tried it though.

Looks like there is a registry key that can be done on each client or a fix for VMware. https://communities.vmware.com/message/2439033I

I am going to try rolling the vmwaretools back to 5.1 RU1 and move a problem user to the new server.  If this works I will be marking MasterKic's reply as the answer as well as this one.

• Marked as answer by BriGy86 Wednesday, February 18, 2015 5:06 PM

This is from the SpiceWorks link posted above. I have not tried it though.

Looks like there is a registry key that can be done on each client or a fix for VMware. https://communities.vmware.com/message/2439033I

I am going to try rolling the vmwaretools back to 5.1 RU1 and move a problem user to the new server.  If this works I will be marking MasterKic's reply as the answer as well as this one.

BriGy86: The link to vmware communities you posted is not working. Can you please provide another link?

EDIT: Found the article in question! https://communities.vmware.com/message/2439033?tstart=0

Thanks!

• Edited by JBHorne Friday, December 19, 2014 8:11 PM Update link

BriGy86: The link to vmware communities you posted is not working. Can you please provide another link?

EDIT: Found the article in question! https://communities.vmware.com/message/2439033?tstart=0

Thanks!

• Edited by JBHorne Friday, December 19, 2014 8:11 PM Update link

Sorry for the late response.  I have since gotten the tools rolled back

TAKE A SNAPSHOT!  removing the tools resulted in the old exchange server inoperable for about a day.  Then all of a sudden it started responding (This is after several reboots.)

After getting them rolled back I ran into some other mail flow issues while testing so I have not completely resolved this issue yet.

After moving some users to the new machine we are not seeing any issues. (The one with the tools version rolled back.)

While building the new server I made sure to install the rolled back version of VMware tools and did not get any errors during the install or SP1 install.  The previous server I built had horrendous issues with the install.

I'm having similar problems with online mode using HTTPS (preparing for our Exchange 2013 migration).   Our Outlook Anywhere and RPC Client Access array host names are the same, so, the suggestion from Brian Day at Microsoft is to put all your clients into HTTPS mode before cutting over DNS to Ex2013.

If we attempt to have our clients (Outlook 2010 and 2013) connect using HTTPS (Outlook Anywhere) versus MAPI in Exchange 2010, the experience is poor at times.   Exchange 2010 is all physical.    We require online mode because of a VDI environment.    

I've been working with Microsoft on this issue for two to three months without any progress.  We've ruled a lot out, but, haven't found the issue.      We've tried the TCPAck setting without any change.

If anyone finds the smoking gun, please post on this.