Hey, We've been chasing down an issue with a customer for the last few weeks without much luck. We get reports every week or two that clients can no longer connect with Outlook or their iPhones. After logging in to the server we find that Microsoft.Exchange.RpcClientAccess.Service.exe is running with 100% CPU utilization. Restarting the Exchange RPC service restores client connectivity. When the RPC service has crashed, I can telnet to tcp/6002 and tcp/6004 and get "ncacn_ip_tcp" as the response, however when I telnet to tcp/6001 telnet connects but there is no output. We're running Exchange 14.1 (Build 218.15) on Server 2008 R2 (6.1 Build 7200). Below is a copy of the last ~20 events from the RPC Client Access Log before the service is restarted. 2011-04-30T09:41:24.197Z,38,0,/o=First Organization/ou=first administrative group/cn=Recipients/cn=[Redacted],,OUTLOOK.EXE,12.0.6554.5000,Cached,192.168.4.100,192.168.0.20,ncacn_ip_tcp,,Connect,0,00:00.0,"SID=S-1-5-21-855859734-562581465-989134940-1248, Flags=None" 2011-04-30T09:41:24.244Z,38,1,/o=First Organization/ou=first administrative group/cn=Recipients/cn=[Redacted],,OUTLOOK.EXE,12.0.6554.5000,Cached,,,ncacn_ip_tcp,,OwnerLogon,0,0:00:00,"Logon: Owner, /o=First Organization/ou=first administrative group/cn=Recipients/cn=[Redacted] in database Mailbox Database last mounted on [red]EX01.[red].local at 28/04/2011 10:05:20 AM, currently Mounted; LogonId: 0" 2011-04-30T09:41:24.291Z,39,0,/o=First Organization/ou=first administrative group/cn=Recipients/cn=[Redacted],,OUTLOOK.EXE,12.0.6554.5000,Cached,192.168.4.100,192.168.0.20,ncacn_ip_tcp,,Connect,0,0:00:00,"SID=S-1-5-21-855859734-562581465-989134940-1248, Flags=None" 2011-04-30T09:41:24.291Z,40,0,/o=First Organization/ou=first administrative group/cn=Recipients/cn=[Redacted],,OUTLOOK.EXE,12.0.6554.5000,Cached,192.168.4.100,192.168.0.20,ncacn_ip_tcp,,Connect,0,0:00:00,"SID=S-1-5-21-855859734-562581465-989134940-1248, Flags=None" 2011-04-30T09:41:24.338Z,39,1,/o=First Organization/ou=first administrative group/cn=Recipients/cn=[Redacted],,OUTLOOK.EXE,12.0.6554.5000,Cached,,,ncacn_ip_tcp,,PublicLogon,0,00:00.0,"Logon: Public, in database 74f7504d-1226-4e87-89ac-8be3b4498d5d last mounted on [red]EX01.[red].local at 28/04/2011 10:05:27 AM, currently Mounted; LogonId: 0" 2011-04-30T09:41:24.353Z,40,1,/o=First Organization/ou=first administrative group/cn=Recipients/cn=[Redacted],,OUTLOOK.EXE,12.0.6554.5000,Cached,,,ncacn_ip_tcp,,OwnerLogon,0,00:00.0,"Logon: Owner, /o=First Organization/ou=first administrative group/cn=Recipients/cn=[Redacted] in database Mailbox Database last mounted on [red]EX01.[red].local at 28/04/2011 10:05:20 AM, currently Mounted; LogonId: 0" 2011-04-30T09:41:24.353Z,40,1,/o=First Organization/ou=first administrative group/cn=Recipients/cn=[Redacted],,OUTLOOK.EXE,12.0.6554.5000,Cached,,,ncacn_ip_tcp,,OwnerLogon,0,00:00.0,"Logon: Owner, /o=First Organization/ou=first administrative group/cn=Recipients/cn=[Redacted] in database Mailbox Database last mounted on [red]EX01.[red].local at 28/04/2011 10:05:20 AM, currently Mounted; LogonId: 1" 2011-04-30T09:41:24.369Z,40,1,/o=First Organization/ou=first administrative group/cn=Recipients/cn=[Redacted],,OUTLOOK.EXE,12.0.6554.5000,Cached,,,ncacn_ip_tcp,,OwnerLogon,0,00:00.0,"Logon: Owner, /o=First Organization/ou=first administrative group/cn=Recipients/cn=[Redacted] in database Mailbox Database last mounted on [red]EX01.[red].local at 28/04/2011 10:05:20 AM, currently Mounted; LogonId: 2" 2011-04-30T09:41:24.369Z,40,2,/o=First Organization/ou=first administrative group/cn=Recipients/cn=[Redacted],,OUTLOOK.EXE,12.0.6554.5000,Cached,,,ncacn_ip_tcp,,OwnerLogon,0,0:00:00,"Logon: Owner, /o=First Organization/ou=first administrative group/cn=Recipients/cn=[Redacted] in database Mailbox Database last mounted on [red]EX01.[red].local at 28/04/2011 10:05:20 AM, currently Mounted; LogonId: 3" 2011-04-30T09:41:24.775Z,40,6,/o=First Organization/ou=first administrative group/cn=Recipients/cn=[Redacted],,OUTLOOK.EXE,12.0.6554.5000,Cached,,,ncacn_ip_tcp,,OwnerLogoff,0,0:00:00,LogonId: 1 2011-04-30T09:41:24.775Z,40,6,/o=First Organization/ou=first administrative group/cn=Recipients/cn=[Redacted],,OUTLOOK.EXE,12.0.6554.5000,Cached,,,ncacn_ip_tcp,,OwnerLogoff,0,0:00:00,LogonId: 2 2011-04-30T09:41:25.119Z,39,4,/o=First Organization/ou=first administrative group/cn=Recipients/cn=[Redacted],,OUTLOOK.EXE,12.0.6554.5000,Cached,,,ncacn_ip_tcp,,PublicLogon,0,00:00.0,"Logon: Public, in database 74f7504d-1226-4e87-89ac-8be3b4498d5d last mounted on [red]EX01.[red].local at 28/04/2011 10:05:27 AM, currently Mounted; LogonId: 1" 2011-04-30T09:41:25.275Z,41,0,/o=First Organization/ou=first administrative group/cn=Recipients/cn=[Redacted],,SearchProtocolHost.exe,12.0.6554.5000,Cached,192.168.4.100,192.168.0.20,ncacn_ip_tcp,,Connect,0,0:00:00,"SID=S-1-5-21-855859734-562581465-989134940-1248, Flags=None" 2011-04-30T09:41:25.322Z,41,1,/o=First Organization/ou=first administrative group/cn=Recipients/cn=[Redacted],,SearchProtocolHost.exe,12.0.6554.5000,Cached,,,ncacn_ip_tcp,,PublicLogon,0,00:00.0,"Logon: Public, in database 74f7504d-1226-4e87-89ac-8be3b4498d5d last mounted on [red]EX01.[red].local at 28/04/2011 10:05:27 AM, currently Mounted; LogonId: 0" 2011-04-30T09:41:25.431Z,42,0,/o=First Organization/ou=first administrative group/cn=Recipients/cn=[Redacted],,SearchProtocolHost.exe,12.0.6554.5000,Cached,192.168.4.100,192.168.0.20,ncacn_ip_tcp,,Connect,0,0:00:00,"SID=S-1-5-21-855859734-562581465-989134940-1248, Flags=None" 2011-04-30T09:41:25.462Z,42,1,/o=First Organization/ou=first administrative group/cn=Recipients/cn=[Redacted],,SearchProtocolHost.exe,12.0.6554.5000,Cached,,,ncacn_ip_tcp,,OwnerLogon,0,0:00:00,"Logon: Owner, /o=First Organization/ou=first administrative group/cn=Recipients/cn=[Redacted] in database Mailbox Database last mounted on [red]EX01.[red].local at 28/04/2011 10:05:20 AM, currently Mounted; LogonId: 0" 2011-04-30T09:41:25.619Z,42,4,/o=First Organization/ou=first administrative group/cn=Recipients/cn=[Redacted],,SearchProtocolHost.exe,12.0.6554.5000,Cached,,,ncacn_ip_tcp,,OwnerLogoff,0,0:00:00,LogonId: 0 2011-04-30T09:41:25.666Z,42,5,/o=First Organization/ou=first administrative group/cn=Recipients/cn=[Redacted],,SearchProtocolHost.exe,12.0.6554.5000,Cached,,,ncacn_ip_tcp,,Disconnect,0,00:00.2, 2011-04-30T09:41:25.681Z,41,5,/o=First Organization/ou=first administrative group/cn=Recipients/cn=[Redacted],,SearchProtocolHost.exe,12.0.6554.5000,Cached,,,ncacn_ip_tcp,,PublicLogoff,0,0:00:00,LogonId: 0 2011-04-30T09:41:25.712Z,41,6,/o=First Organization/ou=first administrative group/cn=Recipients/cn=[Redacted],,SearchProtocolHost.exe,12.0.6554.5000,Cached,,,ncacn_ip_tcp,,Disconnect,0,00:00.4, 2011-04-30T09:41:32.978Z,40,20,/o=First Organization/ou=first administrative group/cn=Recipients/cn=[Redacted],,OUTLOOK.EXE,12.0.6554.5000,Cached,,,ncacn_ip_tcp,,OwnerLogon,0,0:00:00,"Logon: Owner, /o=First Organization/ou=first administrative group/cn=Recipients/cn=[Redacted] in database Mailbox Database last mounted on [red]EX01.[red].local at 28/04/2011 10:05:20 AM, currently Mounted; LogonId: 1" 2011-04-30T09:51:57.865Z,8,224,/o=First Organization/ou=first administrative group/cn=Recipients/cn=[Redacted2],,OUTLOOK.EXE,12.0.6554.5000,Cached,,,ncacn_ip_tcp,,,0,00:00.2,"ImportMessageMove succeeded with warning ec=SyncClientChangeNewer """"" 2011-04-30T10:13:28.936Z,43,0,/o=First Organization/ou=first administrative group/cn=Recipients/cn=[Redacted1],,OUTLOOK.EXE,12.0.6300.5000,Classic,192.168.250.5,192.168.0.20,ncacn_ip_tcp,,Connect,0,00:00.0,"SID=S-1-5-21-855859734-562581465-989134940-1142, Flags=None" 2011-04-30T10:13:28.983Z,43,1,/o=First Organization/ou=first administrative group/cn=Recipients/cn=[Redacted1],,OUTLOOK.EXE,12.0.6300.5000,Classic,,,ncacn_ip_tcp,,OwnerLogon,0,00:00.0,"Logon: Owner, /o=First Organization/ou=first administrative group/cn=Recipients/cn=[Redacted1] in database Mailbox Database last mounted on [red]EX01.[red].local at 28/04/2011 10:05:20 AM, currently Mounted; LogonId: 0" 2011-04-30T10:13:29.342Z,43,11,/o=First Organization/ou=first administrative group/cn=Recipients/cn=[Redacted1],,OUTLOOK.EXE,12.0.6300.5000,Classic,,,ncacn_ip_tcp,,OwnerLogon,0,00:00.0,"Logon: Owner, /o=First Organization/ou=first administrative group/cn=Recipients/cn=[Redacted1] in database Mailbox Database last mounted on [red]EX01.[red].local at 28/04/2011 10:05:20 AM, currently Mounted; LogonId: 1" 2011-04-30T10:13:31.108Z,43,55,/o=First Organization/ou=first administrative group/cn=Recipients/cn=[Redacted1],,OUTLOOK.EXE,12.0.6300.5000,Classic,,,ncacn_ip_tcp,,OwnerLogon,0,00:00.0,"Logon: Owner, /o=First Organization/ou=first administrative group/cn=Recipients/cn=[Redacted1] in database Mailbox Database last mounted on [red]EX01.[red].local at 28/04/2011 10:05:20 AM, currently Mounted; LogonId: 2" 2011-04-30T10:13:31.717Z,43,80,/o=First Organization/ou=first administrative group/cn=Recipients/cn=[Redacted1],,OUTLOOK.EXE,12.0.6300.5000,Classic,,,ncacn_ip_tcp,,PublicLogon,0,00:00.0,"Logon: Public, in database 74f7504d-1226-4e87-89ac-8be3b4498d5d last mounted on [red]EX01.[red].local at 28/04/2011 10:05:27 AM, currently Mounted; LogonId: 3" 2011-04-30T10:17:17.315Z,43,209,/o=First Organization/ou=first administrative group/cn=Recipients/cn=[Redacted1],,OUTLOOK.EXE,12.0.6300.5000,Classic,,,ncacn_ip_tcp,,OwnerLogon,0,00:00.0,"Logon: Owner, /o=First Organization/ou=first administrative group/cn=Recipients/cn=[Redacted1] in database Mailbox Database last mounted on [red]EX01.[red].local at 28/04/2011 10:05:20 AM, currently Mounted; LogonId: 4" 2011-04-30T10:17:17.331Z,43,210,/o=First Organization/ou=first administrative group/cn=Recipients/cn=[Redacted1],,OUTLOOK.EXE,12.0.6300.5000,Classic,,,ncacn_ip_tcp,,OwnerLogon,0,0:00:00,"Logon: Owner, /o=First Organization/ou=first administrative group/cn=Recipients/cn=[Redacted1] in database Mailbox Database last mounted on [red]EX01.[red].local at 28/04/2011 10:05:20 AM, currently Mounted; LogonId: 5" 2011-04-30T10:17:23.050Z,43,228,/o=First Organization/ou=first administrative group/cn=Recipients/cn=[Redacted1],,OUTLOOK.EXE,12.0.6300.5000,Classic,,,ncacn_ip_tcp,,OwnerLogoff,0,0:00:00,LogonId: 0 2011-04-30T10:17:23.050Z,43,228,/o=First Organization/ou=first administrative group/cn=Recipients/cn=[Redacted1],,OUTLOOK.EXE,12.0.6300.5000,Classic,,,ncacn_ip_tcp,,OwnerLogoff,0,0:00:00,LogonId: 1 2011-04-30T10:17:23.050Z,43,228,/o=First Organization/ou=first administrative group/cn=Recipients/cn=[Redacted1],,OUTLOOK.EXE,12.0.6300.5000,Classic,,,ncacn_ip_tcp,,OwnerLogoff,0,0:00:00,LogonId: 2 2011-04-30T10:17:23.050Z,43,228,/o=First Organization/ou=first administrative group/cn=Recipients/cn=[Redacted1],,OUTLOOK.EXE,12.0.6300.5000,Classic,,,ncacn_ip_tcp,,PublicLogoff,0,0:00:00,LogonId: 3 2011-04-30T10:17:23.050Z,43,228,/o=First Organization/ou=first administrative group/cn=Recipients/cn=[Redacted1],,OUTLOOK.EXE,12.0.6300.5000,Classic,,,ncacn_ip_tcp,,OwnerLogoff,0,0:00:00,LogonId: 4 2011-04-30T10:17:23.050Z,43,228,/o=First Organization/ou=first administrative group/cn=Recipients/cn=[Redacted1],,OUTLOOK.EXE,12.0.6300.5000,Classic,,,ncacn_ip_tcp,,OwnerLogoff,0,0:00:00,LogonId: 5 2011-04-30T10:17:23.050Z,43,228,/o=First Organization/ou=first administrative group/cn=Recipients/cn=[Redacted1],,OUTLOOK.EXE,12.0.6300.5000,Classic,,,ncacn_ip_tcp,,Disconnect,0,03:54.2, Any assistance in where to go from here from a debugging perspective would be appreciated. Thanks Blake

Hi Blake It's not clear what Update Rollup you have for SP1. As a starting point I would recommend upgrading to RU3v3 as it has a fix for a known memory leak in the RPC client access service. http://support.microsoft.com/kb/2458543 If you are running any Exchange-aware AV it might be useful to temporarily unhook that from the Exchange services just to remove it as a possible factor.Alexei

Hey, We were at RU2. Will apply the update tonight and let you know how we go. There's no antivirus currently running on the Exchange server. Thanks Blake

If still you have problem then download Exmon and see if any specic prfile is causing the issue for RPC. Microsoft Exchange Server User Monitor : http://www.microsoft.com/downloads/en/details.aspx?FamilyId=9A49C22E-E0C7-4B7C-ACEF-729D48AF7BC9&displaylang=en Anil

Memory leak in "\Process(Microsoft.Exchange.RpcClientAccess.Service)\Private Bytes" http://support.microsoft.com/kb/2458543 http://support.microsoft.com/kb/2529939 Apply RU3 for Exchange 2010 SP1 Thanks, Simon