I am experiencing high CPU usage (98-99%) for MonitoringHost.exe on several servers with managed agents in SCOM R2. Apart from restarting the service and clearing the cache, is there a long term fix? I thought most high CPU problems were fixed in R2.....

Sometimes it is a antivirus trying to scan every script ops mgr is running, and it is running scripts all the time. So if you have a antivirus try disable it for a while. If that dont help, try uninstall management packs and see if the load goes away.Anders Bengtsson | Microsoft MVP - Operations Manager | http://www.contoso.se

are those servers passive nodes of cluster?Marius Sutara My MSDN blog This posting is provided "AS IS" with no warranties, and confers no rights. Use of attachments are subject to the terms specified at http://www.microsoft.com/info/cpyright.htm

Hi. Since this is a pre-prod isolated environment, at the moment there is no anti-virus running. There are also no cluster nodes affected - 2 servers in particular I am seeing- 1 is a Windows 2003 SP2 with SQL 2005 and the other is Windows 2003 SP2 with OCS 2007 R2.

Hi,We had the same problem on one of our servers and fixed this by upgrading to Windows Scripting Host 5.7 on the affected server.Thanksmark

Unfortuately that's not an option as there still isn't a 5.7 version for x64........

5.7 version x64 exists - I had the same issue on Exchange 2007 server with monitoring SCOM 2007 SP1- I opened the case with MS and they sent me right version.

I thought the same till Kevin blogged about script failure issues the other day and noted that there's a knowledge base article for an updated 5.7:WSH binaries are overwritten by Windows File Protection after you install Windows Script 5.7 on a computer that is running Windows Server 2003 or Windows XPAnd sure enough, there's a x64 version of this hotfix available, but not of the original 5.7 download.

Hello, we have upgraded to R2 two days ago. We are having the same issue on windows 2003/2008 clusters. We have applied the fix above about WSH but the issue still exists on 2003. We have removed cluster MP and SQL MP for a while, but the cpu usage was still the same.To make sure, we have also cleared agents' cache several times.. We have also applied some hotfixes for OS, nothing has changed regarding the CPU utilization... (Also, on clusters the agent was restarting itself, per http://blogs.technet.com/kevinholman/archive/2009/06/22/health-service-and-monitoringhost-thresholds-in-r2-how-this-has-changed-and-what-you-should-know.aspx, now the agents are okay, thanks to Kevin.)windows 2003 (sql 05) cluster(x64) Active/Active => high CPU on first node (proxy is node 2)windows 2008 (sql 08) cluster(x64) Active/Active ==> high CPU onFirst node (proxy is node 1)windows 2003 (file) cluster Active/Active ==> high CPU on first node (proxy is node1)windows 2003 (exchange 03) Act/Act/Pass ==> high CPU on first node (proxy is node 3)windows 2003 (exchange 03) Act/Act/Act/Act/Pass ==> high CPU on node 1/3/5 (proxy is node 2)Is this a known issue??? Will there be any hotfixes? I hate to dedicate one core/CPU to scom agent. Upon failing over the cluster (and reboot the server), sometimes the high CPU utilizations jumps to other nodes. Also, I cannot change the proxy agents for the discovered cluster instances using "agentless managed" view. All agents on these clustersare enabled for proxy. Any thoughts?

Hi.Kevin Holman has also blogged about this issue: http://blogs.technet.com/kevinholman/archive/2009/07/20/do-you-randomly-see-a-monitoringhost-exe-process-consuming-lots-of-cpu.aspxBest regards, Marnix Wolf (Thoughts on OpsMgr)

Hi, we already applied the patch http://support.microsoft.com/kb/968967mentioned in Kevin Holman's blog to windows 2008/SQL 2008cluster and rebooted the server two days ago. Health service's CPU utilization is still the same (On average, it is using 6% of CPUof 16 core system, it spikes upto 15%...)

There have been problems reported on SQL Server clusters ... see below for a copy and paste from an old newsgroup. There is a new SQL MP out which you might want to test. "Serge Berat" <srgbrt@hotmail.com> wrote in message news:<e6vW363lJHA.3380@TK2MSFTNGP04.phx.gbl>... > Hi Cory, > > Must say this scares me seriously ... We have a couple of SQL > Clusters, of which a 4-node is the biggest one having 12 instances > hosting almost 400 DB's. > So if I get it right, even with the new MP to come we're still in "trouble". > > Cheers, > Serge > > "Cory Delamarter [MSFT]" <cory.delamarter@online.microsoft.com> wrote > in message news:65363EB0-AF30-4E4B-8B55-4F221DA83C70@microsoft.com... > > The WMI performance issue that you're seeing is a known issue that > > was introduced with the last release of the SQL MP and that we are > > working hard to release a fix for. Out of curiosity, how many > > clustered instances of SQL do you install (average and/or max) on your SQL clusters? > > > > With the fixes that we have in place right now we've been able to > > optimize the performance of our discovery for stand-alone SQL > > servers with many instances, and for clusters with a single > > clustered instance. With multiple clustered instances of SQL > > installed on the same cluster we can start seeing resources > > contention due to the fact that every clustered instance is treated > > as a distinct agent in the eyes of OpsMgr and we therefore run our > > discoveries against each instance simultaneously and the scripts start contending with each other, causing CPU and memory spikes. > > These problems will arise most noticeably when multiple instances > > owned by the same node in the cluster at one point. In these > > configurations we may not be able to do much > > > > > > "Ian Blyth" <IanBlyth@discussions.microsoft.com> wrote in message > > news:188CFA5D-09C8-4449-BF15-83650F761D6E@microsoft.com... > >> Hi > >> > >> my mistake. I put 6.0.6505.0 into the pre production environment as > >> we were seeing the same problem there. In production the cluster MP > >> is still 6.0.6277.1. > >> > >> The SQL MP is 6.0.6460.0 in both. > >> > >> The biggest improvement I saw was by restricting what discoveries > >> are run on the cluster. With many instances the scripts run against > >> each instance spawning many cscript processes. As it is not running > >> SRS, AS etc I stopped those running which helped. The main one > >> running now which takes forever and runs for each SQL instance is > >> DiscoverSQL2005DBEngineDiscovery.vbs. The SQL DBA is complaining of > >> lots of things interogating the databases and the server is now > >> running at 15% with no users on it. It is 8 core. The spikes to > >> 100% have been calmed but it is still not right. In preproduction > >> it is running at 40% but it is in a VM which restricts it. > >> > >> It is wmiprvse.exe which seems to consume the processor as the > >> scripts run. > >> And looking at it with Process Explorer it seems to be calling the > >> 32 version rather than the 64 bit version. > >> > >> Ian > >> > >> "Steve Burkett" wrote: > >> > >>> Hi ya Ian, > >>> > >>> When you say the latest Cluster MP, was that v6.0.6505.0 that they > >>> released on the 12th? > >>> > >>> > >>> "Ian Blyth" <IanBlyth@discussions.microsoft.com> wrote in message > >>> news:A4BEC883-85AC-48C7-8F68-20B1E4FC1F99@microsoft.com... > >>> > Hi, > >>> > > >>> > what was the WMI hotfix? > >>> > > >>> > I am seeing a problem monitoring a SQL 2005 x64 cluster. The problem > >>> > is > >>> > spikes up to 100% CPU with wmi being the main culprit. There are lots > >>> > of > >>> > cscript.exes running and eventually time out. This does not happen on > >>> > the > >>> > passive nodes. Just the active node. > >>> > > >>> > I have created a group and restricted the discovery of anything > >>> > non-essential that is targeted at Windows computer as many discovery > >>> > scripts > >>> > were running. When the SQL 2006 scripts run they spawn multiple > >>> > versions. > >>> > I > >>> > presume for each SQL instance. > >>> > > >>> > I am using the latest versions of the SQL, Cluster, Windows MPs. > >>> > > >>> > Ian > >>> > > >>> > "Harold Buckner" wrote: > >>> > > >>> >> We have been having a nightmare in monitoring are SQL instances on a > >>> >> 64bit > >>> >> cluster. Whenever the SQL Discovery occures it would error out and > >>> >> drop > >>> >> out > >>> >> monitored SQL instances. We finially got that resolved with a WMI > >>> >> hotfix > >>> >> and > >>> >> the new SQL management pack. > >>> >> > >>> >> Now, I install a new instance on the cluster and the virtual SQL > >>> >> Server > >>> >> stays in the not monitored state under discovered inventory. > >>> >> > >>> >> Yes I know about the proxy rights and looking for the is virtual node > >>> >> property. It all there and I have 8 other sql instances running on > >>> >> this > >>> >> cluster and they are monitored. > >>> >> > >>> >> Before I throw out OpsMgr all together....Any one have a suggestion > >>> >> on > >>> >> how > >>> >> to fix this issue now. > >>> >> > >>> >> I have tried un-installing the agents, and re-installing. Next I > >>> >> thought > >>> >> about removeing the magement packs and puting the older version back. > >>> >> > >>> >> Currently on the newest SQL mangement pack 6.0.6460.0 > >>> >> > >>> >> Thanks > >>> >> > >>> >> > >>> >> > > View OpsMgr tips and tricks at http://systemcentersolutions.wordpress.com/

Thank you. We applied the latest SQL MP 6.0.6648 when we upgraded. Also after seeing issues, we removed the MP and cleared the agent's cache. (We also removed cluster MP (6568)), and the servers were still having the same issue. BTW, agents on some of these systems have ~300MB memory in use for healthservice.exe,andHealthServiceStore.edb is ~2.5GB which seems odd.

Update: Regardinghealthservice.exe high cpu issue, I have created a new two node file server failover cluster on windows 2008 x64(to eliminate any possible issues on our existing productions servers), and pushed R2 agent. One node1, healthservice.exe is using 1-50% of available CPU resources (this is a VM with 2 cpu) constantly. Node 1, CPU time of healthservice.exe is 14:50 as of now (60MB), and node2 is 1:48 (30MB)..Anyone having the same high CPU utilization of healthservice.exe on clusters with R2 agent??? I appreciate any comments... Thank you!

by anychance, is the one node with high CPU the one which is passive? There is a known issue with sometimes passive node is hammered. Workaround is to failover to passive node and wait for configuration reload. That should bring down CPU after some time. (Also having server health service cache flushed will likely cause this CPU issue on passive node again)Marius Sutara My MSDN blog This posting is provided "AS IS" with no warranties, and confers no rights. Use of attachments are subject to the terms specified at http://www.microsoft.com/info/cpyright.htm

We also upgrade to SCOM 2007 R2 10 days ago and we observed the same issue in our environment, MonitoringHost.exe consuming 100% CPU, but it's a soft 100%. If we restart the Sys Ctr Mgmt services, it will drop back down to normal... but only last for so long.We do not have any cluster setup and the CPU issue happens randomly with our servers.. W2k3 server, W2k8 Exchange, W2k8 SQL server. Already sent our MPS report to Microsoft support awaiting answer.

Yes indeed. I have the same problem as you. In my opinion it is a product bug. We have opened a case to Microsoft and we are waiting. I think that the RMS is asking to the wrong proxy agent the state of some aggregate monitor roll-ups (and it is doing it in a loop). Could you start a trace in your RMS doing: starttrace INF.If, after some minutes, you seemany lines like this: [0]2348.2668::07/28/2009-17:24:54.604 [InternalTaskManager] [] [Information] :CInternalTask::TaskStatus{InternalTask_cpp893}Recieved job result: <JobStatus ID="{2B106790-D024-432B-B45F-2D256A8324B6}" BatchID="{DBF9B985-D9E6-4F0C-AE1B-B68023F5EE41}" TaskID="{4BE723CD-BA53-F7FB-6A4A-4A5F062E77EF}" TargetInstanceID="{43D87B22-921E-213D-BA96-6515A4EB4DAD}" JobCategory="0"><Result>Failure</Result><TimeStarted>2009-07-28T17:24:51.6641017+02:00</TimeStarted><TimeEnded>2009-07-28T17:24:51.6641017+02:00</TimeEnded><ErrorCode>-2130771918</ErrorCode><ErrorMessage>Get the state of the monitor failed with error: -1601.Monitor ID: 37B4681A-4A09-67A5-C7F3-6F613EF5E7C3Instance ID: 43D87B22-921E-213D-BA96-6515A4EB4DADManagement Group ID: {F42204D4-D7E0-ED86-EA3A-446703B4A2C3} </ErrorMessage></JobStatus> [0]2348.2668::07/28/2009-17:24:54.604 [ModulesLibrary] [] [Information] :XmlCommonDoInitialize{MOMFasterXmlSupport_cpp2007}FasterXML parser for: <JobStatus ID="{2B106790-D024-432B-B45F-2D256A8324B6}" BatchID="{DBF9B985-D9E6-4F0C-AE1B-B68023F5EE41}" TaskID="{4BE723CD-BA53-F7FB-6A4A-4A5F062E77EF}" TargetInstanceID="{43D87B22-921E-213D-BA96-6515A4EB4DAD}" JobCategory="0"><Result>Failure</Result><TimeStarted>2009-07-28T17:24:51.6641017+02:00</TimeStarted><TimeEnded>2009-07-28T17:24:51.6641017+02:00</TimeEnded><ErrorCode>-2130771918</ErrorCode><ErrorMessage>Get the state of the monitor failed with error: -1601.Monitor ID: 37B4681A-4A09-67A5-C7F3-6F613EF5E7C3Instance ID: 43D87B22-921E-213D-BA96-6515A4EB4DADManagement Group ID: {F42204D4-D7E0-ED86-EA3A-446703B4A2C3} </ErrorMessage></JobStatus> and like this: [0]2348.2668::07/28/2009-17:24:54.604 [HealthManager] [] [Information] :HealthManagerInternalTask::TaskCompletionCallback{HealthManagerInternalTask_cpp442}Task will respawn: contributing monitor was not found yet! We are suffering the same problem.

After applying this hotfix http://support.microsoft.com/kb/968967we have now seen the issue go away....

I think that there are two issues in this post. One is the Monitringost CPU Usage (kb968967seems to bethe solution for this issue) and the other is HealthService CPU usage in passive cluster nodes: no sollution until now.

Hi jripoll.For the process HealthService.exe and high cpu usage there is a blogposting of mine, found here: http://thoughtsonopsmgr.blogspot.com/2009/07/opsmgr-sp1-is-process-healthservice-of.htmlPerhaps this helps?Best regards, Marnix Wolf (Thoughts on OpsMgr)

Thank you. I saw your blog some days ago, but I think it is a different issue. It happens only in R2 and in passive cluster nodes.

Yes indeed I can confirm this is a known bug confirmed by PSS, the only issue here is the product team hasn't an ETA for the fix. I want to be clear the failover proposed by Marius is just a workaround that won't last long. I started with one SQL cluster with issues, then an Exchange 2003 added up and finally an old Virtual Server cluster. If I failover it resolves for a while but then the issue arises again,more a failoverisa service outage I canoot afford easily.Add to this that the change proxy option doesn't work for cluster resources (without any error message) and you can reach the conslusion that cluster monitoring is messed up. Fix it, fix it fix it.More info at http://nocentdocent.wordpress.com/2009/07/21/opsmgr-2007-r2-lessons-learned-reprise/Daniele- Daniele This posting is provided "AS IS" with no warranties, and confers no rights. http://nocentdocent.wordpress.com http://www.progel.it

did someone try to rollback problem R2 agents to Sp1? will it help? i means, RMS is R2 and cluster agents are Sp1

hi guys, i have the problem on server 2008 SP2 x64. MonitoringHost.exe - 99% CPU KB968967 changes file versions, but my file versions are newer, so i don't think i will install it. Any other suggestions?