SSRS 2005 API Causing Application Pool Crash
My site has started having issues with the application pool crashing while using Reporting Services manager.
This is happening at more than one installation.
I have seperated both the Reports and ReportServer web sites into their own application pool to single out the issue and have localized the issue to ReportServer as it is the only app pool that is crashing.
This started at the time we implemented the ReportingServices2005 API.
Attached below is a sample call we use to retrieve the list of reports in an SSRS folder.
Currently our application uses javascript (Jquery) to make ajax calls to a C# .NET function which makes the SSRS
API call.
It seems to happen randomly as the app pool will not crash for days or will crash within hours of restarting.
Once this error happens, the app pool rejects all future connections, however
the IIS UI does not show that the app pool is stopped.
The httperr log shows that the app pool was disabled due to rapid-fail protection.
When rapid-fail protection is turned off the crash still occurs and the app pool drops all connections.
We have looked at all logs from app pool as well as reporting services.
We have also added the Debug Diagnostics Tool to IIS to retrieve the dump file from the server.
Nothing obvious has jumped out at us and we’re now at a loss.
Example Reporting Services API Call:
[WebMethod(EnableSession =
false)]
[ScriptMethod]
public List<CatalogItem> GetReportList()
{
CatalogItem[] items =
null;
List<CatalogItem> ret =
new List<CatalogItem>();
try
{
ReportingService2005 rs =
new ReportingService2005();
rs.Url = ConfigurationManager.AppSettings["Reporting_Services_Url"];
rs.Credentials = System.Net.CredentialCache.DefaultCredentials;
// Use the reports path in the app config
string path =
ConfigurationManager.AppSettings["ReportServer_ReportList_Path"];
if (path == null || path ==
"") path = "/";
// send in at least a base root "/"
items = rs.ListChildren(path, true);
foreach (CatalogItem item
in items)
{
if (item.Type ==
ItemTypeEnum.Folder || item.Type == ItemTypeEnum.Report)
{
if (item.Name !=
"Search Reports" && item.Name != "My Reports" && item.Name !=
"Users Folders" && item.Name !=
"Data Sources")
ret.Add(item);
}
}
}
catch (System.Exception ex)
{
Guid? guid =
ExceptionPolicy.HandledException(ex);
// json object to include the guid as well as the exception info, for UI retrieval
string jsonErrData =
AjaxErrorHandling.CreateJsonFormat(guid, ex.Message, ex.GetType().ToString());
throw new
Exception(jsonErrData);
}
return ret;
}
Dump Log Warning:
The following threads in
Crash_Rule_Targets__Date__11_03_2010__Time_09_16_14AM__w3wp.exe__ReportServer__PID__5256__371__Manual Dump.dmp are waiting on data to be returned from another server via WinSock
The call to WinSock originated from 0x02a6a1c3 and is destined for
port 80 at IP address 127.0.0.1
( 62 )
1.12% of threads blocked
Ensure that any remote server this application may be calling is functioning properly and there are no network issues between the two servers. If the problem
continues, please contact the application vendor for further assistance
Dump Thread in question:
Thread 62 - System ID 3024
Entry point
mscorwks!Thread::intermediateThreadProc
Create time
11/2/2010 7:21:00 PM
Time spent in user mode
0 Days 00:00:00.734
Time spent in kernel mode
0 Days 00:00:00.156
This thread is waiting on data to be returned from another server via WinSock.
The call to WinSock originated from 0x02a6a1c3 and is destined for
port 80 at IP address 127.0.0.1
WinDbg Dump access on Thread 62
OS Thread Id: 0xbd0 (62)
ESP
EIP
2b13edb0 7c82860c [NDirectMethodFrameStandalone: 2b13edb0] System.Net.UnsafeNclNativeMethods+OSSOCK.recv(IntPtr, Byte*, Int32, System.Net.Sockets.SocketFlags)
2b13edc8 7a581b04 System.Net.Sockets.Socket.Receive(Byte[], Int32, Int32, System.Net.Sockets.SocketFlags, System.Net.Sockets.SocketError ByRef)
2b13ee00 7a581a00 System.Net.Sockets.Socket.Receive(Byte[], Int32, Int32, System.Net.Sockets.SocketFlags)
2b13ee20 7a58b854 System.Net.Sockets.NetworkStream.Read(Byte[], Int32, Int32)
2b13ee50 7a58b7bb System.Net.PooledStream.Read(Byte[], Int32, Int32)
2b13ee6c 7a58b4aa System.Net.Connection.SyncRead(System.Net.HttpWebRequest, Boolean, Boolean)
2b13eec0 7a58b30a System.Net.Connection.PollAndRead(System.Net.HttpWebRequest, Boolean)
2b13eed8 7a58b29b System.Net.ConnectStream.PollAndRead(Boolean)
2b13eee4 7a58b132 System.Net.HttpWebRequest.EndWriteHeaders(Boolean)
2b13ef10 7a58b056 System.Net.HttpWebRequest.WriteHeadersCallback(System.Net.WebExceptionStatus, System.Net.ConnectStream, Boolean)
2b13ef24 7a58ac11 System.Net.ConnectStream.WriteHeaders(Boolean)
2b13ef78 7a588f82 System.Net.HttpWebRequest.EndSubmitRequest()
2b13efa4 7a588e9f System.Net.HttpWebRequest.SetRequestSubmitDone(System.Net.ConnectStream)
2b13efb8 7a5888c2 System.Net.Connection.CompleteStartRequest(Boolean, System.Net.HttpWebRequest, System.Net.TriState)
2b13efe8 7a588233 System.Net.Connection.SubmitRequest(System.Net.HttpWebRequest)
2b13f02c 7a587bac System.Net.ServicePoint.SubmitRequest(System.Net.HttpWebRequest, System.String)
2b13f064 7a587479 System.Net.HttpWebRequest.SubmitRequest(System.Net.ServicePoint)
2b13f090 7aaaa4d4 System.Net.HttpWebRequest.GetResponse()
2b13f0d0 65dda7e5 System.Web.Services.Protocols.WebClientProtocol.GetWebResponse(System.Net.WebRequest)
2b13f100 65dde895 System.Web.Services.Protocols.HttpWebClientProtocol.GetWebResponse(System.Net.WebRequest)
2b13f104 2f51e76e Microsoft.ReportingServices.UI.Global+RSWebServiceWrapper.GetWebResponse(System.Net.WebRequest)
2b13f114 65e09905 System.Web.Services.Protocols.SoapHttpClientProtocol.Invoke(System.String, System.Object[])
2b13f158 2be6bd5c Microsoft.SqlServer.ReportingServices2005.ReportingService2005.ListSecureMethods()
2b13f164 2f51df12 Microsoft.SqlServer.ReportingServices2005.RSConnection.GetSecureMethods()
2b13f19c 2f51de3a Microsoft.ReportingServices.UI.Global+RSWebServiceWrapper.GetSecureMethods()
2b13f1c4 2f51dd9b Microsoft.SqlServer.ReportingServices2005.RSConnection.IsSecureMethod(System.String)
2b13f1dc 2f51dd13 Microsoft.SqlServer.ReportingServices2005.RSConnection.ValidateConnection()
2b13f204 2f5197d2 Microsoft.ReportingServices.UI.Global.SecureAllAPI()
2b13f20c 2f519720 Microsoft.ReportingServices.UI.ReportingPage.EnsureHttpsLevel(Microsoft.ReportingServices.Diagnostics.HttpsLevel)
2b13f218 2f5191e4 Microsoft.ReportingServices.UI.ReportingPage.ReportingPage_Init(System.Object, System.EventArgs)
2b13f260 02a6b30d [MulticastFrame: 2b13f260] System.EventHandler.Invoke(System.Object, System.EventArgs)
2b13f274 660ab484 System.Web.UI.Control.OnInit(System.EventArgs)
2b13f288 660a72ed System.Web.UI.Page.OnInit(System.EventArgs)
2b13f294 660ab5de System.Web.UI.Control.InitRecursive(System.Web.UI.Control)
2b13f2b4 660a7a3b System.Web.UI.Page.ProcessRequestMain(Boolean, Boolean)
2b13f40c 660a7764 System.Web.UI.Page.ProcessRequest(Boolean, Boolean)
2b13f444 660a7691 System.Web.UI.Page.ProcessRequest()
2b13f47c 660a7626 System.Web.UI.Page.ProcessRequestWithNoAssert(System.Web.HttpContext)
2b13f488 660a7602 System.Web.UI.Page.ProcessRequest(System.Web.HttpContext)
2b13f49c 2f519105 ASP.pages_report_aspx.ProcessRequest(System.Web.HttpContext)
2b13f4a0 660adad6 System.Web.HttpApplication+CallHandlerExecutionStep.System.Web.HttpApplication.IExecutionStep.Execute()
2b13f4d4 6608132c System.Web.HttpApplication.ExecuteStep(IExecutionStep, Boolean ByRef)
2b13f514 6608c5c3 System.Web.HttpApplication+ApplicationStepManager.ResumeSteps(System.Exception)
2b13f564 660808ac System.Web.HttpApplication.System.Web.IHttpAsyncHandler.BeginProcessRequest(System.Web.HttpContext, System.AsyncCallback, System.Object)
2b13f580 66083e1c System.Web.HttpRuntime.ProcessRequestInternal(System.Web.HttpWorkerRequest)
2b13f5b4 66083ac3 System.Web.HttpRuntime.ProcessRequestNoDemand(System.Web.HttpWorkerRequest)
2b13f5c4 66082c5c System.Web.Hosting.ISAPIRuntime.ProcessRequest(IntPtr, Int32)
2b13f7d8 79f9811e [ContextTransitionFrame: 2b13f7d8]
2b13f80c 79f9811e [GCFrame: 2b13f80c]
2b13f968 79f9811e [ComMethodFrame: 2b13f968]
November 5th, 2010 6:12pm
Hi Drew,
Thank you for your posting.
Since this issue is an application crash, in order to get a better assistment, I would suggest you contact Microsoft Custom Services Support to dump analysis.
Thanks,
Jin ChenJin Chen - MSFT
Free Windows Admin Tool Kit Click here and download it now
November 10th, 2010 7:01am