Thread exhaustion

One of the biggest headaches for any web developer is the 503 – “server too busy” error message from IIS. When developing a site, if your code is inefficient in the way it processes data you’ll see CPU spikes that are easy enough to profile, identify and resolve. If your code is inefficient in the way it manages memory then you will see the w3wp.exe process balloon up, which is also fairly easy to identify and resolve. So what causes the 503 – “server too busy”? Inefficient thread management. Now I’m not going to discuss the threading model of IIS 6 and the way it changed with IIS 7, or IIS threads vs CLR threads. I’ll leave that topic to men with far larger beards than I have. The short version is that with .net 2.0 to 3.5 the math was 12 requests per CPU and 1000 requests in the queue waiting to be executed would trigger the 503 (with .net ). IIS is doing everything it can, queuing everything it can’t do and that queue is too big to hold anything more so IIS is just abandoning the request.

The typical scenario I’ve seen for this occurring is when a website is supposed to be consuming a slow web service in real time. In this scenario, the majority of the time is spent with the thread blocked, waiting on the webservice to return the required data. A much more efficient way to handle the request would be to release the thread and continue processing when the service has returned the data you need. Back in the old MVC days you would refactor code like this:

 

public class HomeController : Controller
{
    public ActionResult Index()
    {
        var client = new SlowWebservice();
        ViewData["WebserviceResults"] = client.GetResults();
        return View();
    }
}

Into code like this:

public class HomeController : AsyncController
{
    public void IndexAsync()
    {
        AsyncManager.OutstandingOperations.Increment();
           
        var client = new SlowWebservice();
        client.GetResultsAsyncCompleted += (s, e) =>
            {
                AsyncManager.Parameters["items"] = e.Items;
                AsyncManager.OutstandingOperations.Decrement();
            };
        client.GetResultsAsync();
    }
 
    public ActionResult IndexCompleted(IEnumerable items)
    {
        ViewData["WebserviceResults"] = items;
        return View();
    }
}

Which, in my opinion, turns a very simple operation into an ugly mess of infrastructure code, but you’ve got to do what you’ve got to do. If the difference was between ugly code that worked and neat code that tanked the server, I know which I would be taking.

Thankfully this isn’t the case anymore. Thanks to .net 4.5 and Microsoft’s love of syntactic sugar we now have a much cleaner way to achieve the same result:

public class HomeController : AsyncController
{
    public async Task Index()
    {
        var client = new SlowWebservice();
        ViewData["WebserviceResults"] = await client.GetResultsAsync();
        return View();
    }
 }

Now with this vast simplification of the syntax there is the temptation to go and make everything async, but that’s not the lesson here. There is additional overhead in using async controllers here rather than standard synchronous controllers. I’ve read that the rule of thumb is 2 seconds, if your webservice takes more than 2 seconds to run you should make the call asynchronously. I think that’s probably a little slow and you should look to make asynchronous calls on faster methods than that, but definitely everything taking 2 seconds or more should run asynchronously.