When doing a recycle (scheduled or not) of an application pool it’s possible your System event log shows the following warning: “A process serving application pool ‘APPLICATION_POOL_NAME’ exceeded time limits during shut down. The process id was ‘PID’.” (Source: W3SVC; Category: None; Event ID: 1013), with APPLICATION_POOL_NAME the name of the application pool and PID the Process ID (PID) of the application pool’s worker process.
Some people might find this weird: a shutdown for a recycle? Isn’t a recycle the kind of “refresh the application pool” task without interrupting the users? Indeed. Then why should IIS shutdown the (worker) process of the application pool? This should interrupt the users’ requests, right?
Not necessarily… First of all, when doing a recycle, a new worker process is created to handle the new requests. Secondly, the “old” worker process still exists for a while so it can finish its work decently (= handle the requests present at the time of the recycle). This way new requests will be served without interrupt and the same counts for existing requests, as the “old” process will only terminate after dealing with those requests. When the old process has finished doing its work, it is shutdowned. You see, there is a shutdown going on and it doesn’t cause an interruption. This is a very good concept as it can clean things up without disrupting availability or current requests.
The thing is that the old process gets a maximum amount of time to do its finishing business. When this time has been elapsed, IIS terminates the old process “by force”, whether the process has finished to do its cleanup work or not. The reason for this is quite simple: suppose a bug or design deficiency in the web application prevents the process to finish properly, this process could exist forever, using memory and other resources. Perhaps this is no disaster, but what if a recycle happens 100 times? It’s never a good thing to have dirty stuff on your machine, but it gets problematic if you staple this kind of dirty stuff over time.
When the maximum amount of “shutdown time” has been elapsed, there is a so-called “shutdown timeout” or with other words (used by the warning): the process serving the recycled application pool has exceeded time limits, i.e. the shutdown time limit. The process is terminated by force, meaning you won’t find the process anymore in for example Task Manager after this warning appears in your event log. This implies you probably won’t find a process anymore with the PID mentioned in the warning, as the new process gets another PID (remember: the old and new processes exist both for a while and they can’t have the same PID ofcourse).
This recycle is also called an “overlapped recycle” as 2 worker processes “overlap” each other for a while, that is: they exist both for some time (the shutdown time limit as a maximum). The shutdown time limit can be configured with the “Shutdown time limit” setting in the Health tab of the Properties dialog box for an application pool (so this setting is application pool specific). Its value is expressed in seconds and by default is 90 (seconds).
If the warning mentioned in the beginning of this article occurs, this means your web application (in the broad sense) couldn’t finish its work decently. Sometimes this could be normal and then you can augment the shutdown time limit. Most of the time though it indicates a problem. Ofcourse you could still augment the shutdown time limit, but this should only be considered as a temporary workaround, not as a final solution. And even then it can happen that the old process keeps running and has to be terminated by force (some processes would never finish because of a bug for example).
The bottom line is you don’t have to panic because your recycle seems to imply a shutdown 🙂 The only thing you should try to avoid is terminating the worker process by force: investigate the situation and if there is an acceptable explanation augment the shutdown time limit; otherwise fix your application/bug/architecture/… if you can (in the meanwhile or if such a fix isn’t realistic you could augment the value too, but this shouldn’t be something to be proud of ;-)).