[Mono-bugs] [Bug 472732] mod-mono spawns many process and fails to respond when using AutoRestartMode

[bugzilla_noreply at novell.com]

https://bugzilla.novell.com/show_bug.cgi?id=472732

User mhabersack at novell.com added comment
https://bugzilla.novell.com/show_bug.cgi?id=472732#c16





--- Comment #16 from Marek Habersack <mhabersack at novell.com>  2009-02-11 12:07:47 MST ---
(In reply to comment #15)
> Marek, thanks for the explanation. I'm happy for this to be a configuration
There's undoubtedly room for improvement in mod_mono, but this particular issue
is a configuration one, indeed :)

> issue on my part, and I would happily offer to document it on the mod-mono wiki
> once its setteled :-)
> 
> A couple of clarifications:
> * We are using MPM worker
> * I've set the hard and soft per proccess FD limits to 32768. When I check the
> limits of the backend process using /proc/BACKENDPID/limits it shows 32768 on
> both soft and hard limits. I'm not clear about your comment of this limit being
> set on a group of processes.
I wasn't clear enough when writing that, sorry. What I mean is that all of the
child processes of apache inherit that. While for the child process it's enough
room for opening files it needs (hopefully), for the parent process the space
might be tight. Each instance of a backend consumes at least 3 file descriptors
(that's just mod_mono communications) per request, plus apache descriptors
opened per request, plus logging etc. etc. - with many concurrent requests,
several queued ones the 32k of available slots might not be enough and is the
cause of the error. If your apache has other virtual hosts, then the number of
available descriptors might decrease even quicker.

> * I've set the autorestart request limit to 100 only for the repro. It was set
> to 10,000 on our servers.
Ah, that's good :) I was under impression it was your production value.

> * The repro is most basic ASP.NET app with a single default.aspx page. Shutdown
> of this application should be pretty fast.
> 
> If I understand your explanation, you are proposing that I size up the request
> limits, etc. to *avoid* this issue. It seems that there is a window here where
> apache/mod-mono could start a large number of processes and consume a large
> number of FD. This window happens when backend are shutdown and restarted and
> is a result of the synchornous operations that are performed. Is this correct?
Yes, as explained above it may create a situation when the parent (apache)
process runs out of available file descriptors.

> My issues is that I'm unomfortable running without any limits (time or request)
> based, and I've yet been able to find the right configuration to avoid this
> issue when a limits is reached. I've observed that by setting the request limit
> to 10,000 (and having the FD descriptor limit set to a high number) this
> prolongs the lock-up issue but it still happens.
I think it's just a matter of finding the safe margin of available file
descriptors value for apache.

> The cronjob approach might be the most robust approach at this point.
yes, it will clean all the backend processes. Frankly, there's not much
mod_mono can do if its apache host runs out of file descriptors :)

-- 
Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug.