[Mono-aspnet-list] issues with intermittent apache hangups

[Marek Habersack]

On Wed, 13 Apr 2011 06:42:05 -0700 (PDT)
Dan <parnham at gmail.com> wrote:

> Marek,
Hey Dan,

> Thanks for all your help so far.
You're welcome

> I've applied the patch (0eff07d0fc4ef1ef3c584e71a8dd54ab708543e8) both on my
> development machine (to run further mprof tests) and on our test server. The
> local tests still show many thousands of cache items being generated,
> although I believe at a slightly slower rate.
From your previous mails it seems that you generate lots and lots of session entries, so the rate
at which they appear is not dependent on Mono. What happens now is that we don't create the excess
entries for session items whose timeouts got reset. Note also that each session entry, not only
one with a timeout, will have an associated CacheItem instance. CacheItems are cleared whenever
they are removed from the cache, so their memory overhead is very small after they are no longer
valid. But the responsibility to remove non-timed entries relies upon the application (or
System.Web code in certain cases) - System.Web cannot do anything about them. 
Could you count the total number of Session items your application creates, what percentage of them
are timed ones, log what are the keys of all of the entries and send that data to me off-list?

> The test server has also been set running again for the past couple of days,
> although I think the app got restarted at some point (probably killed by the
> system when it started demanding too much memory). Just watching it for the
> past two hours it has gone from 19.7% to 26.8% memory usage (1GB of memory
Does running with sgen help?

> available in this old test machine), and is still creeping up. This problem
> will cause us major issues in a production kiosk system, since it will reach
> the point where the web app response slows to a halt and then eventually
> gets killed by the os and restarted by apache. 
Have you considered rolling your own session state module? Based e.g. on memcached?

> Unless this can be resolved properly, we may have to resort to setting up a
> cron job that will automatically force the web server to restart on a
> regular basis, but we really do not want to have to do that!
Well, to resolve it properly we need to know what the problem is :) Let's see what we can read from
the data I asked you to collect above.

best,

marek

> 
> Regards,
> Dan
> 
> 
> Marek Habersack wrote:
> > 
> > On Tue, 5 Apr 2011 15:24:25 -0500 (CDT)
> > Dan &lt;parnham at gmail.com&gt; wrote:
> > 
> > Hey Dan,
> > 
> > The issue here was the way the in-proc session handler handled item
> > timeout reset. It would first
> > remove the item from the cache (it uses an internal instance of the Cache
> > class) and then insert
> > it back in the cache. With a big number of items, that would grow the
> > timed items priority queue
> > heap much more than was necessary and generate a lot of duplicate internal
> > cache items. The effects
> > would be different depending on the application's usage of session data.
> > I've just committed a possible fix for the issue to Mono's master branch
> > (I haven't committed it to
> > the mono-2-10 branch since it is a potentially breaking change which needs
> > to be tested in real
> > life first) in commit 0eff07d0fc4ef1ef3c584e71a8dd54ab708543e8 (tests have
> > been committed in
> > c00c5d00fddf18969c9869492f7e1235d960076b if you're interested). The fix
> > applies cleanly to the 2.10
> > branch, so please apply it and test with your application to see whether
> > it fixed the issue. Please
> > let me know about the results, thanks :)
> > 
> > marek
> > 
> 
> 
> --
> View this message in context:
> http://mono.1490590.n4.nabble.com/issues-with-intermittent-apache-hangups-tp3264509p3447198.html
> Sent from the Mono - ASP.NET mailing list archive at Nabble.com.
> _______________________________________________ Mono-aspnet-list mailing list
> Mono-aspnet-list at lists.ximian.com
> http://lists.ximian.com/mailman/listinfo/mono-aspnet-list