[Mono-dev] relaxing the 4GB heap limit (I think implied by LARGE_CONFIG) & bad out-of-memory behavior

[Jonathan Shore]

I have applications that process through terrabytes of timeseries data.  Usually I can limit the amount of memory I use in-process, however, from time to time I need to deal with data larger than 4GB in size.   I run mono both on OSX and linux.   It seems that the LARGE_CONFIG (which is not even the default), maxes out at something close to 4GB of heap.     What would it take to relax this so can use more of a 64bit memory space?

In libgc/include/private/gc_priv.h, there seems to be a hastable of heap pages, indexed by up to 20 bits in the LARGE_CONFIG compilation.  The comment indicates that the 2097151 possible entries corresponds to roughly 4GB +/- of heap.

Regardless of limitations build into the memory model, the mono runtime has the bad behavior of crashing when the maximum # of heap pages is reached instead of throwing OutOfMemoryException.  Particularly for production services it would be useful to catch, say, a condition where most of the memory is used and throw an exception so that the application can exit or clean up gracefully.

Thoughts on this?

Jonathan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ximian.com/pipermail/mono-devel-list/attachments/20120829/a280b55d/attachment.html>