[Mono-dev] Size of thread in Mono (65MB per thread?)
james at mansionfamily.plus.com
Sat Dec 21 14:38:30 UTC 2013
On 20/12/2013 09:27, Marek Habersack wrote:
> Unix systems work based on a bit different principle than the Windows
> ones. Namely, as Nikita mentions in his other mail, the virtual memory
> is nearly free of any limits - your process reserves the right to use
> that much memory, but it doesn't actually use ("commit") it
> physically. The virtual memory reservation is merely a hint of what
> can be consumed by the program, should it need it.
I really think this ought to be challenged. No least because I recently
ran into an issue on some production RedHat servers which had
memory_overcommit set to '2'.
Historically, UNIX systems DID behave like Windows. And some of them
still do. They require that a request for VM can be satisfied from the
free physical+virtual memory.
The default is now to allow lazy programming that treats virtual address
space as free, as you note - but the flip side is that programs can ask
the OS for memory, be given it without error, and then find that they
are summarily terminated if they try to use it. Or that some other
process is summarily terminated.
This mechanism came about in AIX first I believe, an Linux copied it.
Solaris held out for a long time with 'proper' behaviour, I don't know
whether it allows overcommit now.
Whether or not you think allowing overcommit is a good idea is very much
dependent on whether you think a good return from malloc should allow
you to use the memory you just asked for.
In fairness, the problem is typically worst in multi-threaded programs
that can't handle segmented thread stacks - they have to reserve enough
space for reasonable levels of stack depth and stack-local data so they
have to reserve VM space to ensure that malloc doesn't place another
block after the heap that's actually in use. Laziness is actually
allocating the space and relying on the OS to allow overcommit and not
actually back such memory until needed, rather than just reserving a VM
range and allocating on fault.
The prod servers I had showed no use of swap, about 6gig out of 32
physical in use (with the rest as buffers) - and thread starts and
allocs were failing. Needless to say all the machines we'd used for
dev, qa, and load testing had overcommit 0 and we got mugged as a result.
More information about the Mono-devel-list