[Mono-dev] Size of thread in Mono (65MB per thread?)

[Nikita Tsukanov]

The problem is that there is no anything simular to MEM_RESERVE and
MEM_COMMIT from Windows (flags for VirtualAlloc/VirtualAllocEx), so there
is no way to properly reserve a large block of virtual address space. BTW,
windows threads reserve a lot of address space using MEM_RESERVE and then
use SEH/VEH to detect, when memory should be commited, so it uses the same
"overcommit" mechanics. You can create a lot of threads with large stacks,
try to use it and get out of memory exception when, actually, trying to
push something on your stack.


2013/12/21 james <james at mansionfamily.plus.com>

> On 20/12/2013 09:27, Marek Habersack wrote:
>
>> Unix systems work based on a bit different principle than the Windows
>> ones. Namely, as Nikita mentions in his other mail, the virtual memory is
>> nearly free of any limits - your process reserves the right to use that
>> much memory, but it doesn't actually use ("commit") it physically. The
>> virtual memory reservation is merely a hint of what can be consumed by the
>> program, should it need it.
>>
>
> I really think this ought to be challenged.  No least because I recently
> ran into an issue on some production RedHat servers which had
> memory_overcommit set to '2'.
>
> Historically, UNIX systems DID behave like Windows.  And some of them
> still do.  They require that a request for VM can be satisfied from the
> free physical+virtual memory.
>
> The default is now to allow lazy programming that treats virtual address
> space as free, as you note - but the flip side is that programs can ask the
> OS for memory, be given it without error, and then find that they are
> summarily terminated if they try to use it. Or that some other process is
> summarily terminated.
>
> This mechanism came about in AIX first I believe, an Linux copied it.
>  Solaris held out for a long time with 'proper' behaviour, I don't know
> whether it allows overcommit now.
>
> Whether or not you think allowing overcommit is a good idea is very much
> dependent on whether you think a good return from malloc should allow you
> to use the memory you just asked for.
>
> In fairness, the problem is typically worst in multi-threaded programs
> that can't handle segmented thread stacks - they have to reserve enough
> space for reasonable levels of stack depth and stack-local data so they
> have to reserve VM space to ensure that malloc doesn't place another block
> after the heap that's actually in use.  Laziness is actually allocating the
> space and relying on the OS to allow overcommit and not actually back such
> memory until needed, rather than just reserving a VM range and allocating
> on fault.
>
> The prod servers I had showed no use of swap, about 6gig out of 32
> physical in use (with the rest as buffers) - and thread starts and allocs
> were failing.  Needless to say all the machines we'd used for dev, qa, and
> load testing had overcommit 0 and we got mugged as a result.
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ximian.com/pipermail/mono-devel-list/attachments/20131221/043dc69b/attachment.html>