[Mono-dev] Issues with GC due to libgc

[pablosantosluac at terra.es]

Hi all,

After several weeks working on a bunch of mem issues related to the
libgc based garbage collector, we've identified the following issue and
a possible solution (Dick already sent some workarounds to the list):

The libgc garbage collector has a really hard time identifying pointers
to objects since it "guesses" what is a pointer instead of actually
"knowing" by using data passed by the mono runtime.

It means something as simple as introducing a long on the stack (for
instance something like array = new int[1000000]) will block (forever)
the memory at address 1000000. Yes, as incredible as it sounds, it can
cause important mem problems on long living apps (typically servers).

(As a side note, this exact problem is present on sgen, since it also
scans the stack "conservatively").

A small improvement could be made in the current GC with little effort,
and is supplying more class refmaps to libgc.

Libgc is very hard to modify, it contains too many hacks and
optimizations that have made the code a nightmare to understand and
modify, so we don't find useful to make anything here beyond very small
patches.

That said, mono currently can provide reference bitmaps for objects,
it's a matter of providing the right descriptor to the garbage collector.

Libgc supports this kind of descriptors and mono already generates them
for the sgen gc, so it's just a matter of joining those together (which
should beeasy to do). This should improve a great number of scans in the
arking process, leaving only stacks and several minor objects without
precise marking. (Should become similar to the current sgen idea, where
stacks and other roots are scanned conservatively, although not compacting).

Attached is the sample code we use to reproduce the issue on 32 bit
based Linux/Mono systems.

Some notes about the test app below:


=======================================
the program accepts commands like gc, mem, exit, 2, or 1

2 n m       creates n arrays of ints with m elements, and put them in an
arraylist. After the call completes, they are no longer referenced.
1 n m       same, but waiting for a key press after each new array
gc n         performs n gcs
exit         exits

So, the case:

mono test.exe
> 2 2000000 70            creates 2 million int arrays of 70 elements
each (virtual goes up to 777MB)
> gc 10                        should free everything, but around 33MB
remain allocated acording to pmap:

...
bf4b5000     32K      0K      0K ---p [anon]
bfc9e000     88K     32K     28K rwxp [stack]
ffffe000      4K      0K      0K r-xp [vdso]
Total:   777820K  33852K  29336K


> 2 20 25000000          creates 20 int arrays of 25 million elements
each (2.7GB)
> gc 10                         now pmap shows everything is screwed up:

...
b7f2b000      8K      8K      8K rwxp /lib/ld-2.6.1.so
bf4b5000     32K      0K      0K ---p [anon]
bfc9e000     88K     32K     28K rwxp [stack]
ffffe000      4K      0K      0K r-xp [vdso]
Total:   2764356K 1696132K 1691616K


Trying with smaller sizes lets you see that segments are joined and
split, but seems that there is some inability to free everything.

======================================


Regards,


	pablo



-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: Program.cs
Url: http://lists.ximian.com/pipermail/mono-devel-list/attachments/20090928/8d4f1de6/attachment.pl