[Mono-dev] Issues with GC due to libgc
pablosantosluac at terra.es
pablosantosluac at terra.es
Wed Sep 30 03:39:20 EDT 2009
Is it a 64bits machine?
If so, the same is true for us.
But, with 32 bits mono, we can consistently repeat the issue. I can set
up a gotomeeting and show you how it happens.
pablo
Zoltan Varga wrote:
> Hi,
>
> On my machine, after the allocation of the 2.5GB of arrays + 10 gcs,
> the RSS of the
> process goes back to about 100MB. This is perfectly normal, as the GC
> doesn't return
> free memory to the OS in some cases, so it doesn't have to allocate it
> again when the
> program starts allocating memory.
>
> Zoltan
>
> On Mon, Sep 28, 2009 at 9:05 PM, pablosantosluac at terra.es
> <mailto:pablosantosluac at terra.es> <pablosantosluac at terra.es
> <mailto:pablosantosluac at terra.es>> wrote:
>
> Hi all,
>
> After several weeks working on a bunch of mem issues related to the
> libgc based garbage collector, we've identified the following issue and
> a possible solution (Dick already sent some workarounds to the list):
>
> The libgc garbage collector has a really hard time identifying pointers
> to objects since it "guesses" what is a pointer instead of actually
> "knowing" by using data passed by the mono runtime.
>
> It means something as simple as introducing a long on the stack (for
> instance something like array = new int[1000000]) will block (forever)
> the memory at address 1000000. Yes, as incredible as it sounds, it can
> cause important mem problems on long living apps (typically servers).
>
> (As a side note, this exact problem is present on sgen, since it also
> scans the stack "conservatively").
>
> A small improvement could be made in the current GC with little effort,
> and is supplying more class refmaps to libgc.
>
> Libgc is very hard to modify, it contains too many hacks and
> optimizations that have made the code a nightmare to understand and
> modify, so we don't find useful to make anything here beyond very small
> patches.
>
> That said, mono currently can provide reference bitmaps for objects,
> it's a matter of providing the right descriptor to the garbage
> collector.
>
> Libgc supports this kind of descriptors and mono already generates them
> for the sgen gc, so it's just a matter of joining those together (which
> should beeasy to do). This should improve a great number of scans in the
> arking process, leaving only stacks and several minor objects without
> precise marking. (Should become similar to the current sgen idea, where
> stacks and other roots are scanned conservatively, although not
> compacting).
>
> Attached is the sample code we use to reproduce the issue on 32 bit
> based Linux/Mono systems.
>
> Some notes about the test app below:
>
>
> =======================================
> the program accepts commands like gc, mem, exit, 2, or 1
>
> 2 n m creates n arrays of ints with m elements, and put them in an
> arraylist. After the call completes, they are no longer referenced.
> 1 n m same, but waiting for a key press after each new array
> gc n performs n gcs
> exit exits
>
> So, the case:
>
> mono test.exe
> > 2 2000000 70 creates 2 million int arrays of 70 elements
> each (virtual goes up to 777MB)
> > gc 10 should free everything, but around 33MB
> remain allocated acording to pmap:
>
> ...
> bf4b5000 32K 0K 0K ---p [anon]
> bfc9e000 88K 32K 28K rwxp [stack]
> ffffe000 4K 0K 0K r-xp [vdso]
> Total: 777820K 33852K 29336K
>
>
> > 2 20 25000000 creates 20 int arrays of 25 million elements
> each (2.7GB)
> > gc 10 now pmap shows everything is screwed up:
>
> ...
> b7f2b000 8K 8K 8K rwxp /lib/ld-2.6.1.so
> <http://ld-2.6.1.so>
> bf4b5000 32K 0K 0K ---p [anon]
> bfc9e000 88K 32K 28K rwxp [stack]
> ffffe000 4K 0K 0K r-xp [vdso]
> Total: 2764356K 1696132K 1691616K
>
>
> Trying with smaller sizes lets you see that segments are joined and
> split, but seems that there is some inability to free everything.
>
> ======================================
>
>
> Regards,
>
>
> pablo
>
>
>
>
> using System;
> using System.Collections;
>
> namespace test
> {
> class Program
> {
> static void Main(string[] args)
> {
> WaitForEnter();
> }
>
> private static void WaitForEnter()
> {
> Console.WriteLine("Command:");
>
> while (true)
> {
> Console.Write("> ");
> string line = Console.ReadLine();
>
> string[] args = line.Split(' ');
>
> if (args.Length <= 0) continue;
>
> switch (args[0].ToLower())
> {
> case "exit": return;
>
> case "1": Case1(args); break;
>
> case "2": Case2(args); break;
>
>
> case "mem":
> Console.WriteLine("Memory now: {0}",
> GC.GetTotalMemory(false));
> break;
> case "gc":
> Gcs(args);
> break;
> default:
> Console.WriteLine("Unknown command");
> break;
> }
> }
> }
>
> private static void Gcs(string[] args)
> {
> int loop = (args.Length == 2) ? Int32.Parse(args[1]) : 1;
>
> for (int i = 0; i < loop; ++i)
> {
> Console.WriteLine("Memory {1} now : {0}",
> GC.GetTotalMemory(false), i);
> Console.WriteLine("Memory {1} after GC: {0}",
> GC.GetTotalMemory(true), i);
> }
> }
>
> private const int OneMeg = 1024 * 1024;
>
> private static void Case1(string[] args)
> {
> int loop = (args.Length >= 2) ? Int32.Parse(args[1]) : 5;
> int size = (args.Length >= 3) ? Int32.Parse(args[2]) : 10
> * OneMeg;
>
> ArrayList container = new ArrayList();
>
> for (int i = 0; i < loop; ++i)
> {
> int[] s1 = new int[size];
>
> for (int j = 0; j < size; ++j)
> {
> s1[j] = j;
> }
>
> container.Add(s1);
>
> Console.Write("Iteration {0}, press enter for next", i);
> Console.ReadLine();
> }
>
> // Explicit in case it helps
> container = null;
> }
>
> private static void Case2(string[] args)
> {
> int loop = (args.Length >= 2) ? Int32.Parse(args[1]) : 5;
> int size = (args.Length >= 3) ? Int32.Parse(args[2]) : 10
> * OneMeg;
>
> ArrayList container = new ArrayList();
>
> for (int i = 0; i < loop; ++i)
> {
> int[] s1 = new int[size];
>
> for (int j = 0; j < size; ++j)
> {
> s1[j] = j;
> }
>
> container.Add(s1);
>
> Console.WriteLine("Iteration {0}", i);
> }
>
> // Explicit in case it helps
> container = null;
> }
>
>
> }
> }
>
> _______________________________________________
> Mono-devel-list mailing list
> Mono-devel-list at lists.ximian.com
> <mailto:Mono-devel-list at lists.ximian.com>
> http://lists.ximian.com/mailman/listinfo/mono-devel-list
>
>
More information about the Mono-devel-list
mailing list