[Mono-dev] [Mono-gc-list] Mono memory problems!

[David Wolinsky]

FYI, this case is only triggered when using gmcs and not mcs.

David

David Wolinsky wrote:
> We've isolated the problem down to AutoResetEvent...
>
> using System;
> using System.Threading;
>
> namespace Ipop {
>  public class IPOP_Common {
>    public static void Main() {
>      AutoResetEvent re = null;
>      while(true) {
>        re = new AutoResetEvent(false);
>        re.Close();
>      }
>    }
>  }
> }
>
> blows up memory
>
> whereas ...
>
> using System.Security.Cryptography;
> using System;
>
> namespace Ipop {
>  public class IPOP_Common {
>    public static void Main() {
>      RNGCryptoServiceProvider rng = new RNGCryptoServiceProvider();
>      while(true) {
>        byte[] key = new byte[1024];
>        rng.GetBytes(key);
>      }
>    }
>  }
> }
>
> This doesn't.
>
> David Wolinsky wrote:
>> We run this software on system where memory is a concern.  The data 
>> that we presented is our test case system that has 50 nodes all 
>> running in the same mono process.  We run only a single node at each 
>> site which initially starts at ~15 MB, we've seen it swell to well 
>> over 300 MBs in a period of less than a week.  Since this must be 
>> used in production environments and is meant to be extremely 
>> lightweight we can forgive a small memory portion like 15 MB, since 
>> it has relatively no processing overhead, but at over 300 MBs our 
>> processes are often stopped by the remote admin and we are told to 
>> clean up the problem.
>>
>> Since this seems to be a problem of using a non-compacting gc, do you 
>> know where the compacting gc is, so that we could at least test it 
>> out.  I searched the SVN and found no clues of it.
>>
>> Also, I should correct myself, the results for memory consumption 
>> were not directly related to the test that grows at 25kB/sec.  I 
>> found this out after posting the data, I am running heap-shot right 
>> now with the correct test and it has grown 100MB in less than 1 hour.
>>
>> Regards,
>> David
>>
>>
>>
>> Alan McGovern wrote:
>>  
>>> Well, after 12 hours at a consistent 25kB/sec, you'd expect to have 
>>> over 1 gig of memory allocated. As you don't, i think what you're 
>>> seeing is just 'normal usage' for the non-compacting GC that mono 
>>> uses. I have a similar app which uses sockets extensively (50-150 
>>> simultaneous connections)  and i can assure you that memory usage 
>>> doesn't get unbearably large. It'd be interesting to see the logs 
>>> but i don't think there's much to be worried about.
>>>
>>> Alan.
>>>
>>> On 7/18/07, *David Wolinsky* <davidiw at ufl.edu 
>>> <mailto:davidiw at ufl.edu>> wrote:
>>>
>>>     Initially 45 MB, 12 hours later 147 MB
>>>
>>>     Another developer has the heap-shot logs, I'll post those as 
>>> soon as
>>>     possible.
>>>
>>>     David
>>>
>>>     Alan McGovern wrote:
>>>     > Could you post up the detailed stats from heapshot? After the 12
>>>     hour
>>>     > run, how much memory are you using? Are we talking in the 
>>> gigabyte
>>>     > range, or megabyte range?
>>>     >
>>>     > Alan.
>>>     >
>>>     > On 7/18/07, *David Wolinsky* <davidiw at ufl.edu
>>>     <mailto:davidiw at ufl.edu>
>>>     > <mailto:davidiw at ufl.edu <mailto:davidiw at ufl.edu>>> wrote:
>>>     >
>>>     >     My lab works on a peer-to-peer network overlay and we've 
>>> noticed
>>>     >     recently significant memory issues.  Some background...
>>>     >
>>>     >     This application is constantly creating new objects and 
>>> shortly
>>>     >     thereafter deleting (removing reference to) them
>>>     >     Using a sample run with 150 threads running...
>>>     >     Mono on Linux has a growth rate of ~25 KB per second with a
>>>     base
>>>     >     of 50MB
>>>     >     (y = 25K *x + 50M)
>>>     >     .NET on Windows stabilizes at 35 MB
>>>     >
>>>     >     We ran heap-shot with Linux and found that in a 12 hour
>>>     period it
>>>     >     reported this...
>>>     >     start:
>>>     >     objects: 58,823
>>>     >     heap memory: 6,838,426 bytes
>>>     >
>>>     >     end:
>>>     >     objects: 59,925
>>>     >     heap memory: 6,862,336
>>>     >
>>>     >     We have run mono with GC_MAXIMUM_HEAP_SIZE and the memory 
>>> size
>>>     >     (RES) got
>>>     >     significantly bigger than it.
>>>     >
>>>     >     I have searched for the Compacting GC with no luck, we would
>>>     >     really like
>>>     >     to see if it would help our problem.
>>>     >
>>>     >     The only operating system resources we're using are 
>>> Sockets, but
>>>     >     we use
>>>     >     them VERY heavily!
>>>     >
>>>     >     If anyone has any suggestions, we'd be open to test out
>>>     anything
>>>     >     at this
>>>     >     point!
>>>     >
>>>     >     We are leaning towards an issue in unmanaged memory and
>>>     possibly a bug
>>>     >     in mono.
>>>     >
>>>     >     Best regards,
>>>     >     David
>>>     >
>>>     >
>>>     >     ps, I fwded this to gc and devel list because gc list looks
>>>     quite
>>>     >     dead.... sorry for the duplication
>>>     >     _______________________________________________
>>>     >     Mono-devel-list mailing list
>>>     >     Mono-devel-list at lists.ximian.com
>>>     <mailto:Mono-devel-list at lists.ximian.com>
>>>     >     <mailto:Mono-devel-list at lists.ximian.com
>>>     <mailto:Mono-devel-list at lists.ximian.com>>
>>>     >     http://lists.ximian.com/mailman/listinfo/mono-devel-list
>>>     >
>>>     >
>>>
>>>
>>>     
>>
>> _______________________________________________
>> Mono-gc-list maillist  -  Mono-gc-list at lists.ximian.com
>> http://lists.ximian.com/mailman/listinfo/mono-gc-list
>>
>>   
>
>