[Mono-dev] Mono tasklet memory issues

[James Zhao]

Ah here's the problem. I didn't call yield/wait on any of the threads (and
thus store()). I fixed the problem by having every thread call yield at
least once and turning the code back to the version where one continuation
is created per step.

On Sun, Aug 30, 2009 at 12:45 AM, James Zhao <jameszhao00 at gmail.com> wrote:

> Hi,
> I'm currently working on something that uses mono 2.6's
> tasklet/continuations (similar to stackless python/'eve). However, I'm
> experiencing some really bizzare memory issues. I tested on Windows and
> Linux.
>
> I modified the default Mono.MicroThreads library so each microthread reuses
> continuations rather than allocating a new continuation every Start().
>
> I'm experiencing high memory usage when calling Mono Continuation's
> Continuation.store(). Every 4 or 5 seconds a huge GC op/spike happens.
> Here's a comparison of memory/cpu for using Mono Continuations and directly
> calling the method.
>
> You can view the source code
> http://code.google.com/p/mmo-test/source/browse/#svn/trunk/mmo_v2 .
> EntityProcessor.cs/Run() runs the microthreads and
> EntityProcessor.cs/AddQueuedEntities() adds the microthreads.
>
> Here's some data.
>
> Memory, ms spent in iteration
>
> Using Mono Continuations (200k entities/MicroThreads)
>
> 367210496, 410
> 385441792, 418
> 405204992, 346
> 427053056, 363
> 444809216, 333
> 462483456, 339
> 423661568, 1559
> 441327616, 465
> 458985472, 305
> 403214336, 1338
> 419254272, 506
>
> Not sing Mono Continuations (200k entities)
>
> 365662208, 67
> 365662208, 67
> 365662208, 66
> 365662208, 66
> 365662208, 67
> 365662208, 67
>
> Thanks,
> James
>
> Here's my chat log with Rodrigo:
>
> 12:16 AM me: hey rodrigo. do u have a moment?
>  Rodrigo: yes
> 12:17 AM me: i'm currently working on something that uses mono 2.6's
> tasklet/continuations (similar to stackless python/'eve). however, im
> experiencing some really bizzare memory issues
>  Rodrigo: right
>   on linux, right?
> 12:18 AM me: i tested on linux and windows.
>  Rodrigo: ok
>  me: once i start using them, it causes memory usage to continue rising,
>  Rodrigo: what kind of problems are you having?
>  me: until at around 4-5 secs a huge gc op occur
>   i took a look at the source
> 12:19 AM *c source
>  Rodrigo: well, each tasklet holds onto a big block of memory
>  me: and it should be gcing each time store() is called, or it will
> overwrite
>   and im reusing continuations across iterations
> 12:20 AM Rodrigo: why, it shouldn't be required
>  me: sorry. gc if current required > previous size
>   overwrite if otherwise
>  Rodrigo: I still fails to see a reason to trigger a gc on this case
> 12:21 AM me: hmm i guess im not too sure either
> 12:22 AM but the thing is, even if im reusing the continuation
>   the memory, (ms spent) is like this
>   87314432, 161
> 92131328, 128
> 91377664, 315
> 95244288, 88
> 93085696, 330
> 91824128, 319
> 89403392, 292
>   it keeps cycling
> 12:23 AM sorry. it's more like this
>   367210496, 410
> 385441792, 418
> 405204992, 346
> 427053056, 363
> 444809216, 333
> 462483456, 339
> 423661568, 1559
> 441327616, 465
> 458985472, 305
> 403214336, 1338
> 419254272, 506
>  Rodrigo: and 400mb is like the working set of your app?
> 12:24 AM me: at
> 423661568, 1559, there's a huge lag spike
>   lemme check
>  Rodrigo: those spike can happen
>  me: 530 ish on windows
>   thing is, im not reallocating anything major each iteration
>   *allocating
> 12:25 AM i allocate 1 new continuation per step, and call store() on 200k
> others
>  Rodrigo: why are you calling store on those 200k other?
>  me: yield/sleep
> 12:26 AM Rodrigo: oh, ok
>   well, you realize that the working set is largely related to the living
> set of the objects on the many stacks
>   and GC spikes are not really uncommon
> 12:27 AM me: but i tested another scenario
>   i also have a version of microthreads that doesnt use continuations
>   that just runs() and waits for it to return
>   and that one doesnt have any spikes on mono
> 12:28 AM and memory on that one barely increases
>  Rodrigo: do you mind emails the list with your code and CC Paolo Molaro
>   he's the maintainer of tasklets
>   I'm not familiar with the code
>  me: yea sure. do u mind if i link this chat?
>  Rodrigo: no problem
>  me: k. thank you
> 12:29 AM later
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.ximian.com/pipermail/mono-devel-list/attachments/20090830/c5d692de/attachment.html