[Mono-dev] Need help tracking this bug...

[Alan McGovern]

Hi,

I just compiled the latest SVN head, and once again monotorrent works
perfectly (well, as perfectly as to be expected ;) ). No more crazy memory
usage, which means it was a mono bug after all. I just wish i knew what
exactly had gone wrong inside mono. For the amount of time i spent trying to
figure out what it was, i feel cheated that someone else fixed it before i
could ;)

Anyway, thanks for the help joe, i couldn't have gotten as far as i did
without you.

Alan.

On 3/31/07, Alan McGovern <alan.mcgovern at gmail.com> wrote:
>
> Ok, i did a quick bit of testing on my code.
>
> Firstly, the 54 instances of SocketAsyncResult do make sense. Typically
> there will *always* be a pending BeginReceive on every socket i have opened.
> Therefore 54 instances of SocketAsyncResult would correspond to 54 open
> connections, which is pretty standard stuff.
>
> Every call to socket.BeginXXX or socket.EndXXX goes through a
> TCPConnection class, so i added a count there so i could see how many calls
> to each BeginXXX and EndXXX method i actually make. The results were pretty
> much as expected. There was always a difference between BeginReceive and
> EndReceive pretty much equal to the number of sockets i have open. The
> difference between BeginSend and EndSend was always close enough to zero,
> which would also make sense as most messages i'm sending would only be a few
> bytes in size. The difference between the BeginConnects and EndConnects is
> always 5. This is right as i've set it so i only connect to at most 5 people
> at any one instant.
>
> So i can rule out a bug in my code that's causing millions of BeginXXX's
> without corresponding EndXXX's. There'd typically be between 80 and 120
> Begin/EndReceive calls a second, which isn't that much really. A similar
> amount for Begin/EndSend.
>
>
> On 3/31/07, Alan McGovern <alan.mcgovern at gmail.com> wrote:
> >
> > Hi,
> >
> > That leaves me with the question of how the hell a SocketAsyncResult is
> > 10 megs in size! The size of the largest single object in my entire code is
> > a 16kB byte[] buffer. If each SocketAsyncResult is 10 megabytes in size, i
> > have to question the internal workings of mono, as i know from profiling my
> > own code does *not* create objects anywhere near that size.
> >
> > I'm going to do a bit of profiling to count how many socket BeginXXX
> > calls are made from my own code as compared to the EndXXX calls to see how
> > they match up.
> >
> > Is there any way of finding out what exactly is inside those
> > SocketAsyncResults that is 10 megs in size? I can verify that the exact same
> > code running under Mono 1.2 and earlier does *not* exhibit the same
> > behavior, everything works fine. I only came across this bug after updating
> > my mono installation to 1.2.3. This is why i think it's a mono bug,
> > however i can't reproduce the problem in the form of an NUnit test.
> >
> > Thanks again,
> > Alan.
> >
> > On 3/30/07, Joe Shaw < joe at ximian.com> wrote:
> > >
> > > Hi again,
> > >
> > > On 3/30/07, Joe Shaw <joe at ximian.com> wrote:
> > > > That huge object array, in turn, is referenced by
> > > > System.Collections.Queue -> System.Net.Socket.Socket , specifically
> > > the
> > > > readQ member.  So basically it means that the readQ member in
> > > > System.Net.Socket.Socket is a huge Queue, which internally has an
> > > > object array, which apparently has millions of SocketAsyncResult
> > > > objects inside.  So how those are being allocated?
> > > >
> > > > These objects are created a lot[...]
> > >
> > > I got a little ahead of myself here: I'm obviously looking at the code
> > > for System.Net.Sockets.Socket here.
> > >
> > > > I don't know much about the Socket class and how the async IO
> > > > works, but it boils down to the fact that BeginReceive() is being
> > > > called probably millions of times, but it doesn't look like
> > > Complete()
> > > > is being called enough (or possibly at all) to balance the load.
> > >
> > > I actually noticed something else:
> > >
> > > The object array in question has an average size of 10.6 megs, but it
> > > only holds 56 references to SocketAsyncResult at the time of this
> > > snapshot.  So this seems to indicate to me that the enqueues and
> > > dequeues do ultimately match up, but that the allocation pattern is
> > > bad and probably not interspersed.  That is,
> > >
> > > enqueue, enqueue, dequeue, dequeue, enqueue, enqueue, etc.
> > >
> > > would mean that an array could be as small as 2 items and still work
> > > for N items, assuming a 1:1 match..  But if the pattern is instead,
> > >
> > > enqueue, enqueue, enqueue, enqueue, dequeue, dequeue, etc.
> > >
> > > then it would have to be at least N.  Now pretend N is a million. :)
> > >
> > > Joe
> > >
> >
> >
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.ximian.com/pipermail/mono-devel-list/attachments/20070331/58111b81/attachment.html