[Mono-dev] Potential issue with async sockets in Raspberry ARM – Mono 4.4.0

Greg Young gregoryyoung1 at gmail.com
Mon Sep 12 23:59:32 UTC 2016


We dealt with an issue similar to this long ago that was due to
threadpool requests being dropped occasionally. How reproducible is
your issue?

On Mon, Sep 12, 2016 at 11:45 PM, psantosl at codicesoftware.com
<psantosl at codicesoftware.com> wrote:
> Hi Alan,
>
> I'll try to create a small test case. We are experiencing this running a
> Plastic SCM server on raspberry, and a client on a different machine. It
> will take me a few hours to isolate a test case taking just the required
> parts from the plastic network protocol.
>
> Meanwhile: do you have any suggestion to try to check whether my assumption
> is correct? I mean, I suspect somehow the BeginReceive never awakes. I guess
> it is using epoll down the stairs. Any log or something I can build to make
> sure the issue is on my code and not on Mono?
>
> Thanks,
>
> pablo
>
>
> On 9/12/2016 11:37, Alan McGovern wrote:
>
> Can you share the code to repro the issue?
>
> Sent from my iPhone
>
> On 12 Sep 2016, at 09:39, "psantosl at codicesoftware.com"
> <psantosl at codicesoftware.com> wrote:
>
> Thanks for the hint Chris,
>
> We are actually enqueuing the request and doing the actual read on a
> different threadpool.
>
> That being said, we are reproducing this issue with a single client, I mean,
> single thread attending on the server. Not sure what can go wrong, tcp
> connection is still there, client can send, it simply looks like the server
> never wakes up to attend it, randomly.
>
> Any other hints would be appreciated.
>
> Thanks,
>
> pablo
>
> On 9/9/2016 18:11, Chris Swiedler wrote:
>
> From what I understand it’s dangerous to do blocking reads on the callback
> from something like BeginReceive. The callback will happen on a threadpool
> thread, and the blocking reads could then cause the threadpool to be
> exhausted. I don’t know if that’s causing your specific problem (it could,
> if the reads really do block and you have enough of them) but it’s something
> to watch out for.
>
>
>
> chris
>
>
>
> From: Mono-devel-list [mailto:mono-devel-list-bounces at lists.dot.net] On
> Behalf Of psantosl at codicesoftware.com
> Sent: Friday, September 09, 2016 2:42 AM
> To: mono-devel-list at lists.dot.net
> Cc: dude <rdealba at codicesoftware.com>
> Subject: [Mono-dev] Potential issue with async sockets in Raspberry ARM –
> Mono 4.4.0
>
>
>
> Hi,
>
> I’m posting here hoping someone can throw some light into the problem. I'm a
> little bit lost now.
>
> Our server code (Plastic SCM) running on Raspberry on Mono 4.4.0 (exact
> version below) ends up not awaking from socket.BeginReceive after a while.
>
> I mean, client connects and requests data in 4MB chunks, and depending on
> the run, it can transfer a few GB but it ends up not awaking. The client
> just sits waiting on a “socket recv” but the server doesn’t answer.
> Connection is established (can be checked at OS level).
>
> The code could be simplified as follows:
>
>             mSocket.BeginReceive(buffer, 0, 0, SocketFlags.None,
>                 OnMessageReceived, null);
>
> OnMessageReceived => does the EndReceive and then reads data and enqueues
> the request for a threadpool to attend it. Once the request is attented and
> the response sent, BeginReceive is invoked again. Important: all
> “BeginReceive()” calls are done from the same thread which NEVER dies.
>
> All we use the BeginReceive for is to decouple socket from thread, so we do
> not have a 1-1. You see we do pass “zero” as bytes to read, because all we
> want to achieve is to get awakened when new data is received, then just read
> using blocking calls (no async).
>
> I’m asking if there could be something about Mono because I read 4.6 release
> notes (http://www.mono-project.com/docs/about-mono/releases/4.6.0/) and the
> “atomic fixes for ARM64”. Could it be related somehow.
>
> The same code runs on Windows and Linux PCs (even Macs) without issues. We
> use the same code on all our production servers and even our Cloud roles,
> and we are not aware of issues.
> Now we are testing a new faster storage and using Raspberry to check how
> fast it goes on slower hardware.
>
> Complete Mono version:
> Mono JIT compiler version 4.4.0 (tarball Tue Jun 14 13:44:06 UTC 2016)
> Copyright (C) 2002-2014 Novell, Inc, Xamarin Inc and Contributors.
> www.mono-project.com
>         TLS:           __thread
>         SIGSEGV:       normal
>         Notifications: epoll
>         Architecture:  armel,vfp+hard
>         Disabled:      none
>         Misc:          softdebug
>         LLVM:          supported, not enabled.
>         GC:            sgen
> .
>
> Thanks!
>
> Pablo Santos
> www.plasticscm.com
>
>
> _______________________________________________
> Mono-devel-list mailing list
> Mono-devel-list at lists.dot.net
> http://lists.dot.net/mailman/listinfo/mono-devel-list
>
>
>
> _______________________________________________
> Mono-devel-list mailing list
> Mono-devel-list at lists.dot.net
> http://lists.dot.net/mailman/listinfo/mono-devel-list
>



-- 
Studying for the Turing test


More information about the Mono-devel-list mailing list