[Mono-dev] HttpListener stops accepting requests

[Greg Young]

I have to research a bit more and get a local build of mono building
but there are definitely some places here:

https://github.com/mono/mono/blob/master/mcs/class/System/System.Net/EndPointListener.cs#L85

That can break the async loop causing the entire lsitener to stop
accepting requests until its restarted.

Cheers,

Greg

On Wed, Mar 2, 2016 at 4:58 PM, Greg Young <gregoryyoung1 at gmail.com> wrote:
> We have found a situation where HttpListener on mono (reproduced in
> linux/OSX) will permanently stop accepting requests.
>
> The issue involves ulimit. Basically if you get a connection flood
> greater than ulimit the server will stop accepting all requests until
> it is restarted.
>
> After the connection flood you can see the connections were all
> properly killed (I won't put all the ones that were there during the
> flood!):
>
> greg at clown:~$ lsof -p 16651
> lsof: WARNING: can't stat() tracefs file system /sys/kernel/debug/tracing
>       Output information may be incomplete.
> COMMAND     PID USER   FD   TYPE DEVICE  SIZE/OFF     NODE NAME
> eventstor 16651 greg  cwd    DIR    8,2      4096  2105316
> /home/greg/Downloads/EventStore-OSS-Ubuntu-14.04-v3.5.0
> eventstor 16651 greg  rtd    DIR    8,2      4096        2 /
> eventstor 16651 greg  txt    REG    8,2  37443029  2105361
> /home/greg/Downloads/EventStore-OSS-Ubuntu-14.04-v3.5.0/eventstored
> eventstor 16651 greg  mem    REG    8,2     92504 12849525
> /lib/x86_64-linux-gnu/libgcc_s.so.1
> eventstor 16651 greg  mem    REG    8,2     51736 12845148
> /lib/x86_64-linux-gnu/libnss_files-2.21.so
> eventstor 16651 greg  mem    REG    8,2     47616 12845159
> /lib/x86_64-linux-gnu/libnss_nis-2.21.so
> eventstor 16651 greg  mem    REG    8,2     97224 12845153
> /lib/x86_64-linux-gnu/libnsl-2.21.so
> eventstor 16651 greg  mem    REG    8,2     35688 12845149
> /lib/x86_64-linux-gnu/libnss_compat-2.21.so
> eventstor 16651 greg  mem    REG    8,2   8464208  9444549
> /usr/lib/locale/locale-archive
> eventstor 16651 greg  mem    REG    8,2   1869392 12845162
> /lib/x86_64-linux-gnu/libc-2.21.so
> eventstor 16651 greg  mem    REG    8,2    142080 12845083
> /lib/x86_64-linux-gnu/libpthread-2.21.so
> eventstor 16651 greg  mem    REG    8,2     14592 12845158
> /lib/x86_64-linux-gnu/libdl-2.21.so
> eventstor 16651 greg  mem    REG    8,2     31680 12845075
> /lib/x86_64-linux-gnu/librt-2.21.so
> eventstor 16651 greg  mem    REG    8,2   1084840 12845161
> /lib/x86_64-linux-gnu/libm-2.21.so
> eventstor 16651 greg  mem    REG    8,2    154376 12845156
> /lib/x86_64-linux-gnu/ld-2.21.so
> eventstor 16651 greg  mem    REG   0,21      4096       88 /dev/shm/mono.16651
> eventstor 16651 greg    0u   CHR  136,1       0t0        4 /dev/pts/1
> eventstor 16651 greg    1u   CHR  136,1       0t0        4 /dev/pts/1
> eventstor 16651 greg    2u   CHR  136,1       0t0        4 /dev/pts/1
> eventstor 16651 greg    3r  0000   0,11         0     7966 anon_inode
> eventstor 16651 greg    4u   REG    8,2         8  2624038
> /home/greg/Downloads/EventStore-OSS-Ubuntu-14.04-v3.5.0/data/writer.chk
> eventstor 16651 greg    5u   REG    8,2         8  2624039
> /home/greg/Downloads/EventStore-OSS-Ubuntu-14.04-v3.5.0/data/chaser.chk
> eventstor 16651 greg    6u   REG    8,2         8  2624040
> /home/greg/Downloads/EventStore-OSS-Ubuntu-14.04-v3.5.0/data/epoch.chk
> eventstor 16651 greg    7u   REG    8,2         8  2624041
> /home/greg/Downloads/EventStore-OSS-Ubuntu-14.04-v3.5.0/data/truncate.chk
> eventstor 16651 greg    8r   CHR    1,9       0t0       11 /dev/urandom
> eventstor 16651 greg    9u   REG    8,2 268435712  2624042
> /home/greg/Downloads/EventStore-OSS-Ubuntu-14.04-v3.5.0/data/chunk-000000.000000
> eventstor 16651 greg   10r   REG    8,2 268435712  2624042
> /home/greg/Downloads/EventStore-OSS-Ubuntu-14.04-v3.5.0/data/chunk-000000.000000
> eventstor 16651 greg   11r   REG    8,2 268435712  2624042
> /home/greg/Downloads/EventStore-OSS-Ubuntu-14.04-v3.5.0/data/chunk-000000.000000
> eventstor 16651 greg   12r   REG    8,2 268435712  2624042
> /home/greg/Downloads/EventStore-OSS-Ubuntu-14.04-v3.5.0/data/chunk-000000.000000
> eventstor 16651 greg   13r   REG    8,2 268435712  2624042
> /home/greg/Downloads/EventStore-OSS-Ubuntu-14.04-v3.5.0/data/chunk-000000.000000
> eventstor 16651 greg   14r   REG    8,2 268435712  2624042
> /home/greg/Downloads/EventStore-OSS-Ubuntu-14.04-v3.5.0/data/chunk-000000.000000
> eventstor 16651 greg   15u  IPv4 279150       0t0      TCP
> localhost:1113 (LISTEN)
> eventstor 16651 greg   16u  0000   0,11         0     7966 anon_inode
> eventstor 16651 greg   17u  IPv4 278173       0t0      TCP
> localhost:2113 (LISTEN)
>
>
> My guess is that the too many files kills the asynchronous loop around
> the accept in the httplistener code and will take a look there.
> Perhaps someone else has some ideas.
>
> Cheers,
>
> Greg
>
> --
> Studying for the Turing test



-- 
Studying for the Turing test