[Mono-dev] JIT compiler crashes application

Thu Dec 6 17:38:24 UTC 2012

It's not about that particular method, we got earlier same kind of crashes
on different methods, for
instance, ProtoBuf.Serializers.ArrayDecorator:Write ()
If I remember correctly, I got this crash even for some method on List<T>,
though I can't locate that log quickly. If I get it, will post here.
Also, this bug manifests on different versions of Mono (< 3.0).

This behavior is very randomly reproducible. To give you some context, we
constantly run "test scenarios" where we start our TestClient in a loop.
TestClient does some work and then exits. Then our shell script starts
TestClient again. And sometimes TestClient crashes with error I described.
What is interesting, crash mostly occurs not on first run of TestClient
during this test scenario. Maybe that can help somehow.

Can using LLVM back-end help to mitigate this?

Thread 2 (Thread 0x7f2c4cfe7700 (LWP 23781)):
#0  0x00007f2ccf3e488d in waitpid () from
/lib/x86_64-linux-gnu/libpthread.so.0
#1  0x000000000049996b in mono_handle_native_sigsegv (signal=<optimized
out>, ctx=<optimized out>) at mini-exceptions.c:2291
#2  0x00000000004ecd7f in mono_arch_handle_altstack_exception
(sigctx=0x7f2cbc143ac0, fault_addr=<optimized out>, stack_ovf=0) at
exceptions-amd64.c:884
#3  0x000000000041c157 in mono_sigsegv_signal_handler (_dummy=11,
info=0x7f2cbc143bf0, context=0x7f2cbc143ac0) at mini.c:6046
#4  <signal handler called>
#5  emit_move_return_value (cfg=0x7f2ca800c9b0, ins=<optimized out>,
code=0x7f2ca8021314 "") at mini-amd64.c:3577
#6  0x00000000004dbeed in mono_arch_output_basic_block (cfg=0x7f2ca800c9b0,
bb=0x7f2ca8003a28) at mini-amd64.c:4877
#7  0x000000000041d26a in mono_codegen (cfg=0x7f2ca800c9b0) at mini.c:3707
#8  0x000000000041e16c in mini_method_compile
(method="ProtoBuf.Serializers.ArrayDecorator:Write ()", opts=34695679,
domain=0x10d8d60, run_cctors=<optimized out>, compile_aot=0, parts=0) at
mini.c:5002
#9  0x000000000041f972 in mono_jit_compile_method_inner
(jit_ex=0x7f2c4cfe64c8, opt=34695679, target_domain=0x10d8d60,
method="ProtoBuf.Serializers.ArrayDecorator:Write ()") at mini.c:5284
#10 mono_jit_compile_method_with_opt
(method="ProtoBuf.Serializers.ArrayDecorator:Write ()", opt=34695679,
ex=0x7f2c4cfe64c8) at mini.c:5538
#11 0x000000000042035d in mono_jit_compile_method (method=<optimized out>)
at mini.c:5566
#12 0x000000000049b488 in common_call_trampoline (regs=0x7f2c4cfe6798,
code=0x415e14c4 "L\213,$L\213t$\bH\203?\030?",
m="ProtoBuf.Serializers.ArrayDecorator:Write ()",
vt=vtable("ProtoBuf.Serializers.ArrayDecorator"), vtable_slot=<optimized
out>, need_rgctx_tramp=0, tramp=<optimized out>) at mini-trampolines.c:483
#13 0x0000000041570e16 in ?? ()
#14 0x00007f2cce0687e0 in ?? ()
#15 0x00007f2cce068540 in ?? ()
#16 0x00000000415e846c in ?? ()
#17 0x00007f2ca800a660 in ?? ()
#18 0x00007f2cce0687e0 in ?? ()
#19 0x00000000005725c6 in mono_delegate_ctor_with_method (this=0x3,
target=0x7f2c4cfe6840, addr=0x0, method=<optimized out>) at object.c:6092
#20 0x0000000000527f19 in ves_icall_System_Delegate_CreateDelegate_internal
(type=0x7f2ccfe55c08, target=0x7f2cce068540, info=<optimized out>,
throwOnBindFailure=<optimized out>) at icall.c:5911
#21 0x0000000040664333 in ?? ()
#22 0x00007f2ca800af20 in ?? ()
#23 0x00007f2cbf4fd0c8 in ?? ()
#24 0x00007f2ca800af21 in ?? ()
#25 0x00007f2c4cfe7660 in ?? ()
#26 0x0000000000000000 in ?? ()

-- Andrii Nakryiko

2012/12/6 Rodrigo Kumpera <kumpera at gmail.com>

> The method in question is EventStore.Transport.Tcp.TcpConnection:EnqueueSend,
> you can check if calling it directly triggers the bug.
>
>
> On Thu, Dec 6, 2012 at 11:20 AM, Greg Young <gregoryyoung1 at gmail.com>wrote:
>
>> Its not reproducible even in our code base. Sometimes we get a sigfault
>> sometimes not. It may happen 20 times in a row and not happen 20 times on
>> an identical box.
>>
>>
>> On Thu, Dec 6, 2012 at 6:13 PM, Rodrigo Kumpera <kumpera at gmail.com>wrote:
>>
>>> Please file a bug report with a reproducible test case.
>>>
>>>
>>>  On Thu, Dec 6, 2012 at 10:44 AM, Andrii Nakryiko <
>>> andrii.nakryiko at gmail.com> wrote:
>>>
>>>> Hi,
>>>>
>>>> We sometimes get runtime crashes during the application run and it
>>>> seems that it happens inside JIT compiler. The crash is not happening
>>>> constantly, just once in a few runs.
>>>>
>>>> We run under Mono 3.0.1 (no/301b6c6 Tue Dec  4 15:13:23 EET 2012) with
>>>> SGen.
>>>>
>>>> Here is the stack trace:
>>>>
>>>> Thread 5 (Thread 0x7f26da5f4700 (LWP 13042)):
>>>> #0  0x00007f26e2fec88d in waitpid () from
>>>> /lib/x86_64-linux-gnu/libpthread.so.0
>>>> #1  0x000000000049a66b in mono_handle_native_sigsegv (signal=<optimized
>>>> out>, ctx=<optimized out>) at mini-exceptions.c:2289
>>>> #2  0x00000000004ee5ff in mono_arch_handle_altstack_exception
>>>> (sigctx=0x7f26e2015ac0, fault_addr=<optimized out>, stack_ovf=0) at
>>>> exceptions-amd64.c:884
>>>> #3  0x000000000041c427 in mono_sigsegv_signal_handler (_dummy=11,
>>>> info=0x7f26e2015bf0, context=0x7f26e2015ac0) at mini.c:6066
>>>> #4  <signal handler called>
>>>> #5  emit_move_return_value (cfg=0x7f26a8006b10, ins=<optimized out>,
>>>> code=0x7f26a80257d0 "\300W\002\250&\177") at mini-amd64.c:3552
>>>> #6  0x00000000004dd76d in mono_arch_output_basic_block
>>>> (cfg=0x7f26a8006b10, bb=0x7f26a8003678) at mini-amd64.c:4853
>>>> #7  0x000000000041d53a in mono_codegen (cfg=0x7f26a8006b10) at
>>>> mini.c:3727
>>>> #8  0x000000000041e43c in mini_method_compile
>>>> (method="EventStore.Transport.Tcp.TcpConnection:EnqueueSend ()", opts=
>>>> 51472895, domain=0xc5cdf0, run_cctors=<optimized out>, compile_aot=0,
>>>> parts=0) at mini.c:5022
>>>> #9  0x000000000041fc42 in mono_jit_compile_method_inner
>>>> (jit_ex=0x7f26da5f36b8, opt=51472895, target_domain=0xc5cdf0,
>>>> method="EventStore.Transport.Tcp.TcpConnection:EnqueueSend ()") at
>>>> mini.c:5304
>>>>  #10 mono_jit_compile_method_with_opt
>>>> (method="EventStore.Transport.Tcp.TcpConnection:EnqueueSend ()", opt=
>>>> 51472895, ex=0x7f26da5f36b8) at mini.c:5558
>>>> #11 0x000000000042062d in mono_jit_compile_method (method=<optimized
>>>> out>) at mini.c:5586
>>>> #12 0x000000000049c228 in common_call_trampoline (regs=0x7f26da5f3988,
>>>> code=0x40bd8718 "H\203\304", <incomplete sequence \303>,
>>>> m="EventStore.Transport.Tcp.TcpConnection:EnqueueSend ()", vt=0x0,
>>>> vtable_slot=<optimized out>, need_rgctx_tramp=0, tramp=<optimized out>) at
>>>> mini-trampolines.c:483
>>>> #13 0x0000000040e48186 in ?? ()
>>>> #14 0x00007f26a8002560 in ?? ()
>>>> #15 0x00007f26d40025f0 in ?? ()
>>>> #16 0x00007f26da5f3a70 in ?? ()
>>>> #17 0x000000000056c09a in mono_thread_interruption_checkpoint_request
>>>> (bypass_abort_protection=-631293392) at threads.c:4183
>>>> #18 0x0000000040e48193 in ?? ()
>>>> #19 0x00007f26e1c30cd8 in ?? ()
>>>> #20 0x0000000000000000 in ?? ()
>>>>
>>>> By looking at mini-amd64.c:3552 it seems that some internal assertion
>>>> is wrong:
>>>>
>>>> 3541: case OP_VCALL: 3542: case OP_VCALL_REG: 3543: case
>>>> OP_VCALL_MEMBASE: 3544: case OP_VCALL2: 3545: case OP_VCALL2_REG: 3546:
>>>> case OP_VCALL2_MEMBASE: 3547: cinfo = get_call_info
>>>> (cfg->generic_sharing_context, cfg->mempool,
>>>> ((MonoCallInst*)ins)->signature); 3548: if (cinfo->ret.storage ==
>>>> ArgValuetypeInReg) { 3549: MonoInst *loc = cfg->arch.vret_addr_loc; 3550:
>>>> 3551: /* Load the destination address */ 3552: g_assert (loc->opcode ==
>>>> OP_REGOFFSET);
>>>>
>>>> Any thought on what's wrong? Can we somehow work around this issue?..
>>>>
>>>> -- Andrii Nakryiko
>>>>
>>>> _______________________________________________
>>>> Mono-devel-list mailing list
>>>> Mono-devel-list at lists.ximian.com
>>>> http://lists.ximian.com/mailman/listinfo/mono-devel-list
>>>>
>>>>
>>>
>>> _______________________________________________
>>> Mono-devel-list mailing list
>>> Mono-devel-list at lists.ximian.com
>>> http://lists.ximian.com/mailman/listinfo/mono-devel-list
>>>
>>>
>>
>>
>> --
>> Le doute n'est pas une condition agréable, mais la certitude est absurde.
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ximian.com/pipermail/mono-devel-list/attachments/20121206/98c75d30/attachment-0001.html>