[Mono-dev] Fwd: Native stack trace in mono-2-10

[Rolf Bjarne Kvinge]

Sorry, this was meant for the entire list.


---------- Forwarded message ----------
From: Rolf Bjarne Kvinge <rolf at xamarin.com>
Date: Mon, Sep 12, 2011 at 10:30 PM
Subject: Re: [Mono-dev] Native stack trace in mono-2-10
To: Bassam Tabbara <bassam at symform.com>


Hi,

If you can, using Valgrind is usually the best tool to track down
memory corruption, it will slow down everything immensely though.

See: http://www.mono-project.com/Debugging#Using_Valgrind_on_Mono

I also find it slightly strange that managed code (#14 in your stack
trace) calls monoeg_g_hash_table_lookup, so I'd figure out which frame
that is. Trying to figure out what managed code causes this will
likely make it easier to produce a smaller test case.

Rolf

On Mon, Sep 12, 2011 at 10:21 PM, Bassam Tabbara <bassam at symform.com> wrote:
> Thanks Andres.
>
> We build out of the mono-2-10 branch directly (no tags).
>
> The part that is puzzling to me is why monoeg_g_hash_table_lookup calls mono_magic_trampoline (it doesn't).  Also frame #12 looks very suspiciously close in value to the key parameter of monoeg_g_hash_table_lookup (they are 0x30 apart).
>
> #10 0x00000000004977cd in mono_magic_trampoline (regs=0x600311, code=0x600311 "__icall_wrapper_", arg=0x5f, tramp=0x8 <Address 0x8 out of bounds>) at mini-trampolines.c:590
> #11 0x0000000041d5116a in ?? ()
> #12 0x00007fe15cd65158 in ?? ()
> #13 0x00000000005e2d2b in monoeg_g_hash_table_lookup (hash=0x178c180, key=0x7fe15cd653e8) at ghashtable.c:280
> #14 0x0000000041ff0290 in ?? ()
> #15 0x0000000000000000 in ?? ()
>
> Any pointers on how to proceed here would be appreciated. How do folks on the mono team debug stack corruption like this?
>
> Thanks!
> Bassam
>
> -----Original Message-----
> From: mono-devel-list-bounces at lists.ximian.com [mailto:mono-devel-list-bounces at lists.ximian.com] On Behalf Of Andres G. Aragoneses
> Sent: Monday, September 12, 2011 9:57 AM
> To: mono-devel-list at lists.ximian.com
> Subject: Re: [Mono-dev] Native stack trace in mono-2-10
>
> On 09/11/2011 09:42 AM, Bassam Tabbara wrote:
>> Hello,
>>
>> We are seeing the following crash quite frequently on our test servers:
>>
>> #0 0x00007fe188725165 in raise () from /lib/libc.so.6
>>
>> #1 0x00007fe188727f70 in abort () from /lib/libc.so.6
>>
>> #2 0x0000000000493298 in mono_handle_native_sigsegv (signal=2086645120,
>> ctx=<value optimized out>) at mini-exceptions.c:2245
>>
>> #3 0x00000000004e6c8d in mono_arch_handle_altstack_exception
>> (sigctx=0x7fe18390fac0, fault_addr=<value optimized out>,
>>
>> stack_ovf=0) at exceptions-amd64.c:956
>>
>> #4 0x0000000000417704 in mono_sigsegv_signal_handler (_dummy=11,
>> info=0x7fe18390fbf0, context=0x7fe18390fac0) at mini.c:5881
>>
>> #5 <signal handler called>
>>
>> #6 0x00007fe1887fbc0b in ?? () from /lib/libc.so.6
>>
>> #7 0x000000000041e751 in mono_jit_compile_method_with_opt
>> (method=0x178c180, opt=51472895, ex=0x7fe15cd65040) at mini.c:5342
>>
>> #8 0x000000000041eefe in mono_jit_compile_method (method=0x600311) at
>> mini.c:5403
>>
>> #9 0x0000000000496eaf in common_call_trampoline (regs=0x7fe15cd652f0,
>> code=0x41ff0290 "\353", <incomplete sequence \344>,
>>
>> m=0x178c180, tramp=<value optimized out>, vt=0x0, vtable_slot=0x0,
>> need_rgctx_tramp=0) at mini-trampolines.c:488
>>
>> #10 0x00000000004977cd in mono_magic_trampoline (regs=0x600311,
>> code=0x600311 "__icall_wrapper_", arg=0x5f,
>>
>> tramp=0x8 <Address 0x8 out of bounds>) at mini-trampolines.c:590
>>
>> #11 0x0000000041d5116a in ?? ()
>>
>> #12 0x00007fe15cd65158 in ?? ()
>>
>> #13 0x00000000005e2d2b in monoeg_g_hash_table_lookup (hash=0x178c180,
>> key=0x7fe15cd653e8) at ghashtable.c:280
>>
>> #14 0x0000000041ff0290 in ?? ()
>>
>> #15 0x0000000000000000 in ?? ()
>>
>> The servers is an x64 running Debian Linux 6.
>>
>> The back trace looks odd in that g_hash_table_lookup is calling a
>> trampoline. Is this memory corruption? Any ideas on how to debug this?
>
>
> Have you read http://www.mono-project.com/Gdb ?
>
> Also, mono-2-10 is a branch, are you using the branch or particular
> tag/version? (I recall seeing a bug about trampolines and g_hash_tables
> being already fixed upstream a while ago.)
>
>   Andres
>
> --
>
> _______________________________________________
> Mono-devel-list mailing list
> Mono-devel-list at lists.ximian.com
> http://lists.ximian.com/mailman/listinfo/mono-devel-list
> _______________________________________________
> Mono-devel-list mailing list
> Mono-devel-list at lists.ximian.com
> http://lists.ximian.com/mailman/listinfo/mono-devel-list
>