[Mono-dev] Mono 2.4 crashes due to accessing freed data structures

[Ulrich Weigand]

Zoltan Varga wrote:

> I'm pretty sure these problems are already fixed on the mono 2.4 branch,
> and will be in the upcoming 2.4.2.

Rodrigo Kumpera wrote:

> Can you try OpenSim using mono from the tip of the 2.4 branch?

I've checked out revision 135089 from:
svn co svn://anonsvn.mono-project.com/source/branches/mono-2-4
and re-run the OpenSim build tests.

And indeed it does look like the one set of crashes is no longer there.
However, the *second* problem still seems to be present.


>> @@ -2196,7 +2197,8 @@ inflated_method_in_image (gpointer key,
>>        // https://bugzilla.novell.com/show_bug.cgi?id=458168
>>        return method->declaring->klass->image == image ||
>>                (method->context.class_inst && ginst_in_image
>> (method->context.class_inst, image)) ||
>> -               (method->context.method_inst && ginst_in_image
>> (method->context.method_inst, image)) || signature_in_image
>> (mono_method_signature ((MonoMethod*)method), image);
>> +               (method->context.method_inst && ginst_in_image
>> (method->context.method_inst, image)) ||
>> +               (((MonoMethod*)method)->signature && signature_in_image
>> (((MonoMethod*)method)->signature, image));
>  }

> This has been fixed in trunk and 2.4

I'm no longer able to reproduce this type of crash.  (Just for my
curiosity, could you point me to the change that fixes this?  Looking
at the code it seems to me that there is still a chance the call to
mono_method_signature call might causes allocation of new types ...
What am I overlooking here?)


>> The second problem is related to wrapper classes allocated by the
>> routines in marshal.c.  I've been seeing various instances of crashes
>> caused by those routines returning apparently clobbered method structures.

>Your change isn't required because you only remove from image hash tables,
>which are released together with the inflated method.

*This* problem is still present in the 2.4 branch, with identical
symptoms as what I described in my earlier mail.

What I'm seeing is that in fact the inflated method is released much earlier
than the image hash tables.  The inflated method is allocated on the heap
and entered in the generic_method_cache in metadata.c.  From there it is
released as soon as *any* referenced image is released.  For example, if
a generic method from image A is instantiated with types from image B,
it seems the method wrappers are stored in the hash tables associated with
image A, but the inflated methods are released once image *B* is released.

As a typical example, I'm seeing crashes along the lines of:

Core was generated by `/home/uweigand/steve/mono-2-4-install/bin/mono /home/uweigand/steve/mono-2-4-in'.
Program terminated with signal 6, Aborted.
#0  0x0000040000385840 in .raise () from /lib64/ppc-cell-be/libc.so.6
(gdb) bt
#0  0x0000040000385840 in .raise () from /lib64/ppc-cell-be/libc.so.6
#1  0x0000040000387398 in .abort () from /lib64/ppc-cell-be/libc.so.6
#2  0x0000000010081f44 in mono_handle_native_sigsegv (signal=<value optimized out>, ctx=<value optimized out>)
    at ../../../mono-2-4-branch/mono/mono/mini/mini-exceptions.c:1527
#3  0x00000000100a0e3c in mono_arch_handle_altstack_exception (sigctx=0x40000812780,
    fault_addr=<value optimized out>, stack_ovf=0)
    at ../../../mono-2-4-branch/mono/mono/mini/exceptions-ppc.c:659
#4  0x000000001001b24c in sigsegv_signal_handler (_dummy=<value optimized out>, info=0x40000812e58,
    context=0x40000812780) at ../../../mono-2-4-branch/mono/mono/mini/mini.c:4282
#5  <signal handler called>
#6  mono_compile_create_var (cfg=0x4000435e200, type=0x0, opcode=325)
    at ../../../mono-2-4-branch/mono/mono/mini/mini.c:919
#7  0x000000001001ff58 in mini_method_compile (method=0x4000434d480, opts=0, domain=0x40002368db0, run_cctors=1,
    compile_aot=0, parts=0) at ../../../mono-2-4-branch/mono/mono/mini/mini.c:2686
#8  0x0000000010021b1c in mono_jit_compile_method (method=0x4000434d480)
    at ../../../mono-2-4-branch/mono/mono/mini/mini.c:3830
#9  0x00000000100b66ec in mono_compile_method (method=<value optimized out>)
    at ../../../mono-2-4-branch/mono/mono/metadata/object.c:529
#10 0x0000000010153e34 in ves_icall_System_Delegate_CreateDelegate_internal (type=0x4000dc17738,
    target=0x4000d3a2770, info=0x400028f7c08) at ../../../mono-2-4-branch/mono/mono/metadata/icall.c:5994

(gdb) frame 7
#7  0x000000001001ff58 in mini_method_compile (method=0x4000434d480, opts=0, domain=0x40002368db0, run_cctors=1,
    compile_aot=0, parts=0) at ../../../mono-2-4-branch/mono/mono/mini/mini.c:2686
2686                    cfg->ret = mono_compile_create_var (cfg, sig->ret, OP_ARG);
(gdb) print *method
$1 = {flags = 145, iflags = 0, token = 0, klass = 0x104c2f70, signature = 0x40004379af0,
  name = 0x4000434d4b8 "static_rgctx_invoke_object_MonoProperty/Getter`2<TestFixtureAttribute, string>_object",
  inline_info = 1, inline_failure = 0, wrapper_type = 29, string_ctor = 0, save_lmf = 0, dynamic = 0,
  is_generic = 0, is_inflated = 0, skip_visibility = 1, verification_success = 0, slot = 0}
(gdb) print *method->signature
$2 = {hasthis = 0, explicit_this = 0, call_convention = 0, pinvoke = 0, ref_count = 1024, param_count = 1065,
  sentinelpos = -23536, generic_param_count = 256, is_inflated = 0, has_type_parameters = 0, ret = 0x0,
  params = 0x40004379b08}

--- note the stale method; the memory pointed to by "signature" has been reused.

(gdb) frame 10
#10 0x0000000010153e34 in ves_icall_System_Delegate_CreateDelegate_internal (type=0x4000dc17738,
    target=0x4000d3a2770, info=0x400028f7c08) at ../../../mono-2-4-branch/mono/mono/metadata/icall.c:5994
5994                    func = mono_compile_method (method);
(gdb) list
5989
5990            delegate = mono_object_new (mono_object_domain (type), delegate_class);
5991
5992            if (mono_method_needs_static_rgctx_invoke (method, FALSE)) {
5993                    method = mono_marshal_get_static_rgctx_invoke (method);
5994                    func = mono_compile_method (method);
5995            } else if (method->dynamic) {
5996                    /* Creating a trampoline would leak memory */
5997                    func = mono_compile_method (method);
5998            } else {

--- this stale method is a wrapper resulting from the mono_marshal_get_static_rgctx_invoke call.

Bye,
Ulrich


-- 
  Dr. Ulrich Weigand
  GNU Toolchain for Linux on System z and Cell BE
  Ulrich.Weigand at de.ibm.com