[Mono-dev] xbuild crash with mono 4.6.1?

Tue Oct 18 21:10:56 UTC 2016

Hello David,

    Based on the crash site, it would seem that we fail to allocate memory from the OS, which means we have an OOM situation, which we poorly handle as of right now.

     Could you check whether this is the case, whether there are mono instances that use more memory than expected. You could consider trying to limit the heap from growing too much by passing MONO_GC_PARAMS=soft-heap-limit=<heap_limit>, unless something is leaking.

Vlad

From: Mono-devel-list <mono-devel-list-bounces at lists.dot.net> on behalf of David Evans <devans at pacificbiosciences.com>
Date: Tuesday, 18 October 2016 at 20:50
To: mono-devel <mono-devel-list at lists.ximian.com>
Subject: [Mono-dev] xbuild crash with mono 4.6.1?

I just started building and testing with mono 4.6.1 and I’m seeing an intermittent crash (memory corruption?) now during the build. Happens for me about one time out of ten. Anyone else seeing this or is there a solution already available that I could patch with? We have a fairly large project which uses a lot of memory, but I haven’t seen anything like this building with 4.0.4.1 or when I tried out 4.2.x or 4.3.x Thanks!

Appears to be crashing in sgen when allocating, hence my corruption guess, here’s the native portion of the dying thread:

1.       Thread 1 (Thread 0x7f5ac59f0780 (LWP 2273)):
2.       #0  0x00007f5ac4eccee9 in __libc_waitpid (pid=pid at entry=2872, stat_loc=stat_loc at entry=0x7f5ac59ff11c, options=options at entry=0) at ../sysdeps/unix/sysv/linux/waitpid.c:40
3.       #1  0x00000000004b0bae in mono_handle_native_sigsegv (signal=signal at entry=11, ctx=ctx at entry=0x7f5ac59ffac0, info=info at entry=0x7f5ac59ffbf0) at mini-exceptions.c:2427
4.       #2  0x000000000050674e in mono_arch_handle_altstack_exception (sigctx=sigctx at entry=0x7f5ac59ffac0, siginfo=siginfo at entry=0x7f5ac59ffbf0, fault_addr=<optimized out>, stack_ovf=stack_ovf at entry=0) at exceptions-amd64.c:795
5.       #3  0x0000000000425af2 in mono_sigsegv_signal_handler (_dummy=11, _info=0x7f5ac59ffbf0, context=0x7f5ac59ffac0) at mini-runtime.c:2865
6.       #4  <signal handler called>
7.       #5  alloc_sb (desc=0x7f5aa9fd88c0) at lock-free-alloc.c:146
8.       #6  alloc_from_new_sb (heap=0x993310 <allocators+400>) at lock-free-alloc.c:411
9.       #7  mono_lock_free_alloc (heap=0x993310 <allocators+400>) at lock-free-alloc.c:440
10.   #8  0x00000000006079ce in sgen_alloc_internal (type=type at entry=14) at sgen-internal.c:206
11.   #9  0x0000000000606aea in sgen_gray_object_alloc_queue_section (queue=queue at entry=0x992f80 <gray_queue>) at sgen-gray.c:58
12.   #10 0x0000000000606b4c in sgen_gray_object_enqueue (queue=queue at entry=0x992f80 <gray_queue>, obj=obj at entry=0x7f57e1c25530, desc=desc at entry=32762) at sgen-gray.c:108
13.   #11 0x0000000000610025 in GRAY_OBJECT_ENQUEUE (desc=<optimized out>, obj=0x7f57e1c25530, queue=0x992f80 <gray_queue>) at ../../mono/sgen/sgen-gray.h:164
14.   #12 major_copy_or_mark_object_no_evacuation (queue=0x992f80 <gray_queue>, obj=<optimized out>, ptr=0x7f57b2a7f580) at sgen-marksweep-drain-gray-stack.h:176
15.   #13 major_scan_object_no_evacuation (queue=0x992f80 <gray_queue>, desc=<optimized out>, full_object=<optimized out>) at sgen-scan-object.h:56
16.   #14 drain_gray_stack_no_evacuation (queue=<optimized out>) at sgen-marksweep-drain-gray-stack.h:309
17.   #15 drain_gray_stack (queue=<optimized out>) at sgen-marksweep.c:1217
18.   #16 0x00000000006023c2 in finish_gray_stack (generation=generation at entry=1, ctx=...) at sgen-gc.c:1074
19.   #17 0x000000000060313c in major_finish_collection (reason=0x702ca3 "LOS overflow", is_overflow=0, old_next_pin_slot=136, forced=0) at sgen-gc.c:1957
20.   #18 0x00000000006034dc in major_do_collection (reason=0x702ca3 "LOS overflow", is_overflow=0, forced=0) at sgen-gc.c:2083
21.   #19 0x0000000000605c6f in sgen_perform_collection (requested_size=3730512, generation_to_collect=1, reason=0x702ca3 "LOS overflow", wait_to_finish=0, stw=1) at sgen-gc.c:2279
22.   #20 0x000000000060602c in sgen_ensure_free_space (size=<optimized out>, generation=<optimized out>) at sgen-gc.c:2232
23.   #21 0x000000000060803e in sgen_los_alloc_large_inner (vtable=vtable at entry=0x1eb8a38, size=size at entry=3730512) at sgen-los.c:379
24.   #22 0x00000000005f93f8 in sgen_alloc_obj_nolock (vtable=vtable at entry=0x1eb8a38, size=size at entry=3730512) at sgen-alloc.c:175
25.   #23 0x00000000005e6901 in mono_gc_alloc_vector (vtable=vtable("Microsoft.Build.Framework.ITaskItem[]"), size=3730512, max_length=466310) at sgen-mono.c:1744
…

And the managed stack looks like:

1.         at <unknown> <0xffffffff>
2.         at (wrapper managed-to-native) object.__icall_wrapper_mono_gc_alloc_vector (intptr,intptr,intptr) <IL 0x00009, 0x0005d>
3.         at (wrapper alloc) object.AllocVector (intptr,intptr) <IL 0x000b3, 0x00143>
4.         at Microsoft.Build.BuildEngine.BuildItemGroup.ConvertToITaskItemArray (Microsoft.Build.BuildEngine.Expression,Microsoft.Build.BuildEngine.Expression,Microsoft.Build.BuildEngine.ExpressionOptions) [0x0001e] in /home/pbi/git/mono/runtime/sources-4.6.1.0/mono-4.6.1/mcs/class/Microsoft.Build.Engine/Microsoft.Build.BuildEngine/BuildItemGroup.cs:256
5.         at Microsoft.Build.BuildEngine.ItemReference.ConvertToITaskItemArray (Microsoft.Build.BuildEngine.Project,Microsoft.Build.BuildEngine.ExpressionOptions) [0x00013] in /home/pbi/git/mono/runtime/sources-4.6.1.0/mono-4.6.1/mcs/class/Microsoft.Build.Engine/Microsoft.Build.BuildEngine/ItemReference.cs:80
6.         at Microsoft.Build.BuildEngine.ExpressionCollection.ConvertToITaskItemArray (Microsoft.Build.BuildEngine.Project,Microsoft.Build.BuildEngine.ExpressionOptions) [0x001cc] in /home/pbi/git/mono/runtime/sources-4.6.1.0/mono-4.6.1/mcs/class/Microsoft.Build.Engine/Microsoft.Build.BuildEngine/ExpressionCollection.cs:228
7.       …

Full traces here:
http://pastebin.com/vQ4630Dv<https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fpastebin.com%2FvQ4630Dv&data=02%7C01%7Cvlbrez%40microsoft.com%7C14c7f03e35954cd37bf608d3f77f5695%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636124098705789846&sdata=E8axBm%2BOnKauibPynBuXu4En5X9%2FrCvIWJewgApwlJA%3D&reserved=0>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.dot.net/pipermail/mono-devel-list/attachments/20161018/313f110b/attachment-0001.html>