[Mono-bugs] [Bug 49499][Cri] New - mono:: segfaults under Fedora
Linux
bugzilla-daemon at bugzilla.ximian.com
bugzilla-daemon at bugzilla.ximian.com
Thu Jun 30 13:09:13 EDT 2005
Please do not reply to this email- if you want to comment on the bug, go to the
URL shown below and enter your comments there.
Changed by matt at use.net.
http://bugzilla.ximian.com/show_bug.cgi?id=49499
--- shadow/49499 2005-06-30 13:09:13.000000000 -0400
+++ shadow/49499.tmp.27178 2005-06-30 13:09:13.000000000 -0400
@@ -0,0 +1,469 @@
+Bug#: 49499
+Product: Mono: Runtime
+Version: unspecified
+OS: unknown
+OS Details: Fedora Linux 0.94 (Severn)
+Status: NEEDINFO
+Resolution: FIXED
+Severity: 001 One hour
+Priority: Critical
+Component: misc
+AssignedTo: mono-bugs at ximian.com
+ReportedBy: raphael.schmid at gmx.de
+QAContact: mono-bugs at ximian.com
+TargetMilestone: ---
+URL:
+Cc:
+Summary: mono:: segfaults under Fedora Linux
+
+Description of Problem:
+When trying to execute any .exe with mono::, it gives a Segmentation Fault.
+Calling `mono' with no parameters at all correctly displays the usage
+information.
+
+Steps to reproduce the problem:
+1. Install Fedora Linux 0.94 (Severn)
+2. Compile mono:: (mono-0.28)
+3. Try something like mono /usr/bin/mcs.exe
+
+How often does this happen?
+Every time.
+
+Additional Information:
+My machine is an AMD Duron. This Severn installation has been enhanced with
+diverse Multimedia applications, but no core packages have been updated. Here
+is a backtrace with gdb:
+
+---< snip >---
+[root at proactivity mcs-0.28]# gdb --args mono /usr/bin/mcs.exe
+GNU gdb Red Hat Linux (5.3.90-0.20030710.21rh)
+Copyright 2003 Free Software Foundation, Inc.
+GDB is free software, covered by the GNU General Public License, and you are
+welcome to change it and/or distribute copies of it under certain conditions.
+Type "show copying" to see the conditions.
+There is absolutely no warranty for GDB. Type "show warranty" for details.
+This GDB was configured as "i386-redhat-linux-gnu"...Using host
+libthread_db lib rary "/lib/tls/libthread_db.so.1".
+
+(gdb) run
+Starting program: /usr/bin/mono /usr/bin/mcs.exe
+[Thread debugging using libthread_db enabled]
+[New Thread -1084512640 (LWP 23751)]
+[New Thread 5434288 (LWP 23753)]
+
+Program received signal SIGSEGV, Segmentation fault.
+[Switching to Thread -1084512640 (LWP 23751)]
+0x001408d1 in mono_jit_runtime_invoke (method=0x82ad460, obj=0x0, params=0x0,
+ exc=0xbfeede44) at mini.c:7239
+7239 return runtime_invoke (obj, params, exc);
+(gdb) up
+#1 0x0018e642 in mono_runtime_invoke (method=0x82ad460, obj=0x0, params=0x0,
+ exc=0xbfeede44) at object.c:711
+711 return default_mono_runtime_invoke (method, obj, params, exc);
+(gdb) up
+#2 0x0018d3c3 in mono_runtime_class_init (vtable=0x82a9ec4) at object.c:108
+108 mono_runtime_invoke (method, NULL, NULL,
+(MonoObject **) &exc);
+(gdb) up
+#3 0x00140822 in mono_jit_compile_method (method=0x83471f8) at mini.c:7214
+7214 mono_runtime_class_init (mono_class_vtable (target_domain,
+method->klass));
+(gdb) up
+#4 0x0018d5e0 in mono_compile_method (method=0x83471f8) at object.c:180
+180 return default_mono_compile_method (method);
+(gdb) up
+#5 0x001405ce in mono_jit_compile_method (method=0x8345f68) at mini.c:7160
+7160 method->info = mono_compile_method
+(nm);
+(gdb) up
+#6 0x0018d5e0 in mono_compile_method (method=0x8345f68) at object.c:180
+180 return default_mono_compile_method (method);
+(gdb) up
+#7 0x0017e657 in mono_arch_create_jit_trampoline (method=0x8345f68)
+ at tramp-x86.c:405
+405 method->info = mono_compile_method (method);
+(gdb) up
+#8 0x0018dfcc in mono_class_vtable (domain=0x82ecf80, class=0x82c34f8)
+ at object.c:515
+515 vt->vtable [i] = arch_create_jit_trampoline
+(cm);
+(gdb) up
+#9 0x0018fea1 in mono_string_new_size (domain=0x82ecf80, len=5)
+ at object.c:1601
+1601 vtable = mono_class_vtable (domain,
+mono_defaults.string_class);
+(gdb) up
+#10 0x0018fe19 in mono_string_new_utf16 (domain=0x82ecf80, text=0x833cd58,
+ len=5) at object.c:1580
+1580 s = mono_string_new_size (domain, len);
+(gdb) up
+#11 0x0018ffd2 in mono_string_new (domain=0x82ecf80, text=0x2164c0 "dummy")
+ at object.c:1663
+1663 o = mono_string_new_utf16 (domain, ut, items_written);
+(gdb) up
+#12 0x0019002f in mono_string_new_wrapper (text=0x2164c0 "dummy")
+ at object.c:1686
+1686 return mono_string_new (domain, text);
+(gdb) up
+#13 0x001a2d35 in mono_marshal_get_runtime_invoke (method=0x831a730)
+ at marshal.c:1728
+1728 string_dummy = mono_string_new_wrapper ("dummy");
+(gdb) up
+#14 0x001408ab in mono_jit_runtime_invoke (method=0x831a730, obj=0x82f7fc0,
+ params=0xbfeee190, exc=0x0) at mini.c:7237
+7237 invoke = mono_marshal_get_runtime_invoke (method);
+(gdb) up
+#15 0x0018e642 in mono_runtime_invoke (method=0x831a730, obj=0x82f7fc0,
+ params=0xbfeee190, exc=0x0) at object.c:711
+711 return default_mono_runtime_invoke (method, obj, params, exc);
+(gdb) up
+#16 0x001b1829 in mono_domain_fire_assembly_load (assembly=0x8328ed8,
+ user_data=0x0) at appdomain.c:458
+458 mono_runtime_invoke (method, domain->domain, params, NULL);
+(gdb) up
+#17 0x001bcdca in mono_assembly_invoke_load_hook (ass=0x8328ed8)
+ at assembly.c:235
+235 hook->func (ass, hook->user_data);
+(gdb) up
+#18 0x001bd6c1 in mono_assembly_open (
+ filename=0x8325e98 "/usr/lib/System.Xml.dll", status=0xbfeee4c4)
+ at assembly.c:562
+562 mono_assembly_invoke_load_hook (ass);
+(gdb) up
+#19 0x001bd869 in mono_assembly_load (aname=0xbfeee2a0,
+ basedir=0x8323e48 "/usr/lib", status=0xbfeee4c4) at assembly.c:603
+603 result = mono_assembly_open (fullpath, status);
+(gdb) up
+#20 0x001bccc3 in mono_assembly_load_references (image=0x8322ae0,
+ status=0xbfeee4c4) at assembly.c:204
+204 image->references [i] = mono_assembly_load (&aname,
+image->assembly->basedir, status);
+(gdb) up
+#21 0x001bd4f8 in mono_assembly_open (
+ filename=0x8322950 "/usr/lib/System.dll", status=0xbfeee4c4)
+ at assembly.c:520
+520 mono_assembly_load_references (image, status);
+(gdb) up
+#22 0x001bcb15 in load_in_path (basename=0x8321a98 "System.dll",
+ search_path=0x240f4c, status=0xbfeee4c4) at assembly.c:125
+125 result = mono_assembly_open (fullpath, status);
+(gdb) up
+#23 0x001bd8ed in mono_assembly_load (aname=0xbfeee420,
+ basedir=0x8321960 "/usr/bin", status=0xbfeee4c4) at assembly.c:617
+617 result = load_in_path (filename, default_path, status);
+(gdb) up
+#24 0x001bccc3 in mono_assembly_load_references (image=0x8320d90,
+ status=0xbfeee4c4) at assembly.c:204
+204 image->references [i] = mono_assembly_load (&aname,
+image->assembly->basedir, status);
+(gdb) up
+#25 0x001bd4f8 in mono_assembly_open (filename=0xbff40ad5 "/usr/bin/mcs.exe",
+ status=0xbfeee4c4) at assembly.c:520
+520 mono_assembly_load_references (image, status);
+(gdb) up
+#26 0x00162d36 in mono_main (argc=2, argv=0xbfeee674) at driver.c:685
+685 assembly = mono_assembly_open (aname, NULL);
+(gdb) up
+#27 0x08048722 in main (argc=2, argv=0xbfeee674) at main.c:6
+6 return mono_main (argc, argv);
+(gdb) up
+Initial frame selected; you cannot go up.
+(gdb)
+---< snap >---
+
+If I can further assist in resolving this problem, please drop me a mail.
+
+- Raphael
+
+------- Additional Comments From richard.torkar at htu.se 2003-10-17 09:30 -------
+
+This problem can be reproduced with latest Fedora Linux 0.95 (Severn)
+and latest CVS (2003-10-17).
+
+If I try to inspect the method's name I get this in gdb:
+(gdb) inspect ((MonoMethod *)method)->name
+$4 = 0x8904c483 <Address 0x8904c483 out of bounds>
+
+Hmm strange. I'll try to look into this a bit more.
+
+------- Additional Comments From vargaz at freemail.hu 2003-10-17 10:13 -------
+If you try to inspect variables in a core file, it will not work,
+because a lot of the runtime variables point into the metadata inside
+the assemblies, and these are mmap-ed instead of loaded into allocated
+memory. So they do not exist in the core file.
+
+------- Additional Comments From dick at ximian.com 2003-10-17 10:21 -------
+It's to do with how memory segments are now mapped by default, if I
+recall correctly what Alan told me (i need to hassle him again to
+email me the details.)
+
+I think we need to map the dynamically generated code with MAP_EXEC.
+
+
+------- Additional Comments From richard.torkar at htu.se 2003-10-17 14:17 -------
+Ahaa, thanks a lot Zoltan and Dick! No need to spend hours trying to
+debug something that can't be debugged this way then :)
+
+------- Additional Comments From richard.torkar at htu.se 2003-10-17 16:12 -------
+[root@/home/torkar]# echo 0 > /proc/sys/kernel/exec-shield
+[torkar@~]$ mcs
+error CS2008: No files to compile were specified
+Compilation failed: 1 error(s), 0 warnings
+
+That's it... But if it's enabled as default in upcoming releases we
+will have a lot of people complaining about mcs giving a segfault...
+
+------- Additional Comments From tds00mahi at thn.htu.se 2003-10-17 20:26 -------
+Concurring with what was already suggested - that this is because of
+the Exec-Shield implementation in Fedora (
+http://lists.insecure.org/lists/linux-kernel/2003/May/0371.html ), I
+am attaching a patch suggestion below with what I believe could be one
+solution. The call to mmap() will not set execution rights even if
+passed as argument (Exec-Shield prevents implicit executable rights
+commonly assumed from the (at least on x86) merged 'read or execute'
+flag (details in link above)), so by adding a call to mprotect() we
+can set PROT_EXEC to allow execution rights on the mapped area.
+
+------- Additional Comments From tds00mahi at thn.htu.se 2003-10-17 20:26 -------
+Created an attachment (id=5655)
+suggested patch
+
+
+------- Additional Comments From richard.torkar at htu.se 2003-10-17 20:31 -------
+Works for me [tm]
+
+I compiled mono/mcs/gtk# with this patch applied.
+
+------- Additional Comments From vargaz at freemail.hu 2003-10-18 11:36 -------
+I fail to see how that change can help. The mmap is used for
+mapping the assembly file into memory, but the assembly contains
+metadata, not executable code, so mapping it with MAP_EXEC should
+change nothing. What dick was talking about is changing the memory
+protection used for the memory which contains the JITted code. I will
+do that.
+
+Are you sure you didn't turn off the exec-shield stuff you mentioned ?
+That would explain why mono worked with that (inchange) change.
+
+
+------- Additional Comments From richard.torkar at htu.se 2003-10-19 08:05 -------
+
+That patch works for me.
+
+When I have that patch applied everything works even when exec-shield
+is set to 1.
+
+------- Additional Comments From vargaz at freemail.hu 2003-10-19 09:12 -------
+Oops, I meant: 'exec shild set to 2'. The '1' value is
+
+exec-shield=1 - default disabled, except binaries that enable it
+
+Does it work for you if you set it to '2' ?
+
+
+------- Additional Comments From richard.torkar at htu.se 2003-10-19 11:36 -------
+
+echo 0 > /proc/sys/kernel/exec-shield
+
+PATCHED:
+$ mcs
+error CS2008: No files to compile were specified
+Compilation failed: 1 error(s), 0 warnings
+
+UNPATCHED:
+$ mcs
+error CS2008: No files to compile were specified
+Compilation failed: 1 error(s), 0 warnings
+
+
+echo 1 > /proc/sys/kernel/exec-shield
+
+PATCHED:
+$ mcs
+error CS2008: No files to compile were specified
+Compilation failed: 1 error(s), 0 warnings
+
+UNPATCHED:
+
+/home/torkar/foo/bin/mcs: line 2: 11019 Segmentation fault
+/home/torkar/foo/bin/mono /home/torkar/foo/bin/mcs.exe "$@"
+
+
+echo 2 > /proc/sys/kernel/exec-shield
+
+PATCHED:
+$ mcs
+error CS2008: No files to compile were specified
+Compilation failed: 1 error(s), 0 warnings
+
+UNPATCHED:
+/home/torkar/foo/bin/mcs: line 2: 11059 Segmentation fault
+/home/torkar/foo/bin/mono /home/torkar/foo/bin/mcs.exe "$@"
+
+Does this answer your question? ;-)
+I don't know if the patch does it the *right* way, but it definitely
+works.
+
+------- Additional Comments From vargaz at freemail.hu 2003-10-20 08:38 -------
+Well, I patched my kernel with the exec-shield patch, but couldn't make
+mono to crash, so I can't find out what's happening :(. Anyway, your
+patch has been checked in, so marking this fixed.
+
+------- Additional Comments From lupus at ximian.com 2003-10-20 09:36 -------
+malte, torkar: can you check if just adding PROT_EXEC to the mmap call
+is enough? Doing another syscall is expensive, and it should really be
+no different. Still, this is weird, since we don't execute code from
+the PE binaries: there must be some other bug (maybe in RH's kernel).
+Do you get the same backtrace for the segfaults as Raphael? Can you
+print in gdb the variables accessed on the segfault line (method->name
+and klass)?
+
+------- Additional Comments From tds00mahi at thn.htu.se 2003-10-20 12:20 -------
+I had myself confused here for a while why the patch actually worked,
+I think I might have a better solution than mprotect - mostly because
+the mprotect one is downright confusing.
+
+Adding the PROT_EXEC to the mmap call did not work, honestly I don't
+know why but from what I understand of exec-shield it moves mappings
+around and only allows execution rights within certain memory areas
+(or, I'm way off, in which case you may now start to point and laugh :-).
+
+Eventually I came across the exec-shield notes here:
+http://distro.ibiblio.org/pub/Linux/distributions/redhat/rawhide/i386/RELEASE-NOTES
+. And by doing an 'export LDFLAGS=-Wl,-z,execstack' to mark the
+binaries as having executable stacks (I reckon exec-shield also
+prevents exec rights on mmap()ed areas and malloc()ed heap, but I'm
+not sure if this flag revokes those rights as well?) I get successful
+results. So I guess this is the preferred way to do it? I don't know
+if this should be passed all the time or if we only should do it if it
+is compiled with a toolchain that specifies the .note.GNU-stack and
+PT_GNU_STACK.
+
+------- Additional Comments From tds00mahi at thn.htu.se 2003-10-20 13:00 -------
+
+To answer the question, the backtrace I see here without either
+mprotect() or linker flag is the same as Raphaels and printing the
+names look like this:
+
+(gdb) print method->name
+$1 = 0x8904c483 <Address 0x8904c483 out of bounds>
+(gdb) print method->klass
+$2 = (MonoClass *) 0x74c08510
+(gdb) print method->klass->name
+Cannot access memory at address 0x74c08538
+(gdb) print method->klass->name_space
+Cannot access memory at address 0x74c0853c
+
+
+------- Additional Comments From tds00mahi at thn.htu.se 2003-10-20 14:03 -------
+One thing to note about the linker flag variant, is that mono works in
+exec-shield levels 0, 1 and 3(!) only. With the mprotect thing it does
+2 as well. These are the levels;
+
+ exec-shield=0 - always-disabled
+ exec-shield=1 - default disabled, except binaries that enable it
+ exec-shield=2 - default enabled, except binaries that disable it
+ exec-shield=3 - always-enabled
+
+
+Using the linker flag fix:
+
+[root@/home/malte/test-execstack-ldflags/bin]# echo 0 >
+/proc/sys/kernel/exec-shield
+[root@/home/malte/test-execstack-ldflags/bin]# ./mono mcs.exe
+error CS2008: No files to compile were specified
+Compilation failed: 1 error(s), 0 warnings
+
+[root@/home/malte/test-execstack-ldflags/bin]# echo 1 >
+/proc/sys/kernel/exec-shield
+[root@/home/malte/test-execstack-ldflags/bin]# ./mono mcs.exe
+error CS2008: No files to compile were specified
+Compilation failed: 1 error(s), 0 warnings
+
+[root@/home/malte/test-execstack-ldflags/bin]# echo 2 >
+/proc/sys/kernel/exec-shield
+[root@/home/malte/test-execstack-ldflags/bin]# ./mono mcs.exe
+Segmentation fault
+
+[root@/home/malte/test-execstack-ldflags/bin]# echo 3 >
+/proc/sys/kernel/exec-shield
+[root@/home/malte/test-execstack-ldflags/bin]# ./mono mcs.exe
+error CS2008: No files to compile were specified
+Compilation failed: 1 error(s), 0 warnings
+
+
+------- Additional Comments From dick at ximian.com 2003-10-21 08:12 -------
+Here's some notes I got from Dave Jones at Red Hat:
+
+You're probably better off asking Ingo <mingo at redhat.com>
+Here's what I've managed to get out of him so far..
+
+they should either allocate those dynamic buffers via mmap(PROT_EXEC)
+(best method), or should do a mprotect(PROT_EXEC) over the affected
+pages (acceptable method), or should define the app to have an
+executable stack (worst method).
+
+how do these apps work on amd64/ia64?
+(I guess Ingo means given they too have non executable stacks)
+
+ Dave
+
+
+------- Additional Comments From richard.torkar at htu.se 2003-10-21 10:35 -------
+
+Just to get the facts straightened out. Would this mean that:
+
+a) mmap(PROT_EXEC) is broken in glibc 2.3.2? At least the way we use it.
+
+b) malte's patch (using mprotect) is the right way to do it at the moment?
+
+b) we should check to see if this actually is a glibc bug?
+
+
+------- Additional Comments From lupus at ximian.com 2003-10-21 10:50 -------
+We don't execute code from the stack, AFAIK, so we should not be
+affected by the execstack thing.
+Ingo just released an updated exec shield patchset: it's reported to
+fix random segfaults (in programs like ls, no less...) so maybe the
+people that experience the issue should try with that. This looks more
+and more like a kernel bug.We don't execute code from the stack,
+AFAIK, so we should not be affected by the execstack thing.
+
+As for the bullet points:
+a) mprotect just seems to workaround a kernel bug: the inconsistent
+results from the different exec shield level seems to prove somthing
+weird is going on
+b) it's not the right way, since we don't execute code from those
+pages, but it appears to make the kernel behave.
+c) it's likely a kernel bug: people should try the latest ingo patch
+(currently -G4) to see if that fixes it, else this should probably be
+reported to him.
+
+------- Additional Comments From richard.torkar at htu.se 2003-10-21 13:05 -------
+Since the kernel in Fedora Linux is updated fairly regular I will try
+this out every now and then to confirm what you say Paolo.
+
+So stay tuned!
+
+------- Additional Comments From jluke at cfl.rr.com 2003-10-21 18:50 -------
+I was also experiencing this (or something similar) in Gentoo, it was
+fixed when I upgraded glibc, which I believe had updated the nptl part
+
+------- Additional Comments From richard.torkar at htu.se 2003-11-13 08:04 -------
+Just a small note (to show paolo I haven't forgotten this bug, and
+that I'll remove it immediately when it is fixed in Fedora, since it
+is uses an expensive syscall).
+
+I've tested it with Fedora Core 1 (glibc-2.3.2-101,
+kernel-2.4.22-1.2115.nptl).
+
+If I remove the "fix" mcs still crashes, so atm it has not been fixed
+in Fedora. I'll continiously check this, whenever glibc/kernel
+upgrades in Fedora.
+
+------- Additional Comments From dick at ximian.com 2003-11-30 05:19 -------
+*** Bug 51505 has been marked as a duplicate of this bug. ***
+
+------- Additional Comments From matt at use.net 2005-06-30 13:09 -------
+Given that this is on an ancient and unsupported Fedora distro, and
+we know it does work on newer ones, can this be closed?
More information about the mono-bugs
mailing list