[Mono-dev] Shutdown hangs

David Evans devans at pacificbiosciences.com
Mon Jun 13 19:40:09 UTC 2016 

I saw something similar when I was testing 4.2.2 awhile back, a rare hang on Exit with the main thread waiting like yours. I haven't tried 4.2.3 to see if our hang reproduces yet. We're still on 4.0.4 and we don't see it there.

In the hang we saw, I also had one thread in pthread_cond_wait like yours, though we had two threads also in poll() with libusb_handle_events_timeout_completed () further up the stack.

(gdb) info threads
  Id   Target Id         Frame
  16   Thread 0x7f5eb0bff700 (LWP 2763) "mono" pthread_cond_wait@@GLIBC_2.3.2 ()
    at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
  15   Thread 0x7f5eae7e1700 (LWP 2765) "Finalizer" sem_wait () at ../nptl/sysdeps/unix/sysv/linux/x86_64/sem_wait.S:85
  14   Thread 0x7f5ead26b700 (LWP 2784) "Timer-Scheduler" sem_wait () at ../nptl/sysdeps/unix/sysv/linux/x86_64/sem_wait.S:85
 13   Thread 0x7f5eacf2f700 (LWP 2806) "Top" sem_wait () at ../nptl/sysdeps/unix/sysv/linux/x86_64/sem_wait.S:85
  12   Thread 0x7f5eacd2a700 (LWP 2807) "Top" 0x00007f5eb1b48b9d in nanosleep () at ../sysdeps/unix/syscall-template.S:81
  11   Thread 0x7f5e8e07b700 (LWP 2840) "Top" sem_wait () at ../nptl/sysdeps/unix/sysv/linux/x86_64/sem_wait.S:85
* 10   Thread 0x7f5e72fa5700 (LWP 2980) "Top" 0x00007f5eb186112d in poll () at ../sysdeps/unix/syscall-template.S:81
  9    Thread 0x7f5e727a4700 (LWP 2981) "Top" 0x00007f5eb186112d in poll () at ../sysdeps/unix/syscall-template.S:81
  8    Thread 0x7f5e71a7b700 (LWP 3531) "Top" sem_wait () at ../nptl/sysdeps/unix/sysv/linux/x86_64/sem_wait.S:85
  7    Thread 0x7f5e7187a700 (LWP 3532) "Top" sem_wait () at ../nptl/sysdeps/unix/sysv/linux/x86_64/sem_wait.S:85
  6    Thread 0x7f5e706a3700 (LWP 3562) "Top" sem_wait () at ../nptl/sysdeps/unix/sysv/linux/x86_64/sem_wait.S:85
  5    Thread 0x7f5e704a2700 (LWP 3563) "Top" sem_wait () at ../nptl/sysdeps/unix/sysv/linux/x86_64/sem_wait.S:85
  4    Thread 0x7f5e579fc700 (LWP 3594) "Top" sem_wait () at ../nptl/sysdeps/unix/sysv/linux/x86_64/sem_wait.S:85
  3    Thread 0x7f5e577fb700 (LWP 3595) "Top" sem_wait () at ../nptl/sysdeps/unix/sysv/linux/x86_64/sem_wait.S:85
  2    Thread 0x7f5e565f2700 (LWP 3876) "Top" sem_wait () at ../nptl/sysdeps/unix/sysv/linux/x86_64/sem_wait.S:85
  1    Thread 0x7f5eb266f7c0 (LWP 2758) "Top" sem_wait () at ../nptl/sysdeps/unix/sysv/linux/x86_64/sem_wait.S:85

-----Original Message-----
From: mono-devel-list-bounces at lists.ximian.com [mailto:mono-devel-list-bounces at lists.ximian.com] On Behalf Of Chris Swiedler
Sent: Monday, June 13, 2016 9:55 AM
To: mono-devel-list
Subject: [Mono-dev] Shutdown hangs

I'm getting semi-reliable shutdown hangs in a mono command-line app using mono 4.2.3 on Centos 7 64-bit. The app does different things at different times, and the shutdowns only seem to happen when it connects to our SQL server database (using the Mono database bindings, not ODBC). We've also seen an occasional segfault, also seemingly only when doing database work, though those may have gone away since upgrading to 4.2.3.

Does anyone have any thoughts? Pstack output below.

Thanks,
chris


Thread 5 (Thread 0x7f4c777ff700 (LWP 11658)):
#0  pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
#1  0x00000000005f786c in thread_func (thread_data=0x0) at sgen-thread-pool.c:118
#2  0x00007f4c77ffedf5 in start_thread (arg=0x7f4c777ff700) at pthread_create.c:308
#3  0x00007f4c77d2c1ad in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:113

Thread 4 (Thread 0x7f4c75560700 (LWP 11659)):
#0  sem_wait () at ../nptl/sysdeps/unix/sysv/linux/x86_64/sem_wait.S:85
#1  0x000000000061aa37 in mono_sem_wait (sem=sem at entry=0x953c40 <finalizer_sem>, alertable=alertable at entry=1) at mono-semaphore.c:107
#2  0x00000000005a1736 in finalizer_thread (unused=<optimized out>) at gc.c:1096
#3  0x00000000005841e9 in start_wrapper_internal (data=<optimized out>) at threads.c:725
#4  start_wrapper (data=<optimized out>) at threads.c:772
#5  0x0000000000621026 in inner_start_thread (arg=0x7fffda0d42e0) at mono-threads-posix.c:97
#6  0x00007f4c77ffedf5 in start_thread (arg=0x7f4c75560700) at pthread_create.c:308
#7  0x00007f4c77d2c1ad in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:113

Thread 3 (Thread 0x7f4c74fd3700 (LWP 11660)):
#0  sem_wait () at ../nptl/sysdeps/unix/sysv/linux/x86_64/sem_wait.S:85
#1  0x000000000061aa37 in mono_sem_wait (sem=sem at entry=0x7f4c68000940, alertable=alertable at entry=0) at mono-semaphore.c:107
#2  0x000000000061f5ba in mono_thread_info_wait_for_resume (info=<optimized out>) at mono-threads.c:110
#3  mono_thread_info_end_self_suspend () at mono-threads.c:692
#4  0x0000000000583d2c in self_suspend_internal (thread=0x7f4c78a74330) at threads.c:4546
#5  mono_thread_execute_interruption (thread=thread at entry=0x7f4c78a74330) at threads.c:4050
#6  0x0000000000583f50 in mono_thread_interruption_checkpoint_request (bypass_abort_protection=bypass_abort_protection at entry=1) at threads.c:4184
#7  0x0000000000584dae in mono_thread_force_interruption_checkpoint () at threads.c:4221

Thread 2 (Thread 0x7f4c746ff700 (LWP 11661)):
#0  sem_wait () at ../nptl/sysdeps/unix/sysv/linux/x86_64/sem_wait.S:85
#1  0x000000000061aa37 in mono_sem_wait (sem=sem at entry=0x7f4c6c000940, alertable=alertable at entry=0) at mono-semaphore.c:107
#2  0x000000000061f5ba in mono_thread_info_wait_for_resume (info=<optimized out>) at mono-threads.c:110
#3  mono_thread_info_end_self_suspend () at mono-threads.c:692
#4  0x0000000000583d2c in self_suspend_internal (thread=0x7f4c78a74430) at threads.c:4546
#5  mono_thread_execute_interruption (thread=thread at entry=0x7f4c78a74430) at threads.c:4050
#6  0x00000000005851f9 in ves_icall_System_Threading_Thread_Sleep_internal (ms=1000) at threads.c:1206
#17 0x00007f4c756fa2e7 in System_Threading_Thread_Sleep_int (millisecondsTimeout=1000) at /home/cswiedler/mono-main/Extern/mono/source/external/referencesource/mscorlib/system/threading/thread.cs:715

Thread 1 (Thread 0x7f4c78b34780 (LWP 11657)):
#0  0x00007f4c7800599d in nanosleep () at ../sysdeps/unix/syscall-template.S:81
#1  0x0000000000632a5a in monoeg_g_usleep (microseconds=1935890) at gdate-unix.c:53
#2  0x000000000061f2bb in suspend_sync_nolock (id=id at entry=139966347736832, interrupt_kernel=interrupt_kernel at entry=1) at mono-threads.c:870
#3  0x000000000061f732 in mono_thread_info_safe_suspend_and_run (id=139966347736832, interrupt_kernel=interrupt_kernel at entry=1, callback=callback at entry=0x5813c0 <suspend_thread_critical>, user_data=user_data at entry=0x7fffda0d3a10) at mono-threads.c:891
#4  0x0000000000581d29 in suspend_thread_internal (thread=0x7f4c78a74430, interrupt=1) at threads.c:4531
#5  0x00000000005834ff in mono_thread_suspend_all_other_threads () at threads.c:3154
#6  0x0000000000538176 in ves_icall_System_Environment_Exit (result=0) at icall.c:6006
#7  0x0000000040731832 in ?? ()
_______________________________________________
Mono-devel-list mailing list
Mono-devel-list at lists.ximian.com
http://lists.ximian.com/mailman/listinfo/mono-devel-list

More information about the Mono-devel-list mailing list