[Mono-dev] libc heap/object/linked-list failures on ARM platform with/without --trace=all

Alexander Stohr Alexander.Stohr at gmx.de
Fri Aug 6 10:28:11 EDT 2010


running a custom application on Linux (ARM Cortex-A8)
exposed me repeated stability problems. somteimes it
happens in between the first few seconds, other times
it takes much longer. (with a second binary build for x86
the application was run as well but seemed to work fine.)
i am using the OpenEmbedded config flavour
DISTRO="angstrom-2008.1" as the base of my development.

the problem was reproducible for mono 2.6.4, 2.6.7
and daily 2010-08-02. other components like Cairo
or GDIplus were updated in the process but seemed
to have no impact at all - thus no improvement.

in a non-debugged setup it will probably produce
a 1/10 startup failure scenario when upfront the
disk caches are flushed. this is probably leading
to less threads beeing runable at the same time.

errors range from stack traces to glibc complaining
about bad heap to glibc in the form of double free
or other corruption decetion. the most regular
report talks about grafical api calls such aus
putting a bitmap on the screen or selecting a brush.
its in no way deterministic - but it looks like some
linked list in glibc is corrupted by higher levels.

when enabling much of debugging by using the mono
command line option --trace=all then the problem
will happen nearly instantly in the first few
seconds of load.

all together it lets the problem apear as if there
are multiple threads accessing shared resources
and the resources are not locked sufficiently
against possible concurrent access calls from threads.


what methods of debugging and resolving such sort
of a problem do exist? which one raieses fewest
efforts and further is the most promising aproach?

web research showed me that such libc list corruptions
are reported any now and then on misc applications.
but it seemed to me that running an application based
on mono using an openembedded Linux platform has
a quite noticeable deep footprint in the last years.
all i were able to do indicates to me that there
must be a very fundamental problem in there.
to find out if it hits you try out "--trace=all".

regards, Alex.


---------------------------------------------
in the debug case i can see things like this:

Unhandled Exception: System.TypeInitializationException: An exception was thrown by the type initializer for System.Windows.Forms.XplatUI ---> System.ArgumentOutOfRangeException: l
oad factor
Parameter name: loadFactor
  at System.Collections.Hashtable..ctor (Int32 capacity, Single loadFactor, IHashCodeProvider hcp, IComparer comparer) [0x00000] in <filename unknown>:0
  at System.Collections.Hashtable..ctor (Int32 capacity, Single loadFactor) [0x00000] in <filename unknown>:0
  at System.Collections.Hashtable..ctor (Int32 capacity) [0x00000] in <filename unknown>:0
  at System.Windows.Forms.XplatUIX11..ctor () [0x00000] in <filename unknown>:0
  at System.Windows.Forms.XplatUIX11.GetInstance () [0x00000] in <filename unknown>:0
  at System.Windows.Forms.XplatUI..cctor () [0x00000] in <filename unknown>:0
  --- End of inner exception stack trace ---
  at System.Windows.Forms.Application.EnableVisualStyles () [0x00000] in <filename unknown>:0

--------------------
or things like this:

[0x40932490: 1.25394 9] LEAVE: string:CharCopy (char*,char*,int)
[0x40932490: 1.25400 8] LEAVE: string:CharCopy (string,int,string,int,int)
[0x40932490: 1.25409 7] LEAVE: System.Text.StringBuilder:InternalEnsureCapacity (int)
[0x40932490: 1.25418 7] ENTER: string:CharCopy (string,int,string,int,int)() ip: (nil)
[0x40932490: 1.25424 8] ENTER: string:CharCopy (char*,char*,int)() ip: (nil)
[0x40932490: 1.25433 9] ENTER: string:memcpy4 (byte*,byte*,int)() ip: (nil)
[0x40932490: 1.25439 9] LEAVE: string:memcpy4 (byte*,byte*,int)
[0x40932490: 1.25449 8] LEAVE: string:CharCopy (char*,char*,int)
[0x40932490: 1.25455 7] LEAVE: string:CharCopy (string,int,string,int,int)
[0x40932490: 1.25464 6] LEAVE: System.Text.StringBuilder:Append (string)[System.Text.StringBuilder:0x409347d8]
[0x40932490: 1.25479 6] ENTER: System.Text.StringBuilder:Append (string)() ip: (nil)
[0x40932490: 1.25488 7] ENTER: string:CharCopy (string,int,string,int,int)() ip: (nil)
[0x40932490: 1.25497 8] ENTER: string:CharCopy (char*,char*,int)() ip: (nil)
[0x40932490: 1.25504 9] ENTER: string:memcpy2 (byte*,byte*,int)() ip: (nil)
[0x40932490: 1.25513 9] LEAVE: string:memcpy2 (byte*,byte*,int)
[0x40932490: 1.25519 8] LEAVE: string:CharCopy (char*,char*,int)
[0x40932490: 1.25525 7] LEAVE: string:CharCopy (string,int,string,int,int)
[0x40932490: 1.25534 6] LEAVE: System.Text.StringBuilder:Append (string)[System.Text.StringBuilder:0x409347d8]
[0x40932490: 1.25549 6] ENTER: System.Exception:get_StackTrace ()() ip: (nil)
[0x40932490: 1.25558 6] LEAVE: System.Exception:get_StackTrace ()[STRING:null],
[0x40932490: 1.25568 6] ENTER: System.Text.StringBuilder:ToString ()() ip: (nil)
[0x40932490: 1.25574 7] ENTER: string:InternalSetLength (int)() ip: (nil)
[0x40932490: 1.25583 7] LEAVE: string:InternalSetLength (int)
[0x40932490: 1.25592 6] LEAVE: System.Text.StringBuilder:ToString ()[STRING:0x4093e000:System.TypeInitializationException: An exception was thrown by the type initializer for Syste
m.Runtime.Remoting.Contexts.Context ---> System.ArgumentOutOfRangeException: load factor
Prameter name: loadFactor
  at System.Collections.Hashtable..ctor (Int32 capacity, Single loadFactor, IHashCodeProvider hcp, IComparer comparer) [0x00000] in <filename unknown>:0
  at System.Collections.Hashtable..ctor (Int32 capacity, Single loadFactor) [0x00000] in <filename unknown>:0
  at System.Collections.Hashtable..ctor () [0x00000] in <filename unknown>:0
  at System.Runtime.Remoting.Contexts.Context..cctor () [0x00000] in <filename unknown>:0
  --- End of inner exception stack trace ---]
[0x40932490: 1.25620 5] LEAVE: System.Exception:ToString ()[STRING:0x4093e000:System.TypeInitializationException: An exception was thrown by the type initializer for System.Runtime
.Remoting.Contexts.Context ---> System.ArgumentOutOfRangeException: load factor
Parameter name: loadFactor
  at System.Collections.Hashtable..ctor (Int32 capacity, Single loadFactor, IHashCodeProvider hcp, IComparer comparer) [0x00000] in <filename unknown>:0
  at System.Collections.Hashtable..ctor (Int32 capacity, Single loadFactor) [0x00000] in <filename unknown>:0
  at System.Collections.Hashtable..ctor () [0x00000] in <filename unknown>:0
  at System.Runtime.Remoting.Contexts.Context..cctor () [0x00000] in <filename unknown>:0
  --- End of inner exception stack trace ---]
[0x40932490: 1.25650 4] LEAVE: (wrapper runtime-invoke) <Module>:runtime_invoke_object__this__ (object,intptr,intptr,intptr)[System.String:0x4093e000]

Unhandled Exception: System.TypeInitializationException: An exception was thrown by the type initializer for System.Runtime.Remoting.Contexts.Context ---> System.ArgumentOutOfRange
Exception: load factor
Parameter name: loadFactor
  at System.Collections.Hashtable..ctor (Int32 capacity, Single loadFactor, IHashCodeProvider hcp, IComparer comparer) [0x00000] in <filename unknown>:0
  at System.Collections.Hashtable..ctor (Int32 capacity, Single loadFactor) [0x00000] in <filename unknown>:0
  at System.Collections.Hashtable..ctor () [0x00000] in <filename unknown>:0
  at System.Runtime.Remoting.Contexts.Context..cctor () [0x00000] in <filename unknown>:0
  --- End of inner exception stack trace ---

--------------------
or things like this:

[0x40932490: 1.21433 8] LEAVE: string:CharCopy (char*,char*,int)                                                                                                                    
[0x40932490: 1.21457 7] LEAVE: string:CharCopy (string,int,string,int,int)                                                                                                          
[0x40932490: 1.21481 6] LEAVE: System.Text.StringBuilder:Append (string)[System.Text.StringBuilder:0x409347d8]                                                                      
[0x40932490: 1.21512 6] ENTER: System.Exception:get_StackTrace ()() ip: (nil)                                                                                                       
[0x40932490: 1.21539 6] LEAVE: System.Exception:get_StackTrace ()[STRING:null],                                                                                                     
[0x40932490: 1.21564 6] ENTER: System.Text.StringBuilder:ToString ()() ip: (nil)                                                                                                    
[0x40932490: 1.21588 7] ENTER: string:InternalSetLength (int)() ip: (nil)                                                                                                           
[0x40932490: 1.21616 7] LEAVE: string:InternalSetLength (int)                                                                                                                       
[0x40932490: 1.21643 6] LEAVE: System.Text.StringBuilder:ToString ()[STRING:0x4093e000:System.TypeInitializationException: An exception was thrown by the type initializer for System.Runtime.Remoting.Contexts.Context ---> System.ArgumentOutOfRangeException: load factor                                                                                            
Parameter name: loadFactor                                                                                                                                                          
  at System.Collections.Hashtable..ctor (Int32 capacity, Single loadFactor, IHashCodeProvider hcp, IComparer comparer) [0x00000] in <filename unknown>:0                            
  at System.Collections.Hashtable..ctor (Int32 capacity, Single loadFactor) [0x00000] in <filename unknown>:0                                                                       
  at System.Collections.Hashtable..ctor () [0x00000] in <filename unknown>:0                                                                                                        
  at System.Runtime.Remoting.Contexts.Context..cctor () [0x00000] in <filename unknown>:0                                                                                           
  --- End of inner exception stack trace ---]                                                                                                                                       
[0x40932490: 1.21719 5] LEAVE: System.Exception:ToString ()[STRING:0x4093e000:System.TypeInitializationException: An exception was thrown by the type initializer for System.Runtime.Remoting.Contexts.Context ---> System.ArgumentOutOfRangeException: load factor                                                                                                     
Parameter name: loadFactor                                                                                                                                                          
  at System.Collections.Hashtable..ctor (Int32 capacity, Single loadFactor, IHashCodeProvider hcp, IComparer comparer) [0x00000] in <filename unknown>:0                            
  at System.Collections.Hashtable..ctor (Int32 capacity, Single loadFactor) [0x00000] in <filename unknown>:0                                                                       
  at System.Collections.Hashtable..ctor () [0x00000] in <filename unknown>:0                                                                                                        
  at System.Runtime.Remoting.Contexts.Context..cctor () [0x00000] in <filename unknown>:0                                                                                           
  --- End of inner exception stack trace ---]                                                                                                                                       
[0x40932490: 1.21796 4] LEAVE: (wrapper runtime-invoke) <Module>:runtime_invoke_object__this__ (object,intptr,intptr,intptr)[System.String:0x4093e000]                              

Unhandled Exception: System.TypeInitializationException: An exception was thrown by the type initializer for System.Runtime.Remoting.Contexts.Context ---> System.ArgumentOutOfRangeException: load factor                                                                                                                                                              
Parameter name: loadFactor                                                                                                                                                          
  at System.Collections.Hashtable..ctor (Int32 capacity, Single loadFactor, IHashCodeProvider hcp, IComparer comparer) [0x00000] in <filename unknown>:0                            
  at System.Collections.Hashtable..ctor (Int32 capacity, Single loadFactor) [0x00000] in <filename unknown>:0                                                                       
  at System.Collections.Hashtable..ctor () [0x00000] in <filename unknown>:0                                                                                                        
  at System.Runtime.Remoting.Contexts.Context..cctor () [0x00000] in <filename unknown>:0                                                                                           
  --- End of inner exception stack trace ---

--------------------
or things like this:

[0x40932490: 1.22662 8] LEAVE: string:CharCopy (char*,char*,int)
[0x40932490: 1.22672 7] LEAVE: string:CharCopy (string,int,string,int,int)
[0x40932490: 1.22681 6] LEAVE: System.Text.StringBuilder:Append (string)[System.Text.StringBuilder:0x409347d8]
[0x40932490: 1.22696 6] ENTER: System.Exception:get_StackTrace ()() ip: (nil)
[0x40932490: 1.22705 6] LEAVE: System.Exception:get_StackTrace ()[STRING:null],
[0x40932490: 1.22714 6] ENTER: System.Text.StringBuilder:ToString ()() ip: (nil)
[0x40932490: 1.22720 7] ENTER: string:InternalSetLength (int)() ip: (nil)
[0x40932490: 1.22730 7] LEAVE: string:InternalSetLength (int)
[0x40932490: 1.22739 6] LEAVE: System.Text.StringBuilder:ToString ()[STRING:0x4093e000:System.TypeInitializationException: An exception was thrown by the type initializer for System.Runtime.Remoting.Contexts.Context ---> System.ArgumentOutOfRangeException: load factor
Parameter name: loadFactor
  at System.Collections.Hashtable..ctor (Int32 capacity, Single loadFactor, IHashCodeProvider hcp, IComparer comparer) [0x00000] in <filename unknown>:0
  at System.Collections.Hashtable..ctor (Int32 capacity, Single loadFactor) [0x00000] in <filename unknown>:0
  at System.Collections.Hashtable..ctor () [0x00000] in <filename unknown>:0
  at System.Runtime.Remoting.Contexts.Context..cctor () [0x00000] in <filename unknown>:0
  --- End of inner exception stack trace ---]
[0x40932490: 1.22769 5] LEAVE: System.Exception:ToString ()[STRING:0x4093e000:System.TypeInitializationException: An exception was thrown by the type initializer for System.Runtime.Remoting.Contexts.Context ---> System.ArgumentOutOfRangeException: load factor
Parameter name: loadFactor
  at System.Collections.Hashtable..ctor (Int32 capacity, Single loadFactor, IHashCodeProvider hcp, IComparer comparer) [0x00000] in <filename unknown>:0
  at System.Collections.Hashtable..ctor (Int32 capacity, Single loadFactor) [0x00000] in <filename unknown>:0
  at System.Collections.Hashtable..ctor () [0x00000] in <filename unknown>:0
  at System.Runtime.Remoting.Contexts.Context..cctor () [0x00000] in <filename unknown>:0
  --- End of inner exception stack trace ---]
[0x40932490: 1.22797 4] LEAVE: (wrapper runtime-invoke) <Module>:runtime_invoke_object__this__ (object,intptr,intptr,intptr)[System.String:0x4093e000]

Unhandled Exception: System.TypeInitializationException: An exception was thrown by the type initializer for System.Runtime.Remoting.Contexts.Context ---> System.ArgumentOutOfRangeException: load factor
Parameter name: loadFactor
  at System.Collections.Hashtable..ctor (Int32 capacity, Single loadFactor, IHashCodeProvider hcp, IComparer comparer) [0x00000] in <filename unknown>:0
  at System.Collections.Hashtable..ctor (Int32 capacity, Single loadFactor) [0x00000] in <filename unknown>:0
  at System.Collections.Hashtable..ctor () [0x00000] in <filename unknown>:0
  at System.Runtime.Remoting.Contexts.Context..cctor () [0x00000] in <filename unknown>:0
  --- End of inner exception stack trace ---
-- 
GMX DSL SOMMER-SPECIAL: Surf & Phone Flat 16.000 für nur 19,99 ¿/mtl.!*
http://portal.gmx.net/de/go/dsl


More information about the Mono-devel-list mailing list