[Mono-dev] Webapp stuck with "Key duplication when adding: httpModules"

[Robert Jordan]

On 23.10.2014 13:25, Etienne Champetier wrote:
> After some more investigation, reading many lines of code with a collegue,
> we came to the conclusion that this exception is impossible :)
> (yet the web is stuck on it)

Are you sure that this NameObjectCollection isn't mistakenly accessed
from  multiple threads?

Look System.Web.Configuration.WebConfigurationManager.GetSection
all the way up the stacktrace. There are several scary parts that
deal with locking.

Robert


>
> m_ItemsContainer is an hashtable, and a private field of
> NameObjectCollectionBase
> class (new HashTable(0,StringComparer.Ordinal)).
> The exception tell us that there is already a key 'httpModules' in
> m_ItemsContainer.
> Reading the code the only way we go from BaseSet to BaseAdd is if
> FindFirstMatchedItem('httpModules'), so if (_Item)m_ItemsContainer[
> 'httpModules'] is null,
> so if the value is null (the key exists, we got an exception to prove it :))
>
> if i'm right, we are sure that we have ('httpModules',null) in
> m_ItemsContainer.
>
>
> Now m_ItemsContainer only modifications are:
> 0) m_ItemsContainer = new Hashtable (m_defCapacity, equality_comparer);
>
> 1) m_ItemsContainer.Clear ();
>
> 2) _Item newitem=new _Item(name, value);
>     m_ItemsContainer.Add(name,newitem);
>
> 3) m_ItemsContainer.Remove(name);
>
> So there is no way there is ('httpModules',null) in m_ItemsContainer
>
>
> Please someone tell me where i'm wrong
>
> https://github.com/mono/mono/blob/master/mcs/class/System/System.Collections.Specialized/NameObjectCollectionBase.cs
>
> Thanks
> Etienne
>
>
> 2014-10-21 14:54 GMT+02:00 Etienne Champetier <champetier.etienne at gmail.com>
> :
>
>> Hi,
>>
>> At my firm we are running multiple webapplication using mono (nginx +
>> fastcgi-mono-server2.exe) on a cluster of 4 VM, 2 vcpus.
>> One of them is restarted every day at 1h45 (small memory leak).
>> 3 weeks ago we went from 8VM with 1vcpu (no restart problem) to 4VM with
>> 2vcpus, and now,
>> it's the 3rd time in 2 weeks that on one VM, the restart doesn't work well
>> and we have:
>>
>> [System.ArgumentException]: Key duplication when adding: httpModules
>>    at System.Collections.Hashtable.PutImpl (System.Object key,
>> System.Object value, Boolean overwrite) [0x00000] in <filename unknown>:0
>>    at System.Collections.Hashtable.Add (System.Object key, System.Object
>> value) [0x00000] in <filename unknown>:0
>>    at System.Collections.Specialized.NameObjectCollectionBase.BaseAdd
>> (System.String name, System.Object value) [0x00000] in <filename unknown>:0
>>    at System.Collections.Specialized.NameObjectCollectionBase.BaseSet
>> (System.String name, System.Object value) [0x00000] in <filename unknown>:0
>>    at System.Configuration.ConfigurationSectionCollection.get_Item
>> (System.String name) [0x00000] in <filename unknown>:0
>>    at System.Configuration.Configuration.GetSection (System.String path)
>> [0x00000] in <filename unknown>:0
>>    at System.Web.Configuration.WebConfigurationManager.GetSection
>> (System.String sectionName, System.String path, System.Web.HttpContext
>> context) [0x00000] in <filename unknown>:0
>>    at System.Web.Configuration.WebConfigurationManager.GetSection
>> (System.String sectionName, System.String path) [0x00000] in <filename
>> unknown>:0
>>    at
>> System.Web.Configuration.WebConfigurationManager.GetWebApplicationSection
>> (System.String sectionName) [0x00000] in <filename unknown>:0
>>    at System.Web.HttpApplication.InitOnce (Boolean full_init) [0x00000] in
>> <filename unknown>:0
>>
>>
>> The really cool stuff is that even if it's a 500 error, the returned http
>> code is 200 :)
>>
>> I've already checked and all files are equal (dll, exe, conf), all VM conf
>> are equal (vcpu, memory, ...), ...
>>
>> I'm not able to reproduce the bug (restart under high load (ab -c 50 -n
>> 100000)) so can't tell if the bug is there in the newest version.
>>
>> We are using a version of mono before 3.2.7
>> (92887027be801b012846866208e6b4e362a4fc24) and fastcgi-mono-server2 (xsp)
>> version 2.10.2-1 (old version build on 14/09/12 by someone at my firm)
>> I know we are using old version but it seems that the bug is still there:
>> https://bugzilla.xamarin.com/show_bug.cgi?id=18303
>>
>> I've run lsof on the processes (the stuck one and one working), and by
>> comparing the 2, i see that App_global.asax_XXXXXXX.dll
>> is here in both case, but not  App_Web_XXXXXXXX.dll (only on the working
>> one) (see attachment)
>>
>> I have root access on the VM with the stuck process, and i'm on irc right
>> now (champtar), so if you have some ideas to gather more info to kill this
>> bug
>>
>> Thanks in advance
>>
>> Etienne
>>
>
>
>
> _______________________________________________
> Mono-devel-list mailing list
> Mono-devel-list at lists.ximian.com
> http://lists.ximian.com/mailman/listinfo/mono-devel-list
>