[Mono-dev] Using valgrind with Mono

Wed Mar 30 00:14:34 UTC 2016

This is the PR in question: https://github.com/mono/mono/pull/2783

It probably won't make into 4.2, but should definitely be in 4.4.

On Tue, Mar 29, 2016 at 11:55 AM, Zinkevicius, Matt <
matt.zinkevicius at hpe.com> wrote:

> Hi Rodrigo,
>
>
>
> Is the following the PR that you were referring to that fixes the
> “mono_method_get_header” leak?
>
>
>
> https://github.com/mono/mono/pull/2705
>
>
>
> I am eager to find it, as this particular leak seems to be the most
> egregious of the ones we’re experiencing, in terms of both occurrences and
> total bytes leaked (hundreds of MB/hour when under load).
>
>
>
> If so, I’ll try to backport it to Mono 4.2.3, though it is far from
> patching cleanly currently.
>
>
>
> Matt
>
>
>
> *From:* mono-devel-list-bounces at lists.ximian.com [mailto:
> mono-devel-list-bounces at lists.ximian.com] *On Behalf Of *Zinkevicius, Matt
> *Sent:* Monday, March 28, 2016 2:42 PM
> *To:* Rodrigo Kumpera <kumpera at gmail.com>
>
> *Cc:* Straw, David (Storage) <david.straw at hpe.com>;
> mono-devel-list at lists.ximian.com
> *Subject:* Re: [Mono-dev] Using valgrind with Mono
>
>
>
> Hi Rodrigo,
>
>
>
> I am not finding a recent commit in master that mentions
> “mono_method_get_header”. Do you happen to have the commit ID or log
> message handy?
>
>
>
> Sorry for the bother,
>
> Matt
>
>
>
> *From:* Rodrigo Kumpera [mailto:kumpera at gmail.com <kumpera at gmail.com>]
> *Sent:* Monday, March 28, 2016 1:58 PM
> *To:* Zinkevicius, Matt <matt.zinkevicius at hpe.com>
> *Cc:* Zoltan Varga <vargaz at gmail.com>; Straw, David (Storage) <
> david.straw at hpe.com>; mono-devel-list at lists.ximian.com
> *Subject:* Re: [Mono-dev] Using valgrind with Mono
>
>
>
> The memory leak on mono_method_get_header was recently fixed in master.
>
>
>
>
>
>
>
> On Mon, Mar 28, 2016 at 3:31 AM, Zinkevicius, Matt <
> matt.zinkevicius at hpe.com> wrote:
>
> I can confirm that Zoltan’s fix does indeed remove any leak instances with
> a call stack containing “ves_icall_Type_GetNestedTypes”, but unfortunately
> that accounted for only 36 leak instances out of the 19,900 reported.
>
>
>
> I have filed a bug (https://bugzilla.xamarin.com/show_bug.cgi?id=39940)
> and attached the latest valgrind output there.
>
>
>
> Thanks again for any help you can provide,
>
> Matt Zinkevicius
>
>
>
> *From:* mono-devel-list-bounces at lists.ximian.com [mailto:
> mono-devel-list-bounces at lists.ximian.com] *On Behalf Of *Zinkevicius, Matt
> *Sent:* Sunday, March 27, 2016 11:06 PM
> *To:* Zoltan Varga <vargaz at gmail.com>
> *Cc:* Straw, David (Storage) <david.straw at hpe.com>;
> mono-devel-list at lists.ximian.com
>
>
> *Subject:* Re: [Mono-dev] Using valgrind with Mono
>
>
>
> Hi Zoltan,
>
>
>
> Thank you for investigating my findings. I will apply the fix you linked
> to and report back. While I understand the other leaks are small, valgrind
> reported over 19,000 instances of leaked memory in under 5 minutes of our
> app running without load.
>
>
>
> Matt Zinkevicius
>
>
>
> *From:* Zoltan Varga [mailto:vargaz at gmail.com <vargaz at gmail.com>]
> *Sent:* Sunday, March 27, 2016 4:45 PM
> *To:* Zinkevicius, Matt <matt.zinkevicius at hpe.com>
> *Cc:* mono-devel-list at lists.ximian.com; Straw, David (Storage) <
> david.straw at hpe.com>
> *Subject:* Re: [Mono-dev] Using valgrind with Mono
>
>
>
> Hi,
>
>
>
>   Fixed the last one in:
>
>
> https://github.com/mono/mono/commit/b97ac0023256bf7d915552f5f24a7742b28c32b7
>
>
>
> The first two are leaks, but they should be small and bounded. Will look
> into fixing them to decrease the noise.
>
>
>
>           Zoltan
>
>
>
> On Sun, Mar 27, 2016 at 6:23 PM, Zinkevicius, Matt <
> matt.zinkevicius at hpe.com> wrote:
>
> Hello,
>
>
>
> Our backend service running on Mono 4.2.2 on Linux is experiencing an
> unmanaged memory leak. When running our stress tests for several hours, we
> see the managed heap sit around 50 MB, while private memory keeps growing
> until the process is killed because of OOM. I am therefore attempting to
> use valgrind to find the culprit, but I am getting so many leaks detected
> that I think many must be false positives, so I thought I would ask here
> for some guidance about which are safe to suppress or any other valgrind +
> mono tricks you can share.
>
>
>
> The vast majority of leaks reported have call stacks that closely match
> one of the following:
>
>
>
> ==16846== 4 bytes in 1 blocks are definitely lost in loss record 74 of
> 19,903
>
> ==16846==    at 0x4C26FEF: calloc (vg_replace_malloc.c:711)
>
> ==16846==    by 0x62D1D9: monoeg_malloc0 (in /usr/bin/mono-sgen)
>
> ==16846==    by 0x4870F2: decode_exception_debug_info (in
> /usr/bin/mono-sgen)
>
> ==16846==    by 0x488975: mono_aot_find_jit_info (in /usr/bin/mono-sgen)
>
> ==16846==    by 0x53C3A7: mono_jit_info_table_find_internal (in
> /usr/bin/mono-sgen)
>
> ==16846==    by 0x493C04: mini_jit_info_table_find_ext (in
> /usr/bin/mono-sgen)
>
> ==16846==    by 0x4988FB: mini_add_method_trampoline (in
> /usr/bin/mono-sgen)
>
> ==16846==    by 0x499067: common_call_trampoline_inner (in
> /usr/bin/mono-sgen)
>
> ==16846==    by 0x403217C: ???
>
> ==16846==    by 0x10D3FB63: ???
>
> ==16846==    by 0x10D3F41B: ???
>
> ==16846==    by 0x10D3F117: ???
>
>
>
> ==16846== 12 bytes in 1 blocks are definitely lost in loss record 1,172 of
> 19,903
>
> ==16846==    at 0x4C2828A: malloc (vg_replace_malloc.c:299)
>
> ==16846==    by 0x62D221: monoeg_malloc (in /usr/bin/mono-sgen)
>
> ==16846==    by 0x55B8EF: mono_metadata_type_dup (in /usr/bin/mono-sgen)
>
> ==16846==    by 0x49FC4B: get_shared_gparam (in /usr/bin/mono-sgen)
>
> ==16846==    by 0x49FE70: get_shared_inst (in /usr/bin/mono-sgen)
>
> ==16846==    by 0x4A073A: mini_get_shared_method_full (in
> /usr/bin/mono-sgen)
>
> ==16846==    by 0x414723: lookup_method (in /usr/bin/mono-sgen)
>
> ==16846==    by 0x4147FA: mono_jit_compile_method_with_opt (in
> /usr/bin/mono-sgen)
>
> ==16846==    by 0x414B9A: mono_jit_compile_method (in /usr/bin/mono-sgen)
>
> ==16846==    by 0x498DA4: common_call_trampoline_inner (in
> /usr/bin/mono-sgen)
>
> ==16846==    by 0x403405C: ???
>
> ==16846==    by 0x10D2DCA7: ???
>
>
>
> ==16846== 10 bytes in 1 blocks are definitely lost in loss record 739 of
> 19,903
>
> ==16846==    at 0x4C2828A: malloc (vg_replace_malloc.c:299)
>
> ==16846==    by 0x62D221: monoeg_malloc (in /usr/bin/mono-sgen)
>
> ==16846==    by 0x62BA8C: monoeg_g_utf16_to_utf8 (in /usr/bin/mono-sgen)
>
> ==16846==    by 0x5A8646: mono_string_to_utf8_checked (in
> /usr/bin/mono-sgen)
>
> ==16846==    by 0x5A885B: mono_string_to_utf8 (in /usr/bin/mono-sgen)
>
> ==16846==    by 0x52DE3C: ves_icall_Type_GetNestedTypes (in
> /usr/bin/mono-sgen)
>
> ==16846==    by 0x120D4256: ???
>
> ==16846==    by 0xE338A78:
> System_Type_GetMember_string_System_Reflection_BindingFlags (type.cs:806)
>
> ==16846==    by 0x40C09EF: ???
>
> ==16846==    by 0x1259A6AF: ???
>
> ==16846==    by 0x73: ???
>
> ==16846==    by 0x141D191D: ???
>
>
>
> Are these valid leaks or is valgrind confused/misconfigured? I am using
> the following command:
>
> valgrind --tool=memcheck -v --leak-check=full --log-file=val.txt
> --smc-check=all mono program.exe
>
>
>
> Thanks for any input you can offer,
>
> Matt Zinkevicius
>
>
>
>
>
>
> _______________________________________________
> Mono-devel-list mailing list
> Mono-devel-list at lists.ximian.com
> http://lists.ximian.com/mailman/listinfo/mono-devel-list
>
>
>
>
> _______________________________________________
> Mono-devel-list mailing list
> Mono-devel-list at lists.ximian.com
> http://lists.ximian.com/mailman/listinfo/mono-devel-list
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ximian.com/pipermail/mono-devel-list/attachments/20160329/fbd23d21/attachment-0001.html>