[Mono-dev] Solving the profiler loading problem

Thu Sep 28 04:48:01 UTC 2017

This writeup is in support of option 2, and/or supporting build of the
profiler plugins into the monolithic Mono in a static build.

I'm not familiar with all the deployment use cases of Mono, but the
"standard" solution I'm aware of (e.g. Linux kernel, Samba) is that distro
makers build dynamically, thereby allowing for 3rd party to ship binary
plugins (e.g. kernel modules and Samba VFS modules), whereas "canned"
product vendors who ship a customized binary might want a statically-linked
build, but do not expect 3rd party plugins. In this light, you might be
good to go with option 2 - plugin vendors ship plugins against
dynamically-linked Mono.

With respect to long-term transitioning the Mono-API into a vtable - that
may be good for Mono API but not necessarily for plugins. IMO there's a
fundamental difference between "northbound" interfaces - interfaces for
code embedding Mono, and plugins ("southbound" interfaces). Plugins are
most successful when they have full access to the internal APIs, and the
decision of what builds into the monolithic binary and what loads as plugin
becomes a quite arbitrary build-time decision (see Linux modules or even
Windows kernel drivers for that matter).

I would not want plugins to be constrained (in functionality or
performance) just because we want to help vendors ship binary plugins (not
sure vendors would want that either).

Just my 2c,

Uri

On Wed, Sep 27, 2017 at 4:02 PM, Alex Rønne Petersen <alex at alexrp.com>
wrote:

> Hello,
>
> I was recently taking a look at this bug:
> https://bugzilla.xamarin.com/show_bug.cgi?id=16785
>
> The problem is that we link libmono-profiler-log.so to
> libmonosgen-2.0.so, even when the mono executable is statically
> linked. What happens is that, when we load libmono-profiler-log.so
> into the process of a statically linked mono, libmonosgen-2.0.so is
> loaded in addition to the mono executable (which already contains all
> the libmono code). The dynamic linker helpfully figures out that all
> of the libmono symbols that the profiler uses are already present in
> the mono executable, so none of the code or data in the newly loaded
> libmonosgen-2.0.so is actually used. However, this no longer holds
> true when library constructors enter the mix. When libmonosgen-2.0.so
> is loaded, any library constructors that it contains, or which are
> present in any libraries it links to, will be executed.
>
> C code rarely makes use of library constructors (which is why this
> hasn't been a problem for a 'normally' built libmonosgen-2.0.so so
> far), but they're very common in C++ code; constructors for global C++
> objects are implemented with them. LLVM, for example, makes use of
> them in many places. So, when we've already loaded libLLVM*.so from
> the static mono executable and therefore executed the library
> constructors contained within, running those library constructors
> again when we load libmono-profiler-log.so just kinda breaks
> everything.
>
> Below are a few different ways we can solve this problem.
>
> -----
>
> # Always build a dynamically linked mono executable
>
> If we never statically link mono with libmonosgen-2.0.so, we can
> simply keep linking libmono-profiler-log.so to libmonosgen-2.0.so and
> everything will just work. When the profiler is loaded, the dynamic
> linker realizes that libmonosgen-2.0.so is already loaded, so it'll do
> the right thing.
>
> The downside of this approach is that working with a dynamically
> linked executable can be a pain. In particular, you need to mess with
> LD_LIBRARY_PATH to get mono to use a libmonosgen-2.0.so in your
> working tree rather than the one from your prefix. libtool tries to
> alleviate this problem by turning the mono file in the working tree
> into a script that invokes the actual executable with the right
> dynamic linker setup to use the in-tree libraries. It's also possible
> to use libtool --mode=execute gdb mono ... to run in-tree mono in a
> debugger.
>
> If we go this route, we'd probably need to enhance
> runtime/mono-wrapper to support all of our in-tree use cases (such as
> running in a debugger) since nobody wants to type out the libtool
> commands by hand.
>
> There may be performance considerations with this approach, but from
> what I've seen, there's already some interest in always building
> libmonosgen-2.0.so as PIC to speed up the build process (no need for
> building a separate static version of everything). So, I'm not too
> worried about this aspect.
>
> # Link libmono-profiler-log.so to libmonosgen-2.0.so only when needed
>
> The idea here is that, in our build system, we can easily determine
> whether we're building a statically or dynamically linked mono
> executable. So, if we're building a statically linked mono (which is
> the norm currently), we can simply avoid linking
> libmono-profiler-log.so to libmonosgen-2.0.so. This leaves the
> profiler with some undefined symbols, but the dynamic linker will sort
> that out by resolving them to the symbols in the mono executable.
>
> This is definitely the simplest approach and it's probably what I'll
> end up doing at least as a temporary fix. However, it's a pretty poor
> solution for third-party users of the profiler API. Those developers
> cannot possibly know whether their profiler module will end up being
> used with a statically or dynamically linked mono executable. So,
> effectively, they have to build two separate profiler modules, one for
> each case. This is wasteful for obvious reasons.
>
> I care a great deal about making profiler development for Mono as
> simple and painless as possible (see the new profiler API, for
> example). This approach goes completely counter to that, so I'm not a
> big fan of it.
>
> # Pass an API vtable to a profiler module's init function
>
> This would basically be JNI. Just like how JNI passes a JNIEnv pointer
> to native functions, we'd pass a MonoEnv pointer to
> mono_profiler_init_<name> (). The profiler would do all Mono API calls
> through this vtable. Long term, we could even transition the entire
> Mono API to use this mechanism instead of exposing functions directly
> from the library.
>
> I do like this idea since it would mean that profiler modules would
> never have to explicitly link to libmonosgen-2.0.so, but there are at
> least three potential issues with it:
>
> 1. Having all API calls go through a vtable could have some serious
> performance implications. We use a lot of low-level functions from
> libmono in the profiler (e.g. mono_memory_barrier (),
> mono_hazard_pointer_get (), MONO_LLS_FOREACH, etc).
> 2. Macros such as MONO_LLS_FOREACH (which integrates closely with C's
> break/continue statements) can't be expressed in such a vtable.
> 3. The log profiler uses around 160 functions from the Mono API.
> That's just the amount of API functions that would need to be added to
> this vtable for the log profiler to work; exposing the entire Mono API
> through this vtable would be a ton of work.
>
> -----
>
> This list is by no means exhaustive; there are probably solutions I
> haven't thought of, and pros/cons of the above solutions that I
> haven't considered. Please let me know your thoughts.
>
> Regards,
> Alex
> _______________________________________________
> Mono-devel-list mailing list
> Mono-devel-list at lists.dot.net
> http://lists.dot.net/mailman/listinfo/mono-devel-list
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.dot.net/pipermail/mono-devel-list/attachments/20170928/b56c1844/attachment.html>