Ant: Re: Ant: Re: Ant: Re: Ant: Re: [Gtk-sharp-list] gtk-sharp.dll not working on linux because it references win32 dll's

[Jonathan Pryor]

On Wed, 2005-07-20 at 18:27 +0200, Jost Boekemeier wrote:
> Hi,
> 
> > > Why do I need that?  I want to load the library that
> > > the user has placed into his lib directory. 
> 
> > For any assembly, if you want to load it from the GAC 
> 
> Why it this information necessary?  The user has given
> me the name of the library and I have to load the most
> recent version.  This worked fine in gtk-sharp 0.6. 
> Why do I have to care if you have or have not decided
> to declare the library as API stable?

Because that's one of the *fundamental* underpinnings of .NET.  You
might as well ask "why do I need to know the difference between value
types (structs) and reference types (classes)?"  Because it's
fundamental, and you ignore such information at your own peril.

For .NET, *everything* placed in the GAC *must* follow the rules,
including (1) a stable API within a major version, and (2) signed by a
public/private key pair.  Failure to follow the rules will result in
broken code, unhappy customers, lost income, and developers yelling at
others because these fundamental issues are not understood. :-)

Why is all this information necessary?  Because imagine this:

1.  Your application uses Assembly.LoadWithPartialName to load 
    assemblies.  This means that version, culture, and public key token
    information are potentially ignored, especially if you're just 
    doing a Assembly.LoadWithPartialName ("gtk-sharp").

2.  User installs your application, which installs Gtk# 1.0 into the 
    GAC.  Everything works fine.

3.  User installs another application, which installs a "gtk-sharp" 2.0 
    into the GAC.

4.  User tries to run your application.  Here, one of several things 
    can happen:

    a.  Your application continues to run.  This will only happen if 
        gtk-sharp 2.0 is *fully* backward compatible with gtk-sharp 1.0.

    b.  Your application dies catastrophically, because:

        i   The API changed between 1.0 and 2.0 (which *will* happen 
            with Gtk#; even 1.0.x has unstable changes if an API cannot
            be used as previously bound).
        ii  The new gtk-sharp comes from a completely different company
            and bears no relation to Mono's gtk-sharp assembly
            (different public key token, which you're ignoring)

5.  User becomes irate, blames Microsoft for creating a shitty .NET 
    platform, or (less likely) you for being a stupid developer, and
    tells all his friends to avoid Windows and .NET.

The strong-binding rules were created to avoid this situation, to avoid
DLL Hell and the potential to fatally break existing apps by "merely"
installing a new app.  Assembly.LoadWithPartialName() bypasses all this,
allowing for broken programs, which is why it's [Obsolete] in .NET 2.0.

In short, Microsoft doesn't want you to be able to load the latest
version of an Assembly, because it's unstable and prone to easy failure.

Which leaves you with two choices: always bundle the assembly within
your application directory (making it a private assembly, which is
always searched for first), or always use a strong name.  The middle
path of Assembly.LoadWithPartialName() is BAD, and will not be supported
by Microsoft in the future.

(As for why it worked with Gtk# 0.6, it didn't work with Gtk# 0.6, it
worked with Mono 0.30 (or whatever existed before strong-name binding
was implemented in Mono).  Gtk# 0.6 just happened to be shipping with a
"broken" version of Mono, which is why this broken behavior worked for
you.  It's not a Gtk# issue, it's a mono/.NET issue.)

> > fully-qualified assembly name (name, version, culture, public key
> > token).  Otherwise, you can just place your assembly  into the
> > appropriate directory and just Assembly.Load() the name.
> 
> What I have is the library name.  The user says that
> he installed the libraries in /usr/local/lib and he
> verified that they are installed in
> /usr/local/lib/mono/gth-sharp-2.  Now I have to load
> it, whether it is version 0.6 or 20.0, I don't care.

Mono does care, so you have to care....slightly.  .NET also cares.

In the case of Mono, you don't need to care too much about what version
of Gtk# you're using.  That's what pkg-config is for.  Gtk# should
install a /usr/local/lib/pkgconfig/gtk-sharp-2.0.pc file, so to link
against Gtk# you do this:

	$ export PKG_CONFIG_PATH=/usr/local/lib/pkgconfig:$PKG_CONFIG_PATH
	$ mcs -pkg:gtk-sharp-2.0 my-program.cs

mono, mcs, and pkg-config will worry about the assembly binding issues
for you, allowing you to have a peaceful life without worrying about the
complications of strong names.

Alternatively, you could copy /usr/local/lib/mono/gtk-sharp-2/*.dll into
your application's directory, and just reference them normally:

	$ cp /usr/local/lib/mono/gtk-sharp-2/*.dll .
	$ mcs my-program.cs -r:gtk-sharp.dll # and any other -r needed

This isn't ideal, as you also need to copy over the .dll.config files if
you plan on running with the copied assemblies (see below for where
the .dll.config files are).

> The problem is, that loading the library works on windows, but not on Linux.

I seriously doubt that this is the case, UNLESS you've copied the .dll
files into your application's directory (as demonstrated above).

> > Thus the question: how are you using, shipping, and
> > installing Gtk#?
> 
> I don't.  The user has installed the library with make
> install.  If he loads
> /usr/local/lib/mono/gtk-sharp-2/gtk-sharp.dll, he get
> an exception saying that a windows library is missing.

Then in all likelihood something is wrong with the user's environment.
To see where things are going wrong, set the MONO_LOG_LEVEL and
MONO_LOG_MASK environment variables:

	$ MONO_LOG_LEVEL=info MONO_LOG_MASK=dll mono --debug program.exe

I would guess that the actual problem is that libgtksharpglue.so cannot
be loaded because /usr/local/lib isn't part of the dynamic library path.
Either set LD_LIBRARY_PATH:

	$ export LD_LIBRARY_PATH=/usr/local/lib:$LD_LIBRARY_PATH

Or add /usr/local/lib to /etc/ld.so.conf and run /sbin/ldconfig as root.

> > Are you using the one included with Mono?  Then it's
> > in the GAC.
> 
> Again, I don't do anything.  I just have to load the
> library.  Are you saying that since gtk-sharp 1.0 it
> is no longer possible to load a Linux library without
> knowing its hash code?  Okay, your decision, but why
> does the install script install the libraries in
> /usr/local/lib/mono/gtk-sharp2 then?

No libraries are installed into /usr/local/lib/mono/gtk-sharp2.
*symlinks* to libraries are installed there, and the contents of that
directory are for use by pkg-config (and ignored by mono).  *You* don't
need to worry about them, nor care that they exist.  Just use `mcs
-pkg:gtk-sharp-2.0` and let the tools worry about it.

> > Are you copying the Gtk# assemblies into your
> > applications directory?
> > Then an Assembly.Load() will work.
> 
> Hrm.  As I said, not on linux because the .config
> files are missing.

They're not missing, they're just where you're not looking.  As
mentioned above, the contents of /usr/local/lib/mono/gtk-sharp2 are for
use by the pkg-config file, and are ignored by mono.  The actual .dll
files (and the corresponding .config files) are installed into:

	$prefix/lib/mono/gac/gtk-sharp/VERSION__PUBKEYTOKEN/gtk-sharp.dll

For example:

	/usr/lib/mono/gac/gtk-sharp/1.0.0.0__35e10195dab3c99f/gtk-sharp.dll
	/usr/lib/mono/gac/gtk-sharp/1.0.0.0__35e10195dab3c99f/gtk-sharp.dll.config

> Copying the config files from the
> build directory into the application directory solves
> the problem, but a) why do I have to do this, 

Because you're not compiling against Gtk# correctly.  Use `mcs
-pkg:gtk-sharp-2.0`, make sure your environment variables are set
(including PKG_CONFIG_PATH and LD_LIBRARY_PATH), and everything should
Just Work.  (Thus are the perils of installing into a non-standard
prefix.)

> b) why only on linux,

Because .NET doesn't support .dll.config files or dll mapping, so if you
want to provide a single assembly that works everywhere you need to use
Win32 library names (to appease .NET).

>  c) why are these config files necessary at all, they contain
> information which shall never be changed, this information should go
> into the library itself, not into a user visible config file.

Alas, this information *does* change, more frequently than we'd like.
Different Linux distributions ship different (yet often compatible)
shared library versions.  So you'd either need to compile your assembly
for each Linux distribution, or use the .config file support.

(A common Linux example is the gtkhtml library, which has an unstable
API and different versions across most distributions, requiring
constant .dll.config modification...)

Mac OS X is in a similar boat: *all* libraries need <dllmap> statements
on Mac OS X because of a glib bug where ".so" is used as the shared
library extension, not ".dylib".  Plus, the library version is placed in
a different spot, e.g. Foo.A.dylib instead of libFoo.so.1.

Remember, the goal is to permit use of the *same* assembly across *all*
platforms (recompiling is evil).  Dllmaps are the *only* viable
solution.

> > > Depends on how you do it.  I think it should be
> > > possible to check the platform, load the real dll once
> > > and then this dll takes over.
> > 
> > I know of no way to do that.  
> 
> Okay.  I thought that if the dynamic loader has the
> ability to patch a library when it is loaded one could
> as well check for the architecture and then load the
> byte code from the appropriate file or section.

The patches made by a dynamic loader are limited.  The only linker I
know of that would provide such support is the Mach-O linker (used by
NeXTStep and Mac OS X), which has the concept of "fat" binaries.  Win32
and Linux do not support such functionality.

Plus, loading byte code for a specific architecture from the file isn't
what you were suggesting, at least it wasn't how I interpreted it.  You
said "check the platform, load the real dll once and then this [new] dll
takes over."  This implies 2 libraries, the one loaded by Mono and the
platform-specific library.  It is this "load a 2nd library to
effectively replace the 1st library" which isn't supported by any linker
that I know of.

> > > As I understand it, a linux user must place a dll/map
> > > pair into his library directory, while the windows
> > > user can simply copy the gtk-sharp dll's, right?
> > 
> > The user shouldn't need to worry about assemblies
> 
> He needs to, simply because he wants to load one.

As Rafael Teixeira mentioned, what kind of user are we talking about?
End users shouldn't need to care, as the program's installer (or the OS
package manager) should handle such issues for them.  If you mean a
developer "user", then I'm sorry but some background is assumed.  I
would suggest reading the Gtk# Beginners Guide:

	http://www.mono-project.com/GtkSharpBeginnersGuide


> > Those are a development and deployment issues, handled by the 
> > developer and packager.
> 
> I am sorry, I don't understand this at all.

The developer of a library, such as the Gtk# developers, worries about
the .dll.config files.  The packager (i.e. the person who creates RPMs
or installation programs) worries about installing the .dll.config files
into the correct spot.  The end user should only need to use an
appropriate installer, and let the packager worry about getting things
right.

The same holds true for developers using libraries -- a packager should
be worrying about the .dll.config files, and the end-developer should
only need to worry about using pkg-config to link against the
appropriate library.

In no circumstance is the end user expected to write their
own .dll.config files (though they may need to edit them if they want to
run on a currently unsupported platform...).

 - Jon