[Mono-dev] BSTR Marshalling in Mono

[Jonathan Chambers]

Hello,
      COM Interop support in mono works pretty well for most basic uses, but
has some limitations when it comes to strings. Mainly, BSTR marshalling on
non-Windows platforms is just a default implementation. The problem is that
most COM systems (both Mainsoft's COM and Mozilla's XPCOM) have specific
requirements on strings. They need to follow a certain format (usually a
length prefixed string), use special allocators/deallocators, and use the
correct byte size (2-byte vs. 4-byte encoding).

Note, I did try to go down the road of using a custom marshaller, but ran
into some issues. First, any assemblies that worked on Windows now need
extensively modified to work on non-Windows platforms. This is more of an
issue for COM, since MS has nice tools to generate COM Interop assemblies.
Almost no one generates COM Interop code by hand. Second, even if the user
tries to use a custom marshaller, there are places in the runtime that
perform BSTR marshalling as well that the user cannot modify (without
modifying mono and rebuilding). Examples include BSTR util methods in the
Marshal class, VARIANT marshalling, etc.

In order to better support this on non-Windows platforms, I'd like to make
the BSTR marshalling in mono extensible. I have thought of a few ways to do
this that I'll list, and hopefully get some feedback on as to which is best.

1. Expose some methods in the runtime so users could embed mono and adjust
the BSTR marshalling behavior with callbacks. Something like
mono_set_bstr_to_string_marshal, mono_set_bstr_from_string_marshal,
mono_set_bstr_free. This would require users to embed mono.

2. Use the dll map in the config file to let the user specify entry points
to perform BSTR marshalling. This seems a better choice than the first.
There is then a technical question as to how to implement this.

a. Right now all BSTR marshalling occurs in an icall. The current icall
could lookup the info in the config map, and call the entry point itself.
This would require no changes to any other code, but would require logic to
parse the config file, load a library, and get an entry point. I know this
logic exists, but I am not sure how much of it exists outside of the pinvoke
framework and whether we would want to put that logic in the icalls anyway.

b. I could also change all BTSR related routines to go through the Marshal
class. This would intern pinvoke into the runtime. We could then use the
config file and the current pinvoke redirection logic to simply redirect
those pinvokes to a user supplied library. The downside to this would be
having to modify all the icalls/logic in the runtime marshalling code for
BSTRs. It would also make the string marshalling code a bit uglier, as
currently all string marshalling occurs in icalls. This would require a
check for BSTRs to call a managed method rather than an icall.

Thanks,
Jonathan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.ximian.com/pipermail/mono-devel-list/attachments/20071219/8a75974a/attachment.html