[Mono-list] Custom Marshalling

[Neale.Ferguson@SoftwareAG-USA.com]

I'll attempt to think before I type this time :-)

I have a a C routine I wish to call that takes the following parameters:

typedef struct XXXCB {
    short a;
    char  b[2];
};

void scumbag(XXXCB *cb, char *v);

for our purposes v is an arbitrarily long string of bytes in the range =
0x00-0x7f. Depending on the contents of "b" variable "v" can be input to =
or output from the function "scumbag".

In C# I'd code XXXCB as:

public struct XXXCB {
	public short a;
	[MarshalAS(UnmanagedType.ByValArray, SizeConst=3D2)}
	public byte[] b;
}

Now, because the variable "v" can be a different length of each call (up =
to 32K is size) I can't use MarshalAs(UnmanagedType.ByValArray) as mcs =
demands that SizeConst=3D be coded (using SizeParamIndex=3D doesn't =
work, in fact I don't think it's supported).

So what I've done is created a class XXXByteBuffer to Marshal the data =
using ICustomMarshaler.

Therefore, I've coded the call prototype as:

[DllImport(libname, EntryPoint =3D "scumbag")]
private static extern void scumbag (ref XXXCB cb,
  [MarshalAs(UnmanagedType.CustomMarshaler, MarshalTypeRef =3D typeof =
(XXXByteBuffer))] v);

I created the necessary methods (like ManagedToNative and =
NativeToManaged) in XXXByteBuffer. When I call scumbag using parameters =
conforming to the above prototype the ManagedToNative method is being =
driven prior to the call but NativeToManaged is not being called on the =
return. If I change the prototype to specify "ref v" or "out v" then =
neither method is being driven.

Neale

-----Original Message-----
I'm a little confused by your description. I am not trying to be =
difficult,=20
but you use the pronoun "it" in many places where the antecedent is =
unclear.=20
Also, the phrase "on its way out" is not clear.

Perhaps if you demonstrated the problem using a P/Invoke method with =
fewer=20
parameters and explained any problems with "in", "out", and "ref" =
parameters=20
separately, I could better understand.