[Mono-list] Deep Marshaling

[Giuseppe Greco]

Thanks Jon!

Another alternative would be the following:

[StructLayout(LayoutKind.Sequential)]
internal struct MyStruct
{
      public int Id;
      public unsafe byte* Data;
}

public unsafe void FillMyStruct(string s)
{
      //
      // convert the given string into a byte array
      //
      byte[] data = Encoding.UTF8.GetBytes(s);

      //
      // create another byte array for storing
      // a new instance of MyStruct
      //
      byte[] buffer = new byte[
          sizeof(int)    // Size of MyStruct.Id
        + data.Length];  // Length of MyStruct.Data

      //
      // fill the buffer
      //
      fixed (byte* pBuffer = buffer, pData data) {
          MyStruct* pMyStruct = (MyStruct*) pBuffer;
          pMyStruct->Id = 100;
          pMyStruct->Data = pData + sizeof(int);
          UnsafeCopy(
              pData, 0,               // source buffer
              pMyStruct->Data, 0,     // destination buffer
              data.Length);           // number of bytes to copy
      }
}

public static unsafe void UnsafeCopy(
     byte* pSource, int sourceOffset,
     byte* pDest, int destOffset, int count)
{
     byte* ps = pSource + sourceOffset;
     byte* pd = pDest + destOffset;

     //
     // loop over the count in blocks of 8 bytes, copying a
     // long integer (8 bytes) at a time
     //
     int c = count / sizeof(long);
     for (int i = 0 ; i < c; i++) {
         *((long*)pd) = *((long*)ps);
         pd += sizeof(long);
         ps += sizeof(long);
     }

     //
     // complete the copy by moving any bytes that were not
     // moved in blocks of 8
     //
     c = count % sizeof(long);
     for (int i = 0; i < c; i++) {
         *pd = *ps;
         pd++;
         ps++;
     }
}

That works fine and performance is quite good. Of course,
the method UnsafeCopy() is really 'unsafe', but it defined
in an internal class...

j3d.

Jonathan Pryor wrote:
> On Sun, 2005-10-23 at 15:21 +0200, Giuseppe Greco wrote:
> 
>>Jon,
>>
>>that you said is really interesting... and I'm wondering how
>>can I solve another similar problem. Giving the following type:
>>
>>[StructLayout(LayoutKind.Sequential)]
>>internal struct MyStruct
>>{
>>     public int Id;
>>     public byte[] Data;       // of course, that's wrong!!!
>>}
>>
>>... I'd like to fill it like this:
>>
>>public unsafe void FillMyStruct(string s)
>>{
>>     //
>>     // convert the given string into a byte array
>>     //
>>     byte[] data = Encoding.UTF8.GetBytes(s);
>>
>>     //
>>     // create another byte array for storing
>>     // a new instance of MyStruct
>>     //
>>     byte[] buffer = new byte[
>>         sizeof(int)    // Size of MyStruct.Id
>>       + data.Length];  // Length of MyStruct.Data
>>
>>     //
>>     // fill the buffer
>>     //
>>     fixed (byte* pBuffer = buffer) {
>>         MyStruct* pMyStruct = (MyStruct*) pBuffer;
>>         pMyStruct->Id = 100; // OK
>>         pMyStruct->Data = data; // ERROR
>>     }
>>}
>>
>>Any help would be really appreciated,
>>j3d.
> 
> 
> This has a fundamental problem associated with it: the Common Language
> Infrastructure is *not* C or C++.  You can't subvert the type system
> with random casting, as the runtime will *ensure* that all managed types
> are used correctly.  This means that such things as C++ "placement new"
> aren't possible.
> 
> For your particular example, it means that you can't take a pointer to a
> type containing non-blittable types.  Thus declaring a `byte*' is safe,
> but declaring a `System.Collections.ArrayList*' will yield this:
> 
>         error CS0208: Cannot take the address of, get the size of, or
>         declare a pointer to a managed type
>         `System.Collections.ArrayList'
> 
> The reason for this restriction is primarily the garbage collector.
> ArrayList, and your MyStruct structure, contain managed arrays (object[]
> for ArrayList, and byte[] for MyStruct).  The managed array needs to be
> tracked by the garbage collector, lest Bad Things happen.  Consequently,
> you cannot use these types in pointer context, as the GC can't safely
> track it.
> 
> A solution would be to use only blittable types within your structure,
> though since you seem to want to make Data a variable length array,
> inline with the structure memory, this isn't entirely feasible.
> Instead, you'd have to use a named "first" element of the array, and
> then index off that, e.g.:
> 
>         struct VaribleLengthStruct {
>                 public int Size;
>                 public byte data_begin;
>         }
>         
>         void Test ()
>         {
>                 byte* buffer = stackalloc byte [
>                         Marshal.SizeOf (VariableLengthStruct)+10
>                 ];
>                 VariableLengthStruct *p = (VariableLengthStruct*)
>                 buffer;
>                 p->Size = 10;
>                 byte* data = &p->data_begin;
>                 for (int i = 0; i < 10; ++i)
>                         data [i] = (byte) i;
>         }
> 
> The above is in fact how Mono's System.String handles things:
> 
>         public sealed class String // : ...
>         {
>                 [NonSerialized] private int length;
>                 [NonSerialized] private char start_char;
>         }
> 
> Can you provide more information about the problem you're *actually*
> trying to solve?
> 
>  - Jon
> 
> 


-- 
----------------------------------------
Giuseppe Greco
::agamura::

call giuseppe.greco via Skype
phone:  +41 (0)91 604 67 65
mobile: +41 (0)79 590 33 06
email:  giuseppe.greco at agamura.com
web:    www.agamura.com
----------------------------------------