[Mono-devel-list] Prevalence of pointer-integral-pointer casting in mono?

[Jonathan Pryor]

I need to seriously start trimming down this message...but I'll fail
dismally.

On Wed, 2004-08-04 at 01:36, Peter Colson wrote:
<snip/>
> > IntPtr and UIntPtr are supposed to be large enough to hold a pointer
> > value.  That's their entire purpose.  So if you're targeting a 128-bit
> > platform, then IntPtr and UIntPtr should be resized appropriately.
<snip/>
>
> First of all thanks for the response.
> 
> To clarify, the platform concerned is the AS/400. It's not a full-blown
> 128-bit platform as such, however pointers are represented as 128-bit
> entities.

Can you clarify "128-bit entities"?  Is there a *single* 128-bit
register that is used for pointer addressing, or is a combination of
registers used (such as the 80286 segment:offset).

If it's a single register, this should be reasonably straightforward:
when JITting code, place all IntPtrs into the 128-bit registers.  Things
should Just Work (I hope).

If it's a combination of registers, things are probably more hairy. 
You'd need to intercept every load/store of IntPtrs in the JIT and
marshal them into a structure so you don't lose any pointer information.

<snip/>

> Re. pointer-int-pointer conversion, ILE C (the 400's native C compiler)
> allows casting of 128-bit pointers to another pointer, but once a
> pointer has been cast to an integral (int/long, 32 bits, or long long,
> 64 bits) information is lost that results in an invalid pointer when the
> integral value is cast back to a pointer. End result: exception on the
> first attempt to de-ref the pointer. There doesn't appear to be an
> integral type large enough to hold a complete pointer value.

The real question is how your 128-bit pointers are represented.  If
they're a single register, there *should* be an integral type that can
hold a complete pointer value (it may just have a special keyword, such
as __int128 or something).  If pointer->integer->pointer conversions are
a problem, you just need to make sure that the intermediate integer is
of the appropriate type.

If it's not a single register, then ILE C is doing things for you
(behind your back) to properly pack/unpack pointers.  Consequently,
there is no intermediate integer large enough, and you would need to
find every pointer->integer->pointer occurrence and remove it (if
possible).

Personally, in the Mono C Runtime code, I doubt that there are many
instance of pointer->integer->pointer conversion;
integer->pointer->integer (which is safe) is far more likely.

The next issue is then the Managed/Unmanaged interaction layer.  IntPtrs
should be treated as raw pointers from the Unmanaged side, so this
shouldn't complicate things too much (though it all depends on what the
JIT does).

<snip/>

> Part of my research has led me across code that uses unused bits in
> pointers to signify type information. In most cases this can be dealt
> with by rewriting expressions that get and set this information. But I
> was curious if this was a widespread phenomenon under Mono, because if
> it's not, it makes it a more attractive proposition for porting.

I do not believe it is, but I haven't actively worked on the C runtime
portions of Mono either, so take that with the required grains of salt.

<snip/>

> [T]he question in
> my mind is even if the author of a C# program doesn't use unsafe code, whether
> the compiler would make use of unsafe code internally as a matter of course,
> with this same IntPtr-related unsafe code winding up in the IL of any executable
> produced by the compiler.

Short answer: no.  "Safe" vs. "Unsafe" refers to whether the generated
code is verifiable, which is part of the security infrastructure of
.NET.  If it's verifiable, then .NET permits code downloading and
execution within a sandbox.  If it's not verifiable (unsafe), then the
code makes use of things such as pointer arithmetic which are impossible
(or at least very difficult) to verify, so .NET won't permit code
downloading and execution within a sandbox (as the sandbox won't work
too well...).

Verifiability is a key part of .NET.  Consequently, the compiler can't
just go inserting Unsafe code into any program it feels like.  The
programmer must explicitly make use of unsafe code.

There is one slight complication: IntPtr isn't "unsafe"; it can be used
by normal verifiable code.  This isn't a major complication from the
security standpoint, though, as IntPtr is normally used for P/Invoke,
and P/Invoke is outside of the sandbox (so if you're using DllImport the
.NET security infrastructure won't let you run within a sandbox,
disabling code downloading, etc.).

> I think I'm also coming to the realisation that any C# application of substance
> is going to be using unsafe code anyway to get things done... 
> 
> > Let's put it this way: as currently implemented, Mono requires the use
> > of IntPtrs.  Just look at the System.IO source code (IIRC) -- IntPtrs
> > are used as part of the internal call declarations into the Mono
> > runtime.  This is pretty much true for any part of the runtime that uses
> > internal calls (which is a substantial fraction).  If an IntPtr can't
> > actually hold a pointer, you're screwed.  Period.

To revisit an earlier point, if IntPtr is actually represented as a
pointer on the hardware, you're fine.  You might need to see what ILE C
generates when pointers are used, to make sure that IntPtr is
represented the same way.
 
<snip/>

One final point: I wouldn't be surprised that Mono could run in your
environment (though I wouldn't be surprised if you can't; I'm
ambivalent), but you might have a problem with portability of user
code.  Some existing programs/libraries (Gtk#) assume that pointers are
either 32-bit or 64-bit, as (1) they need to perform pointer arithmetic,
and (2) IntPtr doesn't provide the required operators to do pointer
arithmetic.  Consequently existing code will cast an IntPtr to an
appropriately-sized integer, perform the pointer arithmetic, and cast
back -- a managed pointer->integer->pointer conversion.

Obviously such managed code will break with 128-bit pointers.

The solution is to get operator+/operator- added to IntPtr so that
pointer arithmetic can be performed without needing an intermediate
integer type (which can lose pointer information).  I'm not sure how
easy this would be to get into the standard, though.

 - Jon