[Mono-list] Bug in System.Reflection.

[Miguel de Icaza]

On Thu, 2002-11-07 at 14:21, fgonthier@hermes.usherb.ca wrote:
> ** Sorry if there is a double-post but I forgot to enter a subject the first 
> time I sent this message so I feared it might have ended in /dev/null...
> 
> There is an error (I think) in:
> 
> namespace System.Reflection.Emit {
>         public enum PEFileKinds {
>                 Dll = 1,
>                 ConsoleApplication = 2,
>                 WindowApplication = 3
>         }
> }
> 
> >From WinNT.h I see:
> 
> #define IMAGE_SUBSYSTEM_WINDOWS_GUI 2 // Image runs in the Windows GUI 
> subsystem.
> #define IMAGE_SUBSYSTEM_WINDOWS_CUI 3 // Image runs in the Windows character 
> subsystem.
> 
> This is the solution to bug #32287.  PE files with the subsystem field set to 
> Windows GUI won't use the console.
> 
> Perhaps I mixed few things but this look like true because if I change that 
> subsystem field manually in a file produced by .NET, I don't get any console 
> outputs.

The problem is actually a Mono problem (sorry for the false alarm Lee),
the following patch converts the values from PEFileKind to the values
expected by the PE loader.

Index: reflection.c
===================================================================
RCS file: /cvs/public/mono/mono/metadata/reflection.c,v
retrieving revision 1.116
diff -u -r1.116 reflection.c
--- reflection.c	17 Oct 2002 11:15:16 -0000	1.116
+++ reflection.c	7 Nov 2002 20:10:49 -0000
@@ -2591,7 +2591,21 @@
 	size += VIRT_ALIGN - 1;
 	size &= ~(VIRT_ALIGN - 1);
 	header->nt.pe_image_size = GUINT32_FROM_LE (size);
-	header->nt.pe_subsys_required = GUINT16_FROM_LE (assemblyb->pekind); /* 3 -> cmdline app, 2 -> GUI app */
+
+	//
+	// Translate the PEFileKind value to the value expected by the Windows loader
+	//
+	{
+		short kind = assemblyb->pekind;
+
+		// PEFileKinds.ConsoleApplication == 2 means
+		if (kind == 2)
+			kind = 3;
+		else if (kind == 3)
+			kind = 2;
+		
+		header->nt.pe_subsys_required = GUINT16_FROM_LE ();
+	}    
 	header->nt.pe_stack_reserve = GUINT32_FROM_LE (0x00100000);
 	header->nt.pe_stack_commit = GUINT32_FROM_LE (0x00001000);
 	header->nt.pe_heap_reserve = GUINT32_FROM_LE (0x00100000);

Miguel