[Mono-dev] Replacing/Removing I18N

[Andreas Nahr]

> To not let our thoughts go wrong way: the reason why existing
> I18N.*.dll consist of a bunch of IL code and very few static data
> is just because those conversion tables are just made into a bunch
> of switch-cases.

And I wrote quite extensively in the original post why this is something 
that we definatelly wouldn't want to have.

> The encoding conversion tables definitely do not have to reside in
> mscorlib.dll. It is a lot of waste of resources.
>
> They could just be other set of resources like per-encoding dlls
> (which means that the refactoring is totally meaningless, only
> hobbiests' satisfaction) or some kind of tables which have different
> loading rules than managed dlls (for example, something like *.nlp
> on .NET).

Why would embedding them into corelib be a waste of resources? As far as I 
found out this approach should use the minimum amount of resources possible.
It surely uses a LOT less resources than having approx. 150 additional 
assemblies that each contain substantial overhead. And it is faster in 
loading than external assemblies.
I don't know if mono mmaps external resource files. If it does not then 
putting the files into external files (like *.nlp) would be a HUGE waste of 
memory resources. If these files are cross-process memory-mapped then it is 
just a matter of taste if we want to have 150 externally linked files or 
have them embedded (in fact then we could even mix both possibilities 
without any additional work needed).

> I'm not a fan of this specific restructuring anyways: so niche.

Care to explain? Maybe I don't understand niche? See my original post for 
the detailed rationale.

> Atsushi Eno
>
> Andreas Nahr wrote:
>> Hi,
>>
>>> Hello,
>>>
>>>> * Creating the binary data should be simple when generating from a .Net 
>>>> VM.
>>>> Would it be allowed to gernerate directly from MS.Net? From 
>>>> Portable.Net?
>>>> (obviously from Mono is no problem, but would not allow to ADD data)
>>>
>>> I did not understand this question at all.
>>
>> Well the question is: Is it allowed to "extract" Data from .Net if it can 
>> be extracted using a normal application (I attached an application that 
>> might be suited for the task)
>>
>>>> * Size of a memmaped page?
>>>
>>> 4k or 8k, depending on the platform.
>>
>> This would be a perfect size.
>>
>>>> * Growth in *file*size for corelib acceptable? Altogether probably 
>>>> 5-10MB
>>>
>>> Do we really need to grow corlib?   What do you have in mind?
>>>
>>> Couldnt we just use static data, and access that as a resource?  (Mono
>>> uses mmap for resources in the file)
>>
>> Do you mean something like: private static readonly byte[]...?
>> This seems to come with some overhead for me. Also it doesn't seem to 
>> produce shareable memory between processes.
>> So the idea was (see original mail) to embed the datatables as resources 
>> into the corelib assembly.
>>
>>> Am not sure how much code vs tables lives in the I18N libraries, do you
>>> have details?
>>
>> Currently the I18N assemblies are about 1MB IL-compiled in size. About 
>> 98% is IL-Code, 1-2% is static data.
>> With the suggested implementation we would have (for single byte 
>> encodings) basically 0% IL and 100% resource-data.
>> My MS.Net has 145 encodings, mono currently has 95 total.
>> Each (single-byte (approx. 95 in MS.Net)) encoding would have about 65kb 
>> of tables (the idea is to create one resource per encoding)
>>
>>>> * Other sideeffects possible?
>>>>
>>>>
>>>> Greetings
>>>> Andreas
>>>> _______________________________________________
>>>> Mono-devel-list mailing list
>>>> Mono-devel-list at lists.ximian.com
>>>> http://lists.ximian.com/mailman/listinfo/mono-devel-list
>>> -- 
>>> Miguel de Icaza <miguel at ximian.com>
>>>
>> using System;
>> using System.Text;
>> using System.IO;
>>
>> class CreateEncodingData
>> {
>>     private static string path = @"c:\st\out\";
>>     public static void Main ()
>>     {
>>         byte[] bytes = new byte[256];
>>         for (int i = 0; i < 256; i++)
>>             bytes[i] = (byte)i;
>>        char[] chars = new char[65536];
>>         for (int i = 0; i < 65536; i++)
>>             chars[i] = (char)i;
>>        EncodingInfo[] encodings = Encoding.GetEncodings();
>>         Console.WriteLine ("Encoding count: " +  encodings.Length);
>>        foreach (EncodingInfo info in encodings)
>>         {
>>             Encoding e = Encoding.GetEncoding(info.CodePage);
>>             Console.Write ("Encoding: " + e.CodePage + " (" + e.WebName + 
>> ";" + info.DisplayName + ")");
>>            if (e.IsSingleByte)
>>             {
>>                 Console.WriteLine (" - single byte");
>>                 //byte unusedbyte = (e.GetBytes(new char[] { 
>> (char)30000 }))[0];
>>                Stream s = new FileStream (Path.Combine(path, e.CodePage + 
>> ".bin"), FileMode.Create);
>>                char[] dataChars = e.GetChars(bytes);
>>                 byte[] temp = Encoding.Unicode.GetBytes(dataChars);
>>                 s.Write(temp, 0, temp.Length);
>>                byte[] dataBytes = e.GetBytes(chars);
>>                //                for (int i = 0; i < dataBytes.Length; 
>> i++)
>> //                    if (dataBytes[i] == unusedbyte)
>> //                        dataBytes[i] = 0;
>>                s.Write(dataBytes, 0, dataBytes.Length);
>>             }
>>             else
>>             {
>>                 Console.WriteLine ();
>>             }
>>         }
>>     }
>> }
>>
>>
>> ------------------------------------------------------------------------
>>
>> _______________________________________________
>> Mono-devel-list mailing list
>> Mono-devel-list at lists.ximian.com
>> http://lists.ximian.com/mailman/listinfo/mono-devel-list
>
>