[Mono-dev] Replacing/Removing I18N

[Andreas Nahr]

Additionally:

If the 10MB are too large there could be the option to split the tables into 
a lower and upper table (the middle range is unused by single-byte 
encodings) reducing the table size to about 12-16kb.
I don't think its worth it though because in 99% of the cases this just 
means trading HDD space for speed (for the more complex handling) which IMHO 
is no good deal.


> To not let our thoughts go wrong way: the reason why existing
> I18N.*.dll consist of a bunch of IL code and very few static data
> is just because those conversion tables are just made into a bunch
> of switch-cases.
>
> The encoding conversion tables definitely do not have to reside in
> mscorlib.dll. It is a lot of waste of resources.
>
> They could just be other set of resources like per-encoding dlls
> (which means that the refactoring is totally meaningless, only
> hobbiests' satisfaction) or some kind of tables which have different
> loading rules than managed dlls (for example, something like *.nlp
> on .NET).
>
> I'm not a fan of this specific restructuring anyways: so niche.
>
> Atsushi Eno
>
> Andreas Nahr wrote:
>> Hi,
>>
>>> Hello,
>>>
>>>> * Creating the binary data should be simple when generating from a
>>>> .Net VM.
>>>> Would it be allowed to gernerate directly from MS.Net? From
>>>> Portable.Net?
>>>> (obviously from Mono is no problem, but would not allow to ADD data)
>>>
>>> I did not understand this question at all.
>>
>> Well the question is: Is it allowed to "extract" Data from .Net if it
>> can be extracted using a normal application (I attached an application
>> that might be suited for the task)
>>
>>>> * Size of a memmaped page?
>>>
>>> 4k or 8k, depending on the platform.
>>
>> This would be a perfect size.
>>
>>>> * Growth in *file*size for corelib acceptable? Altogether probably
>>>> 5-10MB
>>>
>>> Do we really need to grow corlib?   What do you have in mind?
>>>
>>> Couldnt we just use static data, and access that as a resource?  (Mono
>>> uses mmap for resources in the file)
>>
>> Do you mean something like: private static readonly byte[]...?
>> This seems to come with some overhead for me. Also it doesn't seem to
>> produce shareable memory between processes.
>> So the idea was (see original mail) to embed the datatables as resources
>> into the corelib assembly.
>>
>>> Am not sure how much code vs tables lives in the I18N libraries, do you
>>> have details?
>>
>> Currently the I18N assemblies are about 1MB IL-compiled in size. About
>> 98% is IL-Code, 1-2% is static data.
>> With the suggested implementation we would have (for single byte
>> encodings) basically 0% IL and 100% resource-data.
>> My MS.Net has 145 encodings, mono currently has 95 total.
>> Each (single-byte (approx. 95 in MS.Net)) encoding would have about 65kb
>> of tables (the idea is to create one resource per encoding)
>>
>>>> * Other sideeffects possible?
>>>>
>>>>
>>>> Greetings
>>>> Andreas
>>>> _______________________________________________
>>>> Mono-devel-list mailing list
>>>> Mono-devel-list at lists.ximian.com
>>>> http://lists.ximian.com/mailman/listinfo/mono-devel-list
>>> -- 
>>> Miguel de Icaza <miguel at ximian.com>
>>>
>> using System;
>> using System.Text;
>> using System.IO;
>>
>> class CreateEncodingData
>> {
>>     private static string path = @"c:\st\out\";
>>
>>     public static void Main ()
>>     {
>>         byte[] bytes = new byte[256];
>>         for (int i = 0; i < 256; i++)
>>             bytes[i] = (byte)i;
>>
>>         char[] chars = new char[65536];
>>         for (int i = 0; i < 65536; i++)
>>             chars[i] = (char)i;
>>
>>         EncodingInfo[] encodings = Encoding.GetEncodings();
>>         Console.WriteLine ("Encoding count: " +  encodings.Length);
>>
>>         foreach (EncodingInfo info in encodings)
>>         {
>>             Encoding e = Encoding.GetEncoding(info.CodePage);
>>             Console.Write ("Encoding: " + e.CodePage + " (" + e.WebName
>> + ";" + info.DisplayName + ")");
>>
>>             if (e.IsSingleByte)
>>             {
>>                 Console.WriteLine (" - single byte");
>>                 //byte unusedbyte = (e.GetBytes(new char[] { (char)30000
>> }))[0];
>>
>>                 Stream s = new FileStream (Path.Combine(path, e.CodePage
>> + ".bin"), FileMode.Create);
>>
>>
>>                 char[] dataChars = e.GetChars(bytes);
>>                 byte[] temp = Encoding.Unicode.GetBytes(dataChars);
>>                 s.Write(temp, 0, temp.Length);
>>
>>                 byte[] dataBytes = e.GetBytes(chars);
>>
>> //                for (int i = 0; i < dataBytes.Length; i++)
>> //                    if (dataBytes[i] == unusedbyte)
>> //                        dataBytes[i] = 0;
>>
>>                 s.Write(dataBytes, 0, dataBytes.Length);
>>             }
>>             else
>>             {
>>                 Console.WriteLine ();
>>             }
>>         }
>>     }
>> }
>>
>>
>> ------------------------------------------------------------------------
>>
>> _______________________________________________
>> Mono-devel-list mailing list
>> Mono-devel-list at lists.ximian.com
>> http://lists.ximian.com/mailman/listinfo/mono-devel-list
>
> _______________________________________________
> Mono-devel-list mailing list
> Mono-devel-list at lists.ximian.com
> http://lists.ximian.com/mailman/listinfo/mono-devel-list
>