[Mono-dev] Unhandled Exception in Normalization.cs Combine()

[Tom Hindle]

Hi Guys,

With regard to recent Normalization changes I have just run our test
suite with recent mono r136422 - and are getting a number of
regressions.


For example:

{
string styleName = "\u00e1bc";
StStyle style = new StStyle();
Cache.LangProject.StylesOC.Add(style);
style.Name = styleName;

FwStyleSheet.StyleInfoCollection styleCollection = new
FwStyleSheet.StyleInfoCollection();
styleCollection.Add(new BaseStyleInfo(style));

Assert.IsTrue(styleCollection.Contains(styleName.Normalize(NormalizationForm.FormC)));	Assert.IsTrue(styleCollection.Contains(styleName.Normalize(NormalizationForm.FormD)));			Assert.IsTrue(styleCollection.Contains(styleName.Normalize(NormalizationForm.FormKC)));	Assert.IsTrue(styleCollection.Contains(styleName.Normalize(NormalizationForm.FormKD)));
}

is now failing, as well as other larger unit tests.

I will look info this further to try and produce an example test program
that doesn't contain references to our code base.

Thanks
Tom

On Thu, 2009-06-18 at 15:01 +0900, Atsushi Eno wrote:
> Hi,
> 
> If you mean the test cases by the previous email, then that's what
> (I said) includes raw native encoding in your land (Latin1?) and is
> what I cannot read. Replace them all with ASCII representation (\uxxxx).
> 
> Even if the attachment includes encoding (you mean BOMs?), it is
> not readable in some environment (like the text editor I use on
> Windows). Let me repeat, Latin1 is not universal. Don't depend on it
> (if you do).
> 
> Atsushi Eno
> 
> 
> Tom Philpot wrote:
> > Atsushi,
> > 
> > Thanks for the feedback. For some reason, the Mac when displaying 
> > unicode always composes strings before display. I'll look at the test 
> > case in corlib tomorrow when I get in to work. Would it be helpful for 
> > the test cases if I gave you both the formD bytes and the formC bytes 
> > that I think are correct for the test case I sent? Perhaps the encoding 
> > did not come across in the attachment.
> > 
> > We have a workaround for the Mac port of our app which would require 
> > overriding string.Normalize to p/invoke to Mac OS X's NSString library 
> > to do normalization. It would work, but we would prefer not to have to 
> > ship a custom build of Mono. The normalization on .NET appears to be 
> > "good enough" for our purposes and we'd just like our Mac version to be 
> > consistent.
> > 
> > Tom
> > 
> > -----Original Message-----
> > From: Atsushi Eno [mailto:atsushieno at veritas-vos-liberabit.com]
> > Sent: Wed 6/17/2009 7:51 PM
> > To: Tom Philpot
> > Cc: mono-devel-list at ximian.com
> > Subject: Re: [Mono-dev] Unhandled Exception in Normalization.cs Combine()
> > 
> > You seem to have embedded raw native encoding in your land that
> > is *not* understandable in Japan. Anyways the input string you
> > posted in the previous sample was already in FormC which will
> > look like "doing nothing" as the conversion results.
> > 
> > There is a standalone normalization test generated from normalization
> > conformance test in corlib/Mono.Globalization.Unicode. We fail
> > about 26000. Far from good, but still better than 35000 on .NET.
> > 
> > Atsushi Eno
> > 
> > Tom Philpot wrote:
> >  > Now, string.Normalize(NormalizationForm.FormC) doesn't do anything using
> >  > mono (r136228).
> >  >
> >  > I've attached some test cases which will hopefully help in tracking down
> >  > what doesn't work.
> >  >
> >  > On 6/15/09 1:58 AM, "Atsushi Eno" <atsushieno at veritas-vos-liberabit.com>
> >  > wrote:
> >  >
> >  >> Hi again,
> >  >>
> >  >> It should be now fixed in trunk.
> >  >>
> >  >> Atsushi Eno
> >  >>
> >  >> Atsushi Eno wrote:
> >  >>> I'll have a look. However since 4 years have passed since I wrote it,
> >  >>> I'll have to revisit the spec and will take not a little time.
> >  >>>
> >  >>> Atsushi Eno
> >  >>>
> >  >
> > 
> > 
> 
> _______________________________________________
> Mono-devel-list mailing list
> Mono-devel-list at lists.ximian.com
> http://lists.ximian.com/mailman/listinfo/mono-devel-list