[Mono-dev] NUnit migration and test failure status

Atsushi Eno atsushi at ximian.com
Sun Jan 4 22:58:58 EST 2009

We still have a *lot* of test failures in our unit tests, and I'd like
to give some updates on analysis.

(Note: I didn't walk through every failure. Also my build is not always
clean. Verify it at your side before reading them blindly.)

- System.Threading.ExecutionContextTest.Copy_FromThread:
   it does not fail on .NET, so it's likely a bug in runtime or corlib.
- System.Data.DataColumnTest.ExpressionAggregates:
   it is rather likely due to nunit change with related to current
   culture handling. This test does not fail when I give TESTNAME or
   FIXTURE option at run-test.
- System.Web.UI.WebControls.MenuTest.Menu_DefaultRender:
   similarly it does not fail when I run only this test (with TESTNAME).
- Mono.Data.Sqlite.SqliteCommandUnitTests.Select:
   I copied Mono.Data.Sqlite.dll to mcs/class/Mono.Data.Sqlite directory
   and did run-test-ondotnet. The result was the same failure as on mono,
   so I assume it is rather the test case that is wrong.
- the same applies to Mono.Data.SqliteClient.
- System.Runtime.Remoting: we have no remedy. We will have to either
   fully disable the tests for this assembly or revert NUnit to 2.2.
   FWIW they fail on .NET either.
- Microsoft.Build.Tasks
   It does not fail on mono/w32. Instead some tests in the assembly do
   not run successfully on mono/w32. Another set of tests fail on .NET.
- System.Transactions.AsyncTest.AsyncFail3:
   It doesn't fail on mono/w32 either.
- System.Web.UI.WebControls.ListViewTest.ListView_Edit:
   It doesn't, either.

Our nunit fails to handle System.Web_test_net_2_0.dll under .NET with
nunit-console, so I cannot verify if those sys.web tests are fine.

Thanks to Marek (grendel), the floating point issues are gone.
Also thanks to Sebastien, the drawing tests are fixed to pass.

Atsushi Eno

Atsushi Eno wrote:
> Hello,
> As some of you may have noticed, I have upgraded our local use of nunit
> from 2.2.0 to 2.4.8. It involved some changes in our test run, but
> should be invisble to most of you.
> The build has been broken for a while (am sorry for that; it was a set
> of make dist / packaging mess), and now we have a couple of test
> failures. So before Marc dives into the hell list of unknown bugs to
> him, I wanted to give some survey on them:
> - some JIT stuff: test-verifier, test-System_Web-2.0
> - mere build failure: test-System_Data-2.0
> - some floating point failures: test-corlib-*, test-System_Drawing-*,
>   test-System_Data-1.0, test-System_Runtime_SFS-*
> - possible NUnit run changes: test-corlib-2.0
>   (MonoTests.System.Threading.ExecutionContextTest.Copy_FromThread),
>   test-System_Runtime_Remoting-*
> - possibly gdiplus: test-System_Drawing-*, test-Windows_Forms-2.0
>   (MonoTests.System.Drawing.PenTest.DashCap_Valid)
> - some sys.data changes: test-Mono_Data_Sqlite-*,
>   test-Mono_Data_SqliteClient*
> The only pattern I am concerned is the one with (possible) NUnit run
> changes. NUnit 2.4.8 now requires -noshadow to get our tests run fine.
> This also happens under .NET (i.e. make run-test-ondotnet) even without
> -noshadow, so I am rather afraid that NUnit 2.4.8 unlike 2.2.0 cannot
> handle it.
> I'd like to hear inputs on what could be done to fix this issue.
> Atsushi Eno
> _______________________________________________
> Mono-devel-list mailing list
> Mono-devel-list at lists.ximian.com
> http://lists.ximian.com/mailman/listinfo/mono-devel-list

More information about the Mono-devel-list mailing list