[Mono-devel-list] NUnit Regression test suites.
vargaz at gmail.com
Wed Jan 12 18:35:32 EST 2005
I think we should mark failing tests with [Ignore("FAIL-ON-MONO")].
On Wed, 12 Jan 2005 12:34:14 -0500, Miguel de Icaza <miguel at novell.com> wrote:
> We have a problem with our NUnit regression test suites: today we
> have a mix of tests that pass and some that do not pass. The later are
> bugs that have existed forever in our runtime.
> The problem with having these failures in our test suites is that we
> unlike the runtime or the C# compiler, we assume that having errors
> there is fine and we do not actively track those after a build, because
> we can not tell the difference between known bugs or recently introduced
> I would personally like to move tests that are known to fail into a
> separate test assembly, so we at least have `positive' tests that are
> supposed to work never fail. This means that we can put them on the
> tinderbox and track when we introduce a regression on the runtime,
> currently days might pass before we find out about a regression.
> There are various possible approaches here:
> * Build system changes to support two different assemblies
> (known passing, and known failures).
> * Ifdef tests that fail (this makes it hard to actively track
> down bugs and fix them).
> * Ifdef tests that fail, but file a bugzilla report for each
> failure (might clog Bugzilla).
> * Fix every failure (not likely to happen anytime soon ;-)
> Mono-devel-list mailing list
> Mono-devel-list at lists.ximian.com
More information about the Mono-devel-list