[Mono-devel-list] NUnit Regression test suites.

Zoltan Varga vargaz at gmail.com
Wed Jan 12 18:35:32 EST 2005


  I think we should mark failing tests with [Ignore("FAIL-ON-MONO")].


On Wed, 12 Jan 2005 12:34:14 -0500, Miguel de Icaza <miguel at novell.com> wrote:
> Hey,
>     We have a problem with our NUnit regression test suites: today we
> have a mix of tests that pass and some that do not pass.  The later are
> bugs that have existed forever in our runtime.
>     The problem with having these failures in our test suites is that we
> unlike the runtime or the C# compiler, we assume that having errors
> there is fine and we do not actively track those after a build, because
> we can not tell the difference between known bugs or recently introduced
> bugs.
>     I would personally like to move tests that are known to fail into a
> separate test assembly, so we at least have `positive' tests that are
> supposed to work never fail.  This means that we can put them on the
> tinderbox and track when we introduce a regression on the runtime,
> currently days might pass before we find out about a regression.
>     There are various possible approaches here:
>         * Build system changes to support two different assemblies
>           (known passing, and known failures).
>         * Ifdef tests that fail (this makes it hard to actively track
>           down bugs and fix them).
>         * Ifdef tests that fail, but file a bugzilla report for each
>           failure (might clog Bugzilla).
>         * Fix every failure (not likely to happen anytime soon ;-)
>     Thoughts?
> miguel.
> _______________________________________________
> Mono-devel-list mailing list
> Mono-devel-list at lists.ximian.com
> http://lists.ximian.com/mailman/listinfo/mono-devel-list

More information about the Mono-devel-list mailing list