[Mono-devel-list] NUnit Regression test suites.

[Zoltan Varga]

                       Hi,

  I think we should mark failing tests with [Ignore("FAIL-ON-MONO")].

              Zoltan


On Wed, 12 Jan 2005 12:34:14 -0500, Miguel de Icaza <miguel at novell.com> wrote:
> Hey,
> 
>     We have a problem with our NUnit regression test suites: today we
> have a mix of tests that pass and some that do not pass.  The later are
> bugs that have existed forever in our runtime.
> 
>     The problem with having these failures in our test suites is that we
> unlike the runtime or the C# compiler, we assume that having errors
> there is fine and we do not actively track those after a build, because
> we can not tell the difference between known bugs or recently introduced
> bugs.
> 
>     I would personally like to move tests that are known to fail into a
> separate test assembly, so we at least have `positive' tests that are
> supposed to work never fail.  This means that we can put them on the
> tinderbox and track when we introduce a regression on the runtime,
> currently days might pass before we find out about a regression.
> 
>     There are various possible approaches here:
> 
>         * Build system changes to support two different assemblies
>           (known passing, and known failures).
> 
>         * Ifdef tests that fail (this makes it hard to actively track
>           down bugs and fix them).
> 
>         * Ifdef tests that fail, but file a bugzilla report for each
>           failure (might clog Bugzilla).
> 
>         * Fix every failure (not likely to happen anytime soon ;-)
> 
>     Thoughts?
> 
> miguel.
> 
> _______________________________________________
> Mono-devel-list mailing list
> Mono-devel-list at lists.ximian.com
> http://lists.ximian.com/mailman/listinfo/mono-devel-list
>