[Mono-devel-list] Detecting Mono

[Ben Maurer]

On Mon, 2004-02-16 at 13:57, Charlie Poole wrote:
> So - using your example - if an application stored settings somewhere it
> should have acceptance tests to determine that settings are saved and
> stored correctly. The test should work whether the registry or a file or
> some other approach is used to saving the settings.
> 
> However, the same app would probably have a unit test for it's registry
> access code. That test would only be run in the win32 environment. If
> you're not heavily into unit testing and particulary TDD, this may not
> make sense. However, a large part of the Nunit user community is into
> those things and so are the developers. :-)

I don't really think that is valid. Tests should probe how the code
works on the *outside* not the inside. For example, you would not write
a test that used Reflection to get the fields of a class and test if
they were set to the right value. You would use a publicly exposed API.
Likewise, you shouldnt go about probing the storage format of your
settings.

Tests really shouldnt assert things about how the code looks inside.

I can see the argument here in a setup where, say, you could use a MySQL
db or a MSSQL db and were expected to interop with other programs.
However, I think that the best way to do this case is just to build
seperate assemblies for each provider and only run the tests on
platforms where the provider is supported. However, you dont need to
change NUnit to support this.

> Your thoughts on known failures are interesting. It had not occurred to
> me that there may be a difference between something that doesn't work on
> a platform - and never will - and something that is expected (hoped?) to
> work some time in the future. For some projects, the distinction is not
> important but it probably is for others. I'll have to think on that.

Actually, I am not thinking of this in terms of platforms, I am talking
about known failures in general. I really dont like [Ignore] because it
makes you *not* run the test.

You could extend this to allowing the decleration of groups by saying:

[Group ("FILED_BUGS")]
[Note ("see bug#12345"]
..

And then run on the command line like:

nunit /ignore-group:FILED_BUGS

or you could do:

nunit /known-fail:FILED_BUGS
(so maybe you would get a gold star or something if one of the known
failures was made green ;-).

-- Ben