[Mono-dev] NUnit version for mono source tests

[Charlie Poole]

Hi Guys,

I hope I didn't speak up too quickly. Confession: I thought I was on
the monodevelop list, not the mono-devel list. Delivering a new
version of NUnit to MD users is relatively easy. Switching actual mono
development is another matter, as I know from past discussions.

Nevertheless...

What version of NUnit is installed with MonDevelop, what version is
installed (for users) with Mono and what version is used for Mono's
own tests are actually orthogonal decisions... except... that Mono
developers are also MonoDevelop and NUnit users...

Some stuff you probably already know about NUnit, but just in case...

1. The guts of NUnit's test discovery and execution reside in the
nunit.util, nunit.core and nunit.core.interfaces assemblies.
2. Users writing tests reference the nunit.framework assembly (and
should not be referencing the others).
3. The assemblies in #1 don't reference nunit.framework.
NUnit.framework doesn't reference the others.
4. Any version of util, core, core.interfaces > 2.0 can run tests
against any version of framework <= its own version.

So... barring bugs we have not discovered, NUnit 2.6.3 can run tests
written against the 2.4.8 framework. It should be very easy to
determine if there are any undiscovered bugs, since you have a large
set of tests. Just take a copy of NUnit 2.6.3 and unzip it into a
directory. Run the console specifying the path to your set of tests,
which reference the 2.4.8 framework. It will either work or not, and
will give us a measure of the effort involved.

I remember when you were using a modified version of (I think) NUnit
2.2 to test mono, It took a lot of effort to get it up to 2.4.8, but
that was changing the tests, not merely the runner.

A possible path to an upgrade would be...

1. Find out (starting with the experiment above) if your tests will
run under a 2.6.3 runner.

2. Fix MonoDevelop to use the 2.6.3 runner - since I'm sure many mono
devs run tests that way. For work with the console runner, this can be
skipped. I've read comments from users who have merely dropped in the
assemblies and they said it worked. I haven't tried it myself, but I
think I will when I get a chance. It would be necessary allow the user
to choose which version of nunit.framework they want to reference, but
that should be pretty easy.

3. Iteratively...
    - Pick some test assembly
    - Change it to reference the 2.6.3 framework.
    - Fix any problems
So long as you are using the 2.6.3 runner, there is no reason all your
tests have to use the same framework version.

I'm around to answer questions an I'm personally intererested in
working on step #2. I'd love to have someone working on the nunit
addin for monodevelop step up to help.

Charlie

PS: I'll post a link to a page with breaking changes since 2.4.8 when
I get it done.


On Sun, Oct 26, 2014 at 6:36 AM, Martin Thwaites
<monoforum at my2cents.co.uk> wrote:
> I'm sure Charlie can tell us what considerations there need to be with
> upgrading from 2.4.8 to 2.6.3 (We have the best resource to do that!).
>
> My perception is that the jump to 3.0 might be too big.
>
> On 26 October 2014 13:29, Miguel de Icaza <miguel at xamarin.com> wrote:
>>
>> There are.   Someone needs to survey in depth all the ramifications.
>> Once that happens, then we might consider it.
>>
>>
>> On Sunday, October 26, 2014, Martin Thwaites <monoforum at my2cents.co.uk>
>> wrote:
>>>
>>> Hi Miguel/Charlie,
>>>
>>> If there is no fundamental reason not to use a more recent version, then
>>> I'll add it to my list.
>>>
>>> I seem to recall that Nunit 3.0 is removing support for older framework
>>> versions, so if it removes 2.0 we won't be able to do it until "the big
>>> purge" happens in mono.
>>>
>>> Once I've got these bits done for the webstack, I'll have a look.  Unless
>>> someone beats me to it.
>>>
>>> Couple of questions.
>>>
>>> Charlie, were there any breaking changes between 2.4.8 and 2.6.3?  and
>>> dovyou have a link to the specific breaking changes between 2.6 and 3.0?
>>>
>>> Miguel, would you accept a PR from the community for this? Given that it
>>> may mean your company needs to update their tests too?
>>>
>>> Thanks
>>> Martin
>>>
>>> On 26 Oct 2014 06:20, "Charlie Poole" <charlie at nunit.com> wrote:
>>>>
>>>> Is there any way we can help?
>>>>
>>>> 1) With 2.6.3 / 2.6.4 (coming up) - should be a small effort.
>>>> 2) With 3.0 - likely to be a bigger job but easier to maintain
>>>> thereafter.
>>>>
>>>> Maybe some monodevelop guy wants to work with us?
>>>>
>>>> Charlie
>>>>
>>>> On Sat, Oct 25, 2014 at 8:55 PM, Miguel de Icaza <miguel at xamarin.com>
>>>> wrote:
>>>> > Time.
>>>> >
>>>> > On Sat, Oct 25, 2014 at 3:22 PM, Martin Thwaites
>>>> > <monoforum at my2cents.co.uk>
>>>> > wrote:
>>>> >>
>>>> >> Hi all,
>>>> >>
>>>> >> I've been writing some unit tests recently and I've noticed (the hard
>>>> >> way,
>>>> >> by using things that don't exist like Assert.Throws and having to
>>>> >> re-write
>>>> >> them) that we're using an old version of NUnit (2.4.8).  Is there a
>>>> >> reason
>>>> >> for not upgrading to something more current?
>>>> >>
>>>> >> The current version is 2.6.3 (and I believe it's mono compliant),
>>>> >> however,
>>>> >> very soon there will be an amazing new version that will enable tests
>>>> >> to run
>>>> >> in parallel out of the box (by amending some of the unit tests).
>>>> >>
>>>> >> Thanks,
>>>> >> Martin
>>>> >>
>>>> >> _______________________________________________
>>>> >> Mono-devel-list mailing list
>>>> >> Mono-devel-list at lists.ximian.com
>>>> >> http://lists.ximian.com/mailman/listinfo/mono-devel-list
>>>> >>
>>>> >
>>>> >
>>>> > _______________________________________________
>>>> > Mono-devel-list mailing list
>>>> > Mono-devel-list at lists.ximian.com
>>>> > http://lists.ximian.com/mailman/listinfo/mono-devel-list
>>>> >
>
>