[Mono-list] SPAM-LOW: Re: NUnit Version - Upgrade soon?

[Gert Driesen]

> -----Original Message-----
> From: Charlie Poole [mailto:charlie at nunit.com] 
> Sent: dinsdag 25 november 2008 22:18
> To: 'Gert Driesen'; 'Charlie Poole'; 'Juan C. Olivares'; 'Sebastien
Pouliot'
> Cc: mono-list at lists.ximian.com
> Subject: RE: [Mono-list] SPAM-LOW: Re: NUnit Version - Upgrade soon?
>
> Hi Gert, 

Hi Charlie,

Sorry for not responding any sooner (sleep/work/child, you know the deal).

> <snipped my earlier note/>
>
> > I actually wanted to ship a NAnt wrapper task for 
> > nunit-console (packaged in either the NAnt or NUnit 
> > distribution) that allows users to run unit tests on 
> > different target frameworks.
>
> It's true that a process command-line interface is an easy
> way to provide compatibility. But nunit-console is designed
> for direct user usage and could easily change to invalidate
> what you want to do.

Apart from the framework issue, I don't see why a wrapper task would not be
an option.

The command line interface should of course remain compatible, but breaking
changes there would affect not just this wrapper task.

If that wrapper task were part of NUnit, then you could keep both in sync
anyway.

> Alternatively, nunit-agent is intended for running tests
> but has the disadvantage that it communicates via remoting,
> requiring at least one assembly in common between it and
> the client.

To be honest, I never looked at that option. Should I consider this?

> So maybe a special-purpose runner that functions logically
> the way nunit-console does is a better approach.

Sure, but then NUnit needs to know how to run a test on a given CLR
(version).

> Still, I haven't yet given up on a client/server approach.
>
> > To make things easier for users (and to avoid bug reports 
> > where users use the wrong nunit.framework flavor with a given 
> > CLR, eg. 2.0 nunit.framework on 1.0 CLR), I was hoping not to 
> > require them to copy the right flavor of nunit.framework (or 
> > any other framework) in the loader path.
>
> Hmmm... it seems as if such a problem should only impact users
> who already use multiple versions of NUnit in their application.
> And they have to deal with it in any case.

People use a single version of NUnit, and want to run tests on multiple
CLRs.

> My assumption is that NUnit would not be installed on a 
> build machine in such a case anyway, but that the proper
> assemblies would be included in the tree. I agree that
> their will be errors if the user has to go out to an
> install directory and find assemblies.
>
> > However, for this to work with xcopy deployment NUnit needs 
> > to fall back to the framework that it ships with.
>
> NUnit could fall back on the framework it shipped with if it
> actually used the framework. However, it doesn't. NUnit has
> no reference to the framework. It executes the user tests
> directly and those tests execute framework code.
>
> We could modify NUnit, of course, to know about it's
> framework, but that would eliminate the ability to run
> against more than one framework.

Why would it? If you'd only fall back to loading the nunit.framework.dll
that ships with NUnit (in case of an assembly load failure for
nunit.framework), then there would be no problem.

> AFAIK, NUnit is not xcopy-deployed under Mono. And its
> not xcopy-deployed by default under Windows. The Mono
> install has NUnit in it's GAC - if I understand correctly.
> The Windows installer doesn't use the GAC, but does other
> things besides just copying files. OTOH, NUnit can be
> xcopy-deployed, as is now done by teams that copy the 
> binaries into their tree.

Yes, that's what NAnt does too. We actually have multiple CLR (version)
flavors in our tree.

> I'm feeling a little fuzzy about the benefits you see
> from xcopy deployment.
>
> > Automatically copying the nunit.framework to the directory 
> > containing the test assembly is definitely not an option, as 
> > we might not even have write access there.
>
> Agreed. And even if we did have write access, copying files
> around without being asked is pretty intrusive.
>
> > Right now people use the <nunit2> task - which uses the NUnit 
> > API - and expect it to run their tests on the current target 
> > framework, while in fact it will always run them on the CLR 
> > in which NAnt is running.
>
> That's because the current API doesn't allow you to specify
> what framework to use. But the new API does.

I'll have to check this out then. Is it extensible, or does it only allow
NUnit to run against a predefined set of frameworks/CLRs?

> > Do you have other ideas on how we can improve the experience 
> > for users?
>
> Under the current API, you would have to locate the correct 
> NUnit assemblies and make use of them. I believe that's what
> is currently done in tasks like the csc task. Of course, it's
> possible that the user may have installed only the 2.0 version
> of NUnit, in which case it's just not possible to run tests
> in 1.0.
>
> This is fixed in 2.5, since you get both builds of NUnit in 
> one installation package.

This is not an issue for NAnt anyway, as we ship with all necessary CLR
flavors of NUnit (and if the NUnit tasks were part of NUnit, then users
could drop in others versions).

> <snip/>
> > > I do prefer using the GAC to rolling my own loading mechanism, btw.
> > 
> > I don't think NUnit should rely on the GAC, but that doesn't 
> > mean its assemblies cannot be installed in the GAC (when its 
> > installed through package management systems or an installer).
>
> So you're suggesting that NUnit should detect whether certain
> assemblies are installed in the GAC and act differently?

No, definitely not. I'm only saying that you should not (meaning, I'd prefer
if you wouldn't) introduce changes that force users to install NUnit
assemblies in the GAC (or copy them locally). An example of such "breakage"
would be to stop preloading the nunit.core assembly in the test domain.

Gert