spam: suspected: Re: [Mono-devel-list] Graphics merge-in status

[Jonathan Pryor]

On Thu, 2005-08-04 at 04:31 -0700, Andrew Skiba wrote:
> I understand that ability to run tests without windows box is an
> advantage. I thought to commit expected results in similar way we did
> with XML tests.

Alas, I'm not familiar with the XML tests.

> By explicit "update" mechanism do you mean any particular technique?

Technique?  None in particular.

For regression tests I write that rely on external files, I usually have
a "XXX-update" target.  For example, if "make check" runs the regression
tests, then "check-update" would update the external resources.

For example, in my (uncommitted) lb regression test makefile, I have:

	check:
		# Clear out previous `check' output
		-rm -Rf test/tmp
		# run lb for new output, placing output into test/tmp
		mono --debug lb.exe --blog-directory=`pwd`/test/in \
			--prefix=test/tmp # ...
		# compare new output to saved output
		diff -r test/out test/tmp

	check-update:
		# update saved output
		-rm -Rf test/out
		mono --debug lb.exe --blog-directory=`pwd`/test/in \
			--prefix=test/out # ...

How would you do this for System.Drawing?  Probably extract all the .NET
Compare() code into Update() methods, place all the Update() methods
into a new .cs file, build that into a .exe during `make test`, and then
a `test-update` makefile target which executes the new program.

There are likely better ways, but this is a simple approach. :-)

> Also, how do you suggest to solve the .NET bug problem?

For anything that would fail under .NET, mark the test with a
[Category("NotDotNet")] attribute declaration.  That way we ensure the
test keeps working under Mono but it won't be run under .NET (thus
avoiding spurious failures).

 - Jon