[Mono-list] Some random notes about writing tests

Martin Baulig martin@gnome.org
01 Mar 2002 23:59:09 +0100

Hi guys,

after doing the testing work for several days on Linux I start to get a feeling
about how to write good tests - and I also found examples for bad tests. So I
try to write a little style guide here.

1.) Constructors

    When writing your testcase, please make sure to provide a constructor which
    takes no arguments:

        public class DateTimeTest : TestCase

                public DateTimeTest() : base ("MonoTests.System.DateTimeTest") {}
                public DateTimeTest (string name): base(name) {}

        	public static ITest Suite
                        get {
                                TestSuite suite = new TestSuite ();
        			return suite;

    This is important since we derive from the DateTimeTest class in the automatically
    generated TheTests.cs like this:

	public class RunDateTimeTest : DateTimeTest
		protected override void RunTest ()
                        // ....

2.) Don't compare two values with Assert(), always use AssertEquals() - this gives you
    a better feeling what went wrong when the test fails.


        Assert ("A01", myTicks[0] == t1.Ticks);


        AssertEquals ("A01", myTicks[0], t1.Ticks);

3.) Use unique labels for your Assert* () statements - the label is to tell you _which_
    test failed, not _what_ failed.


	AssertEquals("array match", compare[0], i1[0]);
	AssertEquals("array match", compare[1], i1[1]);
	AssertEquals("array match", compare[2], i1[2]);
	AssertEquals("array match", compare[3], i1[3]);


	AssertEquals("#A01", compare[0], i1[0]);
	AssertEquals("#A02", compare[1], i1[1]);
	AssertEquals("#A03", compare[2], i1[2]);
	AssertEquals("#A04", compare[3], i1[3]);

    Remember, you get a test report which looks like this:

    2) MonoTests.System.ArrayTest(MonoTests.System.RunArrayTest) "array match"

    And now how do you find out which of the 4 AssertEquals failed.

    Better is

    2) MonoTests.System.ArrayTest(MonoTests.System.RunArrayTest) "#A03"

    It's also much easier to write "ArrayTest/A03" failed in a bug report.

4.) Don't change the labels after they're in use.

    If you already have

        Assert ("#A01", something);
        Assert ("#A02", something_else);

    and want to insert something between them, choose a new number but don't rename the
    existing ones - people might be using the labels in bug reports, Bugzilla or on
    mailing lists and get confused if bug ArrayTest/A02 is suddently something else.

That's it for the moment. Comments, suggestions, feedback is very welcome :-)

Hmm, maybe we should put this document into CVS and edit it there.

Martin Baulig