[Mono-devel-list] Test policy proposition

[RafaelMizrahi]

Ok, I have no problems with stylesheet'ing the stand_alone outputs into
NUnit xml report.
In fact, at mainsoft, we already did that before. We transformed the xml
tests catalog output to our reports system.
The least I can ask for is that they will be CSV format (comma separated
value).

rafi

-----Original Message-----
From: Atsushi Eno [mailto:atsushi at ximian.com] 
Sent: Monday, May 30, 2005 10:16 PM
To: RafaelMizrahi
Cc: 'Andrew Skiba'; 'mono-devel mailing list'; Oved
Subject: Re: [Mono-devel-list] Test policy proposition

Hi Rafi,

Ohh, ok, I should first say that I was talking about the results
on comparing NUnit's resulting xml (TestResult-*.xml) structures.

>>We don't have "time" data.
> 
> If I understand you, "time" can be Now().

Well, when it is about <test-results> element, you are right.

"time" attribute on each <test-suite> element is duration.

>>"asserts" are always "1".
> 
> Why is that ? the stand_alone test suite report will have one test with an
> assert for each xml test, that can pass or fail.

So here I meant "asserts" attribute in tests.

>>There are "known failures" rather than "success"
> 
> In this case, Maybe Andrew proposal should remove the "known failures" 

Oh, no it is useful (actually I dare say it is "required"). We should
not *remove* important bits just to make output to be equivalent to
NUnit stuff.

> And analyzing the status of the test suite should be done through the
NUnit
> reports system.
> My major goal is to have a unified report system, and not to have a
special
> report for each stand_alone test suite.
> I choose NUnit report because we (mainsoft and mono) already created a
> report system for NUnit reports.

Why not just write a stylesheet or another simple result aggregator
that assembles test results (known failures, ignore-lists etc.)
into NUnit-like output? It would be simpler than modifying existing
output and (more importantly) keep things more human-readable.

Atsushi Eno