[Mono-devel-list] no regression policy for System.XML

[Jose Cornado]

Yes, I tried cumulative test runs, etc. I did not get stuck with my idea. 

I was just suggesting something I have seen work successfully before
in a really taxing environment (data base engine development where
millions of SQL queries with a lot network, disk IO and sql
compilation complexity were executed and verified in really short
periods of time)

The benefit of that approach was that the code was always near release
quality with little extra effort/invesment in terms of people or time.
This benefit expands to every future release.

We all have seen/expeirienced the scrambling that goes on when
shipping time nears.

Test/unit of time ratio:

Simple economics: maximize the number of outputs while minimizing the
number of inputs.

Then, this was just a suggestion.

José

On 5/30/05, Atsushi Eno <atsushi at ximian.com> wrote:
> Hello,
> 
> > I think this should apply to the rest of the code base: the only real
> > solution is to run everything with each build. The Mythical Man Month
> > comes to mind.
> 
> It is not only about automated builds but also about hackers' builds.
> 
> Have you ever tried bugfixing and accumulative test run? Am pretty
> sure that you will get stuck with your idea.
> 
> Also, some of the buildbot machines runs slowly which is inevitable,
> e.g. cygwin build. Similarly, on my previous slower machine I needed
> a couple of minutes to just run OASIS XSLT tests (no MSFT tests).
> The builds frequently run whenever we want to verify the build sanity.
> 
> So it is enough that the entire tests runs just *often*, say, once
> in a day (daily test).
> 
> > Excluding the slowest test will be risky. They are slow because they
> > hit critical/complex paths more often than not. A bug caught here is
> > worth more than anywhere else.
> 
> Actually differentiating "slower test" and "not working tests" makes
> little sense (we don't differentiate now).
> 
> What Andrew had in mind was (AFAIK, only) two testcases from MSFT
> xslt tests whose output is megabytes of outputs.
> 
> > The problem to solve is to increase the test/unit of time ratio so the
> > slow and critical tests can be included.
> 
> I have no idea what you have in mind, but I don't think there are
> such concrete thresholds.
> 
> Atsushi Eno
>