[Mono-devel-list] System.Drawing Tests - image recognitionsimilarities

[RafaelMizrahi]

Ben,
* Manually comparing the difference between 2 outputs (mono and msft) can
suffer from the same problems that the automatic comparer will (anti-alias
etc')
* A binary comparison (pixel to pixel) is too strict, and can fail a lot of
test.
This is why I wish to combine (or to GPL ;) a collection of comparers which
together, can measure the similarities of 2 images.

Rafi
Paid for being pessimistic ;)

-----Original Message-----
From: Ben Maurer [mailto:bmaurer at ximian.com] 
Sent: Monday, May 30, 2005 5:15 PM
To: RafaelMizrahi
Cc: jordi at ximian.com; Oren Gurfinkel; mono-devel-list at lists.ximian.com
Subject: Re: [Mono-devel-list] System.Drawing Tests - image
recognitionsimilarities

On Mon, 2005-05-30 at 11:05 +0300, RafaelMizrahi wrote:
> Hi Jordi, (and mono devs)
> Mainsoft wish to invest some effort in testing System.Drawing.
> We wish to enhance the test suite with an image compare component which
will
> compare the difference between the expected .NET image and the mono image.

> 
> The comparer (which we just started prototyping it) will be customized and
> produce a result according to several algorithms:
> * Prior to compare: format, size, colors, resolution
> * Color - strip boarders; compare histograms to get top colors and areas. 
> * Edge Detect - edge detection, and line generation, compare line lists.
> * Histogram to detect shapes
> * FFT (or cosine transform) of sub-regions or the whole image.
> * Tangent space Vectors

What if, rather than comparing the mono and .net image -- which could be
different due to, eg, differences in the anti-alias methods used -- you
did the following:

      * Run the test suites on both mono and msft
      * Compare the images. If the images are similar enough to consider
        the test as passing, mark the test as "pass". Obviously this is
        a manual process.
      * Check in the binary images that are generated by *mono* for all
        sucessful runs
      * Test that the images stay the same pixel by pixel (ie, that the
        binary data of the image is the same).

This makes the assumption that in general, Cairo is deterministic in how
it renders an image. This is probably a reasonable assumption.

-- Ben