[Mono-dev] The State Of Mono Assembly Verification?

[Paolo Molaro]

On 02/16/06 Jim Purbrick wrote:
> OK, I've uploaded a first batch of ~230
> verifier tests based on ECMA-335 III here:
> 
> http://homepage.ntlworld.com/james.purbrick/VerifierTests.tar

Thanks ! This is a good start.
I looked a a few of them and they look fine.

> I'd appreciate it if you could take a look and let me
> know if they look OK, especially the III,3.1 and
> III,3.3 tests which I'll mechanically copy and modify
> to make most of the remaining tests. Feel free to

I'd like this to be done a bit differently. Take the existing tests
you made and change (for example for the 3.1 ones)
add with <BINARY_NUM_OP>. Then a little script will
copy and replace and create tests for each of a list of opcodes.
This way, we only have to check N test case + the list of opcodes
(+ the script, of course;-), instead of checking N * numopcodes tests.

This pattern should be used as much as possible: there are many
tests that look the same but apply to a number of similar opcodes
and we want the test to check all the cases for all the opcodes
while keeping the test source readable.
It would be good to put different kind of tests in different
directories, too.

> Most of these tests assemble with both Mono and MS
> ILASM except in the cases where the CIL is so borked
> that it doesn't make sense (I've left these tests in
> for completeness though).

We should put these in a separate directory, so that later we
can make our ilasm compile them, maybe with a special command line flag.

> Happily most of the tests are caught by the Mono
> runtime which reports that invalid CIL has been found
> and aborts (Mono 1.1.13). There are cases that cause
> the runtime to crash and others which just run
> silently though.

I used this makefile as a starting point:

compile: *.cil
	for i in *.cil; do ilasm $$i; done

test:
	for i in *.exe; do mono $$i >/dev/null 2>/dev/null && echo $$i "NOT ok"; done

We'll take a look at the ilasm failures (it is also important that ilasm
doesn't leave behind 0-length files when it fails).
I will probably add a mono option like --debug-expect-exception=System.InvalidProgramException
so we can more easily check for cases where mono exits with an error and
when it exits because of the correct exception.

lupus

-- 
-----------------------------------------------------------------
lupus at debian.org                                     debian/rules
lupus at ximian.com                             Monkeys do it better