[Mono-dev] Hacking mono to make it more deterministic

[Lucas Meijer]

Hey,

At Unity we have this internal testing system where we

1) play a game, and record all inputs
2) can replay that game, based on the recorded inputs
3) whenever we do a checkin on our sourcecode, we run all games in the 
testing setup against this new unity build, by making screenshots every 
5 seconds, and comparing those to the previous run.

This has proven to be very useful, as it finds regressions in our users 
games that we had not anticipated when committing a certain change to 
the unity codebase.

Currently, not all games are playing back deterministically on a single 
unity build. We don't think we'll ever reach full deterministicality (is 
that a word?) for 100% of our games, but the higher we get the number, 
the better we like it.

In order to do this, we mock out things like System.Random, 
DateTime.Now, to return the recorded values.

A frequent case where games fail to run identically is when our users 
iterate over a dictionary which has a custom type as its key. Since the 
dictionary uses GetHashCode() internally, and Object.GetHashCode() is 
based on the memory address of the object, it means that the order in 
which the KeyValuePair's get returned can differ per run.

Naturally our users shouldn't be doing this. But they do, as they don't 
realize that consistent ordering of iterating over a dictionary isn't 
guaranteed. it just works most of the time so most people don't notice.
I didn't realize this myself untill I started investigating this issue :)

I'm looking for advice on what would be the best solution to this.

So far the best idea I've came up with is to make mono_object_hash just 
return 0. That probably has some severe performance implications, but we 
can live with those. I'm hoping this will only make things go a lot 
slower, but wanted to ask this list:

- if I should expect other problems than slowdown
- if you can think of a better/different way to make our users code 
deterministic

Thanks, Lucas