[Mono-devel-list] Language Shootout - problems with Mono times

[Ben Maurer]

Hi,

Whenever I do performance testing, I take the `real' time. the user time
is unreliable.

My standard procedure is this:

1) Make sure my CPU is pretty idle.
2) Run my test 5-10 times
3) Take the *lowest* time (wall clock time)

Now why the lowest time? The lowest time corresponds to the time when my
computer was doing the least amount of non-benchmark related work.

> Sorry if this is somewhat OT.
>
> We're having problems getting consistent times for the Mono C# programs
> in  "The Great Computer Language Shootout"
> (Mono C# has been pulled from the shootout once already because of this
> problem - it would be good to fix it this time.)
>
> Mono C# is good but not this good ;-)
> http://shootout.alioth.debian.org/bench/sieve/
>
> Even Linux time gives inconsistent results:
>
>    root at lin mono_shootout # time mono sieve.exe 1000
>    Count: 1028
>
>    real    0m0.982s
>    user    0m0.920s
>    sys     0m0.030s
>
>    root at lin mono_shootout # time mono sieve.exe 1000
>    Count: 1028
>
>    real    0m0.977s
>    user    0m0.030s
>    sys     0m0.020s
>
>    root at lin mono_shootout # time mono sieve.exe 1000
>    Count: 1028
>
>    real    0m0.979s
>    user    0m0.920s
>    sys     0m0.020s
>
>    root at lin mono_shootout # time mono sieve.exe 1000
>    Count: 1028
>
>    real    0m0.978s
>    user    0m0.020s
>    sys     0m0.040
>
> Explanations and suggestions appreciated.
> (The shootout Perl script grabs user & sys times, forks, execs the mono
> program, and grabs user & sys times.)
>
>
>
>
> _______________________________
> Do you Yahoo!?
> Win 1 of 4,000 free domain names from Yahoo! Enter now.
> http://promotions.yahoo.com/goldrush
> _______________________________________________
> Mono-devel-list mailing list
> Mono-devel-list at lists.ximian.com
> http://lists.ximian.com/mailman/listinfo/mono-devel-list