[Mono-list] Computer Language Shootout Benchmarks

[Ben Maurer]

On Sat, 2005-05-14 at 15:42 -0400, Miguel de Icaza wrote:
> Hello,
> 
> > now using Mono 1.1.6
> > 
> > http://shootout.alioth.debian.org/great/benchmark.php?test=all〈=csharp&sort=fullcpu
> > 
> > We really welcome improvements/alternatives to the existing programs,
> > and contributions for the micro-benchmarks we haven't done yet.

Am wondering about how you guys measured memory size. For example, you
claim that for a Hello World, mono uses 7.2 MB of ram. However:

[benm at omega benm]$ cat zzz.cs
using System;
class X {
        static void Main ()
        {
                Console.ReadLine ();
        }
}
[benm at omega benm]$ mcs zzz.cs
[benm at omega benm]$ mono zzz.exe

<on another term>

[benm at omega benm]$ ps u -C mono
USER       PID %CPU %MEM   VSZ  RSS TTY      STAT START   TIME COMMAND
benm      6557  0.0  0.3 13572 3648 pts/12   S+   16:25   0:00 mono zzz.exe

That puts it nowhere near as much as your measurements. This is SVN
head. Maybe the IO-layer daemon is messing things up, and 1.1.7 will fix
things up.

One other comment about memory usage -- it would be interesting to look
at what the non-shared startup cost of each program is. A decent way to
approximate this is to do the following

<run one mono process>
pre=<record amount of free ram>
<run another 100 processes>
post=<record amount of free ram>

non_shared_mem_per_proc = (post - pre) / 100

Of course, this assumes that your machine has enough physical memory so
that nothing gets swapped out while you run the extra processes. A much
better way to measure this would be to insert code into the kernel that
tells the RSS per mapping, so that you can discount the RSS of shared
mappings. RML did a hack like this for me, but I don't quite trust it
yet.

I'd also like to see some benchmarks that focus on allocation behavior.
For example:

      * Do tons of allocations, and then free everything except for
        randomly scattered things. Then do 10 mb more of allocation and
        free them all (focuses on fragmentation -- moving gcs should be
        able to compact the heap)
      * Do enough allocations so that cache locality matters:
        http://blogs.msdn.com/ricom/archive/2005/01/05/347361.aspx is a
        really good benchmark, it'd be cool to have something like it.

-- Ben