[Mono-list] mono performance, 20x differential with Java (what am i doing wrong)

[Jon Harrop]

On Friday 29 January 2010 02:00:07 Jonathan Shore wrote:
> My main interest is in Ocaml, particularly the F# variant as the basis for
> my numerical work.

Note that F# uses ILX that Mono does not implement correctly, e.g. TCO. So F# 
code is not yet reliable on Mono.

> One of the first things I do when considering a platform is run benchmarks,
> as performance is critical for what I do.

One of the first things I do when considering a platform is run tests, as 
correctness is critical for what I do. Mono failed so I use .NET.

> I have heard only good things about LLVM performance, so hoping that this
> will help address this gap. 

To really benefit from LLVM you need to design the VM properly from the ground 
up. My HLVM project aims to do this:

  http://www.ffconsultancy.com/ocaml/hlvm/

I haven't benchmarked it against Mono but it is already thrashing Java on 
numerical benchmarks:

http://flyingfrogblog.blogspot.com/2010/01/hlvm-on-ray-tracer-language-comparison.html

HLVM fully supports TCO and has an accurate GC.

> using System;
>
> namespace Performance
> {
>
> 	public class ArrayTest
> 	{
>
> 		public static double test1 (double[] vec)
> 		{
> 			double sum = 0;
> 			for (int i = 8 ; i < vec.Length ; i++)
> 			{
> 				vec[i] = 2*vec[i] - vec[i-1];

The above line is dead code. The JVM is probably eliminating it and .NET does 
not. Removing this dead code by hand, I obtain the same result from .NET in 
the same time that the JVM takes.

> 				for (int j = 1 ; j < 8 ; j++)
> 					sum += 1.3 * vec[j-1];
> 			}
>
> 			return sum;
> 		}

Porting solutions between languages can be bad science in the context of 
benchmarks because it taints your results with the original language.

To build a useful benchmark you should set an irreducibly-complex problem to 
solve and let people solve it freely in different languages using whatever 
features and characteristics of the language or VM they choose.

For example, in the context of technical computing the JVM's lack of value 
types is a crippling problem that afflicts everything from complex numbers 
and low dimensional vectors to hash tables. I haven't tested it but you 
should find that Mono's hash table implementation destroys the JVM's. Java's 
generics are also crippled and it doesn't even support tail call 
elimination...

-- 
Dr Jon Harrop, Flying Frog Consultancy Ltd.
http://www.ffconsultancy.com/?e