[Mono-list] mono performance, 20x differential with Java (what am i doing wrong)

[Jonathan Shore]

On Jan 29, 2010, at 2:32 PM, Jon Harrop wrote:

> On Friday 29 January 2010 02:00:07 Jonathan Shore wrote:
>> My main interest is in Ocaml, particularly the F# variant as the basis for
>> my numerical work.
> 
> Note that F# uses ILX that Mono does not implement correctly, e.g. TCO. So F# 
> code is not yet reliable on Mono.
> 

Jon, I saw your post about that on your blog some time ago.   Someone familiar with Mono claimed otherwise, was therefore uncertain as to whether was addressed or not.    I can live some some inefficiency in tail calls provided one does not get stack overflow or some other fatal issue.


> 
>> I have heard only good things about LLVM performance, so hoping that this
>> will help address this gap. 
> 
> To really benefit from LLVM you need to design the VM properly from the ground 
> up. My HLVM project aims to do this:
> 
>  http://www.ffconsultancy.com/ocaml/hlvm/
> 

I've seen your posts on this and is very impressive.    To be honest I would get more value out of a Ocaml variant wedded to the .NET platform.   There is just so much momentum and available libraries on the two major VMs (CLR and JVM), that would be a huge risk for me at the moment.    I also have a significant body of imperative VM-bound code that I need to get access to.    If HLVM could interact with java bytecode or .NET bytecode, would work for me.

> I haven't benchmarked it against Mono but it is already thrashing Java on 
> numerical benchmarks:
> 
> http://flyingfrogblog.blogspot.com/2010/01/hlvm-on-ray-tracer-language-comparison.html
> 

Again, very impressive stuff.   Do you see bridging between the .NET world and your VM in the future?    For instance the IKVM project that maps Java bytecode to .NET built up a joint project with the mono team to provide the ability to run Java byetcode in mono.    A similar concept could be done in this setting.


> The above line is dead code. The JVM is probably eliminating it and .NET does 
> not. Removing this dead code by hand, I obtain the same result from .NET in 
> the same time that the JVM takes.

Yes, thanks realize this.  The benchmark is flawed.    I had meant to place i - j in the inner loop.       I'm going to test this again with the correction.  Will post results.

> 
>> 				for (int j = 1 ; j < 8 ; j++)
>> 					sum += 1.3 * vec[j-1];
>> 			}
>> 
>> 			return sum;
>> 		}
> 
> To build a useful benchmark you should set an irreducibly-complex problem to 
> solve and let people solve it freely in different languages using whatever 
> features and characteristics of the language or VM they choose.
> 

I've seen the language shootouts, but they appear flawed to me.   The length of the tests is too short to allow the Java or Mono VMs to reap benefits from optimisation.    The main problem I have found in VM environments (aside from the GC, boxing / unboxing) is array access.    One can work around issues of GC, boxing, etc, but cannot work around array issues if they exist.

Jonathan