[Mono-list] mono performance, 20x differential with Java (what am i doing wrong)

[Jon Harrop]

On Friday 29 January 2010 21:28:58 Jonathan Shore wrote:
> On Jan 29, 2010, at 2:32 PM, Jon Harrop wrote:
> > On Friday 29 January 2010 02:00:07 Jonathan Shore wrote:
> >> My main interest is in Ocaml, particularly the F# variant as the basis
> >> for my numerical work.
> >
> > Note that F# uses ILX that Mono does not implement correctly, e.g. TCO.
> > So F# code is not yet reliable on Mono.
>
> Jon, I saw your post about that on your blog some time ago.   Someone
> familiar with Mono claimed otherwise, was therefore uncertain as to whether
> was addressed or not.

You should be able to verify my results easily: just run the 8-line example F# 
program I gave and Mono will stack overflow.

> I can live some some inefficiency in tail calls provided one does not get
> stack overflow or some other fatal issue. 

TCO is broken on Mono, not merely inefficient.

> >> I have heard only good things about LLVM performance, so hoping that
> >> this will help address this gap.
> >
> > To really benefit from LLVM you need to design the VM properly from the
> > ground up. My HLVM project aims to do this:
> >
> >  http://www.ffconsultancy.com/ocaml/hlvm/
>
> I've seen your posts on this and is very impressive.

Thanks.

> To be honest I 
> would get more value out of a Ocaml variant wedded to the .NET platform.

Yes. F# is awesome but only on Windows/.NET and not on Mono.

> There is just so much momentum and available libraries on the two major VMs
> (CLR and JVM), that would be a huge risk for me at the moment.

I was actually disappointed with .NET's libraries in the context of technical 
computing. I felt OCaml had better libraries and it turns out that .NET was 
about as popular for technical computing as OCaml was when I started. The 
main exception is WPF but you don't get that with Mono.

> I also 
> have a significant body of imperative VM-bound code that I need to get
> access to.    If HLVM could interact with java bytecode or .NET bytecode,
> would work for me.

You should be able to compile plain numerical code from JVM/CIL to HLVM easily 
enough, particularly when HLVM is more complete.

> > I haven't benchmarked it against Mono but it is already thrashing Java on
> > numerical benchmarks:
> >
> > http://flyingfrogblog.blogspot.com/2010/01/hlvm-on-ray-tracer-language-co
> >mparison.html
>
> Again, very impressive stuff.   Do you see bridging between the .NET world
> and your VM in the future?

No.

> For instance the IKVM project that maps Java 
> bytecode to .NET built up a joint project with the mono team to provide the
> ability to run Java byetcode in mono.    A similar concept could be done in
> this setting.

That doesn't really interest me. F# is so far ahead now that everything else 
is a toy in comparison from my point of view. HLVM is just a hobby project 
designed to bring some of the benefits of F# to the open source world for fun 
but it is a massive undertaking because the open source world doesn't even 
have any reliable foundations like .NET, let alone decent libraries like WPF 
built upon them. So I have to build everything from scratch myself. I'm not 
even sure I will be able to use hardware acceleration due to the poor state 
of OpenGL drivers on Linux.

> > To build a useful benchmark you should set an irreducibly-complex problem
> > to solve and let people solve it freely in different languages using
> > whatever features and characteristics of the language or VM they choose.
>
> I've seen the language shootouts, but they appear flawed to me.

I've found k-nucleotide, spectralnorm and regex-dna interesting but you have 
to really study the code and be *very* careful when drawing conclusions.

For example, the Haskell implementation of k-nucleotide is only 20% slower 
than OCaml but the Haskell code is a joke: where the OCaml uses its stdlib's 
hash table, the Haskell had to do memory allocation manually using "malloc" 
directly in order to work around serious design flaws in their garbage 
collector. Not only is this hidden from the casual shootout reader but they 
even comment their code with total nonsense like "Hash tables are not 
generally used in functional languages" when what they really mean is "When 
hash tables are a good solution, Haskell will suck uniquely even among 
functional languages".

> The 
> length of the tests is too short to allow the Java or Mono VMs to reap
> benefits from optimisation.

Yes. You can crank up the resolution on my ray tracer benchmark to make it 
take as long as you like but I found it made little difference above the 
value I used.

> The main problem I have found in VM 
> environments (aside from the GC, boxing / unboxing) is array access.    One
> can work around issues of GC, boxing, etc, but cannot work around array
> issues if they exist.

You cannot work around boxing on the JVM because it lacks value types. Indeed, 
that is a major advantage of .NET on the JVM that Mono should inherit.

-- 
Dr Jon Harrop, Flying Frog Consultancy Ltd.
http://www.ffconsultancy.com/?e