[Mono-list] JVM performance: JVM as a basis for CLR

[Jay Freeman (saurik)]

Tom:

Regardless of the fact that I agree 100% on the comments of starting with a
JVM (especially ORP, as I've been arguing with Miguel about for months now),
this comparison isn't really that fair.  You are comparing Beta 2 of a
product that has _specifically_ been mentioned in the past by Microsoft to
not have been optimized for speed for purposes of the beta tests against a
standing JVM.

Secondly, the comparison also doesn't really mean anything.  I don't see
what you are trying to show by this... that Java byte-codes are a better
intedmediate representation than Microsoft IL for writing JITs?  I bet I
could find a JVM that performs worse than your JVM.  Does that mean that
Java byte-codes are a better intermediate representation than Java
byte-codes?  Just because a specific implementation of the CLR is worse off
than a specific implementation of a JVM doesn't mean that Mono couldn't be a
better implementation.  Whether that is feasible or not for Ximian with the
rushed planning process they've been using is another question entirely.

Worse yet, you are using a rather abstract test case that you don't actually
run into much in the real world (even in low-level stuff like 3D engines).
If you want a better comparison, check out
http://www.codeproject.com/useritems/sharphsql.asp .  This guy ported a
relational database to C# from Java and ran various benchmarks and profilers
on the result.  Charts and graphs are posted on the page.  Note that there
is a noticable speed difference between C# and Java (C# not performing as
well) with the data in memory (but not when it is on disk).  Maybe there is
an actual performance issue in MS's VM here, but once again this is an
implementation issue, and nothing can be inferred from it about the
viability of the CLR as a platform.  Personally, my interpretation is the
same as this person's.

At this point I'd normally mention the project I started a few months ago to
work off of ORP (the one I couldn't get any interest from anyone in), but
I'm quite soured on it at this point and _I'm_ not even interested in it
anymore :-).

Sincerely,
Jay Freeman (saurik)
saurik@saurik.com

----- Original Message -----
From: "Tom" <tom7ca@yahoo.com>
To: <mono-list@ximian.com>
Sent: Saturday, July 21, 2001 7:17 PM
Subject: [Mono-list] JVM performance: JVM as a basis for CLR


> Given that there have been lots of arguments against
> basing a Mono runtime (CLI/CLR) on a JVM based on
> performance, I was curious to see how well Microsoft's
> CRL compares to Java VMs in terms of performance.  So,
> without claiming that these are anywhere near complete
> benchmarks, I tested two simple cases: "fib" and
> "convolution".
>
> On the "fib" benchmark (mostly function calls), CLR
> performs considerably worse than Sun's JVM.  On the
> convolution benchmark, they perform about the same.  I
> gave the CLR the benefit of the doubt and compiled
> with "/unsafe+ /checked-".  Based on these little
> benchmarks, I see no performance advantage to CLR.
> This seems in agreement with other recent comparisons
> of CLR and JVM.  I have yet to see any reproducible
> evidence that the CLR actually performs significantly
> better on any task; if you know of any (including
> source code), I'd like to know about it.
>
> What does this mean?  I don't want to bring up the
> issue of creating a Java foundation for Gnome, since
> that's not where the project is going.  But I think it
> suggests again that starting with a JVM implementation
> may lead to a high quality CLR implementation much
> faster than starting from scratch.  And starting with
> a JVM may also make it easy to provide CLR and JVM
> support in the same runtime from the start.  I think
> that would be a really great contribution, and it
> would allow the open source community to hedge its
> bets on the C#-vs.-Java issue.
>
> As people have pointed out, Intel's ORP
> (http://orp.sourceforge.net/) indeed has a non-trivial
> optimizing JIT.  OpenJIT (http://www.openjit.org/) may
> also be worth a look.  I'm going to take a look and
> see how they compare performance-wise to Sun's JDK
> (this is a little easier since programs don't need to
> be ported from one to the other).
>
> Tom.
>
> Here is the output from the benchmarks (this uses
> JDK1.4beta, but JDK1.3 gives the same results):
>
...