[Mono-list] JVM performance: JVM as a basis for CLR

[Tom]

Given that there have been lots of arguments against
basing a Mono runtime (CLI/CLR) on a JVM based on
performance, I was curious to see how well Microsoft's
CRL compares to Java VMs in terms of performance.  So,
without claiming that these are anywhere near complete
benchmarks, I tested two simple cases: "fib" and
"convolution".

On the "fib" benchmark (mostly function calls), CLR
performs considerably worse than Sun's JVM.  On the
convolution benchmark, they perform about the same.  I
gave the CLR the benefit of the doubt and compiled
with "/unsafe+ /checked-".  Based on these little
benchmarks, I see no performance advantage to CLR. 
This seems in agreement with other recent comparisons
of CLR and JVM.  I have yet to see any reproducible
evidence that the CLR actually performs significantly
better on any task; if you know of any (including
source code), I'd like to know about it.

What does this mean?  I don't want to bring up the
issue of creating a Java foundation for Gnome, since
that's not where the project is going.  But I think it
suggests again that starting with a JVM implementation
may lead to a high quality CLR implementation much
faster than starting from scratch.  And starting with
a JVM may also make it easy to provide CLR and JVM
support in the same runtime from the start.  I think
that would be a really great contribution, and it
would allow the open source community to hedge its
bets on the C#-vs.-Java issue.

As people have pointed out, Intel's ORP
(http://orp.sourceforge.net/) indeed has a non-trivial
optimizing JIT.  OpenJIT (http://www.openjit.org/) may
also be worth a look.  I'm going to take a look and
see how they compare performance-wise to Sun's JDK
(this is a little easier since programs don't need to
be ported from one to the other).

Tom.

Here is the output from the benchmarks (this uses
JDK1.4beta, but JDK1.3 gives the same results):

+ javac fibj.java
+ java -server -showversion fibj
java version "1.4.0-beta"
Java(TM) 2 Runtime Environment, Standard Edition
(build 1.4.0-beta-b65)
Java HotSpot(TM) Server VM (build 1.4.0-beta-b65,
mixed mode)
starting
fib(40)=165580141 took 12.63
fib(40)=165580141 took 12.530000000000001

+ csc /o /unsafe+ /checked- fibs.cs
Microsoft (R) Visual C# Compiler Version 7.00.9254
[CLR version v1.0.2914]
Copyright (C) Microsoft Corp 2000-2001. All rights
reserved.

+ ./fibs.exe
starting
fib(40)=165580141 took 00:00:17.3000000
fib(40)=165580141 took 00:00:17.2500000
+ javac convj.java
+ java -server -showversion convj
java version "1.4.0-beta"
Java(TM) 2 Runtime Environment, Standard Edition
(build 1.4.0-beta-b65)
Java HotSpot(TM) Server VM (build 1.4.0-beta-b65,
mixed mode)
starting
convolve=75.88665 took 13.67
convolve=80.14561 took 13.68

+ csc /o /unsafe+ /checked- convs.cs
Microsoft (R) Visual C# Compiler Version 7.00.9254
[CLR version v1.0.2914]
Copyright (C) Microsoft Corp 2000-2001. All rights
reserved.

+ ./convs.exe
starting
convolve=75.88665 took 00:00:12.3500000
convolve=80.14561 took 00:00:12.3000000

Here is the source code:

=== convj.java ===
public class convj {
    public static double rseed =
0.198095803940820482309384;
    public static double random() {
	rseed = rseed * 9.2935802938408293480923 +
1.203985032850982902384;
	while(rseed>1.0) rseed -= 1.0;
	return rseed;
    }
    public static void fill_random(float[] out) {
	for(int i=0;i<out.length;i++) out[i] =
(float)random();
    }
    public static void convolve(float[] out,float[]
in,float[] mask) {
	for(int i=0,n=in.length;i<n;i++) {
	    double total = 0.0;
	    for(int j=0,m=mask.length;j<m;j++) {
		int index = j-m/2;
		index = index<0?0:index>=m?m-1:index;
		float value = in[index];
		total += value * mask[j];
	    }
	    out[i] = (float)total;
	}
    }
    public static void main(String args[]) {
	System.err.println("starting");
	float[] out = new float[65536];
	float[] in = new float[65536];
	float[] mask = new float[256];
	for(int iter=0;iter<2;iter++) {
	    long start = System.currentTimeMillis();
	    for(int i=0;i<10;i++) {
		fill_random(in);
		fill_random(mask);
		convolve(out,in,mask);
	    }
	    double delta = 1e-3 *
(System.currentTimeMillis()-start);
	    System.out.println("convolve="+out[0]+" took
"+delta);
	}
    }
}

=== fibj.java ===
public class fibj {
    public static int fib(int i) {
	if(i<2) return 1;
	return fib(i-1)+fib(i-2);
    }
    public static void main(String args[]) {
	System.err.println("starting");
	for(int iter=0;iter<2;iter++) {
	    int i = 40;
	    long start = System.currentTimeMillis();
	    int result = fib(i);
	    double delta = 1e-3 *
(System.currentTimeMillis()-start);
	    System.out.println("fib("+i+")="+result+" took
"+delta);
	}
    }
}

=== convs.cs ===
using System;

class App {
    public static double rseed =
0.198095803940820482309384;
    public static double random() {
	rseed = rseed * 9.2935802938408293480923 +
1.203985032850982902384;
	while(rseed>1.0) rseed -= 1.0;
	return rseed;
    }
    public static void fill_random(float[] outv) {
	for(int i=0;i<outv.Length;i++) outv[i] =
(float)random();
    }
    public static void convolve(float[] outv,float[]
inv,float[] mask) {
	for(int i=0,n=inv.Length;i<n;i++) {
	    double total = 0.0;
	    for(int j=0,m=mask.Length;j<m;j++) {
		int index = j-m/2;
		index = index<0?0:index>=m?m-1:index;
		float value = inv[index];
		total += value * mask[j];
	    }
	    outv[i] = (float)total;
	}
    }
    public static void Main() {
	Console.WriteLine("starting");
	float[] outv = new float[65536];
	float[] inv = new float[65536];
	float[] mask = new float[256];
	for(int iter=0;iter<2;iter++) {
	    DateTime start = DateTime.Now;
	    for(int i=0;i<10;i++) {
		fill_random(inv);
		fill_random(mask);
		convolve(outv,inv,mask);
	    }
	    TimeSpan span = DateTime.Now.Subtract(start);
	    Console.WriteLine("convolve="+outv[0]+" took
"+span);
	}
    }
}

=== fibs.cs ===
// -*- Java -*-

using System;

class App {
    public static int fib(int i) {
	if(i<2) return 1;
	return fib(i-1)+fib(i-2);
    }
    public static void Main() {
	Console.WriteLine("starting");
	for(int iter=0;iter<2;iter++) {
	    int i = 40;	
	    DateTime start = DateTime.Now;
	    int result = fib(i);
	    TimeSpan span = DateTime.Now.Subtract(start);
	    Console.WriteLine("fib("+i+")="+result+" took
"+span);
	}
    }	
}



__________________________________________________
Do You Yahoo!?
Make international calls for as low as $.04/minute with Yahoo! Messenger
http://phonecard.yahoo.com/