[Mono-list] lies and microbenmarks (Was: Mono 0.23 windows installer)
Stefan Matthias Aust
Sat, 29 Mar 2003 13:19:00 +0100
David Jeske wrote:
> I mean the few benchmarks I've written and plenty of the benchmarks in
> The Win32 version of Doug's Language Shootout.
It greately depends on the benchmarks I guess. I benchmarks two more
complex applications - a Smalltalk-Interpreter I once wrote for fun and
a Scheme interpreter (actually an older version of Peter Norvigs SILK
aka Jscheme) which I tried to port 1:1 from Java to C#. Both
applications run much faster in Java than in C#. .NET was nearly as
fast as the Java client VM, but the Server VM was much faster.
Both benchmarks make heavy (heaviest) use of garbage collection, object
allocation and method calling and do nothing of the "usual" benchmarks -
looping over arrays, doing small arithmetics etc. I consider such
application benchmarks more useful as they better mirror the typical
work of a application server or complex OO application.
Suns Hotspot VM is definitely not the best (there's an Intel resarch VM
which is - according to Intel - 20% better than the client VM for
example) and Microsofts VM is pretty fast. However, it's more tailored
towards faster one-shot execution (which is what most benchmarks
measure) and doesn't do (AFAIK) agressive long term, profile based
optimations like the hotspot architecture.
One example are precompiled libraries. Linking and therefore starting
time is improved, but as you (probably, I don't know for sure, perhaps
the precompiled libraries still contain the IL code) try to gain speed
by inlining and devirtualising method calls, precompiled code is an
> My original post served two purposes: (a) to show that 1-2 year old C#
> implementations are doing well relative to much older Java
> implementations already.
Actually, C# doesn't matter here. You compare the performance of the
Microsoft VM (aka VES) with the JVM and Microsoft is working on that VM
since the early days of Java. I'm pretty sure that the .NET VM shares
the same technology (and perhaps even the code) of Microsofts Java VM
which also wasn't new technology back than but based on existing rearch
> IMO, the multi-language thing itself is mostly hype. JVM still
> supports more languages than .NET. Neither of them support "different"
> languages very well.
Actually, the JVM doesn't support multiple languages. Bytecode is
nearly a 1:1 representation of the Java semantics. The one exception are
gotos which would be possible in JVM bytecode and which are not part of
Java. Therefore, all other languages which different semantics (which
aren't just a syntactic skin as C# and VB.NET are) must emulate their
semantics in Java.
This meants, stuff that isn't fast in Java can't be cast in these other
languages and those languages are always second level citizen. So it's
no real wonder for me that other languages never cought up.
For .NET things a slightly better. The CIL is still architectured
toward one (boring) kind of language but at least, there're some ways
out. The tail-end-recursion-call in the CIL might be the most prominent
example. Efficient tail-end-recursion is a requirement for most
functional languages. Values types, automatic boxing and unmanaged code
can probably also help to implement other languages for .NET.
Still you won't get efficient dynamic languages, no Lisp, no Smalltalk,
no Ruby or Python. Fortunatley, computers are probably fast enough to
sacrify one order of magnitute of performance for better languages with
better development performance (using python for rapid prototyping
instead of C for example).
> However, from what I've seen, MSIL simply has better performance
> characteristics. Today it is because of things like structs and
> references (i.e. safe pointers). Tommorow it will be because of CIL
> support for Generics. Which means those "workarounds" for non-C#/Java
> languages will probably run faster. Here are some references:
This paper emphazises the fact mentioned above, the CIL is slightly
better for supporting other languages - specially stically typed
functional languages - probably because that's what most Microsoft
research guys love and use.
Unfortunately, the (only IMHO of couse) practically more important group
of dynamic languages isn't supported well on neither platform.
The example they pickec, a Pascal dialekt, is IMHO nearly meanless for
all "interesting" modern languages with high level abstraction, object
orientation and so on. For fast execution of OO lanuages, it doesn't
matter how fast you can add two numbers but how fast GC and method
calling works in very dynamic environments where you will reload
classes, deal with large amounts of differently sized memory chunks,
where you need to run applications for days without failure and so on.
Stefan Matthias Aust //
www.3plus4software.de // Inter Deum Et Diabolum Semper Musica Est