[Mono-dev] IronPython Performance

[Brent Fulgham]

A while ago (nearly two years) an interesting paper was published by Jim Hugunin
(http://www.python.org/pycon/dc2004/papers/9/) crowing about the significant
speed advantage observed in this interpreter running on top of Microsoft's .NET
VM stack.  I remember being surprised by these results, as Python has always
seemed fairly fast for an interpreted language.

I've been working on the Programming Language Shootout for a few years now, and
after growing tired of the repeated requests to include scores for IronPython I
finally added the implementation this week.

Comparison of IronPython (1.0 Beta 5) to Python 2.4.3 [1] and IronPython (1.0 Beta 1)
to Python 2.4.2 [2] do not match the results reported in the 2004 paper.  In fact,
IronPython is consistenly 3 to 4 times slower (in some cases worse than that), and
seemed to suffer from recursion/stack limitations.

Since the shootout runs on a Debian Linux box, I was not able to use Microsoft's
CLR; instead, I was pleased to find that Mono successfully ran most of the
Shootout testing suite.  The 1.0 Beta 1 version of IronPython was built using the
compiler chain that ships with Mono 1.1.13.6; the 1.0 Beta 5 version is an 'official' build
from the IronPython download site running on Mono 1.1.13.4.

I am aware of the following issues that might affect the results:
1)  Mono vs. Microsoft's CLR.
2)  Python 2.1 vs. Python 2.4
3)  Changes in IronPython over the last two years.

In addition, there is a small startup cost paid by IronPython since the py_compile
libraries were not present in the IronPython beta, preventing me from precompiling
the *.py files to *.pyo.  However, most of our benchmarks are now sufficiently long
that initial startup costs are less significant.

I thought these findings might be interesting to some of the Python, IronPython,
and Mono developers.  Let the flames begin!  ;-)

-Brent

===============================================================
Current results for Python versus IronPython are posted here:
[1] http://shootout.alioth.debian.org/sandbox/benchmark.php?test=all〈=iron&lang2=python
[2] http://shootout.alioth.debian.org/gp4sandbox/benchmark.php?test=all〈=iron&lang2=python