[Mono-list] Speed of MethodInfo.Invoke?

[Chad Robinson]

Jonathan Pryor wrote:
> This was done under .NET, but I doubt that mono is significantly
> different for relative performance:
> 
> http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dncscol/html/csharp02172004.asp
> http://blogs.msdn.com/ericgu/archive/2004/03/02/83053.aspx
> http://gglaze.homeip.net/code_samples/msdn/function_call_benchmark/

Thanks, I stumbled into that last night. It's interesting though. I converted 
the example to use DateTime to collect its performance counters. It's totally 
false under Mono. Check it out:

DateTime: 325421910 ticks, 32.54219 seconds, 307293.4 calls per second
Direct: 261540 ticks, 0.026154 seconds, 3.823507E+08 calls per second
Interface: 721620 ticks, 0.072162 seconds, 1.385771E+08 calls per second
Delegate: 1832990 ticks, 0.183299 seconds, 5.455567E+07 calls per second
CustomDelegate: 175811750 ticks, 17.58117 seconds, 568790.2 calls per second
CustomClass: 1044910 ticks, 0.104491 seconds, 9.570202E+07 calls per second

This is running 10M times on a P4 2.8GHz dual proc + HT Xeon Noconas with 1MB 
caches, 1GB RAM. No other tasks running except SSH, not even X. All 
optimizations turned on in both compiler and RT.

DateTime is how long it takes to call DateTime.Now - I was curious since I 
need to do that a lot, too. The middle value, seconds elapsed per 10M calls, 
is the easiest number to use for comparison.

It's fascinating. A custom class is only a fifth the speed of a direct call. 
That's easily fast enough for my purposes. It's done by using Reflection to 
emit a class wrapper that implements an interface for the function to call, 
then call via the interface. It's shockingly fast despite the obvious 
overhead, at least compared to the alternatives.

Note the dismal performance of the CustomDelegate. Invoke is even worse - it 
takes so long to run that I turned it off. =)

For the curious, the same test with optimizations turned off:

DateTime: 358940330 ticks, 35.89403 seconds, 278597.8 calls per second
Direct: 632120 ticks, 0.063212 seconds, 1.581978E+08 calls per second
Interface: 725880 ticks, 0.072588 seconds, 1.377638E+08 calls per second
Delegate: 1841970 ticks, 0.184197 seconds, 5.42897E+07 calls per second
CustomDelegate: 184668060 ticks, 18.46681 seconds, 541512.1 calls per second
CustomClass: 1229430 ticks, 0.122943 seconds, 8.13385E+07 calls per second

It looks like CustomClass is the way to go for me.

Also for the curious, here are the results on Windows, in a VMWare session on 
a single-CPU Pentium 1.4M box with 384MB RAM. These are the optimized/Release 
version's results. .NET 2.0, Visual Studio 2005.

DateTime: 89062500 ticks, 8.90625 seconds, 1122807 calls per second
Direct: 156250 ticks, 0.015625 seconds, 6.4E+08 calls per second
Interface: 781250 ticks, 0.078125 seconds, 1.28E+08 calls per second
Delegate: 468750 ticks, 0.046875 seconds, 2.133333E+08 calls per second
CustomDelegate: 605781250 ticks, 60.57813 seconds, 165076.1 calls per second
CustomClass: 781250 ticks, 0.078125 seconds, 1.28E+08 calls per second

The results are still very interesting. First, collecting DateTime information 
is almost FIVE times faster than in Mono on Linux. Anybody know of a faster 
way to poll the system clock?

Also of interest, almost every operation became faster. BUT, CustomDelegate 
became FOUR times slower. The conclusion I'd draw from this is that the Mono 
compiler/RT is generally producing good code - not always the best, but 
predictably "good". As the devel team works on more optimizations I'm sure 
that will improve (it would at least be nice to see Direct calls on par with 
.NET).

In Mono, a Delegate is a third the speed of an Interface. In .NET, an 
Interface is half the speed of a Delegate - it's reversed. Not clear why that 
would be, but it's repeatable on my system.

.NET is faster for most of the general-purpose work, and would be a good 
choice if you really needed performance. However, your development time would 
increase because you'd have to be very careful to benchmark your techniques. 
Their standard deviation on peformance is much wider if you use the wrong 
pattern, and it's not always obvious which one is wrong. The author of one of 
the blog articles doing this benchmarking suggested that it wouldn't really 
matter and that good OO practices should continue to be the rule. My view 
based on the results above is exactly the opposite - you can easily develop 
good OO code that has the worst possible performance compared to similar, 
still-good-OO code that has the best.

I've considered setting up a Web site that maintains a database of performance 
metrics for techniques. I don't want it to be a .NET vs. Mono thing, more of a 
focus on good design patterns. Would anybody else be interested in that, and 
if so, do they have any techniques they'd like to add to the list? Is there 
something like this already?

Regards,
Chad