[Mono-devel-list] Embedding Mono in a Virtual World

[Tim Fries]

>Yes, which means that it may require lots of changes
>to the rest of the system to support calls from
>multiple threads.

In my experience with Microsoft's .NET runtime in a scenario almost
identical to your virtual world plans -- the overhead of locking and
synchronization, especially with user-supplied code, is easily going to
overwhelm you.  Plus, you won't be able to scale up linearly to "one thread
per task" as you've already realized, because at some point you're going to
run out of address space for those thread stacks.  I haven't done any
testing on Mono yet, but I assume the same problems would present.

I ended up deciding on a more traditional single-thread approach, with a
couple twists; briefly summarized:

All I/O is, of course, asynchronous. This shouldn't be a surprise, and the
Framework makes this easy.  User input parsing can even take place on the
ThreadPool asynchronous callbacks; but is limited in order to avoid
unnecessary context switches from doing too much work in extra threads.

The work queue processing is not *strictly* single-threaded; instead, the
virtual world is divided into "regions" of which contained objects are not
likely to interact with objects in a different region. Each region can have
its own work thread, or as I've found more performant: regions and threads
can be m/n (with the number of threads limited by the number of physical
processors in the machine), with a task running regularly to migrate the
regions around to keep the load spread evenly across the available CPUs.
Any time an "inter-region" task needs to be performed, which should be
relatively rare, the appropriate threads (more accurately, the regions
within their owner threads) can synchronize long enough for that to happen.

Certain types of common inter-region tasks, such as chat, can be special
cased to avoid having to synchronize across threads.

The threads, and the regions within them, cooperatively multitask; for the
scalability reasons already described -- and because preemptive multitasking
is incredibly expensive.

Making this all work with user-supplied code is a hurdle I haven't had to
overcome, myself; but it *is* a problem that MUDs have already explored
rather exhaustively.  My initial instinct would be to go along the
interpreted scripts model; with maybe some perf counters to JIT commonly
used scripts into DynamicMethods.  The tricky part is keeping their run
times short enough to not gum up the cooperative environment without having
to thrust that problem onto the authors of the user scripts; but
interpreting/compiling as a state machine that you can enter and exit at
will would probably solve that problem well enough.