[Mono-dev] Heap snapshots in the loggin profiler

[Massimiliano Mantione]

The logging profiler has support for heap snapshots (like heap-shot),
but it has never been perfect.

This mail wants to explain what's wrong, and why I think it's better
concentrating on making it work with sgen instead of insisting in trying
it with Bohem (unless we are willing to implement a proper profiling API
for Bohem).

The heap profiling API in Mono is simply non-existant.
There are hooks for object creation and GC, but there is no API to
interact with the heap as a whole.
So it's not possible to iterate through all the current heap objects,
and it's not possible to be notified when the GC frees some object.

Therefore a profiler has to do all the bookkeeping by itself.

When heap profiling is requested, currently the logging profiler keeps a
"mirror" of the heap in internal buffers: for each allocated MonoObject
a pointer to it is stored there.
At each collection, just after the "mark" phase, the profiler scans
these buffers and for each object it calls "mono_object_is_alive".
The problem is that this is not reliable: "mono_object_is_alive" was not
meant to be a public function.
And in fact sometimes the heap snapshots are wrong (or the profiler
crashes).

A "workaround", suggested by Paolo, is to use GC handles to tell if
objects are alive.

This would be reliable, and in fact it mostly works.
And it would have the nice effect of making it easy to cope with sgen
when it moves objects: mono_gchandle_get_target would give the profiler
the new object location.

However, this approach has its downsides...

For each object, in principle it would be necessary to only store the GC
handle instead of the MonoObject* pointer.
In practice the profiler needs to emit information also when the object
has been freed, therefore the profiler must store at least the GC handle
and the MonoClass*, and in practice also the MonoObject*.
Moreover, working with GC handles I had some very strange deadlock
during collections.

The deadlocks could of course be solved.

But the resulting profiler keeps one GC handle for *every* allocated
object (this makes the GC handle tables huge), and on top of that it
stores three pointer sized values for each object in the profiler
tables.
Of course everything works anyway in "common" cases, but IMHO the memory
overhead is a bit too large.


The solution would be having a proper profiling API which allows the
profiler to walk the heap, object by object.

Or, even better, to have an API that provides the following hooks:
- Allocation-deallocation of a "block" of memory by the GC (the callback
would receive the block address and size, as well as its "intended use"
in the GC).
- Info of the fact that an object has been moved (old and new address
would be provided).
- Given a block of memory, the ability to "walk" it iterating all the
MonoObject it contains.
- The possibility to iterate among all the "blocks" currently used by
the GC (in case the profiler does not want to keep track of the blocks
by itself).

The last two API entry points (iteration of all heap "memory blocks" and
of all objects inside a given block) should be used only when the world
is stopped (during a collection).

The above API would have the following advantages:
- it would make implementing heap snapshots trivial, with no memory
overhead in the profiler, and
- it would also allow to profile how the GC is efficient in its block
usage.

I've had a chat with Mark, and I happily sow that he already implemented
most of this by itself while developing sgen.
His hooks write into a file that is then post processed by a script.

Of course in principle it would be possible to do this with Bohem.
But I really don't think it would be worth doing at this point.
It would be useful only if it worked out of the box on an "old" Mono
runtime, to profile production systems with the new heap profiler.
But I don't think this is going to be easy.

I am attaching the patch that makes the logging profiler use GC handles
to track objects, simply because I did it, and I don't want it to get
lost.
But I am not going to commit it, I don't think it's worth it.

Ciao!
  Massi

-------------- next part --------------
A non-text attachment was scrubbed...
Name: gc-handles-heap-shot.patch
Type: text/x-patch
Size: 14304 bytes
Desc: not available
Url : http://lists.ximian.com/pipermail/mono-devel-list/attachments/20091101/70088d37/attachment.bin