[Mono-dev] New profiler is "ready"

[Massimiliano Mantione]

On Fri, 2008-03-28 at 11:51 +0100, Paul Melis wrote:
> Is the new profiler supposed to replace tools like heap-buddy and 
> heap-shot or is it a supplement to them?

In the long (well, even medium) term, it will replace them.

Most of the low level code is already functional.
It works this way: every object allocation is recorded in
a permanent set of buffers, which represents the heap (it
is the "heap image" of the profiler).
So at each moment we know all the objects present in the
heap, and can iterate on them.
Then, at each collection, we actually do the iteration,
and for each object a couple of things happen:
- If it has been marked "unreachable" (and therefore is
  going to be collected), we remove it from the heap image.
  If the "free" option has been given to the profiler, this
  event is written to the log file.
  This way the log file has all the info needed to implement
  the functionality of heap-buddy.
- If the "heap" option has been given, each reachable object
  is dumped to the log file with all its references to live
  objects. All these dumped objects form the heap snapshot
  (and therefore heap-shot can be reimplemented using them).

The only missing piece is the dumping of static fields, which
heap-shot does so it can have an approximation of the root
set we have just after the gc happened.
The problem is that we would still miss the references on
the thread stacks (which also heap-shot doesn't track), so
the root set is anyway only partial.
I wonder if this partial root set is really useful...

The decoder already correctly recognizes all these data.
What's missing is some analysis to present them to the user.
Recreating heap-buddy is relatively easy, and I'll do it first.
Of course the "heap-diff" functionality of heap-shot will take
a bit longer...


> And is the statistical profiler similar to what oprofile can offer?

Conceptually, yes, and anyway it is similar to the statistical
profiler included in the default Mono profiler: the low level
support for it is the same as before (as exposed by the Mono
profiling API).

The biggest thing we miss from oprofile is call graphs.
We intend to support them, but later.

And of course we only sample the program IP at a certain time
frequency, we don't use all the other counters that oprofile
supports!

Plus, as explained in the previous message, there's the
difference of when the elf files are examined (for unmanaged
code):
- oprofile does it "off line",
- the default Mono profiler does it in process (but after the
  application has ended or the internal buffer is full),
- the logging profiler:
  - for now, uses dladdr at run time, and if it cannot find
    the symbol it behaves like oprofile,
  - next week will likely do everything at run time.

Have fun,
  Massi