[Mono-list] Profiler todo list

[Massimiliano Mantione]

Here is the profiler todo list.
If somebody wants to suggest priorities, or propose other
features... ;-)

- Choose a name for the tool, and stick to that.
  The two candidates are "mprof" and "monoprof".
  I like mprof more, but a simple google search shows that it
  has been used by other minor profiler in the past (a Modula2
  profiler, a "minimalist" profiler on bsd, a memory profiler,
  and the Mercury (the language) profiler).
  Anyway, the actual tool names would be longer, like:
  - "mprof-decoder" for the console decoder,
  - "mprof-heap" for the heap-shot-like GUI,
  - "mprof-whatever" for any future tool...
  So I'd like to use the shorter mprof root for the name.
- Commit the man page (once the tool has a proper name).
  I would commit an "mprof" man page for the flags one can
  pass to the profiler at application start, then each tool
  should have its own page.
- Make it build on Windows and Mac OS X (initially with no
  support for unmanaged symbols in the stat profiler).
- Should the Mono.Profiler svn module be moved in mono-tools?
  If yes, when?

IHMO, all the above things are required for the 2.0 branch as
well (if we want to ship this officially with Mono 2.0).

There is one more thing that I don't know if must be pushed in
2.0: the extension for tracking trampolines and thunks.
It would be nice, it is not disruptive, but is it not mandatory
at all (and not finished).
Maybe it would be nice to have at least the runtime hooks in
place, so that a standard 2.0 Mono installation could use this
new feature by just recompiling the profiler plugin, and-or
upgrading the decoder.


Then, the future.

All the other changes will break the profiler file format, and
each of them breaks it in a different way.
This, BTW, has been one of the reasons why I have been reluctant
to commit them up to now: I didn't want to risk a decoder and
file mismatch too many times, so I was waiting to pile up all
the changes and introduce them together.
This didn't work, small little things to do keep popping up, so
I wanted to have a sane way to handle those changes.

There is a "version" field in the file START block, and I could
easily bump the version every time there's a change.
Then, in the decoder I should check the version and do what's
needed (keeping both code paths, to be able to decode also old
files).

What I don't really like of this approach is that some of the
changes I wanted to add are optional features, that one could
want in some runs but not in all runs, but which are not easily
implemented with entirely new "file block" types.
Instead, they would change the encoding of existing blocks,
mostly adding data.
The goal is also that these additions to existing blocks should
be optional.

Another thing I didn't like was having to code "if (version)"
in the decoder, because it is not "self describing": the
version number that introduces each feature should be looked
up in a document all the time...

So, I have this idea: add a new "directives" block type.
This block would contain an extensible list of "tag-value"
pairs, and the latest decoder is always guaranteed to
understand it fully.
Each of those tags can switch on and off an optional feature,
or can convey other information do the decoder.
This is *really* easy to implement, and would give me all the
flexibility I need (yes, the famous last words...).

So, if in a particular profiler run a certain piece of data
has been added, the directives block will contain its own
specific flag, and the decoder will "if" on it, and not
on a global "format version".

This new block type could have also other purpose, like
allowing the users to put "user inserted markers" in the
log file.
Something like this: the user signals the executed process,
and the profiler puts a timestamped marker in the file.
Later, the decoder (or the GUI) will be able to split the
profile data in time regions based on the markers.
Right now we can start and stop the profiler with a signal,
but this system could be even more useful, analyzing the
application behavior in different "areas".

In short, having an extensible "directives" block will
allow the profiler to put any kind of "low bandwidth" data
in the log file in an easy way.
Since the "high bandwidth" data (the bulk of the profiler
events) are already handled well in the existing blocks,
this could be the last block kind we need, and keep the
code understandable.

That said, here are the additions I have in the todo list.
First of all to the "mapping block":
- Each method could carry the "method flags", so that the
  decoder knows if it's a wrapper (and can skip it in the
  "caller attributions").
- Each method could also carry the "declaring method" or
  similar info, so that the decoder can aggregate multiple
  instances of the same generic method in attributions.
  Maybe we could also encode the method name in a more
  "composite" way, and not just a big string, so that the
  decoder can choose the level of detail it wants to show.
- Thunks and trampolines "declarations" should be placed in
  the "mapping" block of the file: while it is true that a
  new, different block type would work, they really belong
  there.
- Each class could contain the instance size (only arrays
  and strings would miss it), so that allocation encoding
  can be more compact.
Then to the "events" block:
- Each allocation could also contain a timestamp.
- Allocations could also contain the caller method
  directly (computed by the profiler): this would allow
  the user to have this info without the overhead of a
  full enter-exit log.
Anyway, all of the above are clearly minor incremental
improvements: the bulk of the tool functionality is already
there.

Another thing on the profiler todo list is a Shark-like GUI:
http://blog.vlad1.com/2008/07/16/why-cant-vtune-be-more-like-shark/
The information provided by our profiler is almost the same
as those provided by Shark, but we miss a way to present
them well (which would make them actually useful to a larger
portion of our users...).

So, users should vote their priorities, or point out
anything I missed :-)

Ciao,
  Massi