[Mono-devel-list] Profiler Idea

[Ben Maurer]

On Tue, 2004-01-13 at 14:55, Paolo Molaro wrote:
> On 01/13/04 Ben Maurer wrote:
> > Recently, while doing a profile, I saw the following lines:
> > 
> > >      433 KB System.String::ToCharArray(int,int)
> > >          433 KB      247 System.Char[]                                   
> > >   Callers (with count) that contribute at least for 1%:
> > >          247  100 % System.String::ToCharArray()
> > > ########################
> > 
> > Now, obviously, I have seen this method before. But for the first time
> > today, I reflected on how useless this data was. You get no idea *WHO*
> > is calling the method.
> 
> The first part of the profile output has the call info, so you just need
> to see which methods call ToCharArray(int,int).
yes, however this gets really hard to do, and makes it hard to get a
general overview of things.

> 
> > So, I was thinking `maybe we should make a list of common allocators
> > like that that are just called by other methods.' For example, in
> > Hashtable, you often get profiles like:
> > 
> > >      316 KB System.Collections.Hashtable::Rehash()
> > >          316 KB       29 .Slot[]                                         
> > >   Callers (with count) that contribute at least for 1%:
> > >           29  100 % System.Collections.Hashtable::PutImpl(object,object,bool)
> > 
> > However, it is hard to track down who is causing the rehashes because
> > not all calls to PutImpl cause a rehash.
> 
> It does not matter which specific call causes the rehash, since the
> rehash is implementation dependent. You can just look at which method
> causes the most items to be added to the hash. But usually it doesn't
> make sense to reduce the number of items added to an hash table. In most
> cases, instead, you should check that the hash functions are good enough
> so that the keys are spread evenly over the internal array.

Well, the idea is to detect `who is causing the most space to be
allocated in the hashtable.' For example this code:

for (..) {
	for (..) {
		ht.Add (...);
	}
	ht.Clear ();
}

is much better than:


for (..) {
	for (..) {
		ht.Add (...);
	}
}

if the bounds of the for loop are the same in both cases. However, there
is no way to see that right now.


> >       * For other select methods and classes, we shall apply a similar
> >         rule. For example, ToCharArray would be attributed to the method
> >         that called the 0 args overload.
> 
> I don't think adding special case hacks is a good thing.
> It would be better if, for each method, we could also output the total
> amount of memory allocated (by the method and by the called methods as
> well).
Hmmm, i am on the fence about including child methods. On the one hand,
I usually try to catch single methods that are causing problems, but
this method also provides a good approach.

For example, take the following code:

class MyList {
	object [] stuff;

	public MyList (int sz) { stuff = new Object [sz]; }
	void Add (object) {...}
	object Get (int) { ... } 
}

class Driver {
	static void Main () {
		for (int i = 0; i < 500; i ++) Foo ();
	}
	
	static void Foo () {
		MyList l = new MyList (3);
		Bar ();
		...
	}
	static void Bar () {
		MyList l = new MyList (50000);

	}
}

In this code, Bar is obviously taking up the most memory. With your
suggestion, It would be easy for Bar to get lost in the shuffle, if it
were only responsible for 10% of the allocations of Foo. Right now, you
cant tell the difference between Foo and Bar becuase they make the same
number of calls to the ctor. The kind of data that would be most useful
in this case is:



#######################
Driver::Bar
             500 object [] (xxxxxxxx KB) (via MyList.ctor)
...

#######################
Driver::Foo
             500 object [] (xxx KB) (via MyList.ctor)
...

The allocations from MyList are attributed to the caller, because
MyList.ctor is allocating resources for the MyList object, which was
created by its caller.

Maybe we need a profiler dedicated to memory profiling. The type of data
that is useful for memory profiling work isnt really provided right now.

-- Ben