[Mono-dev] Cecil improvement

[Roei Erez]

I think we should differentiate between two types of users, those who
only want to read and explore the assembly and those who use cecil to
write the assembly.
When writing an assembly a user needs to build the full object model, so
I don't think there is a lot of room for improvement at this area.
This is not the case when reading an assembly, which is a common case,
and especially when the user is interested only in part of the assembly.

I agree that cecil should expose an interface to load an assembly as
'Read Only', where the optimizations should be activated, and by that
save the worrying of the changes to the assembly writing part.
I have played with the code a little bit and saw that when loading some
assemblies in a total size of 25M the memory usage has reached 250M.
When I have omitted the building of the object model the memory was
reduced to 120M, and the loading time was improved enormously.
I really like the idea that the object model will access the raw tables
directly and in lazy way.
This idea will work only if the access to the raw table will be fast,
and in order to do so, the code needs to build more relations between
these tables.
We can take the CustomAttributes table for example, each raw at this
table has the 'RID' of the owner element, and the TokenType of the
owner, and in order for a 'Field', for example, to load its
CustomAttibutes lazily, it needs that opposite relation, from its 'RID'
to its CustomAttributes, will exist.

I basically want to suggest the following:
The methods of the ReflectionReader will be changed, and instead of
building an object model, they will only build the desired relations
between elements (Methods->Types, Fields->Constants,
Fields->InitialValues ...)
The higher object model classes (TypeDefinition, MethodDefinition,
FieldDefinition ...), will have their properties access lazily those
built relations in order to fetch the data on first access.
My favorite solution is to write a 'ReadOnlyReflectionReader' that will
contain the above changes, and leave the current implementation as is
for writing purposes.
I think we should also consider removing the 'sealed' modifier from the
higher level object model classes (like TypeDefinition, MethodDefinition
...), by that give the used ReflectionReader full control over the
object model implementation, building its own read-only one.

Now is a good time for us to start such a process, and I would really
want to push things forward.
What do you think about the suggestion?
If you think about major issues that the above solution will encounter,
I would be happy to hear about them. 

Regares, 
Roei Erez

-----Original Message-----
From: Rolf Bjarne Kvinge [mailto:rolflists at ya.com] 
Sent: Tuesday, August 21, 2007 5:38 PM
To: 'Jb Evain'; Roei Erez
Cc: mono-devel-list at lists.ximian.com
Subject: RE: [Mono-dev] Cecil improvement



> -----Original Message-----
> From: mono-devel-list-bounces at lists.ximian.com
[mailto:mono-devel-list-
> bounces at lists.ximian.com] On Behalf Of Jb Evain
> Sent: martes, 21 de agosto de 2007 17:08
> To: Roei Erez
> Cc: mono-devel-list at lists.ximian.com
> Subject: Re: [Mono-dev] Cecil improvement
> 
> Hey,
> 
> On 8/21/07, Roei Erez <roeie at mainsoft.com> wrote:
> > Hi all,
> >
> > As you may know we, at mainsoft, use cecil library.
> >
> > We have made some analysis for the performance and found that
loading
> > assemblies with cecil has a lot of memory consuming impact.
> >
> > Investigating this thing more had shown that the code creates a full
> object
> > model on top of the assembly row tables.
> >
> > We understand that for writing/creating assembly using cecil ,
> building a
> > full object model  might be unavoidable, but for a lot of users that
> only
> > wants to read an assembly, and event interested only in part of the
> assembly
> > it might be not necessary.
> >
> > We believe that improving this area will give theses users a better
> > experience that will be realized in less memory consumption and
> faster
> > loading time.
> >
> > Nowadays, we are looking for ways to improve cecil in this area.
> >
> > I will be happy to know what you think, and whether you have had
> thoughts
> > about this issue.
> >
> > Any thoughts, suggestions and questions are welcome.
> 
> I'm also interested in improving Cecil's memory usage, and assembly
> load time, as it affects basically every user.
> 
> The easier way I can think of right now, to save some memory, is to
> discard the Cecil's representation of the table heap after an assembly
> is loaded, as it is not need afterwards. But that won't improve the
> load time.
> 
> A real and full optimization would be to read Cecil's higher level
> object model straight from the raw bytes. But that requires quite a
> big amount of work. And extra work to keep the assembly writing part
> working.
> 

Maybe an option to load an assembly read-only would make sense, as well
as
an option to not load anything until requested (a compiler for instance,
would only need the names of all the types when loading the assembly, no
need to load type members until the type is actually referenced in code
somehow).

Rolf 

> There's also a few people using only the Mono.Cecil.Binary part, to
> only read the PE file. So there are quite a few things to take into
> consideration before doing anything.
> 
> --
> Jb Evain  <jb at nurv.fr>
> _______________________________________________
> Mono-devel-list mailing list
> Mono-devel-list at lists.ximian.com
> http://lists.ximian.com/mailman/listinfo/mono-devel-list
> 
> 
> --
> No virus found in this incoming message.
> Checked by AVG Free Edition.
> Version: 7.5.484 / Virus Database: 269.12.1/963 - Release Date:
> 20/08/2007 17:44