[MonoDevelop] Code Completion Database - The big patch

[Lluis Sanchez]

I forgot to add this file. It is now in the patch.

On dt, 2004-05-25 at 04:33, Todd Berman wrote:
> also, cdb.diff doesnt cleanly compile, as it needs:
> 
> `Internal/Project/Combine/CombineEntryEventArgs.cs'
> 
> --Todd
> 
> On Tue, 2004-25-05 at 03:41 +0200, Lluis Sanchez wrote:
> > Hi!
> > 
> > I finally found some time this weekend to finish this patch that fixes
> > several problems that the current implementation of the code completion
> > database has.
> > 
> > My goal for the patch was to make it possible to create a MonoDevelop
> > project with all files of corlib, consuming a reasonable amount of
> > memory and with fast startup and operation. With the old implementation,
> > only a with for corlib would take more than 15 minutes to open and it
> > would easily take 350Mb of memory.
> > 
> > It seems that I've made it since with the patch I can open a combine
> > with projects for corlib, System, System.Runtime.Remoting, System.Web
> > and System.Web.Services (around 3500 files), with a startup time of 10
> > seconds, and fitting in 70Mb of memory, with full support for code
> > completion and complete class view tree.
> > 
> > Issues with the old implementation
> > ----------------------------------
> > 
> > The two big problems of the old implementation are the following:
> > 
> > 1) Project files are parsed every time the project is opened, and parse
> > information is kept all in memory. This is unacceptable if you have a
> > combine with 3500 files.
> > 
> > 2) The class tree is fully loaded with all class data. For corlib that
> > means around 25.000 tree nodes. It takes a lot of time and memory to
> > build such a tree.
> > 
> > The fix
> > -------
> > 
> > There are a lot of changes. I'm sorry because the patch is really big. I
> > hope the following notes can help in understanding what the changes are
> > about:
> > 
> > * Class information parsed from project files is now stored in a data
> > file, so it is not necessary to parse again all project files every time
> > a project is opened.
> > 
> > * Each project and each assembly has its own class information database.
> > The project database is stored in the same folder as the project.
> > Assembly databases are stored in the MonoDevelop code completion folder
> > and shared among all projects that reference them. The lifetime of those
> > databases is automatically managed by the Parser Service (i.e. if an
> > assembly is deleted, the corresponding shared database will also be
> > deleted).
> > 
> > * The format of the database is the same for projects and assemblies. It
> > has three parts: a set of headers (with information such as the version
> > of the database), an index of the classes stored in the database, and
> > the class data. When opening a database, only the headers and the index
> > is loaded in memory. Class information is retrieved from the data
> > section as it is requested. The old implementation already had something
> > like this, but the new format is more compact an can be used both for
> > projects and assemblies.
> > 
> > * The class tree is now loaded under demand. Class information is only
> > retrieved when the project or namespace node that contains it is
> > expanded.
> > 
> > * The initial generation of the code completion database is not needed.
> > Databases for assemblies are generated when needed. The generation is
> > made in an external process using the new tool dbgen, so it has no
> > impact on the memory consumed by the main MonoDevelop process.
> > 
> > * The parser service thread now keeps track of modified files. It will
> > update class information of externally modified files (assemblies and
> > project files). So, no worries when you cvs up.
> > 
> > * Class information is now "project-sensitive". It means that the code
> > completion window will show only class information taken from the
> > project to which the file being edited belongs, including all project
> > references (in other words, ctrl+space will include Gtk classes only if
> > the project includes a reference to the Gtk assembly). Most of methods
> > of IParserService now take an IProject as parameter, and this additional
> > parameter has propagated to many other classes, specially in
> > Addins/BackendBindings.
> > 
> > Some other info about the implementation:
> > 
> > * Improved the handling of events fired when adding or removing files
> > and references from a project. Now those events are fired at project
> > level and propagated to the combine and to the project service.
> > 
> > * Databases are handled by the new CodeCompletionDatabase class. It
> > keeps in memory the class index data structure and controls the access
> > to class data (caching that data when needed, and flushing to disk when
> > too much data is loaded). There are two subclasses:
> > ProjectCodeCompletionDatabase and AssemblyCodeCompletionDatabase.
> > 
> > * The class persistence layer has also many changes. Methods for reading
> > and writing class information are now static, which allows for example
> > to write any IClass, no matter what the implementation class it is. The
> > implementation has been also simplified by reusing more code from the
> > AbstractXXX classes.
> > 
> > There is also more information in the ChangeLogs.
> > 
> > The patches
> > -----------
> > 
> > To ease the review, I've divided the big patch in several small patches.
> > All of them must be applied for the whole thing to work. You can also
> > find everything in a single patch here: 
> > 
> > http://primates.ximian.com/~lluis/temp/cdb.diff
> > 
> > The patches are the following:
> > 
> > * changelogs.diff: all ChangeLogs
> > 
> > * parserservice.diff: Changes in the parser service. I also attach
> > DefaultParserService.cs for clarity, since almost everything changed.
> > 
> > * project.diff: Changes related to the addition of the IProject
> > parameter to IParserService methods (now class information is
> > per-project).
> > 
> > * events.diff: Changes related to the new event infrastructure added to
> > Project and Combine.
> > 
> > * persistence.diff: Changes in the persistence layer of class
> > information, and also some changes in the implementation layer.
> > 
> > * classscout.diff: Changes in the Class Scout (lazy load tree).
> > 
> > * dbgen.diff: The new database generation tool.
> > 
> > * other.diff: other unrelated fixes.
> > 
> > 
> > I'm sorry for the reviewers ;-)
> > Lluis.
> > 
> 
> _______________________________________________
> Monodevelop-list mailing list
> Monodevelop-list@lists.ximian.com
> http://lists.ximian.com/mailman/listinfo/monodevelop-list