[MonoDevelop] Code Completion Database - The big patch

[Todd Berman]

also, cdb.diff doesnt cleanly compile, as it needs:

`Internal/Project/Combine/CombineEntryEventArgs.cs'

--Todd

On Tue, 2004-25-05 at 03:41 +0200, Lluis Sanchez wrote:
> Hi!
> 
> I finally found some time this weekend to finish this patch that fixes
> several problems that the current implementation of the code completion
> database has.
> 
> My goal for the patch was to make it possible to create a MonoDevelop
> project with all files of corlib, consuming a reasonable amount of
> memory and with fast startup and operation. With the old implementation,
> only a with for corlib would take more than 15 minutes to open and it
> would easily take 350Mb of memory.
> 
> It seems that I've made it since with the patch I can open a combine
> with projects for corlib, System, System.Runtime.Remoting, System.Web
> and System.Web.Services (around 3500 files), with a startup time of 10
> seconds, and fitting in 70Mb of memory, with full support for code
> completion and complete class view tree.
> 
> Issues with the old implementation
> ----------------------------------
> 
> The two big problems of the old implementation are the following:
> 
> 1) Project files are parsed every time the project is opened, and parse
> information is kept all in memory. This is unacceptable if you have a
> combine with 3500 files.
> 
> 2) The class tree is fully loaded with all class data. For corlib that
> means around 25.000 tree nodes. It takes a lot of time and memory to
> build such a tree.
> 
> The fix
> -------
> 
> There are a lot of changes. I'm sorry because the patch is really big. I
> hope the following notes can help in understanding what the changes are
> about:
> 
> * Class information parsed from project files is now stored in a data
> file, so it is not necessary to parse again all project files every time
> a project is opened.
> 
> * Each project and each assembly has its own class information database.
> The project database is stored in the same folder as the project.
> Assembly databases are stored in the MonoDevelop code completion folder
> and shared among all projects that reference them. The lifetime of those
> databases is automatically managed by the Parser Service (i.e. if an
> assembly is deleted, the corresponding shared database will also be
> deleted).
> 
> * The format of the database is the same for projects and assemblies. It
> has three parts: a set of headers (with information such as the version
> of the database), an index of the classes stored in the database, and
> the class data. When opening a database, only the headers and the index
> is loaded in memory. Class information is retrieved from the data
> section as it is requested. The old implementation already had something
> like this, but the new format is more compact an can be used both for
> projects and assemblies.
> 
> * The class tree is now loaded under demand. Class information is only
> retrieved when the project or namespace node that contains it is
> expanded.
> 
> * The initial generation of the code completion database is not needed.
> Databases for assemblies are generated when needed. The generation is
> made in an external process using the new tool dbgen, so it has no
> impact on the memory consumed by the main MonoDevelop process.
> 
> * The parser service thread now keeps track of modified files. It will
> update class information of externally modified files (assemblies and
> project files). So, no worries when you cvs up.
> 
> * Class information is now "project-sensitive". It means that the code
> completion window will show only class information taken from the
> project to which the file being edited belongs, including all project
> references (in other words, ctrl+space will include Gtk classes only if
> the project includes a reference to the Gtk assembly). Most of methods
> of IParserService now take an IProject as parameter, and this additional
> parameter has propagated to many other classes, specially in
> Addins/BackendBindings.
> 
> Some other info about the implementation:
> 
> * Improved the handling of events fired when adding or removing files
> and references from a project. Now those events are fired at project
> level and propagated to the combine and to the project service.
> 
> * Databases are handled by the new CodeCompletionDatabase class. It
> keeps in memory the class index data structure and controls the access
> to class data (caching that data when needed, and flushing to disk when
> too much data is loaded). There are two subclasses:
> ProjectCodeCompletionDatabase and AssemblyCodeCompletionDatabase.
> 
> * The class persistence layer has also many changes. Methods for reading
> and writing class information are now static, which allows for example
> to write any IClass, no matter what the implementation class it is. The
> implementation has been also simplified by reusing more code from the
> AbstractXXX classes.
> 
> There is also more information in the ChangeLogs.
> 
> The patches
> -----------
> 
> To ease the review, I've divided the big patch in several small patches.
> All of them must be applied for the whole thing to work. You can also
> find everything in a single patch here: 
> 
> http://primates.ximian.com/~lluis/temp/cdb.diff
> 
> The patches are the following:
> 
> * changelogs.diff: all ChangeLogs
> 
> * parserservice.diff: Changes in the parser service. I also attach
> DefaultParserService.cs for clarity, since almost everything changed.
> 
> * project.diff: Changes related to the addition of the IProject
> parameter to IParserService methods (now class information is
> per-project).
> 
> * events.diff: Changes related to the new event infrastructure added to
> Project and Combine.
> 
> * persistence.diff: Changes in the persistence layer of class
> information, and also some changes in the implementation layer.
> 
> * classscout.diff: Changes in the Class Scout (lazy load tree).
> 
> * dbgen.diff: The new database generation tool.
> 
> * other.diff: other unrelated fixes.
> 
> 
> I'm sorry for the reviewers ;-)
> Lluis.
>