[Monodevelop-devel] Using SQLite as parser database

[Christian Hergert]

On Tue, Jul 29, 2008 at 1:09 AM, "Andrés G. Aragoneses"
<aaragoneses at novell.com> wrote:
> Hi. Just my 2 cents on this.
>
> Mike Krüger wrote:
>> Hi
>>
>>
>>> This discussion should have been done before committing anything to
>>> trunk, but here it is anyway.
>>>
>>> Migrating to SQLite only makes sense if it provides noticeable
>>> improvements in performance and memory use. Guessing that it will be
>>> better is not enough. We need real numbers before taking the decision
>>> to
>>> switch, and only do it if the numbers are so much better that pay off
>>> the burden of having a dependency on SQLite.
>>>
>>> I might be wrong, but I don't believe that SQLite will be better than
>>> the ad-hoc database we are using in MD 1.0. I spent a lot of time
>>> tuning
>>> up the parser database, and I'm quite happy about how is it
>>> performing.
>>>
>>>
>>
>> Some more benefits for using a real database over an own implementation:
>>
>> - It's reliable. Atomic transactions, threading - all solved.
>> - It's easy to look into the data using a command line client and SQL
>> statements
>>
>> Its not just pure performance. Using a database will allow us for
>> example to switch the database software to a new implementation. And
>> databases and SQL are very easy to understood and to change.
>>
>> I had to think about it too (I implemented the #develop database some
>> years ago which monodevelop inherited (but optimized I admit ^^)) -
>> after thinking about it the decision was easy - A database makes it
>> easier to change the model and to make complicated querys more
>> efficient.
>>
>> There are many more reasons using database software instead of own data
>> storage solutions - otherwise the whole database software would be
>> obsolete.
>>
>
> There are a lot of reasons too why programs don't use SQL database
> software. Maintainability gets hurt also by adopting such a solution in
> a OOP world; I guess you have heard about the Impedance mismatch [1],
> hence the existance of complicated ORM [2] frameworks or OOP databases
> like DB4O [3] (which in particular could be a better bet, but I guess
> the license is not adequate for MD).
>
> I'm also worried about the fact that your opinion about SQLite may be
> biased because I guess you only used Win32 OS when you were working in
> SharpDevelop, right? There has been huge performance problems (locking
> problems) with SQLite in Linux, which recently caused a lot of
> discussions due to its usage in Firefox (which BTW is performing for me
> a lot worse than other browsers in Linux). I can't give you the exact
> reference of where I read (but I can give you some first google hits
> like [4] and [5]) it but AFAIK this is yet a problem to be solved
> upstream (don't remember if it's in the kernel or in the extX filesystem).

Again, I'm completely non-biased to our outcome. I do like the
completion prediction idea however. Not that its necessarily tied to
Sqlite though.

However, note that Sqlite queries would most likely be delegated to a
single thread for thread-safety (hopefully not the gui thread), the
fsync() issue you notice in firefox would be a non-issue. It would
just mean some async callbacks from the sqlite query would take a
little longer if that level of atomic commit support is desired (which
i doubt is the case). Sqlite prides itself on being able to safely
abort a transaction at any cpu-cycle up to the last block write (which
is atomic on all the platforms it runs on).

> Regards,
>
>    Andrés
>
> [1] http://en.wikipedia.org/wiki/Object-Relational_impedance_mismatch
> [2] http://en.wikipedia.org/wiki/Object-relational_mapping
> [3] http://www.db4o.com/
> [4]
> http://digg.com/linux_unix/Firefox_3_has_system_killing_performance_problem_for_Linux
> [5]
> http://news.softpedia.com/news/Who-Freezes-The-System-Firefox-or-ext3-86242.shtml
> --
>
> _______________________________________________
> Monodevelop-devel-list mailing list
> Monodevelop-devel-list at lists.ximian.com
> http://lists.ximian.com/mailman/listinfo/monodevelop-devel-list
>

-- Christian