[MonoDevelop] Lucene-based full-text search

Sergey Lobko-Lobanovsky serge.lobanovsky at gmail.com
Thu Oct 14 15:46:31 EDT 2010


OK folks, I have some progress to report. I was able to get the latest
Lucene.Net source tar.gz, build it, and do some prototyping to clarify my
vision of how it may fit into MD. Here's my stream of consciousness on the
topic:

1. The easiest part is to search on everything except for the source code.
Lucene has a notion of Document that can contains Fields. As the index is
constructed, the content inside Fields can be queried. Fields can either
contain tokenized content from sources, or can just provide an index.
Basically integrating Lucene into MD will mean writing Document classes for
all items that should be searchable throughout the IDE, and constructing
mutable (project source files) and immutable (IDE commands, for example)
indices . The search mechanism itself is pretty simple, and can be
integrated quite easily.

2. However, when it comes to searching the source code a dilemma arises. On
the one hand, there is the AST tree in MD which is used for all operations
including usage search, etc. On the other hand, if Lucene is integrated into
the product, it naturally (at least in my opinion) should become the central
search service, to be used by other subsystems, such as refactoring, usage
search, etc.

3. Probably the AST tree should become the part of the Lucene project index,
but I am not quite sure whether that is needed, and if so - exactly how it
should be done.

I'd like to solicit the others' opinion on how to go about integrating
Lucene in the most efficient and desired way,

Cheers,
Sergey

On Tue, Oct 12, 2010 at 1:10 AM, Michael Hutchinson <
m.j.hutchinson at gmail.com> wrote:

> Hi Sergey,
>
> On Mon, Oct 11, 2010 at 5:34 PM, Sergey Lobko-Lobanovsky
> <sergey at bitfold.net> wrote:
> >> I can't think of anything offhand, it was just an idea that had been
> >> thrown around ("wouldn't it be awesome if searches were instant?").
> >> Other ideas in the same area would be
> >> * indexing all references to make "find references" instant - though
> >> maybe thsi could be done in pidb files rather than lucene
> >
> > Been thinking of that, too. Great minds think alike, lol
> > In my previous experience I had been a total VS.NET fan, until recently
> when
> > I picked up ActionScript, Java/Scala and the IntelliJ IDEA IDE.
> > IDEA rocks, enough said :) So I am going to use it as the source of
> ideas.
> >  And it totally looks like the fulltext index + probably some AST parsing
> > are the cornerstone for most productivity tools. That's why I decided to
> > start with the Lucene integration task.
>
> That makes sense. I'm sure Mike Krueger could give some input on that
> area. We do a get a lot of people asking for Resharper (also made by
> JetBrains) features for MD.
>
> >> * some kind of global search (project, files, panels, docs, commands)
> >> like QT Creator and VS Productivity Tools
> >>
> >> (
> http://blogs.msdn.com/b/visualstudio/archive/2010/08/03/quick-access-extension.aspx
> )
> >
> > Will need to take a look.
>
> VS productivity tools and power tools addin have a bunch of neat
> ideas. I particularly like the Solution Navigator pad.
>
> >> - lucene would totally make sense for this, especially as MonoDoc docs
> >> already have NLucene indexes
> >>
> >> I'd love to hear any more ideas you have for improving usability and
> >> productivity!
> >
> > The very basic ideas I have found very useful are:
> > 1) Quick open (Ctrl-Shift-N in IDEA), where you start typing either the
> > capital letters or the full name of the file you wanna find, and it gives
> > you a live list of files to jump to.
>
> We've had this for ages - "Go to File" (Alt-Shift-O) and "Go to Type"
> (Control-Shift-T) - and in MD 2.4 we added "Navigate To..."
> (control-,) which behaves like VS2010, i.e. essentially go to any
> file/type/member :)
>
> > 2) IDEA-style of the Find all references feature - as opposed to a flat
> list
> > similar to VS.NET/MD, it gives you a hierarchical list grouped by usage
> type
> > and source file
>
> That would be nice.
>
> > 3) Then I'd love to introduce the smart tags feature - MD lacks it and
> it's
> > a pain to manually include all the namespaces you need :), but that's a
> > longer term shot.
>
> Yup, definitely. The hard problem's not the UI though so much as the
> background analysis and suggestions engine. I started on some
> background analysis stuff a few months ago -
> http://twitter.com/mjhutchinson/status/19875030462 - but I never got
> around to integrating MD's type resolver and implementing more complex
> analyses. Another related idea could be to attach quick fixes to
> compiler errors.
>
> > In case you need some testing from me, my setup is:
> > Ubuntu 10.04 on a 4GB 2-core HP ProBook 4710s
> > MonoDevelop from the trunk (not sure how this is called in the Git terms,
> I
> > don't have previous Git experience so learning something new here as well
> :)
>
> The main git branch is called "master".
>
> --
> Michael Hutchinson
> http://mjhutchinson.com
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.ximian.com/pipermail/monodevelop-list/attachments/20101014/7d94b0b1/attachment-0001.html 


More information about the Monodevelop-list mailing list