[Monodevelop-devel] Changes to resoving/ast/type system

[Mike Krüger]

Hi

Monodevelop joins forces with the SharpDevelop team in the area 'how do 
we represent source code'.
(Some of you may know that I've good contacts to the #develop team - I 
started #develop 2000 and left 2006 - and joined monodevelop 2007).

This will change the monodevelop source code a bit:

- The old nrefactory and it's DOM is depreceted and we wont see it 
again. (It lived for more than 10 years now)

It  will be replaced with a shiny new Ast + Parser. The parser most of 
you know as 'mcs' and the Ast is inside the c# binding - and inside the 
new nrefactory on github.

For consuming code only little issues change (I translated much old 
nrefactory code). Most should be 'the same' on user side. With one 
exception: you can't 'reparent' ast nodes:

Doesn't work:
     ifStatment.EmbeddedStatment = forStatment.EmbeddedStatment;

You can do that using the 'Clone' method.

Works:
     ifStatment.EmbeddedStatment = forStatment.EmbeddedStatment.Clone ();

All ast nodes have links to their parents & siblings - that's why a node 
can have only one place in the tree. The tree contains all tokens, 
comments etc. and it is able to hold new lines/white spaces or any other 
sort of information.
The tree infrastructure is pretty mature (thanks to marek providing the 
parser) it contains exact text locations for any c# token inside the ast 
- it's pretty nice to write refactorings with that.

- The DOM changes - in fact it's not called DOM anymore we call it now 
'Type system'. I've always had problems explaining the differences 
between MD DOM and C# DOM. Now we call it abstract syntax tree on the 
language level and give the high level representation the name 'type 
system'.

#Develop once started with a type system like monodevelop has from the 
model - I used something like reflection - but didn't really care about 
that much. The #develop team made some great effort on that layer. 
They've a better representation & resolving strategy than we had. Here 
some things change.

MD Dom: Getting all methods on a type:
GetAllMethods ()....
     foreach (var type in dom.GetInheritanceTree (origType))
         foreach (var method in type.Methods)
             yield return method;

NRefactory dom:
GetAllMethods ()....
     origType.GetMethods (ctx);

I think you'll get the point - their type system is more high level than 
ours was. It's easier to get the accisibility (finding out if something 
is private wasn't really easy with the monodevelop dom since the private 
modifier was syntactic).
The type names will sound all familiar - they're the same. Whats mostly 
changing is that the 'dom' as object will not be so important anymore - 
instead it'll be a parameter of most methods.
Even if the usage is 'mostly' the same the model is very different:
- CompilationUnits (are now called IParsedFile) are now always stored 
correctly (this info is never lost).
- The resolving of return types is done at call time. There is no real 
way to do it clean without that, but the resolver is lighting fast and 
correctness > speed. This resolves some issues we had (like changing 
namespaces, makes it possible to use 'external aliases')
- IType and the definition are now 2 classes
IType is that what 'InstantiatedType' was and ITypeDefinition is the 
real definition in file. You won't need ITypeDefinition very often - 
most stuff is done with IType.

While consuming code should be easy to translate dom parsers are a bit 
harder to translate (but again - mostly the same). Atm we've c# and 
cecil as type system source - and I think that nrefactory.vb has a type 
system converter + parser + ast for vb.net. Ovearall the new type system 
feels a lot more like a 'high level' api, which is a good thing.

- Resolver changes
The new resolver infrastructure changes internally very much - the old 
one basically took an expression and a position in the file. Then it 
gave you a resolve result. The new resolver resolves a 'set' of ast 
nodes. And then it analyzes the ast with the 'set' (the set should 
contain all parents of the node to resolve, all other nodes are 
skipped). I intend to hide the resolver internals from the users, so 
that they do not need to care about that (In fact I keep the resolver 
backend mostly intact - but the ResolveResults change - the new resolver 
brings own results, but consuming code is very easy to translate).
What I can say is that the new one + infrastructure is very fast to use :)

... I don't know when I do the commit, but the ast/type system/resolver 
is all in NRefactory - I just translate our infrastructure over. Atm I 
broke some things - for ex. the refactoring infrastructure & c# code 
completion. But semantic highlighting works and it seems to be usable 
now :) - context actions are in place as well. I think that I'll commit 
around friday (y, I want to get sure I didn't break anything and I need 
to fix some issues that are only working 80%  - I commented out some 
functions that I didn't find not so important for ex. in the code 
metrics or unit tests) + I can't tell how long I need for the code 
completion.

Regards
Mike