[Mono-dev] mcs patch to say goodbye to SeekableStreamReader

[Atsushi Eno]

Hello,

Thanks to Kornél's msg I noticed that I shouldn't reset cached token
on identifier "partial". Thus I created fixed patch.

>> Now I blame SeekableStreamReader on bringing UTF8 related bug
>> (without proof ;-) so I made a patch to eliminate this class.
>>
>> Additionally I made tiny modification for 'MZ' executable check
>> (even with a test case bom-mz.cs that differentiates csc and mcs).
>>
>> I'm not sure if it really is the culprit, but now we don't have
>> tricky stream usage, so now the code should be a bit healthy.
> 
> In general adding more tests to the xtoken() routine hurts performance,
> that is why we have avoided it.  I also believe that the new code is a
> bit more difficult to follow: just how many saved variables are we going
> to have?

I agree that xtoken() could be performance sensitive, but on the
other hand, current code moves back the stream, which results in
extraneous parse on every parenthesized expressions. Actually
as mentioned later, the patched code seems faster. Note that
SeekableStreamReader has to call encoding GetCharCount() when
we reposition the stream.

For complexity, SeekableStreamReader is much more difficult to
debug; it requires us to debug not only tokenizer but also
Stream repositioning, Encoding conversion sanity, relationship
between Stream repositioning and conversion in-process bytes
(incomplete blocks).

I needed to introduce just one cached token and one boolean flag
to support putback from parser (the latter one is required because
there could be another putback_char).

> Before we can do something about this, I would like two tests to be
> done:
> 
> 	time mcs --parse sourcefiles
> 
> Where sourcefiles means a lot of source code, measure before and after.
> 
> Then we should measure the impact of the allocation of SavedToken with
> mono --profile before and after.

Actually the resulting performance is better than existing code:

existing code at *best*:
real    0m2.711s
user    0m0.080s
sys     0m0.050s

Total memory allocated: 34835 KB

patched code at *worst*:
real    0m2.635s
user    0m0.020s
sys     0m0.030s
Total memory allocated: 34083 KB

Most of the memory consumption difference comes from
SeekableStreamReader:

########################
     731 KB Mono.CSharp.SeekableStreamReader::.ctor(StreamReader)
         731 KB      363 System.Char[]
  Callers (with count) that contribute at least for 1%:
         363  100 % Mono.CSharp.SeekableStreamReader::.ctor(Stream,Encoding)

Atsushi Eno
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: goodbye-seekablestreamreader.patch
Url: http://lists.ximian.com/pipermail/mono-devel-list/attachments/20050831/f0c36db4/attachment.pl