[Mono-bugs] [Bug 76218][Nor] Changed - Tokenizer/Seekable stream bugs.

[bugzilla-daemon at bugzilla.ximian.com]

Please do not reply to this email- if you want to comment on the bug, go to the
URL shown below and enter your comments there.

Changed by rharinath at novell.com.

http://bugzilla.ximian.com/show_bug.cgi?id=76218

--- shadow/76218	2005-09-27 12:40:38.000000000 -0400
+++ shadow/76218.tmp.22089	2005-09-28 02:40:29.000000000 -0400
@@ -142,6 +142,23 @@
 Created an attachment (id=15812)
 don't bother trying to keep in sync with underlying stream.  Maintain a moving window
 
 
 ------- Additional Comments From martin at ximian.com  2005-09-27 12:40 -------
 Does your patch work with GMCS, or should I have a look ?
+
+------- Additional Comments From rharinath at novell.com  2005-09-28 02:40 -------
+It works on gmcs, it's passed a full bootstrap, and doesn't crash on
+the test case above.
+
+Couple of points: I have a minor fix to make it somewhat less eager
+about shifting things off.  This'll ensure that a look-ahead of 2048
+bytes is guaranteed.
+
+I instrumented parse_less_than (but lost the log :-) -- it typically
+needs a lookahead of 1 tokens, the largest I remember seeing is 4
+tokens.  Since source code typically has alternating identifiers and
+short punctuation, this translates to about 30 chars of lookahead.
+
+Allowing for one order of magnitude in both number of lookahead tokens
+and avg. token size, I think 2048 is reasonable.
+