[Mono-dev] Should we replace MemoryStream?

[Steve Bjorg]

An updated version of ChunkedMemoryStream has been checked in.  It's  
not production ready yet due to insufficient test coverage.  But it  
should provide some insights into how the code handles both the  
variable and fixed buffer cases, and consolidates the chunks when  
needed.  Also, the code is now available under both Apache License 2.0  
and X11.

Feedback welcome.

http://viewvc.mindtouch.com/public/dream/trunk/src/mindtouch.dream/IO/

- Steve

--------------
Steve G. Bjorg
http://mindtouch.com
http://twitter.com/bjorg
irc.freenode.net #mindtouch

On Nov 10, 2009, at 10:15 AM, Steve Bjorg wrote:

> I have an updated ChunkedMemoryStream implementation that mimics  
> MemoryStream behavior up to the default chunk size, at which point  
> it starts to use chunks.  GetBuffer() consolidates all chunks into  
> the main buffer, which means that any direct changes would be  
> reflected.
>
> Where can I find the mono unit tests for MemoryStream?  That will  
> save me some time from having to write my own.  At that point, I  
> would encourage everyone to look at it and try it out with real  
> world data.
>
> - Steve
>
> --------------
> Steve G. Bjorg
> http://mindtouch.com
> http://twitter.com/bjorg
> irc.freenode.net #mindtouch
>
> On Nov 10, 2009, at 9:56 AM, Avery Pennarun wrote:
>
>> On Tue, Nov 10, 2009 at 12:42 PM, Robert Jordan <robertj at gmx.net>  
>> wrote:
>>> An algorithm based on a MemoryStream implemented with chunks will
>>> perform better in average. I fully agree with that.
>>>
>>> The problem is that one method (GetBuffer) *will be* unexpected
>>> slower,
>>
>> I just don't believe this is true.  I think we're moving the slowness
>> from "add to buffer" into GetBuffer().  However, it is not
>> *additional* slowness.  It is simply displaced slowness, and it's
>> potentially *less* slowness overall.
>>
>> I'm not sure I can imagine a program that would be negatively  
>> affected
>> by this.  Doesn't the gc cause random slowness sometimes anyway?
>>
>>> and another one, much harder to fix: it is allowed to change
>>> the buffer even before the stream has been closed. This means that
>>> after every GetBuffer call, the implementation must behave  
>>> differently
>>> because it must somehow deal with a changed underlying buffer.
>>
>> I don't think this is a problem either.  Since you're now using the
>> returned buffer as your one-and-only chunk, you can use it just as  
>> you
>> always would.  If someone then pushes so much new data into the  
>> stream
>> that you would exceed the buffer size, you would have to do what you
>> would do in the non-chunked implementation; either a) reject it, or  
>> b)
>> not guarantee that it ends up in the array from the earlier
>> GetBuffer().  I'm not sure which is the correct behaviour, but both
>> are easily implemented in the chunked implementation too,  
>> particularly
>> since it has to support user-supplied fixed-length buffers anyhow.
>>
>> Perhaps I'm missing something...
>>
>> Avery
>> _______________________________________________
>> Mono-devel-list mailing list
>> Mono-devel-list at lists.ximian.com
>> http://lists.ximian.com/mailman/listinfo/mono-devel-list
>