[Mono-dev] Should we replace MemoryStream?

[Gladish, Jacob]

It seems that the original motivation was to deal with fragmentation. In my opinion, work should be concentrated on sgen and not cherry-picking framework classes. If efficient memory buffering is required in the framework for remoting, etc, Then why not have an internal implementation of the chunked stream that does exactly what's need to enhance performance instead of trying to force the MemoryStream to be all things to all people.

After looking at the MSDN docs for MemoryStream, my feeling is that the intention of MemoryStream is to provide an adapter for byte arrays in places where the framework apis require a Stream, and not to provide an efficient way to deal with very large buffers and fragmentation.


I also think introducing things like zero'ing of buffers and pooling is adding functionality where it doesn't belong.

I know there isn't a hard requirement precise compatibility with MS.NET, but MemoryStream is "core" class that people moving from one environment to another are going to expect to perform and behave in a very similar manner. Again, referring to the MSDN docs, its very clear the MemoryStream is very naïve, and that its just an API on top of an array.

Quote from GetBuffer() docs: " To create a MemoryStream instance with a publicly visible buffer, use MemoryStream, MemoryStream(array<Byte>[]()[], Int32, Int32, Boolean, Boolean), or MemoryStream(Int32). If the current stream is resizable, two calls to this method do not return the same array if the underlying byte array is resized between calls. For additional information, see Capacity"


Just my two cents.

-jake


> -----Original Message-----
> From: mono-devel-list-bounces at lists.ximian.com [mailto:mono-devel-list-
> bounces at lists.ximian.com] On Behalf Of Avery Pennarun
> Sent: Tuesday, November 10, 2009 10:05 AM
> To: Robert Jordan
> Cc: mono-devel-list at lists.ximian.com
> Subject: Re: [Mono-dev] Should we replace MemoryStream?
>
> On Tue, Nov 10, 2009 at 8:48 AM, Robert Jordan <robertj at gmx.net> wrote:
> > MemoryStream.GetBuffer's docs indirectly suggest that no copy
> > will be performed:
> >
> > "Note that the buffer contains allocated bytes which might be unused.
> > For example, if the string "test" is written into the MemoryStream
> > object, the length of the buffer returned from GetBuffer is 256, not 4,
> > with 252 bytes unused. To obtain only the data in the buffer, use the
> > ToArray method; however, ToArray creates a copy of the data in memory."
> >
> > So MemoryStream.GetBuffer must remain an O(1) operation in any case,
> > defeating any kind of optimization a chunked memory stream
> > implementation may introduce.
>
> Although this might be strictly true if you want to react exactly as
> Microsoft's documentation claims (I thought 100% compatibility with
> .Net was not the primary goal of mono?), there may be other options
> that result in similar performance
>
> For example, the first call to GetBuffer() could "coagulate" the
> chunks into a single big array (perhaps with extra space at the end),
> and then *keep that array*.  Subsequent calls to GetBuffer() could
> avoid the copy.
>
> In the event that your initial chunk wasn't big enough when pushing
> data into the buffer in the first place, a non-chunked implementation
> would have had to make an extra copy *anyway* at the time of the push.
>  So in the chunked implementation, the extra copy on the first
> GetBuffer() is actually not an *extra* copy at all vs. the naive
> single-buffer implementation.
>
> (I've written an efficient implementation of chunked buffering in C++,
> and these were the conclusions I drew after a lot of benchmarking of
> my library.  YMMV in C#, etc.)
>
> Have fun,
>
> Avery
> _______________________________________________
> Mono-devel-list mailing list
> Mono-devel-list at lists.ximian.com
> http://lists.ximian.com/mailman/listinfo/mono-devel-list

IMPORTANT: The information contained in this email and/or its attachments is confidential. If you are not the intended recipient, please notify the sender immediately by reply and immediately delete this message and all its attachments. Any review, use, reproduction, disclosure or dissemination of this message or any attachment by an unintended recipient is strictly prohibited. Neither this message nor any attachment is intended as or should be construed as an offer, solicitation or recommendation to buy or sell any security or other financial instrument. Neither the sender, his or her employer nor any of their respective affiliates makes any warranties as to the completeness or accuracy of any of the information contained herein or that this message or any of its attachments is free of viruses.