[Mono-dev] Should we replace MemoryStream?

[Miguel de Icaza]

Hello,

> What do you think about extending ChunkedMemoryStream to include an
> option based on a chunk pool, so memory is actually reused instead of
> freed and re-allocated? Wouldn't it be good to reduce fragmentation (I'm
> still thinking on remoting)??

This would work;

Additionally, it might make sense to use the suggestion from Steve to
use plain memory allocation up to a certain point (below some threshold)
and then switch to chunk after that to avoid allocating 16k even for
memory streams that might only have 100 bytes for example.

> 
> 
> pablo
> 
> 
> www.plasticscm.com
> 
> 
> 
> Steve Bjorg wrote:
> > The implementation could be adapted so that if the chunked memory  
> > stream is initialized with an existing byte array it behaves like it  
> > did in the past.  It's possible that the best approached can be  
> > derived for the various MemoryStream constructors.
> > 
> > The question is what is the most common usage pattern?  If it's  
> > GetBuffer(), then there will be a performance and overhead hit.   
> > However, if it's using Write()/Read() as we do, then there are some  
> > significant gains to be had.  GetBytes() would also benefit, though  
> > not as much.
> > 
> > - Steve
> > 
> > --------------
> > Steve G. Bjorg
> > http://mindtouch.com
> > http://twitter.com/bjorg
> > irc.freenode.net #mindtouch
> > 
> > On Nov 9, 2009, at 11:41 AM, Andreas Nahr wrote:
> > 
> >> Are you talking about System.IO.MemoryStream?
> >> Then imho this would be a problematic move.
> >> Most people are used to new MemoryStream (someByteArray) being O(0)  
> >> time,
> >> but with ChunkedStream would be O(n). In fact in those cases  
> >> ChunkedStream
> >> would need twice the memory because it would still need to retain the
> >> original byte array (e.g. for GetBuffer).
> >>
> >> Happy hacking
> >> Andreas
> >>
> >> -----Ursprüngliche Nachricht-----
> >> Von: mono-devel-list-bounces at lists.ximian.com
> >> [mailto:mono-devel-list-bounces at lists.ximian.com] Im Auftrag von  
> >> Miguel de
> >> Icaza
> >> Gesendet: Montag, 9. November 2009 19:10
> >> An: mono-devel-list
> >> Betreff: [Mono-dev] Should we replace MemoryStream?
> >>
> >> Hello folks,
> >>
> >>    I just blogged about a memory fragmentation issue here:
> >>
> >> 	http://tirania.org/blog/archive/2009/Nov-09.html
> >>
> >>    And I am wondering: since MemoryStream is one of these sources of
> >> problems, we could replace this implementation with MindTouch's
> >> ChunkedStream.
> >>
> >> Miguel.
> >>
> >> _______________________________________________
> >> Mono-devel-list mailing list
> >> Mono-devel-list at lists.ximian.com
> >> http://lists.ximian.com/mailman/listinfo/mono-devel-list
> >>
> >> _______________________________________________
> >> Mono-devel-list mailing list
> >> Mono-devel-list at lists.ximian.com
> >> http://lists.ximian.com/mailman/listinfo/mono-devel-list
> > 
> > _______________________________________________
> > Mono-devel-list mailing list
> > Mono-devel-list at lists.ximian.com
> > http://lists.ximian.com/mailman/listinfo/mono-devel-list
> > 
> 
> _______________________________________________
> Mono-devel-list mailing list
> Mono-devel-list at lists.ximian.com
> http://lists.ximian.com/mailman/listinfo/mono-devel-list