[Mono-dev] Should we replace MemoryStream?

[pablosantosluac at terra.es]

Hi,

I think a very important place were this chunked mem streams HAVE TO be
placed (in fact I'll be trying them tomorrow under heavy load on our
testing cluster) is in remoting: there's a huge number of mem streams
being created and destroyed (one for each call) and this can help...
only if the new ChunkedStream is based on a pool, which I don't think is
the case but probably could be easily doable.

What do you think about extending ChunkedMemoryStream to include an
option based on a chunk pool, so memory is actually reused instead of
freed and re-allocated? Wouldn't it be good to reduce fragmentation (I'm
still thinking on remoting)??


pablo


www.plasticscm.com



Steve Bjorg wrote:
> The implementation could be adapted so that if the chunked memory  
> stream is initialized with an existing byte array it behaves like it  
> did in the past.  It's possible that the best approached can be  
> derived for the various MemoryStream constructors.
> 
> The question is what is the most common usage pattern?  If it's  
> GetBuffer(), then there will be a performance and overhead hit.   
> However, if it's using Write()/Read() as we do, then there are some  
> significant gains to be had.  GetBytes() would also benefit, though  
> not as much.
> 
> - Steve
> 
> --------------
> Steve G. Bjorg
> http://mindtouch.com
> http://twitter.com/bjorg
> irc.freenode.net #mindtouch
> 
> On Nov 9, 2009, at 11:41 AM, Andreas Nahr wrote:
> 
>> Are you talking about System.IO.MemoryStream?
>> Then imho this would be a problematic move.
>> Most people are used to new MemoryStream (someByteArray) being O(0)  
>> time,
>> but with ChunkedStream would be O(n). In fact in those cases  
>> ChunkedStream
>> would need twice the memory because it would still need to retain the
>> original byte array (e.g. for GetBuffer).
>>
>> Happy hacking
>> Andreas
>>
>> -----Ursprüngliche Nachricht-----
>> Von: mono-devel-list-bounces at lists.ximian.com
>> [mailto:mono-devel-list-bounces at lists.ximian.com] Im Auftrag von  
>> Miguel de
>> Icaza
>> Gesendet: Montag, 9. November 2009 19:10
>> An: mono-devel-list
>> Betreff: [Mono-dev] Should we replace MemoryStream?
>>
>> Hello folks,
>>
>>    I just blogged about a memory fragmentation issue here:
>>
>> 	http://tirania.org/blog/archive/2009/Nov-09.html
>>
>>    And I am wondering: since MemoryStream is one of these sources of
>> problems, we could replace this implementation with MindTouch's
>> ChunkedStream.
>>
>> Miguel.
>>
>> _______________________________________________
>> Mono-devel-list mailing list
>> Mono-devel-list at lists.ximian.com
>> http://lists.ximian.com/mailman/listinfo/mono-devel-list
>>
>> _______________________________________________
>> Mono-devel-list mailing list
>> Mono-devel-list at lists.ximian.com
>> http://lists.ximian.com/mailman/listinfo/mono-devel-list
> 
> _______________________________________________
> Mono-devel-list mailing list
> Mono-devel-list at lists.ximian.com
> http://lists.ximian.com/mailman/listinfo/mono-devel-list
>