[Mono-list] System.Net.HttpWebRequest using a System.IO.MemoryStream as a buffer

Fri Jan 27 08:45:40 UTC 2012

https://github.com/mono/mono/blob/master/mcs/class/System/System.Net/HttpWebRequest.cs

On 01/27/2012 01:50 AM, Maurício Linhares wrote:
> BTW, I tried to look at the source, but the comments say the
> implementation is somewhere else, but I could not find this "somewhere
> else" ->  https://github.com/mono/mono/blob/master/mcs/class/System.Net/System.Net/HttpWebRequest_2_1.cs
>
> -
> Maurício Linhares
> http://techbot.me/ - http://twitter.com/#!/mauriciojr
>
>
>
> 2012/1/26 Maurício Linhares<mauricio.linhares at gmail.com>:
>> Hello guys,
>>
>> I think maybe I'm doing something stupid here, but I'm trying to
>> upload a somewhat large file (350 mb, in fact) and mono is growing in
>> memory usage to more than 2GB in memory (as it is obviously loading
>> the WHOLE file into memory using the MemoryStream).
>>
>> Here is the trace that I see:
>>
>> [0xb042d000: 47,97481 22] ENTER: System.Buffer:BlockCopy
>> (System.Array,int,System.Array,int,int)([System.Byte[]:0x17b4000],
>> [0xb052f000: 47,97482 22] 4096, LEAVE: System.Net.WebAsyncResult:CB
>> (object)[System.Byte[]:0x27a6000],
>> 0, 4096, )
>> [0xb052f000: 47,97490 21] LEAVE: (wrapper runtime-invoke)
>> <Module>:runtime_invoke_void__this___object
>> (object,intptr,intptr,intptr)[OBJECT:0x0]
>> [0xb042d000: 47,97494 21] ENTER: (wrapper managed-to-native)
>> System.Buffer:BlockCopyInternal
>> (System.Array,int,System.Array,int,int)([System.Byte[]:0x17b4000],
>> 4096, [System.Byte[]:0x27a6000], 0, 4096, )
>> [0xb042d000: 47,97497 21] LEAVE: (wrapper managed-to-native)
>> System.Buffer:BlockCopyInternal
>> (System.Array,int,System.Array,int,int)TRUE:1
>> [0xb042d000: 47,97500 20] LEAVE: System.Buffer:BlockCopy
>> (System.Array,int,System.Array,int,int)
>> [0xb042d000: 47,97501 19] LEAVE: System.IO.FileStream:ReadSegment
>> (byte[],int,int)result=4096
>> [0xb042d000: 47,97503 18] LEAVE: System.IO.FileStream:ReadInternal
>> (byte[],int,int)result=4096
>> [0xb042d000: 47,97504 17] LEAVE: System.IO.FileStream:Read
>> (byte[],int,int)result=4096
>> [0xb042d000: 47,97505 17] ENTER: System.Net.WebConnectionStream:Write
>> (byte[],int,int)(this:0x27a3f50[System.Net.WebConnectionStream
>> SyncDropProfiler.exe], 0x27a6000, 0, 4096, )
>> [0xb042d000: 47,97507 18] ENTER:
>> System.Net.WebConnectionStream:BeginWrite
>> (byte[],int,int,System.AsyncCallback,object)(this:0x27a3f50[System.Net.WebConnectionStream
>> SyncDropProfiler.exe], 0x27a6000, 0, 4096,
>> [System.AsyncCallback:0x2712d20], 0x0, )
>> [0xb042d000: 47,97509 19] ENTER: (wrapper remoting-invoke-with-check)
>> System.Net.HttpWebRequest:get_Aborted
>> ()(this:0x2710ed8[System.Net.HttpWebRequest SyncDropProfiler.exe], )
>> [0xb042d000: 47,97511 20] ENTER: System.Net.HttpWebRequest:get_Aborted
>> ()(this:0x2710ed8[System.Net.HttpWebRequest SyncDropProfiler.exe], )
>> [0xb042d000: 47,97512 20] LEAVE: System.Net.HttpWebRequest:get_Aborted ()FALSE
>> [0xb042d000: 47,97513 19] LEAVE: (wrapper remoting-invoke-with-check)
>> System.Net.HttpWebRequest:get_Aborted ()FALSE
>> [0xb042d000: 47,97514 19] ENTER: (wrapper managed-to-native)
>> object:__icall_wrapper_mono_object_new_fast (intptr)(0xc64568, )
>> [0xb042d000: 47,97516 19] LEAVE: (wrapper managed-to-native)
>> object:__icall_wrapper_mono_object_new_fast
>> (intptr)[System.Net.WebAsyncResult:0x3a8cc80]
>> [0xb042d000: 47,97517 19] ENTER: System.Net.WebAsyncResult:.ctor
>> (System.AsyncCallback,object)(this:0x3a8cc80[System.Net.WebAsyncResult
>> SyncDropProfiler.exe], [System.AsyncCallback:0x2712d20], 0x0, )
>> [0xb042d000: 47,97519 20] ENTER: (wrapper managed-to-native)
>> object:__icall_wrapper_mono_object_new_ptrfree (intptr)(0x1bc4a54, )
>> [0xb042d000: 47,97520 20] LEAVE: (wrapper managed-to-native)
>> object:__icall_wrapper_mono_object_new_ptrfree
>> (intptr)[System.Object:0x3a73328]
>> [0xb042d000: 47,97521 19] LEAVE: System.Net.WebAsyncResult:.ctor
>> (System.AsyncCallback,object)
>> [0xb042d000: 47,97523 19] ENTER:
>> System.Net.HttpWebRequest:get_ContentLength
>> ()(this:0x2710ed8[System.Net.HttpWebRequest SyncDropProfiler.exe], )
>> [0xb042d000: 47,97524 19] LEAVE:
>> System.Net.HttpWebRequest:get_ContentLength
>> ()lresult=0xffffffffffffffff
>> [0xb042d000: 47,97525 19] ENTER:
>> System.Net.WebConnectionStream:CheckWriteOverflow
>> (long,long,long)(this:0x27a3f50[System.Net.WebConnectionStream
>> SyncDropProfiler.exe], 0xffffffffffffffff, 0x0000000002c570a0,
>> 0x0000000000001000, )
>> [0xb042d000: 47,97527 19] LEAVE:
>> System.Net.WebConnectionStream:CheckWriteOverflow (long,long,long)
>> [0xb042d000: 47,97528 19] ENTER: System.IO.MemoryStream:Write
>> (byte[],int,int)(this:0x277eed0[System.IO.MemoryStream
>> SyncDropProfiler.exe], 0x27a6000, 0, 4096, )
>> [0xb042d000: 47,97529 20] ENTER:
>> System.IO.MemoryStream:CheckIfClosedThrowDisposed
>> ()(this:0x277eed0[System.IO.MemoryStream SyncDropProfiler.exe], )
>> [0xb042d000: 47,97530 20] LEAVE:
>> System.IO.MemoryStream:CheckIfClosedThrowDisposed ()
>> [0xb042d000: 47,97531 20] ENTER: System.IO.MemoryStream:Expand
>> (int)(this:0x277eed0[System.IO.MemoryStream SyncDropProfiler.exe],
>> 46497952, )
>> [0xb042d000: 47,97532 20] LEAVE: System.IO.MemoryStream:Expand (int)
>> [0xb042d000: 47,97533 20] ENTER: System.Buffer:BlockCopy
>> (System.Array,int,System.Array,int,int)([System.Byte[]:0x27a6000], 0,
>> [System.Byte[]:0xab6e000], 46493856, 4096, )
>> ^C[0xb042d000: 47,98481 21] ENTER: (wrapper managed-to-native)
>> System.Buffer:BlockCopyInternal
>> (System.Array,int,System.Array,int,int)([System.Byte[]:0x27a6000], 0,
>> [System.Byte[]:0xab6e000], 46493856, 4096, )
>> [0xb042d000: 47,98484 21] LEAVE: (wrapper managed-to-native)
>> System.Buffer:BlockCopyInternal
>> (System.Array,int,System.Array,int,int)TRUE:1
>>
>> And here is the code I'm using ->  https://gist.github.com/1676771
>>
>> Since once I start writing to the request stream the HttpWebRequest
>> class does not allow me to change any headers, I was led to believe
>> the request had already started for real and I was streaming the
>> content to the HTTP server, which isn't true since my activity monitor
>> does not show any kind of network activity.
>>
>> So, am I doing something wrong here or is this object really trying to
>> load a 350mb file into memory to make the HTTP request?
>>
>> -
>> Maurício Linhares
>> http://techbot.me/ - http://twitter.com/#!/mauriciojr
> _______________________________________________
> Mono-list maillist  -  Mono-list at lists.ximian.com
> http://lists.ximian.com/mailman/listinfo/mono-list
>