[Mono-dev] Problem handling more the 1024 file handle

[Torello Querci]

Looking in the strace output file this is the setrlimit call:

setrlimit(RLIMIT_NOFILE, {rlim_cur=20000, rlim_max=20000}) = 0

so seems to be correct ....


2011/11/4 Torello Querci <tquerci at gmail.com>:
> Sorry, You are right .... I'm on 32bit arch
>
> 2011/11/4 Robert Jordan <robertj at gmx.net>:
>> On 04.11.2011 10:17, Torello Querci wrote:
>>> This is the first test that I realized.
>>> On the same machine, same kernel .... same calls ....
>>
>> You don't say which Mono arch you're using. On 32 bit,
>> the following might not be correct:
>>
>>        struct rlimit {
>>                public IntPtr rlimit_cur;
>>                public IntPtr rlimit_max;
>>        }
>>
>> because rlim_t is 8 byte long whereas IntPtr is 4 bytes.
>> This is because Mono is compiled with LARGEFILE64 support.
>>
>> Robert
>>
>>
>>>
>>> Below the source code:
>>>
>>> =====================================================================
>>> #include<linux/resource.h>
>>>
>>> //#include<stdlib.h>
>>> #include<stdio.h>
>>>
>>>
>>> int main(int argc, char* argv[]) {
>>>
>>>      FILE** fp;
>>>      int nFiles = atoi(argv[1]);
>>>      int i;
>>>      char filename[1024];
>>>
>>>      if (argc!=2) {
>>>          printf("Error. You need to specify the number of file that
>>> need to create.\n");
>>>          return -1;
>>>      }
>>>
>>>      struct rlimit data;
>>>
>>>      data.rlim_cur = 20000;
>>>      data.rlim_max = 20000;
>>>
>>>      i = setrlimit (RLIMIT_NOFILE,&data);
>>>      if (i != 0) {
>>>          printf ("Error during changine files limits.\n");
>>>          return 1;
>>>      }
>>>
>>>      fp = calloc (nFiles, sizeof(FILE*));
>>>
>>>      for (i=0; i<nFiles; ++i) {
>>>          sprintf (filename, "file%i.out", i);
>>>
>>>          fp[i] = fopen (filename, "w+");
>>>          if (fp [i] == NULL) {
>>>              printf ("Error in %s file creation.\n", filename);
>>>              return -1;
>>>          }
>>>      }
>>>
>>> }
>>>
>>> =============================================================
>>>
>>> 2011/11/4 Alan<alan.mcgovern at gmail.com>:
>>>> Hi,
>>>> Mono itself has no trouble opening 1000's of files. Would you be able to
>>>> create an equivalent C program and see if that works as expected? My guess
>>>> would be that the equivalent C program will fail in the same way. If it does
>>>> not, then it's likely to be a mono bug.
>>>> Alan
>>>>
>>>> On 4 November 2011 07:19, Torello Querci<tquerci at gmail.com>  wrote:
>>>>>
>>>>> Hi al,
>>>>>
>>>>> trying to handle more that 1024 handle file I got an exception even if
>>>>> I increase the max number of file limit using "setrlimit". The example
>>>>> source code below.
>>>>> Of course mono have the right pcap permission.
>>>>>
>>>>> Using strace on both this code and no setrlimit version code I have
>>>>> some differences.
>>>>> On the no setrlimit version I have this syscall:
>>>>>
>>>>> open("file1021.out", O_WRONLY|O_CREAT|O_TRUNC|O_LARGEFILE, 0666) = -1
>>>>> EMFILE (Too many open files)
>>>>>
>>>>> and this seems to be correct.
>>>>> On the setrlimit version code I have this syscalls:
>>>>>
>>>>> open("file1021.out", O_WRONLY|O_CREAT|O_TRUNC|O_LARGEFILE, 0666) = 1024
>>>>> close(1024)                             = 0
>>>>>
>>>>> so seems that mono close the file because open  return value is 1024.
>>>>>
>>>>> Is this a bug, a feature, or something else?
>>>>>
>>>>>
>>>>> ==================================================================================================================
>>>>> using System;
>>>>> using System.IO;
>>>>> using System.Text;
>>>>> using System.Runtime.InteropServices;
>>>>>
>>>>> namespace TestFiles
>>>>> {
>>>>>
>>>>>         struct rlimit {
>>>>>                 public IntPtr rlimit_cur;
>>>>>                 public IntPtr rlimit_max;
>>>>>         }
>>>>>
>>>>>         class MainClass
>>>>>         {
>>>>>                 public static unsafe void Main (string[] args)
>>>>>                 {
>>>>>                         FileStream[] streams;
>>>>>                         string path;
>>>>>                         if (args.Length == 0) {
>>>>>                                 Console.WriteLine ("You need to specify the
>>>>> number of files that
>>>>> needs to be created.");
>>>>>                                 return;
>>>>>                         }
>>>>>
>>>>>                         int nFiles = Int32.Parse (args[0]);
>>>>>
>>>>>                         streams = new FileStream[nFiles];
>>>>>
>>>>>                         rlimit data = new rlimit ();
>>>>>                         data.rlimit_cur = (IntPtr) 20000;
>>>>>                         data.rlimit_max = (IntPtr) 20000;
>>>>>
>>>>>                         int result = setrlimit (RLIMIT_NOFILE,&data);
>>>>>                         if (result != 0) {
>>>>>                                 throw new Exception ("Cannot change limit
>>>>> on open files");
>>>>>                         }
>>>>>
>>>>>                         for (int i=0; i<nFiles; ++i) {
>>>>>                                 path = string.Format ("file{0}.out", i);
>>>>>                                 try {
>>>>>                                         streams[i] = File.Open(path,
>>>>> FileMode.Create, FileAccess.Write,
>>>>> FileShare.None);
>>>>>                                 } catch (Exception ex) {
>>>>>                                         Console.WriteLine ("Unable to write
>>>>> file {0}", path);
>>>>>                                         Console.WriteLine (ex.Message);
>>>>>                                         Console.WriteLine (ex.StackTrace);
>>>>>                                         return;
>>>>>                                 }
>>>>>                         }
>>>>>                 }
>>>>>
>>>>>                 const int RLIMIT_NOFILE = 7;
>>>>>
>>>>>                 [DllImport ("libc", SetLastError = true)]
>>>>>                 unsafe extern static int setrlimit(int resource, rlimit*
>>>>> rlim);
>>>>>
>>>>>         }
>>>>> }
>>>>> _______________________________________________
>>>>> Mono-devel-list mailing list
>>>>> Mono-devel-list at lists.ximian.com
>>>>> http://lists.ximian.com/mailman/listinfo/mono-devel-list
>>>>
>>>>
>>
>>
>> _______________________________________________
>> Mono-devel-list mailing list
>> Mono-devel-list at lists.ximian.com
>> http://lists.ximian.com/mailman/listinfo/mono-devel-list
>>
>