[Mono-devel-list] UDP 40%+ loss
marcusmonaghan at f2s.com
marcusmonaghan at f2s.com
Mon Apr 4 17:39:34 EDT 2005
Some more information:
It seems my testing was slightly incorrect. I still think there is a problem but
not with mono. When I did my initial testing I used a program on one machine to
send 51000 messages on a broadcast group. I then had two programs running, the
first was the C program and the second was the c# program. When I checked the
results I checked the output of the C program (the file that was generated) and
the count of messages generated by the c# program. I accidentally checked the
input file instead of the output file and thus thought the C program wasn't
missing any messages. However, after running the test several more times I have
discovered that the C program is actually missing data, probably more data than
the c# program. This is to be expected as the C program is doing a lot more
work. So I think I can safely say that c# and mono on linux is just as bad as C
:)
However, the interesting thing is that running the same C program on Unixware
(and i'm doubly sure) doesn't miss a message. Infact on Unixware it seems to
buffer the messages automatically. I can see that when the sending process
finishes the C program is still receiving and keeps receiving for about 1min
45secs.
So I'm off to investigate the differences between Linux (Suse 9.2 Pro) and
Unixware.
Thanks Jason for replying!!
Kind regards,
Marcus
Quoting marcusmonaghan at f2s.com:
> Thanks for the reply Jason.
>
> I didn't post code initially as I was just seeing if there was a "ah yes, we
> know about that one" answer.
>
> So as there doesn't seem to be that, here is the code snippets. I have
> exluded
> obtaining settings:
>
> C code:
> while (TRUE) /* receive (and pass on) loop */
> {
> /* Now clear buffers ready for next iteration */
> memset(buffer, '\0', BUFLEN);
> memset(outbuf, '\0', BUFLEN);
>
> /* Let select know which file descriptor we want to wait on */
> FD_SET(multicast_socket, &read_set);
> /* If we are not using a file then add the tcp socket to the set
> */
> if (message_file <= FAILURE) FD_SET(tcp_socket, &read_set);
>
> /* Select needs to know the maximum file descriptor to look for.
> We
> could specify FS_SETSIZE which is defined as 256 but
> this would meen the kernal would be checking for extra file
> descriptors that did not exist. */
> if (multicast_socket > tcp_socket || message_file >= VALID)
> max_fd = multicast_socket + 1;
> else
> max_fd = tcp_socket + 1;
>
> /* Now select and wait for a signal */
> now_msg(0, "Waiting on select ...\n", errno);
> selected_fd = select(max_fd, &read_set, NULL, NULL, NULL);
> now_msg(1, "After select ...\n", errno);
>
> /* We have a connection so reset the connection_try indicator */
> connection_try=0;
>
> if (selected_fd == FAILURE)
> {
> now_msg_err(0, errno, "Error on select");
> break;
> }
> if (selected_fd == TIMEOUT)
> {
> /* Currently this isn't supported so shouldn't be called yet
> */
> now_msg(0, "Select timed out ... POLL");
>
> /* Need to clear here as we are resetting at the top of the
> loop
> */
> FD_CLR(multicast_socket, &read_set);
> FD_CLR(tcp_socket, &read_set);
> continue;
> }
>
> if ((message_file <= FAILURE) && FD_ISSET(tcp_socket, &read_set))
> /* Data available on tcp socket, only if not using a file */
> {
>
> .. Doesn't fire!!!
>
> } /* End of data available on TCP socket */
>
> if (FD_ISSET(multicast_socket, &read_set)) /* Data available on
> udp
> socket */
> {
> bytes_received=recvfrom(multicast_socket, buffer, BUFLEN, 0,
> NULL, NULL); /* get one message at a time (as it is UDP) */
> if(bytes_received==FAILURE)
> {
> now_msg_err(0, errno, "Error receiving multicast
> messages");
> break;
> }
> /* Let the user know what we have received (remove a linefeed
> if
> one is already there due to testing) */
> if (buffer[bytes_received - 1] == '\n')
> {
> now_msg(5, "removing LF from input\n");
> buffer[--bytes_received] = 0;
> }
> now_msg(0, "<<<[%d][%s]\n", bytes_received, buffer);
> if (add_ascii_header == TRUE)
> {
> .. No longer used
> }
> else
> {
> sprintf(outbuf, "\"%s\"\n", buffer);
> }
> bytes_to_send = strlen(outbuf);
>
> /* Depending on whether or not a file has been specified send
> to
> to a tcp server or write it to a file */
> if (message_file >= VALID) /* File provided */
> {
> now_msg(5, "Writing to file [%d]\n", bytes_to_send);
> bytes_sent=write(message_file, outbuf, bytes_to_send);
> }
> else
> {
> now_msg(5, "Writing to socket [%d]\n", bytes_to_send);
> bytes_sent=send(tcp_socket, outbuf, bytes_to_send, 0
> ); /*
> write the message - 0 = no options */
> }
>
> outbuf[bytes_to_send] = 0; /* for the record, remove the \n,
> but
> retain real length for checking purposes */
> now_msg(0, ">>>[%d][%s]\n", bytes_sent, outbuf);
> } /* End of data available on udp socket */
> } /* listener failed */
>
> c# Code
>
> public class MulticastClient
> {
> #region Member Variables
> private string _multicastAddress = string.Empty;
> private int _multicastPort = 0;
> #endregion Member Variables
>
> #region Variables
> private int messageCount = 0;
> private IPAddress multicastAddress;
> private Socket multicastSocket;
> #endregion Variables
>
> #region Constructors
> public MulticastClient(string MulticastGroup, int MulticastPort)
> {
> new MulticastClient(MulticastGroup, MulticastPort, true);
> }
>
> public MulticastClient(string MulticastGroup, int MulticastPort, bool
> StartImmediately)
> {
> _multicastAddress = MulticastGroup;
> _multicastPort = MulticastPort;
>
> // Do we want to start?
> if (StartImmediately) { StartListening(); }
> }
>
> #endregion Constructors
>
> #region Properties
> public string HostAddress
> {
> get
> {
> return _multicastAddress;
> }
> set
> {
> // TODO: Only allow the setting of the property if the client is not
> active
>
> if (value != _multicastAddress)
> {
> _multicastAddress = value;
> }
> }
> }
>
> public int Port
> {
> get
> {
> return _multicastPort;
> }
> set
> {
> // TODO: Only allow the setting of the property if the client is not
> active
> _multicastPort = value;
> }
> }
> #endregion
>
> #region Methods
>
> private void ConfigureMulticastSocket()
> {
> // Assign the local address automatically
> EndPoint localEndPoint;
> MulticastOption multicastOption;
> IPAddress localIPAddress = IPAddress.Any; // TODO: Possibly need to change
> this to be specific
>
> // Configure the multicast address object
> multicastAddress = IPAddress.Parse(_multicastAddress);
>
> // Create the socket object
> // TODO: Make sure the socket is not already active
> multicastSocket = new Socket(AddressFamily.InterNetwork,
> SocketType.Dgram, ProtocolType.Udp);
>
> // Now create an end point for this end. Local IP address with the
> multicast
> port
> localEndPoint = (EndPoint)new IPEndPoint(localIPAddress, _multicastPort);
>
> // Bind the socket object to the end point
> multicastSocket.Bind(localEndPoint);
>
> // Now set the multicast options
> multicastOption = new MulticastOption(multicastAddress, localIPAddress);
> multicastSocket.SetSocketOption(SocketOptionLevel.IP,
> SocketOptionName.AddMembership,
> multicastOption);
>
> }
>
> private void StartListening()
> {
> bool complete = false; // Flag to indicate completion of processing loop
> IPEndPoint groupEndPoint; // The end point to the group
> EndPoint remoteEndPoint; // The other end of the socket
> byte[] recievedBytes = new byte[10000]; // Bytes to receive
> int recievedByteCount = 0; // Number of bytes recieved
>
> // Configure the socket and start listening
> ConfigureMulticastSocket();
>
> // Configure the end point for the group
> groupEndPoint = new IPEndPoint(multicastAddress, _multicastPort);
>
> // Configure the other end point or where we want to retrieve data from.
> // For this example we are going to accept data from anywhere, but
> // we should be able to specify where we want to receive data from.
> // Port 0 is given to allow the framework/OS to decide what is used.
> remoteEndPoint = (EndPoint) new IPEndPoint(IPAddress.Any, 0);
>
> // Wait for data
> // TODO: This should be asynchronous
> try
> {
> while(!complete)
> {
> Console.WriteLine("Waiting for multicast messages ...");
>
> // Wait for some data
> recievedByteCount = multicastSocket.ReceiveFrom(recievedBytes, ref
> remoteEndPoint);
>
> // Show we have got something
> Console.WriteLine("Received {2}: {0} from {1}",
> Encoding.ASCII.GetString(recievedBytes, 0, recievedByteCount),
> remoteEndPoint.ToString(),
> ++messageCount);
> }
> }
> catch(Exception e)
> {
> Console.WriteLine(e.ToString());
> }
>
>
> }
>
> #endregion Methods
>
> }
>
>
> As you can see the c# code is no where near complete compared to the C code.
> It
> has less processing, but still manages to miss the messages. The only thing I
> can think of is to remove the Console.Writelines and log the messages to a
> log
> file. Just incase this is actually causing the issue (overhead of writing out
> to the console is more than writing to a file perhaps).
>
> Marcus.
>
>
>
> Quoting Jason Starin | Giant Head <jasonstarin at giant-head.com>:
>
> >
> >
> >
> >
> > Hi Marcus,
> >
> > First off, without code, I feel like I'm going into this question blind,
> > but I do have a similar example that I did find a solution for.
> >
> > I had a web service that was capturing web requests from several different
> > forms, it then incremented a database, and then sent confirmation mails as
> > well as lead emails to different targets, based upon the web data. I had
> > all exceptions mailed to me, and it allowed me a little time to figure out
> > about how to configure my application to work well with xsp.
> >
> > Here's the general idea:
> >
> > 1. I made receipt of requests a top priority process, running on a
> separate
> > service, so that I always received requests. I had those requests write to
> > a common xml file that was hosted in memory. Whenever my requests exceeded
> > 1000 or so a minute, the service would crash if I was trying to write a
> file
> > to disk, so instead, I had a single write at the end of each minute as part
> > of a separate thread.
> >
> > 2. Database updates seem to be slow using the ByteFX and the MySql.Data
> > ADO.net connections. I built a timer that updated with the common XML file
> > every 4 minutes. That reduced the total number of database requests to a
> > manageable number that gave them plenty of time to finish (all of my
> > requests seem to take between 9 and 15 seconds to run almost regardless of
> > their size).
> >
> > 3. Because mail services (I'm using sendmail on RedHat Fedora Core 2, but
> I
> > also tested SMTP on a Win2k3 with Mono box) seemed to flake out on repeated
> > connections, I batched emails, and sent them in groups, every ten minutes.
> > If I allowed each process to send an email message, immediately, my dual
> > processor server seemed to choke on all of the connections, and eventually
> > XSP would crash and take apache with it. On Windows, I'd have a great big
> > collection of emails bogged down in my logs, needing to be resent.
> >
> > Frankly, I don't think these are bugs, but more along the lines of
> > enviornmental constraints, that you can adapt to if you map out the process
> > and look for where things are flaking out. Perhaps establish a separate
> log
> > file to watch each step through your process, and save at the end of your
> > test?
> >
> > Send code if you get a chance, I'm interested to see where I can help, and
> > have you thought about testing your C# app on a windows machine?
> >
> > Interesting problem,
> > Jason
> >
> >
> >
> >
> > From: mono-devel-list-admin at lists.ximian.com
> > [mailto:mono-devel-list-admin at lists.ximian.com] On Behalf Of
> > marcusmonaghan at f2s.com
> > Sent: Monday, April 04, 2005 10:37 AM
> > To: Mono Help
> > Subject: [Mono-devel-list] UDP 40%+ loss
> >
> > All,
> >
> > I am trying to re-creating a C program in C# mono (as there are more C#
> > developers in our company now) and it seems it's not behaving in the
> > expected
> > way.
> >
> > The program is very simple. It listens on a multicast group, receives data
> > and
> > writes it to a file. The problem is that I'm seeing roughly 40%+ missed
> > messages when compared to the C program. I ran the C# and C program on the
> > same
> > box. On another box I send out 100 messages in 10 second bursts. The C
> > program
> > receives all of them whereas the C# program only receive 60 on the first
> > burst
> > 50 on the second 55 on the third and so on.
> >
> > I then (just for a laugh) unthrottled the sending program. It sent 51,000
> > messages, all of which where received by the C program. The c# program
> > recorded
> > 7152.
> >
> > Anyone got any ideas? Possibly a bug?
> >
> > Regards,
> > Marcus
> > _______________________________________________
> > Mono-devel-list mailing list
> > Mono-devel-list at lists.ximian.com
> > http://lists.ximian.com/mailman/listinfo/mono-devel-list
> >
> >
> > _______________________________________________
> > Mono-devel-list mailing list
> > Mono-devel-list at lists.ximian.com
> > http://lists.ximian.com/mailman/listinfo/mono-devel-list
> >
>
>
> Regards,
> Marcus
> _______________________________________________
> Mono-devel-list mailing list
> Mono-devel-list at lists.ximian.com
> http://lists.ximian.com/mailman/listinfo/mono-devel-list
>
Regards,
Marcus
More information about the Mono-devel-list
mailing list