[Mono-dev] ConcurrentQueue<T> issues (repros)

[Greg Young]

Here are some tests that show some of the failure modes of concurrent
queue (all work in CLR impl)

On my machine (8 cores within 1 second of running for all failures).
The worst is the last one where anything larger than a reference gives
partial reads.

in gist if you prefer: https://gist.github.com/3755979

Unhandled Exception:
System.NullReferenceException: Object reference not set to an instance
of an object
  at System.Collections.Concurrent.ConcurrentQueue`1[T].TryDequeue
(System.Collections.Concurrent.T& result) [0x00000] in <filename
unknown>:0
  at ConsoleApplication1.Program+<TestMonoConcurrentQueueReference>c__AnonStorey1.<>m__0
(System.Object x) [0x00000] in <filename unknown>:0
  at System.Threading.Thread.StartInternal () [0x00000] in <filename unknown>:0
[ERROR] FATAL UNHANDLED EXCEPTION: System.NullReferenceException:
Object reference not set to an instance of an object
  at System.Collections.Concurrent.ConcurrentQueue`1[T].TryDequeue
(System.Collections.Concurrent.T& result) [0x00000] in <filename
unknown>:0
  at ConsoleApplication1.Program+<TestMonoConcurrentQueueReference>c__AnonStorey1.<>m__0
(System.Object x) [0x00000] in <filename unknown>:0
  at System.Threading.Thread.StartInternal () [0x00000] in <filename unknown>:0



        private static void TestMonoConcurrentQueueReference()
        {
            var queue = new ConcurrentQueue<object>();
            var waits = new List<AutoResetEvent>();
            for (int i = 0; i < 5; i++)
            {
                int q = i;
                waits.Add(new AutoResetEvent(false));
                var t = new Thread(x =>
                {
                    for (int j = 0; j < 100000000; j++)
                    {
                        if (j % 1000000 == 0) Console.Write(".");
                        queue.Enqueue(new object());
                        object item;
                        if (queue.TryDequeue(out item))
                        {

                        }
                    }
                    waits[q].Set();
                });
                t.Start();
            }
            Console.WriteLine("waiting.");
            waits.ForEach(x => x.WaitOne());
            Console.WriteLine("done.");

        }


I end up in the else {} on try dequeue here don't think I should ever
be allowed to (and dont end up there in MS impl)

        struct TestStruct
        {
            public long X;
            public long Y;

            public TestStruct(long x, long y) : this()
            {
                X = x;
                Y = y;
            }
        }

        private static void TestMonoConcurrentQueueBiggerThanReference()
        {
            var queue = new ConcurrentQueue<TestStruct>();
            var waits = new List<AutoResetEvent>();
            for(int i=0;i<5;i++)
            {
                int q = i;
                waits.Add(new AutoResetEvent(false));
                var t = new Thread(x =>
                                       {
                                           for(int j=0;j<100000000;j++)
                                           {
                                               if(j% 1000000 == 0)
Console.Write(".");
                                               queue.Enqueue(new
TestStruct(0x11223344, 0x99887766));
                                               TestStruct item;
                                               if(queue.TryDequeue(out item))
                                               {
                                                   if(item.X !=
0x11223344) throw new Exception("bad x");
                                                   if(item.Y !=
0x99887766) throw new Exception("bad y");
                                               } else
                                               {
                                                   throw new
Exception("unable to read."); <~~~~ should never hit this.
                                               }
                                           }
                                           waits[q].Set();
                                       });
                t.Start();
            }
            Console.WriteLine("waiting.");
            waits.ForEach(x => x.WaitOne());
            Console.WriteLine("done.");

        }


For good measure lets take out the exception in the last about not
being able to read when it should be able to (eg comment out the
throw)

Unhandled Exception:
System.Exception: bad y
  at ConsoleApplication1.Program+<TestMonoConcurrentQueueBiggerThanReference>c__AnonStorey3.<>m__2
(System.Object x) [0x00000] in <filename unknown>:0
  at System.Threading.Thread.StartInternal () [0x00000] in <filename unknown>:0

Unhandled Exception:
System.NullReferenceException: Object reference not set to an instance
of an object
  at System.Collections.Concurrent.ConcurrentQueue`1[ConsoleApplication1.Program+TestStruct].TryDequeue
(ConsoleApplication1.TestStruct& result) [0x00000] in <filename
unknown>:0
  at ConsoleApplication1.Program+<TestMonoConcurrentQueueBiggerThanReference>c__AnonStorey3.<>m__2
(System.Object x) [0x00000] in <filename unknown>:0
  at System.Threading.Thread.StartInternal () [0x00000] in <filename unknown>:0

Unhandled Exception:
System.NullReferenceException: Object reference not set to an instance
of an object
  at System.Collections.Concurrent.ConcurrentQueue`1[ConsoleApplication1.Program+TestStruct].TryDequeue
(ConsoleApplication1.TestStruct& result) [0x00000] in <filename
unknown>:0
  at ConsoleApplication1.Program+<TestMonoConcurrentQueueBiggerThanReference>c__AnonStorey3.<>m__2
(System.Object x) [0x00000] in <filename unknown>:0
  at System.Threading.Thread.StartInternal () [0x00000] in <filename unknown>:0

Unhandled Exception:
System.NullReferenceException: Object reference not set to an instance
of an object
  at System.Collections.Concurrent.ConcurrentQueue`1[ConsoleApplication1.Program+TestStruct].TryDequeue
(ConsoleApplication1.TestStruct& result) [0x00000] in <filename
unknown>:0
  at ConsoleApplication1.Program+<TestMonoConcurrentQueueBiggerThanReference>c__AnonStorey3.<>m__2
(System.Object x) [0x00000] in <filename unknown>:0
  at System.Threading.Thread.StartInternal () [0x00000] in <filename unknown>:0

Unhandled Exception:
System.NullReferenceException: Object reference not set to an instance
of an object
  at System.Collections.Concurrent.ConcurrentQueue`1[ConsoleApplication1.Program+TestStruct].TryDequeue
(ConsoleApplication1.TestStruct& result) [0x00000] in <filename
unknown>:0
  at ConsoleApplication1.Program+<TestMonoConcurrentQueueBiggerThanReference>c__AnonStorey3.<>m__2
(System.Object x) [0x00000] in <filename unknown>:0
  at System.Threading.Thread.StartInternal () [0x00000] in <filename unknown>:0
[ERROR] FATAL UNHANDLED EXCEPTION: System.Exception: bad y
  at ConsoleApplication1.Program+<TestMonoConcurrentQueueBiggerThanReference>c__AnonStorey3.<>m__2
(System.Object x) [0x00000] in <filename unknown>:0
  at System.Threading.Thread.StartInternal () [0x00000] in <filename unknown>:0
[ERROR] FATAL UNHANDLED EXCEPTION: System.NullReferenceException:
Object reference not set to an instance of an object
  at System.Collections.Concurrent.ConcurrentQueue`1[ConsoleApplication1.Program+TestStruct].TryDequeue
(ConsoleApplication1.TestStruct& result) [0x00000] in <filename
unknown>:0
  at ConsoleApplication1.Program+<TestMonoConcurrentQueueBiggerThanReference>c__AnonStorey3.<>m__2
(System.Object x) [0x00000] in <filename unknown>:0
  at System.Threading.Thread.StartInternal () [0x00000] in <filename unknown>:0
[ERROR] FATAL UNHANDLED EXCEPTION: System.NullReferenceException:
Object reference not set to an instance of an object
  at System.Collections.Concurrent.ConcurrentQueue`1[ConsoleApplication1.Program+TestStruct].TryDequeue
(ConsoleApplication1.TestStruct& result) [0x00000] in <filename
unknown>:0
  at ConsoleApplication1.Program+<TestMonoConcurrentQueueBiggerThanReference>c__AnonStorey3.<>m__2
(System.Object x) [0x00000] in <filename unknown>:0
  at System.Threading.Thread.StartInternal () [0x00000] in <filename unknown>:0


    private static void TestMonoConcurrentQueueBiggerThanReference()
        {
            var queue = new ConcurrentQueue<TestStruct>();
            var waits = new List<AutoResetEvent>();
            for(int i=0;i<5;i++)
            {
                int q = i;
                waits.Add(new AutoResetEvent(false));
                var t = new Thread(x =>
                                       {
                                           for(int j=0;j<100000000;j++)
                                           {
                                               if(j% 1000000 == 0)
Console.Write(".");
                                               queue.Enqueue(new
TestStruct(0x11223344, 0x99887766));
                                               TestStruct item;
                                               if(queue.TryDequeue(out item))
                                               {
                                                   if(item.X !=
0x11223344) throw new Exception("bad x");
                                                   if(item.Y !=
0x99887766) throw new Exception("bad y");
                                               } else
                                               {
                                                   //throw new
Exception("unable to read."); <~~~~ should never hit this.
                                               }
                                           }
                                           waits[q].Set();
                                       });
                t.Start();
            }
            Console.WriteLine("waiting.");
            waits.ForEach(x => x.WaitOne());
            Console.WriteLine("done.");

        }

-- 
Le doute n'est pas une condition agréable, mais la certitude est absurde.