[Mono-dev] Volatile fields don't enforce acquire - release semantics like Volatile.Read() and Volatile.Write()

Thu Jul 7 22:35:11 UTC 2016

I am attaching a zip file that contains the original test (program.cs) and
the more advanced test (VolatileTest.cs) that reproduces the error in both
iOS and Android. It also prints the total time needed to complete the
process.

The second test uses a circular buffer (RingBufferLockFree.cs) in order to
chive 50,000,000 repetitions without allocating such a big array. I have
added a macro that can switch the implementation between:
1) Volatile field
2) Volatile class
3) Full memory barrier

The last 2 work correctly. The first one prints errors.

By the way, the "full memory barrier" has better performance in iOS than
the Volatile class in this test. I would expect the Volatile Class to be
less expensive than the Full Barrier since it emits half fences.

I will filed the bug here:
https://bugzilla.xamarin.com/show_bug.cgi?id=42413

Thank for looking into this.

Best,

On Fri, Jul 8, 2016 at 12:12 AM, Rodrigo Kumpera <kumpera at gmail.com> wrote:

> A better test case? Awesome! Thanks for looking into this.
>
> We're aware of the issue so it's up to you on filing a bug.
>
>
>
> On Thu, Jul 7, 2016 at 12:53 PM, Petros Douvantzis <petrakeas at gmail.com>
> wrote:
>
>> Hello Rodrigo,
>>
>> Sure you can. However this specific test does not reproduce the error in
>> iOS most of the time. I will send another one that has more repetitions
>> (and wraps-around the array).
>>
>> I should file the bug, right?
>>
>> Best,
>>
>> On Thu, Jul 7, 2016 at 6:38 PM, Rodrigo Kumpera <kumpera at gmail.com>
>> wrote:
>>
>>> Hi Petros,
>>>
>>> It does look like a bug in our end. We do enforce ECMA's load-acquire,
>>> store-release semantics for volatiles.
>>>
>>> Can we integrate your test case into mono?
>>>
>>>
>>> --
>>> Rodrigo
>>>
>>> On Thu, Jul 7, 2016 at 11:05 AM, Petros Douvantzis <petrakeas at gmail.com>
>>> wrote:
>>>
>>>> Hello Miguel,
>>>>
>>>> The initial code does *not *have the field marked as volatile.
>>>> However, it may work in Net 2.0 because it has stronger memory guarantees
>>>> than the ECMA.
>>>>
>>>> So, the articles continues saying "*Making the instance variable
>>>> volatile can make it work*". So, *if* the field were volatile, it
>>>> would work in every ECMA implementation.
>>>>
>>>> Also, I tried using:
>>>> adb shell setprop debug.mono.env "'MONO_ENV_OPTIONS=-O=-intrins'"
>>>> with no difference in the outcome.
>>>>
>>>> Best,
>>>>
>>>> On Thu, Jul 7, 2016 at 5:59 PM, Miguel de Icaza <miguel at microsoft.com>
>>>> wrote:
>>>>
>>>>> Hello Petros,
>>>>>
>>>>>
>>>>>
>>>>> That blog post on double-check-locking explicitly states that without
>>>>> extra steps the pattern would not work.   Maybe I missed something?
>>>>>
>>>>>
>>>>>
>>>>> From that post:
>>>>>
>>>>> ·         Without any memory barriers, it's broken in the ECMA CLI
>>>>> specification too. It's possible that under the .NET 2.0 memory model
>>>>> (which is stronger than the ECMA spec) it's safe, but I'd rather not rely
>>>>> on those stronger semantics, especially if there's any doubt as to the
>>>>> safety. Making the instance variable volatile can make it work, as
>>>>> would explicit memory barrier calls, although in the latter case even
>>>>> experts can't agree exactly which barriers are required. I tend to try to
>>>>> avoid situations where experts don't agree what's right and what's wrong!
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> *From: *Petros Douvantzis <petrakeas at gmail.com>
>>>>> *Date: *Thursday, July 7, 2016 at 3:54 AM
>>>>> *To: *"mono-devel-list at lists.ximian.com" <
>>>>> mono-devel-list at lists.ximian.com>, Miguel de Icaza <
>>>>> miguel at microsoft.com>
>>>>> *Cc: *Rodrigo Kumpera <kumpera at gmail.com>
>>>>>
>>>>> *Subject: *Re: [Mono-dev] Volatile fields don't enforce acquire -
>>>>> release semantics like Volatile.Read() and Volatile.Write()
>>>>>
>>>>>
>>>>>
>>>>> Hello Miguel,
>>>>>
>>>>>
>>>>>
>>>>> I see your point. Even worse, it's a bit ambiguous of what guarantees
>>>>> does the volatile field make. For example in MSDN
>>>>> <https://na01.safelinks.protection.outlook.com/?url=https%3a%2f%2fmsdn.microsoft.com%2fen-us%2flibrary%2fx13ttww7.aspx&data=01%7c01%7cmiguel%40microsoft.com%7cbdcbcb26fc8744e9b67d08d3a63c01c8%7c72f988bf86f141af91ab2d7cd011db47%7c1&sdata=s02wpedE5%2fUhB9yawFzf3QlY51QePjK5rUj1c16Selk%3d>
>>>>>  nothing is mentioned about fences or memory re-orders. In that sense, Mono
>>>>> is correct. However, in ECMA 335 they mention acquire-release semantics
>>>>> such as the ones in the Volatile class you mentioned.
>>>>>
>>>>>
>>>>>
>>>>> One this to consider is that if the Volatile field does not have these
>>>>> semantics, then* lazy initialization* that relies on a volatile field
>>>>> and a lock ( double-check locking
>>>>> <https://na01.safelinks.protection.outlook.com/?url=http%3a%2f%2fcsharpindepth.com%2fArticles%2fGeneral%2fSingleton.aspx&data=01%7c01%7cmiguel%40microsoft.com%7cbdcbcb26fc8744e9b67d08d3a63c01c8%7c72f988bf86f141af91ab2d7cd011db47%7c1&sdata=yk%2bj4W775WX%2b82pOGWXA4xyURhfDzV1XSvJle2p3L2w%3d> )
>>>>> is *broken *in Mono for iOS and Android, because there is a chance
>>>>> that a half created object is viewed by another thread. The way the
>>>>> volatile field is supposed to help, is explained in this post
>>>>> <https://na01.safelinks.protection.outlook.com/?url=https%3a%2f%2fmsdn.microsoft.com%2fen-us%2fmagazine%2fjj883956.aspx&data=01%7c01%7cmiguel%40microsoft.com%7cbdcbcb26fc8744e9b67d08d3a63c01c8%7c72f988bf86f141af91ab2d7cd011db47%7c1&sdata=Edq89e1H1sGysBYfBQFrb9WUTXczZe0ZlQfh1FQJvJc%3d>
>>>>> .
>>>>>
>>>>>
>>>>>
>>>>> The only way to make this work right now is using the Volatile class,
>>>>> but most probably someone would expect the volatile field to work.
>>>>>
>>>>>
>>>>>
>>>>> Best,
>>>>>
>>>>>
>>>>>
>>>>> On Wed, Jul 6, 2016 at 9:24 PM, Miguel de Icaza <miguel at microsoft.com>
>>>>> wrote:
>>>>>
>>>>> Hello,
>>>>>
>>>>>
>>>>>
>>>>> I am not convinced that this is a bug worth fixing.
>>>>>
>>>>>
>>>>>
>>>>> I think this requires some thinking.  While this might have been the
>>>>> intended visible behavior from C#, this predates the extensive use of C#
>>>>> beyond the x86 platform.   I believe this is why the Volatile APIs were
>>>>> introduced.
>>>>>
>>>>>
>>>>>
>>>>> Consder the documentation for it:
>>>>>
>>>>>
>>>>>
>>>>> https://msdn.microsoft.com/en-us/library/gg712971(v=vs.110).aspx
>>>>> <https://na01.safelinks.protection.outlook.com/?url=https%3a%2f%2fmsdn.microsoft.com%2fen-us%2flibrary%2fgg712971(v%3dvs.110).aspx&data=01%7c01%7cmiguel%40microsoft.com%7cbdcbcb26fc8744e9b67d08d3a63c01c8%7c72f988bf86f141af91ab2d7cd011db47%7c1&sdata=U4RIuTFZa6oqRlI3uSyP2q8by90V0mpKDfgmnqQNUZI%3d>
>>>>>
>>>>>
>>>>>
>>>>> If the language/runtime already provided this support, there would be
>>>>> no need for these volatile methods in the first place.
>>>>>
>>>>>
>>>>>
>>>>> Miguel.
>>>>>
>>>>>
>>>>>
>>>>> *From: *<mono-devel-list-bounces at lists.ximian.com> on behalf of
>>>>> Rodrigo Kumpera <kumpera at gmail.com>
>>>>> *Date: *Wednesday, July 6, 2016 at 1:27 PM
>>>>> *To: *petrakeas <petrakeas at gmail.com>
>>>>> *Cc: *"mono-devel-list at lists.ximian.com" <
>>>>> mono-devel-list at lists.ximian.com>
>>>>> *Subject: *Re: [Mono-dev] Volatile fields don't enforce acquire -
>>>>> release semantics like Volatile.Read() and Volatile.Write()
>>>>>
>>>>>
>>>>>
>>>>> Yes, it looks like a bug.
>>>>>
>>>>>
>>>>>
>>>>> On Wed, Jul 6, 2016 at 11:13 AM, petrakeas <petrakeas at gmail.com>
>>>>> wrote:
>>>>>
>>>>> According to C#  specification
>>>>> <https://msdn.microsoft.com/en-us/library/ms228593.aspx
>>>>> <https://na01.safelinks.protection.outlook.com/?url=https%3a%2f%2fmsdn.microsoft.com%2fen-us%2flibrary%2fms228593.aspx&data=01%7c01%7cmiguel%40microsoft.com%7cf3c960accdeb43d8500208d3a5c2d9ae%7c72f988bf86f141af91ab2d7cd011db47%7c1&sdata=66AJc2tU2gcy4vutTkC%2b4bPl3MxAnAiXd4POGNZ3ivA%3d>>
>>>>> :
>>>>>
>>>>> •       A read of a volatile field is called a volatile read. A
>>>>> volatile read has
>>>>> “acquire semantics”; that is, it is guaranteed to occur prior to any
>>>>> references to memory that occur after it in the instruction sequence.
>>>>> •       A write of a volatile field is called a volatile write. A
>>>>> volatile write
>>>>> has “release semantics”; that is, it is guaranteed to happen after any
>>>>> memory references prior to the write instruction in the instruction
>>>>> sequence.
>>>>>
>>>>> The spec presents  an example
>>>>> <https://msdn.microsoft.com/en-us/library/aa645755(v=vs.71).aspx
>>>>> <https://na01.safelinks.protection.outlook.com/?url=https%3a%2f%2fmsdn.microsoft.com%2fen-us%2flibrary%2faa645755(v%3dvs.71).aspx&data=01%7c01%7cmiguel%40microsoft.com%7cf3c960accdeb43d8500208d3a5c2d9ae%7c72f988bf86f141af91ab2d7cd011db47%7c1&sdata=cFpmsRLE5a248vj3svbpsmOBouE%2bOxE%2fwDMWjd0YkhU%3d>>
>>>>>  where
>>>>> one thread writes "data" on a non volatile variable and "publishes" the
>>>>> result by writing on a volatile variable that acts as a flag. The other
>>>>> thread checks the volatile flag and if set, it accesses the
>>>>> non-volatile
>>>>> variable that is now *guaranteed* to contain the data.
>>>>>
>>>>> It seems that Mono 4.4 (the one used in Xamarin) does not enforce these
>>>>> semantics or in other words does not prevent memory re-ordering in
>>>>> Android
>>>>> and iOS that have relaxed memory models due to their CPU.
>>>>>
>>>>> I have created an a test that reproduces the problem in iOS and Android
>>>>> Program.cs <http://mono.1490590.n4.nabble.com/file/n4668111/Program.cs
>>>>> <https://na01.safelinks.protection.outlook.com/?url=http%3a%2f%2fmono.1490590.n4.nabble.com%2ffile%2fn4668111%2fProgram.cs&data=01%7c01%7cmiguel%40microsoft.com%7cf3c960accdeb43d8500208d3a5c2d9ae%7c72f988bf86f141af91ab2d7cd011db47%7c1&sdata=H7V6%2bpq4jV8Kw7QdgZMVJRav%2b1XovSCuIY3PgRFaJrk%3d>>
>>>>> .
>>>>>
>>>>> If the access to the volatile field is replaced by Volatile.Read() and
>>>>> Volatile.Write(), then no-problems occur. It seems that
>>>>> Volatile.Read() and
>>>>> Volatile.Write() implement half fences in Mono, but the volatile
>>>>> keyword
>>>>> does not.
>>>>>
>>>>> Is this a bug?
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> View this message in context:
>>>>> http://mono.1490590.n4.nabble.com/Volatile-fields-don-t-enforce-acquire-release-semantics-like-Volatile-Read-and-Volatile-Write-tp4668111.html
>>>>> <https://na01.safelinks.protection.outlook.com/?url=http%3a%2f%2fmono.1490590.n4.nabble.com%2fVolatile-fields-don-t-enforce-acquire-release-semantics-like-Volatile-Read-and-Volatile-Write-tp4668111.html&data=01%7c01%7cmiguel%40microsoft.com%7cf3c960accdeb43d8500208d3a5c2d9ae%7c72f988bf86f141af91ab2d7cd011db47%7c1&sdata=sqJVi9saxf7EEGpn6Wpf%2bFEeZX5yCpzK4%2b38x670OEw%3d>
>>>>> Sent from the Mono - Dev mailing list archive at Nabble.com.
>>>>> _______________________________________________
>>>>> Mono-devel-list mailing list
>>>>> Mono-devel-list at lists.ximian.com
>>>>> http://lists.ximian.com/mailman/listinfo/mono-devel-list
>>>>> <https://na01.safelinks.protection.outlook.com/?url=http%3a%2f%2flists.ximian.com%2fmailman%2flistinfo%2fmono-devel-list&data=01%7c01%7cmiguel%40microsoft.com%7cf3c960accdeb43d8500208d3a5c2d9ae%7c72f988bf86f141af91ab2d7cd011db47%7c1&sdata=Sb1mXUpzvfBCP0RJh%2bB2orCGoBH3eV8Z8CY10t1NbC0%3d>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>>
>>>>> Petros Douvantzis
>>>>>
>>>>> Co-founder
>>>>>
>>>>> Horizon Video Technologies
>>>>>
>>>>> *horizon.camera
>>>>> <https://na01.safelinks.protection.outlook.com/?url=http%3a%2f%2fhorizon.camera&data=01%7c01%7cmiguel%40microsoft.com%7cbdcbcb26fc8744e9b67d08d3a63c01c8%7c72f988bf86f141af91ab2d7cd011db47%7c1&sdata=k%2bCLVnzGEb%2fX6zRRD4SfroHMqrvOcV7WJaGEOt2KYqM%3d>*
>>>>>
>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>>
>>>> Petros Douvantzis
>>>>
>>>> Co-founder
>>>>
>>>> Horizon Video Technologies
>>>>
>>>> horizon.camera
>>>>
>>>>
>>>
>>
>>
>> --
>>
>> Petros Douvantzis
>>
>> Co-founder
>>
>> Horizon Video Technologies
>>
>> horizon.camera
>>
>>
>

-- 

Petros Douvantzis

Co-founder

Horizon Video Technologies

horizon.camera
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ximian.com/pipermail/mono-devel-list/attachments/20160708/60d491ba/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: volatile_field_tests.zip
Type: application/zip
Size: 4991 bytes
Desc: not available
URL: <http://lists.ximian.com/pipermail/mono-devel-list/attachments/20160708/60d491ba/attachment-0001.zip>