[atomic-primops] Please clarify whether atomic operations are barriers or not

[jberryman]

It would be nice to know whether or not casIORef, fetchAddByteArray, etc. represent memory barriers, or not, at the compiler level. In other words, may GHC reorder stores or loads across these operations, regardless of whether the underlying primop is a barrier on a particular architecture? Or like atomicModifyIORef do they guarantee some ordering?

There's mention of how the read operations don't imply any barriers, which suggests that maybe the atomic ops do. If this isn't defined in GHC then it would be nice to clarify that there, obviously. Thanks!

[jberryman]

Also, this commit seems to suggest that at least GHC's cas might be intended to be a barrier.

[rrnewton]

Good question,

@tibbe actually fixed this up so that GHC will not reorder across them. So I think we're all good on this front.

[jberryman]

Thanks for the response. I'm not sure what that means though. When GHC 7.10 is out then that will be the behavior of this library under 7.10 (but presumably different for 7.6?)? Or that's the behavior now?

[rrnewton]

Johan made the changes around the time of this commit I think (though I don't know the exact commit):

ghc/ghc@4ee4ab0

The timing was such that it could possibly have been in GHC 7.8.3, but I just checked out the ghc-7.8 branch of the ghc repo and I don't think Johan's changes are in there. So I think you're right and it missed 7.8.3.

I was unaware of this issue before Johan fixed it (I didn't know GHC would do those kinds of reorderings on primitive World-modifying ops). The behavior before that fix is well... I suppose GHC could break our barriers, Johan (@tibbe) do we have a confirmed case of that happening?

[tibbe]

GHC shouldn't move read/writes across C calls of callish machops (i.e. fat primops). This is guaranteed at the moment I will likely stay that way. Other than that GHC does limited movement of reads/writes in the Cmm optimizer.

[rrnewton]

Great, ok -- so "at the moment" includes GHC 7.8 then?

[jberryman]

FWIW I revisited ezyang's post here which caused me to re-assess how I was using these functions in a few places. I just want to be able to use this as a library, without worrying about what "fat primops" are, how these things change in GHC over time, etc. I don't really care whether they're full barriers or whether I should be required to add explicit barriers, as long as atomic-primops can keep the behavior consistent and the behavior is well-documented.

If we can figure this out here, I'd be happy to submit a pull request for the documentation changes, or help in any other ways.

[rrnewton]

Yes, it would be good to update the documentation of this package. There's also some info here on the GHC wiki:

https://ghc.haskell.org/trac/ghc/wiki/AtomicPrimops

And it was discussed on the ghc-devs thread here.

The intent is for each atomic operation to imply a full barrier, because it only currently has the goal of supporting x86, and because it follows the GCC atomic builtins. That intent should be clearly stated in the docs, I agree. And any failure to provide a full barrier at each atomic op is a bug in either this lib or GHC.

Further, eventually it would be great to have an overhaul and support a more sophisticated model like LLVM or C++11. And I believe Johan and others have been amenable to the idea of coopting an existing design as much as possible -- but in any case I think that it will require quite a bit more compiler support.

More concretely, if anyone wants to use this package on ARM, we need to figure out how that's going to work. We could make sure the GHC primops on those architectures emit full barriers, even if it is overkill...

[tibbe]

The goal for the primops is to provide the "sequential consistency for data-race-free programs" semantics which all languages (i.e. C, C++, and Java) seem to have settled on. It would be nice to also offer the more relaxed semantics that these languages offer.

Implementation wise this might imply various kinds of memory barriers. For example, on x86 you only need to put a barrier on either the reads or the write (and we do this on the write in GHC) as the guarantees on these operations are quite strong.

[jberryman]

The intent the primops is for each atomic operation to imply a full
barrier, because it only currently has the goal of supporting x86

Oh wait, so atomic-primops doesn't support non-x86(32 and 64 bit) at all
then? I thought it was supposed to work cross-platform.
On Nov 7, 2014 6:47 AM, "Ryan Newton" notifications@github.com wrote:

Yes, it would be good to update the documentation of this package. There's
also some info here on the GHC wiki:

https://ghc.haskell.org/trac/ghc/wiki/AtomicPrimops

And it was discussed on the ghc-devs thread here
https://www.haskell.org/pipermail/ghc-devs/2014-May/004882.html.

The intent the primops is for each atomic operation to imply a full
barrier, because it only currently has the goal of supporting x86, and
because it follows the GCC atomic builtins
https://gcc.gnu.org/onlinedocs/gcc-4.1.2/gcc/Atomic-Builtins.html. That
intent should be clearly stated in the docs, I agree. And any failure to
provide a full barrier at each atomic op is a bug in either this lib or GHC.

Further, eventually it would be great to have an overhaul and support a
more sophisticated model like LLVM or C++11. And I believe Johan and others
have been amenable to the idea of coopting an existing design as much as
possible -- but in any case I think that it will require quite a bit more
compiler support.

More concretely, if anyone wants to use this package on ARM, we need to
figure out how that's going to work. We could make sure the GHC primops on
those architectures emit full barriers, even if it is overkill...

—
Reply to this email directly or view it on GitHub
#39 (comment)
.

[rrnewton]

Well, if we have the sequential-consistency-for-racefree programs actually working right now with GHC on ARM backends then we're good to go in terms of running "atomic-primops"-using programs on ARM and getting the same answers as on x86.

What I should have said is that there has been no attempt yet to provide something can make good use of ARM's more relaxed mem model. If our programs run on ARM and compute the same answers, then we're in a pretty good state, but it might be at the cost of doing more synchronization than we strictly need to.

[jberryman]

Oh I see, thanks. Well I guess for now I'll assume that the atomic ops are
full barriers (both for compiler reordering and at the hardware level) on
all architectures, on all supported GHCs. And blame you guys if that turns
out to be wrong :-P Does that sound reasonable? Thanks for answering all
these questions
On Nov 7, 2014 12:12 PM, "Ryan Newton" notifications@github.com wrote:

Well, if we have the sequential-consistency-for-racefree programs actually
working right now with GHC on ARM backends then we're good to go.

What I should have said is that there has been no attempt yet to provide
something can make good use of ARM's more relaxed mem model. If our
programs run on ARM and compute the same answers, then we're in a pretty
good state, but it might be at the cost of doing more synchronization than
we strictly need to.

—
Reply to this email directly or view it on GitHub
#39 (comment)
.