arm_linux: Add missing memory barrier to atomic_cmpxchg

[taiki-e]

This PR contains three commits. The main fix is the last one, and the first two address minor issues such as documentation that I noticed when writing the main fix. I would suggest reviewing by commit, but I can split them into separate PRs if necessary.

---

Currently, atomic_cmpxchg returns the value obtained from the first relaxed load as-is without calling the kuser helper when comparison fails:

compiler-builtins/compiler-builtins/src/sync/arm_linux.rs

Lines 130 to 134 in 5af7a65

	let curval_aligned = unsafe { atomic_load_aligned::<T>(aligned_ptr) };
	let curval = extract_aligned(curval_aligned, shift, mask);
	if curval != oldval {
	return curval;
	}

This is equivalent to compare_exchange(SeqCst, Relaxed), but the __sync builtins are expected to have SeqCst semantics in both the success and failure paths, so this is unsound.

This PR fixes this by adding a memory barrier to failure path:

compiler-builtins/compiler-builtins/src/sync/arm_linux.rs

Lines 134 to 141 in c0cdefb

	let curval_aligned = unsafe { atomic_load_aligned::<T>(aligned_ptr) };
	let curval = extract_aligned(curval_aligned, shift, mask);
	if curval != oldval {
	// __sync builtins must have SeqCst semantics. So, failure path, which returns early
	// without calling __kuser_cmpxchg, must emits a memory barrier.
	__kuser_memory_barrier();
	return curval;
	}

Context and Disclosure: This bug had been observed for several years as a rare stress test failure in portable-atomic's CI when testing pre-v6 ARM Linux on ARM hardware (https://gist.github.com/taiki-e/31962f6efe8cc9bc06aa6869bbceb656). In my past investigations, I suspected the bug was with the algorithm used in portable-atomic, so I was unable to identify the cause, but I recently identified the cause by investigating a wider scope with the help of LLMs. (All changes in this PR were written by me.)

[tgross35]

This looks reasonable to me, couple questions for my understanding:

1. This only shows up when multiple atomics are involved, right (hence seqcst) ? I think the single-atomic pathing was fine here but can't convince myself of it.
2. Is there a C implementation somewhere that we can reference?

Also I'm planning to add the middle bit of your PR message into the third commit's description so there's some context in the history, feel free to do this yourself if you prefer.

View changes since this review

[taiki-e]

1. This only shows up when multiple atomics are involved, right (hence seqcst) ? I think the single-atomic pathing was fine here but can't convince myself of it.

ARM is an architecture with a weak memory model, so the absence of a fence here affects SeqCst, AcqRel, and Acquire.

Actually, the code that encountered this bug that led to this investigation used Acquire load (which, in pre-v6 Arm, this is implemented in CAS(0, 0)).

2. Is there a C implementation somewhere that we can reference?

It appears that libgcc has the same bug: https://github.com/gcc-mirror/gcc/blob/32657f29f918712ad7110bd54d8ebb3bf6b0a2a8/libgcc/config/arm/linux-atomic.c#L209

@Amanieu, who added these to compiler-builtins in #115, might know more about this.

[Amanieu]

The implementation here should match libgcc, I would be more comfortable if this was first reported upstream to get a confirmation.