Skip to content

test: fix address-dependent pointer tests and macOS toolchain noise - #242

Merged
vtjnash merged 1 commit into
masterfrom
fix-flaky-tests
Aug 11, 2026
Merged

test: fix address-dependent pointer tests and macOS toolchain noise#242
vtjnash merged 1 commit into
masterfrom
fix-flaky-tests

Conversation

@vtjnash

@vtjnash vtjnash commented Aug 10, 2026

Copy link
Copy Markdown
Member

test_char_sized_ptr_math_{incr,decr} truncate two addresses four bytes apart to unsigned char and subtract them, expecting a difference of 4. That only holds when the subtraction does not borrow out of the low byte, so the tests fail whenever the stack happens to place a at an address whose low byte is 0x00 (decr) or 0xfc (incr) -- roughly a 1 in 64 chance, since a is 4-byte aligned. This is what failed the Windows job on #240:

FAILED test_consistent_return_value_c[test_char_sized_ptr_math_decr--O1]
- assert 4294967046 == 6

Truncating the difference back to unsigned char makes the result 4 for every address while still exercising the same char-sized pointer math at -O0. At -O1 and above the optimizer now folds main() to return 6, which is itself a proof that the result no longer depends on the address.

Separately, check_no_output() treats any stderr as a failure, including diagnostics that describe the toolchain rather than the code under test. When the macOS SDK and the Homebrew bottles are built for different OS versions, every compile emits

clang: warning: overriding deployment version from '16.0' to '26.0'

which failed all 540 tests on #237 despite an exit code of 0. Filter that class of environmental warning out before deciding whether the process misbehaved; real diagnostics are unaffected.

Assisted-by: Claude Opus 5 (1M context) noreply@anthropic.com

test_char_sized_ptr_math_{incr,decr} truncate two addresses four bytes
apart to unsigned char and subtract them, expecting a difference of 4.
That only holds when the subtraction does not borrow out of the low
byte, so the tests fail whenever the stack happens to place `a` at an
address whose low byte is 0x00 (decr) or 0xfc (incr) -- roughly a 1 in
64 chance, since `a` is 4-byte aligned. This is what failed the Windows
job on #240:

  FAILED test_consistent_return_value_c[test_char_sized_ptr_math_decr--O1]
    - assert 4294967046 == 6

Truncating the difference back to unsigned char makes the result 4 for
every address while still exercising the same char-sized pointer math at
-O0. At -O1 and above the optimizer now folds main() to `return 6`,
which is itself a proof that the result no longer depends on the address.

Separately, check_no_output() treats any stderr as a failure, including
diagnostics that describe the toolchain rather than the code under test.
When the macOS SDK and the Homebrew bottles are built for different OS
versions, every compile emits

  clang: warning: overriding deployment version from '16.0' to '26.0'

which failed all 540 tests on #237 despite an exit code of 0. Filter
that class of environmental warning out before deciding whether the
process misbehaved; real diagnostics are unaffected.

Assisted-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
@vtjnash
vtjnash merged commit a13fcf3 into master Aug 11, 2026
3 checks passed
@vtjnash
vtjnash deleted the fix-flaky-tests branch August 11, 2026 06:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant