Skip to content

test(gates): check comptime C equivalence - #1705

Open
vicmcorrea wants to merge 2 commits into
bendlang:mainfrom
vicmcorrea:test/comptime-c-equivalence
Open

vicmcorrea wants to merge 2 commits into
bendlang:mainfrom
vicmcorrea:test/comptime-c-equivalence

Conversation

@vicmcorrea

@vicmcorrea vicmcorrea commented Oct 18, 2026 •

Copy link
Copy Markdown
Contributor

Problem and fix

The comptime twins print the same value, but their documented same-C claim was untested, as noted in #1314. Compare their emitted C after replacing declared FID names with their numeric IDs, preserving the identifier prefix and every other byte.

Load hand and tmpl explicitly, once each, in comptime_run, before any state-fixture work. A mismatch reports the first differing line number and both normalized C lines, including an end-of-file marker when needed.

Blast radius

gates/test.ts adds the local comparison and updates its comments; tests/comptime/hand.bend only corrects its stale report/benchmark comment. The existing four-fixture state sequence, raw output hashes and probe-count checks remain intact. The twins stay outside those repeated compiles and term-normalization steps. Compiler/runtime code, fixture behavior, APIs and token caps are unchanged.

Regression evidence

The original local state check accepts changing the template from ~U32.inc to ~(x => U32.sub(U32.add(x, 2), 1)). The revised check rejects its different C and shows both statements at the first differing line. Hand, template and mutant still print 50005000 in source execution, emitted JavaScript under Bun and native C. Unmodified twins and a consistent definition rename pass.

Checks

  • Repository gate: 54/54 locally with CI-pinned ttok 0.3; GitHub repo-gate and git diff --check pass.
  • Actual local gate functions pass healthy/rename controls and reject the same-output mutant. The revision also passes on current main f5cee564 and rejects that mutant there.
  • All 48 prior raw C/JS state observations and post-emission probe counts match the original gate. Tracing confirms each twin loads and validates once, before all state-fixture work.
  • Focused comparison controls cover first-line diagnostics, EOF, final-newline/whitespace differences and prefix-preserving FID renames.
  • Targeted The root file gets its own namespace, so sugar's bare names can only mean Base's #1585 (22764c3) loader/compiler probe passes with its external namespace/main-alias adaptation and rejects the mutant.
  • Project typecheck has the same 11 pre-existing diagnostics as the base, with identical normalized logs and none in this gate.

Validation limits

Taelin's sign-off on gate placement is still pending, as the review requests; this revision does not claim that approval. Local checks used macOS arm64 and Bun 1.3.11. Full cluster, GPU and performance gates were not rerun for this revision. This checks emitted C text, not machine-code identity or runtime performance. The #1585 probe covers only the adapted parser/compiler dependencies; its existing safe.ts merge conflict prevents a full-branch integration claim. Node runtime checks were not rerun.

@Lorenzobattistela Lorenzobattistela self-assigned this Oct 19, 2026

@Lorenzobattistela Lorenzobattistela left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, this closes the gap left in #1314. Merged onto main f5cee56: repo, test (1670/1670), perf, ping and safe are all green, and the new check passes (hand and tmpl C are byte-identical after the FID rename).

A few changes before this can go in:

  1. Load the twins by name, not by position. books.slice(-2) and books.slice(0, files.length) depend on the twins being the last two entries of the inline array. Appending or reordering a fixture would silently compare the wrong pair. Load hand and tmpl separately (one copy each) instead of through the copies loop.

  2. Drop the second copies. Each twin is loaded and validated twice, but only the first copy is used.

  3. Compare the twins in a clean spot. Right now they compile after the 44 state steps in the same process, so any compiler state left over from the earlier compiles (the #976 class state_run exists to catch) can make the comparison fail falsely or hide a real difference. Compile them before the state sequence, or in their own function.

  4. Make a failure actionable. On a mismatch, report the first differing line (or line number plus both lines) of the normalized C instead of two hashes.

  5. Update the docs. The file header and the state_run comment still describe only the four state fixtures, and tests/comptime/hand.bend says "the report diffs the two programs' C". This check is now that report.

Also, this changes gates/, which needs Taelin's sign-off. He may prefer the check somewhere other than the test gate.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants