Skip to content

ci/wasm: report executed profile coverage - #229

Closed
cpunion wants to merge 2 commits into
codex/wasm-r3-runner-timeouts-20260905from
codex/wasm-r3-ci-reporting-20260905
Closed

cpunion wants to merge 2 commits into
codex/wasm-r3-runner-timeouts-20260905from
codex/wasm-r3-ci-reporting-20260905

Conversation

@cpunion

@cpunion cpunion commented Sep 4, 2026

Copy link
Copy Markdown
Owner

Stacked on #227.

This R3 slice makes the wasm runtime job report what it actually executed instead of treating one green shell step as sufficient coverage.

  • every EC32, EC64, WC32, wasm, and wasip1 acceptance case writes an append-only result record;
  • the job summary reports checks, completed package executions, expected failure paths, timeout paths, unexpected failures, skipped cases, and inapplicable cases per profile;
  • planned coverage is recorded only after a case reaches its acceptance boundary; an unexpected failure records zero completed package runs;
  • an EXIT trap publishes the partial summary even when set -e stops the suite;
  • a deterministic self-test covers retry-independent success/failure accounting and simulates a missing compiler to exercise the real suite failure trap and GitHub summary path.

Observed full-suite result:

  • 30 checks / 30 executed package runs
  • 7 expected failure paths
  • 1 expected timeout path
  • 0 unexpected failures, skips, or inapplicable cases

Validation:

  • bash -n and shellcheck for all changed scripts
  • dev/wasm_ci_report_test.sh
  • full dev/test_wasm_single_worker.sh with build cache disabled and Binaryen enabled

@cpunion
cpunion force-pushed the codex/wasm-r3-ci-reporting-20260905 branch from ae84c77 to 2e9289b Compare September 5, 2026 15:22
@cpunion
cpunion force-pushed the codex/wasm-r3-runner-timeouts-20260905 branch from 0d070ad to 0061e3a Compare September 5, 2026 15:22
@cpunion

cpunion commented Sep 5, 2026

Copy link
Copy Markdown
Owner Author

Rebased in dependency order onto LLVM 22 main bf3071fdd.

  • This PR's new head: 2e9289be3; current dependency: codex/wasm-r3-runner-timeouts-20260905.
  • R2's 28 patches and R2.1's six lifecycle patches remain equivalent. R3/R4 inherit the independent fork-runner repair ci: keep hosted runners available to forks #236; upstream continues using Qiniu, forks use hosted runners. R4's separate stdlib workflow now also installs LLVM 22.
  • LLVM 22 integration validation passed: 30 single-worker checks (7 expected failure paths, 1 timeout, no unexpected failures/skips); nine R3 stdlib package runs; five R4 lanes with six packages / 74 top-level tests per lane. Go-reference lanes remain Go output, not LLGo official-Go-ABI evidence.

To keep CI load bounded, this intermediate PR retains Format/Targets. Repeated functional matrices are canceled here and kept on the updated stack tip #235. Per-layer benchmark reruns are deferred, not replaced by the top PR's performance comparison. Canceled checks are not passing checks, and the integration result is not independent complete validation of every intermediate revision.

@cpunion

cpunion commented Sep 23, 2026

Copy link
Copy Markdown
Owner Author

Retiring this old R3 stack while its still-useful changes are extracted into a new PR based on the current Wasm direction. The branch remains available for reference.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant