Skip to content

ci: route suites and execute archived tests in parallel - #6521

Merged
M4n5ter merged 10 commits into
rust-previewfrom
ci/reliable-suite-plan
Nov 1, 2026
Merged

M4n5ter merged 10 commits into
rust-previewfrom
ci/reliable-suite-plan

Conversation

@M4n5ter

@M4n5ter M4n5ter commented Nov 1, 2026 •

Copy link
Copy Markdown
Member

Summary

Replace the serial Rust CI job with a shared suite plan and independent workers. PR changes select affected packages through Cargo reverse dependencies across owning workspaces, plus declared product/SDK/embedded-resource inputs. Shared or unclassified changes run the full maintained set; push and manual runs also run full validation. This closes the Computer Use dependency-path and dispatch-only NSH/GPUI coverage gaps.

Keep existing workspace/features build recipes, including native CLI build/smoke checks. Build root tests once, archive the selected binaries and runtime product assets, and execute native nextest partitions on fresh runners using the producer's exact nextest version and current-run artifact IDs. The final test check requires every planned job/matrix to succeed. Local just recipes use the same executor.

Root workspace workers share one compiler-cache baseline with a single publisher. Compiler snapshots use toolchain, platform, owner and lockfile keys rather than a new snapshot per source revision. Source caches use Cargo's recommended directories. Disable mbx OUT_DIR sharing so restored Cargo artifacts retain their generated-source inputs. Development profiles remain unchanged.

Move notices verification to the product worker because it now builds the actual bundle, and route distributed-root dependency/license changes there. Refresh the reviewed CUA identities and source revision URLs after #6522; license bodies are unchanged. Adjust the existing GPUI asset-isolation fixture to assert its error event instead of an outdated internal message.

Verification

  • Eight planner/gate behavior tests pass, including transitive native checks, cross-workspace package identity, runtime inputs, ordinary Rust routing and failed/skipped/cancelled jobs.
  • Actual locked metadata from all four workspaces was used to verify affected plans.
  • Shared tooling executor after synchronizing the current base: 141 tests passed under Node.js 26.13.0.
  • Local prototype archive before the subsequent base updates: 3.5 GiB, 54.25 seconds; 1,955 selected tests split into disjoint partitions of 978 and 977 with complete coverage. This is an archive measurement, not an end-to-end CI speedup claim.
  • Termina archive replay from a separate extraction directory: all 46 tests passed.
  • Format, JavaScript lint, ASF headers, notices, actionlint, Bash syntax and diff checks passed.
  • Full workspace Clippy with warnings denied passed locally after fixing the new-base retirement-condition lint.
  • Independent gpt-6-astra high advisor review found no blocking design or implementation issue.

Runner acceptance

  • Fresh-runner root archive execution and all selected suite/OS checks pass.
  • Cold/warm compiler reuse, archive/cache transfer, storage and shard durations are measured.
    Fresh-runner archive acceptance and final cold/warm measurements are still running. Follow-up PRs will address any failures they expose.

The GPUI suite retains its selected headless platform/terminal scope; standalone host webview/video configurations are not newly included.

AI use

  • Generative tooling made a substantive contribution

Tool(s) and scope: OpenAI Codex implemented and verified the CI architecture and documentation. A gpt-6-astra high advisor reviewed the design and implementation.

Generated-by: OpenAI Codex

Route affected packages across owning Cargo workspaces and include real product/runtime inputs. Keep full validation for shared or unclassified changes and require planned jobs to succeed.

Generated-by: OpenAI Codex
Share suite execution with just, route cross-platform and nested-workspace coverage, and require every selected job to succeed. Keep root compiler cache publication under one owner and transfer current-run test artifacts by immutable IDs.

Generated-by: OpenAI Codex
Regenerate the source-bound notices manifest after the SDK changes in #6517. Only the two source hashes change; the reviewed license text is unchanged.

Generated-by: OpenAI Codex
@github-actions github-actions Bot added the effort/XL Under 2500 readable lines label Nov 1, 2026
@M4n5ter
M4n5ter marked this pull request as ready for review November 1, 2026 05:27
@M4n5ter
M4n5ter merged commit d513a72 into rust-preview Nov 1, 2026
4 of 12 checks passed
@M4n5ter
M4n5ter deleted the ci/reliable-suite-plan branch November 1, 2026 05:27

@Astro-Han Astro-Han left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review notice: This comment was posted by an automated review agent. It is not an independent human review and does not replace one.

I reviewed the final 21-file diff at a11e85ce0ab852112e8760abcbde118884cfb1dd, its merged result d513a72e7ebdc11842442341bbf680deac886835, and the preceding fixture/tool/cache/nextest changes in #6511, #6515, #6516 and #6518. The 20 retained changed files match the merged result byte for byte; the removed Computer Use workflow is absent in both.

The implementation selects suites through reverse Cargo dependencies plus declared embedded/product inputs, retains the workspace/feature build recipes, archives root test binaries and product resources, pins replay workers to the producer's nextest version, and makes the final test job reject failed, skipped or cancelled required work. I checked scripts/ci/{plan,suites,run,setup,command}.mjs, scripts/ci.mjs, both composite actions, the workflow, local recipes, documentation, notice changes and the two small fixture/retirement adjustments. I found no confirmed P0–P2 defect in this source review.

Executed locally: all eight existing planner/gate behavior tests pass under Node 22.23.2. These exercise transitive and cross-workspace routing, SDK/Markdown/license inputs, unknown-input fallback and failed/skipped/cancelled results. They use fixture Cargo metadata; I did not obtain real Cargo metadata locally.

Hosted evidence, distinct from my local execution: run 41277034369 produced a full-suite plan for d513a72e. Its completed producer log records 741 archived files in 52.98 seconds, a successfully uploaded 3,705,103,275-byte artifact, and saved compiler/download caches. Its product worker ran the actual notices.mjs check successfully. The root replay partitions are still running at this check; successful archive production and individual workers do not establish a successful aggregate run, archive replay acceptance or a cold/warm speedup.

P3 — the rolling Node selection does not currently establish stable tooling. The completed product-worker log selected /opt/hostedtoolcache/node/27.0.0-alpha.0.0.0/x64/bin/node through node-version: latest. This is a prerelease despite #6516's stated stable-runtime goal. If stable is the intended contract, select a stable channel and retain a version check in the execution evidence. I have not observed a failure caused by that prerelease.

No Rust, native desktop, archive replay or real cache restore was executed on this machine. In particular, the launch-fixture hard links, Root cleanup expectations, all seven GPUI selections and nested-workspace doctests were checked in source, not independently run here. Final runner acceptance and performance measurements remain unverified.


Automated review notice: This comment was posted by an automated review agent operated by Astro-Han. It is not an independent human review and does not replace one.

@Astro-Han

Copy link
Copy Markdown
Contributor

Automated review notice: This comment was posted by an automated review agent. It is not an independent human review and does not replace one.

Follow-up on the pending replay acceptance: run 41277034369 completed successfully on d513a72e. I retrieved both actual worker logs: each reports 991 PASS identities, with no overlap and a union of 1,982, using artifact 13071718708 and nextest 0.9.150.

A separate same-commit run 41278633526 failed: its second slice had 990 passed and one failed, two_captured_requests_reach_send_ready_and_last_output_retains_full_charge, with the 10-second progress watchdog elapsed. This prevents claiming repeatable all-green execution, but does not invalidate the observed successful replay or prove the CI architecture caused the timeout. #6529 changes the relevant test fixture/watchdog scope; current Rust 50687a74 CI is still running. These are hosted results, not my local Rust execution.


Automated review notice: This comment was posted by an automated review agent operated by Astro-Han. It is not an independent human review and does not replace one.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

effort/XL Under 2500 readable lines

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants