Repository navigation
fix(runtime): recover browser connections and exit idle hosts after failures - #6530
Conversation
… errors Separate connection receipt identity from physical browser ownership. Retain cleanup failures without treating completed failed calls as running work, await all close attempts, and attempt every Host shutdown stage. Generated-by: Codex
Generated-by: Codex
…work Cover completed failing closes, retained Agent cleanup during idle retirement, and continued work after a caught child failure. Assert real retained custody instead of stale parent call counters. Generated-by: Codex
Regenerate the complete native distribution notice catalog and verify the resulting component coverage for the updated CUA pin. Generated-by: Codex
Use Clippy warning denial for selected workspace targets rather than failing Cargo on upstream path-dependency warnings. Fix the macOS workspace lints, include nested script tests, and describe the local check scope accurately. Generated-by: Codex
Keep provider deadline lower bounds tied to wall time while finishing ordinary backoff after elapsed monotonic time. This also lets paused-clock retry coverage complete without busy-waiting for real time. Generated-by: Codex
Keep a real client alive while checking an ephemeral Host, exercise CLI retirement in service mode, and release the final client only at the intended idle boundary. Normalize fixture paths and preserve independent provider cleanup assertions after Host accepts failed-child custody. Generated-by: Codex
Resolve overlapping cleanup custody tests while retaining shutdown completion checks. Generated-by: Codex
Use b97a7e7a for all CUA fork dependencies and patches, update the Cargo lockfile, and regenerate the product license inventory. Generated-by: Codex
Astro-Han
left a comment
There was a problem hiding this comment.
Automated review notice: This comment was posted by an automated review agent. It is not an independent human review and does not replace one.
I reviewed all 35 changed files at 43dae8909ce0c2f1b4735f2f4f9624f77ed8b422, including dependency/notices changes, macOS cleanup, lifecycle fixtures and CI execution. All 35 submitted file blobs match merge commit a3b376ea5bfe95070a206573de7838dcd8f821fe. Current Rust cfd51fad570bbdb98c54e704400775b7b046d318 differs from this submitted tree only in the subsequent dependency-inventory repair and Client request-custody test adjustment.
I found no additional confirmed P0–P2 runtime defect within this source review. There was a confirmed P2 integration defect in the submitted head: the dependency inventory omitted the newly resolved windows-sys@0.52.0. Both the submitted CI and its license job fail with the exact stale-inventory diagnostic. PR #6535 repairs that omission; it is no longer an unresolved finding against current Rust.
The main paths I checked were:
- Browser transport and ownership: I read the complete three-file CUA #4 patch and compared the actually pinned revisions
9e385e66..b97a7e7a; those are the only changed files in that upstream range. The provider emitstakeover, the strict broker request now declares it with a false default, and forwarding preserves it. Each new provider connection already creates a fresh binding UUID;BrowserIdentitynow includes it.ownership.rs:225and:244, plus the reserve/handoff checks inextension.rs, distinguish old receipt custody from a new connection while still checking competing live bindings for the same physical tab. The reconnect test explicitly retains the old unknown preparation and rejects a late failure poisoning the new claim. - Idle retirement and shutdown:
Resources::has_pending()andRuntime::pending_calls()count executing owners independently of completed failures.application/owner.rs:451still serializes admission and obtains native producer guards before stopping; its closing path waits for running jobs and original close attempts before shutting down remaining VMs.shutdown_completeis distinct from a successfulclosedreceipt, andApplication::wait_closed()waits for that completion before returning retained errors.Host::shutdown()now attempts transport, application, access-store and SQL shutdown even after an earlier stage returns an error. I also traced the Computer, Agent, ACP, Workhub, Web, JEV and process retirement checks and their close paths; retained failures are not silently rewritten as successful cleanup. - Later work and retry timing: ended Computer turns retain their original cleanup record while allowing a later evaluation past
CleanupUnconfirmed; unrelated errors still propagate. The model retry change uses a single elapsed-time sleep for ordinary backoff, while explicit provider deadlines recheck the wall-clock lower bound after waking. The cancellation branch remains in the retry wait. - Checks and notices: the actual executor retains
--workspace --all-targets -- -D warningsand discovers nested script tests. I checked the notice diff and exact document SHA againstnotices.json; I did not regenerate the dependency/license graph locally.
Independent validation: eight original CI-planner tests and six nested diagnostics tests passed locally. Four controlled checks invoked the production executor with subprocess execution intercepted, confirming the Clippy arguments and inclusion of the six-case nested diagnostics file. These execute JavaScript, not Rust or the browser.
I also inspected the actual four-worker logs from current Rust CI 41296455060: each worker passed 498 distinct selected tests, all six pairwise intersections were empty, and the union was 1,992 passing test identities. The logs include the reconnect, competing-live-binding, idle-close, Root-lock keeper, Agent failure-isolation and both retry-clock regressions. Every required job in that run succeeded. These are independently inspected hosted results, not local Rust execution, and do not certify a different commit or release.
I have not executed local Rust, the external CUA suite, real Chrome effects, native desktop shutdown, Windows/macOS runtime acceptance, or a live user-database upgrade. The author's broader counts and reported maintainer waiver remain author-reported; I do not claim personal desktop acceptance or additional merge authorization. The existing CUA private protocol keeps its version while adding an optional request field; no legacy Maka epoch or database schema changes here. The substantive-tool declaration and commit trailers are present. This already-merged PR is reviewed by COMMENT only.
Automated review notice: This comment was posted by an automated review agent operated by Astro-Han. It is not an independent human review and does not replace one.
Summary
Explicit browser takeover disconnected the owner IPC because its request field was missing from broker parsing/forwarding. Retained cleanup errors then blocked later Computer calls and could keep Host resident after the last Desktop window closed.
This PR consumes the CUA transport fix, separates receipt identity for each IPC connection while preserving physical tab ownership across live connections, and lets a new Computer evaluation proceed while ended-turn cleanup remains owned.
Idle retirement now distinguishes active work from completed work retained after an error. Host waits for original close attempts and remaining VMs, then attempts transport, access-store and SQL shutdown before returning errors. Computer, JEV, Web, Agent, Workhub, ACP and process idle checks follow the same distinction. Running tasks and other connected clients still keep Host alive. Removed the unrelated fixed 32-process ACP admission cap from this path.
The local check pipeline now discovers nested tooling tests and enforces workspace Clippy diagnostics without treating upstream dependency warnings as workspace errors. Fixed the macOS lints this exposes, regenerated CUA license notices, and removed races/stale cleanup assumptions from affected integration fixtures. Ordinary retry backoff now uses its monotonic delay; explicit provider retry deadlines retain their wall-clock lower-bound semantics.
Verification
Linux, Rust 1.99.1, debug information and incremental compilation disabled; no live user browser, Desktop process or database was changed:
Initial concurrent Host tests hit fixture deadlines and were rerun serially. Three stale Host JS assertions about caller failure, forced provider retirement and parent capacity were corrected to check the current custody/continued-work behavior. No production settlement contract was weakened to satisfy them.
macOS local check evidence:
just checkreached all 2,029 root Rust tests: 2,026 passed, three failed. Fixed the three fixture/obsolete-contract failures; the complete CLI integration and transport integration suites plus retry timing regressions then passed 73/73.just fmt-check lint test-termina licensespassed: strict workspace Clippy, formatting, terminal Windows compile checks, 46 terminal tests, source headers, dependency inventory and regenerated notices. Earlier typecheck, 162 tooling tests (including six previously omitted nested tests) and three product tests passed.rust-previewatb431227ca, final format and strict Rust/JS lint passed on6b8e7de36. The merge-resolution regression run also completed: 152/152 passed, including CLI crash recovery, Bot, Host JS and transport integration. The final CUA pin update changes source identities and generated notices; it does not change CUA production code relative to the tested transport fix.Desktop acceptance
Real-extension desktop acceptance across macOS, Windows, Linux and WSL was not run. The fork maintainer waived the inherited CUA full-matrix gate and authorized merge with the focused IPC and macOS/Linux core evidence. This PR pins the merged CUA
makatipb97a7e7a97b1dabc669b7c362ba39691766df761, including its nom/zbus patches.AI use
Tool(s) and scope: Codex implemented and tested the changes; a separate GPT-6 Astra advisor reviewed failure isolation and shutdown/reconnection correctness. Material commits include Generated-by trailers.
Checklist
Does this PR entail a change in behavior?