Skip to content

feat(runner): hot apply Job capacity through fenced config reload - #1325

Merged
yyjeqhc merged 3 commits into
mainfrom
feat/runner-capacity-hot-reload
Oct 27, 2026
Merged

yyjeqhc merged 3 commits into
mainfrom
feat/runner-capacity-hot-reload

Conversation

@yyjeqhc

@yyjeqhc yyjeqhc commented Oct 27, 2026 •

Copy link
Copy Markdown
Owner

Changing max_concurrent_jobs now applies through the existing Check → Reload transaction and Unix SIGHUP. Growth immediately schedules the original queue; shrink keeps running Jobs intact and waits for occupied slots to fall below the new limit. Desktop stages, checks and hot-applies the same setting without restarting.

Stacked on #1324. Merge B first, then retarget this PR to main. Independent Trace PR: #1326. No automatic merge or deployment.

Ownership and activation

  • All JobManager clones share the effective limit. The existing lifecycle lock serializes capacity changes with admission and slot reservation; the atomic alone does not authorize admission. Config activation commits before queue draining, with spawn/I/O outside config/provider locks.
  • Growth wakes queued Jobs immediately. Shrink never mutates an existing Job's identity/context or cancels its process. FIFO admission cannot be bypassed by a new enqueue. Shutdown owns queued cancellation and now retains terminal evidence instead of silently clearing accepted queued identities.
  • Optional capacity metadata carries the active generation and limit over Polling and WS/QUIC. Registration uses the activated snapshot. Registry checks process instance, current transport/connection lease, disconnected state, monotonic generation and 1–64 bounds. Delayed polls cannot mutate a same-instance stream lease. Old readers ignore the additive field; old senders preserve unknown/startup observations.
  • Desktop reuses owned-local-Runner checks and the existing staged transaction, including definite-failure rollback and uncertain/conflicting-result reconciliation. The panel distinguishes saved settings from Server-reported effective capacity; explicit restart remains a separate warned operation.
  • Other static fields remain restart-required. Invalid candidates, conflicting generations and shutdown fail closed. Polling's ordinary-request limit of 4, ACP and Persistent Shell limits are unchanged.

Validation on mini, using isolated fixtures

  • JobManager suite: 80 passed, including gated 2 → 6 → 2 and enqueue/complete/stop/shutdown races. Additional real interactive/detached/structured capacity regression passed, preserving exact Job/request/context identities and one launch per Job. Explicit real-process shutdown/stop fixtures: 7 passed; input replay/EOF: 5 passed.
  • Config reload: 18 passed plus concurrent generation fencing. Transport suite: 107 passed. Final Registry suite: 398 passed, 1 pre-existing ignored.
  • Optional-wire compatibility, metadata retry after backpressure, actual QUIC TLS metadata/reconnect, and 18 WebSocket tests passed.
  • Desktop: 29 React/helper tests, 10 Tauri staged-transaction tests, 9 settings tests; TypeScript passed.
  • Updated reload E2E passed with real Server/Runner binaries over both Polling and WebSocket after the final lease fix. Both passed seven checks: eight exact Jobs, actual limit/running/queued, growth without completion, shrink without interruption, stable Runner PID, and clean processes/ports/temp state. One concurrent-edit run tripped the repository-residue guard after its capacity assertions passed; the complete clean-checkout rerun passed without changing assertions.
  • dogfood build, formatting, Markdown links and whitespace checks passed.

Stream metadata following SIGHUP converges on the existing keepalive cadence; formal tool reload also triggers metadata submission. Legacy Servers cannot live-update the new capacity field. QUIC uses the existing isolated TLS integration harness, while the standalone reload script covers Polling and WebSocket. No production configuration edits, service restarts or deployment were performed.

Final CI for 23205c253: every selected check passed, including Linux Server/Runner/workspace/tooling, macOS core/Desktop, Windows core/Runner/Desktop, contract, MCP conformance and aggregate gates. Docker Server and the separate Windows package job were skipped by the workflow matrix. Workflow: https://github.com/yyjeqhc/webcodex/actions/runs/40493574239 .

Base automatically changed from fix/runner-default-capacity-32 to main October 27, 2026 11:57
@yyjeqhc
yyjeqhc force-pushed the feat/runner-capacity-hot-reload branch from 23205c2 to 61e5b76 Compare October 27, 2026 12:02
@yyjeqhc
yyjeqhc merged commit 7cf6442 into main Oct 27, 2026
14 checks passed
@yyjeqhc
yyjeqhc deleted the feat/runner-capacity-hot-reload branch October 27, 2026 12:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant