Skip to content

Windows service BelowNormal priority causes Offline health timeouts under CPU contention #3634

Description

@S0RYUASUKA

Client or integration

Codex App

Area

Service lifecycle

Summary

On Windows, the Task Scheduler backend installs the proxy with <Priority>7</Priority> (BelowNormal). Under sustained CPU contention, the proxy can remain alive but stop answering loopback health probes within the tray's 700 ms deadline. The tray reports Offline and model requests stall. This looks like a crash, but the captured failures did not involve a process exit.

Expected: a latency-sensitive local proxy should run at normal scheduling priority and remain competitive with ordinary local workloads. Existing installations need an explicit repair migration, since rewriting the task XML file alone does not update the registered task.

Reproduction

  1. On Windows, install the Task Scheduler service with ocx service install and run concurrent agent work alongside CPU-intensive local workloads. The reported workload had approximately 15 main/subagents in total; that count is an observation, not a required threshold.
  2. Inspect (Get-ScheduledTask -TaskName opencodex-proxy).Settings.Priority and the proxy's process priority. The affected installation reported task priority 7 and Bun BelowNormal.
  3. During sustained near-100% CPU utilization, poll the actual loopback /healthz endpoint with the same 700 ms deadline as the tray. Observe intermittent timeouts and Offline while the proxy PID and start time remain unchanged.

This is load-dependent. A controlled scheduling experiment isolated the proposed cause: two identical minimal Bun HTTP servers, one Normal and one BelowNormal, were pinned to the same CPU core during an eight-second normal-priority CPU load. Both were polled with a 700 ms deadline. A second run swapped the priorities assigned to the two server labels. These were disposable helper processes; the production proxy's affinity was not changed.

Controlled run Normal failures / probes BelowNormal failures / probes Normal maximum latency
Initial assignment 0 / 12 8 / 12 1.369 ms
Swapped assignment 0 / 16 4 / 16 28.4 ms

Timeouts followed priority rather than the server label. This is evidence for scheduler starvation, not a claim that normal priority guarantees responsiveness under every possible load.

Version

Observed in @bitkyc08/opencodex 2.42.0, bundled Bun 1.4.0. The same priority setting remains in dev at a537751 (package version 2.43.0).

Operating system

Windows 11, 12 logical CPUs.

Provider and model

Not provider-specific: the failure was measured on the local health endpoint without a model call.

Logs or error output

Sanitized aggregate of a ten-minute live observation:

Loopback health probes: 2631
Timeouts at the 700 ms deadline: 43
Slow-response process/resource snapshots: 88
Proxy PID/start time: unchanged throughout
Median CPU utilization in slow-response snapshots: 100%
Available physical RAM in those snapshots: approximately 3-8 GiB
Scheduled task priority: 7
Proxy process priority class: BelowNormal
Tray status during failures: Offline

Physical memory exhaustion is not sufficient to explain these captured failures. No process-exit or out-of-memory signature was established for them.

Screenshots and supporting files

Raising the live proxy's priority was followed by 30 successful health probes (maximum 18.94 ms), and the reporter subsequently observed no further Offline episodes. That production mitigation used High; the controlled experiment above supports the less aggressive Normal default for upstream.

Raw logs and screenshots are omitted because they contain local paths and unrelated application/session information.

Redacted configuration

No provider-specific configuration is required. The relevant generated task setting is <Settings><Priority>7</Priority></Settings>.

Checks

  • I searched existing issues and documentation.
  • I removed secrets, tokens, account details, request credentials, and personal data.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingplatformOS/service/tray/ACL (Windows-heavy, not Windows-only)serviceService lifecycle (WinSW/launchd/scheduler)

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions