Skip to content

feat(container-runner): log unexpected platform SIGTERM as an error - #5536

Open
abcxff wants to merge 1 commit into
container-runner-fixesfrom
stack/feat-container-runner-log-unexpected-platform-sigterm-as-an-error-zsrurkrz
Open

feat(container-runner): log unexpected platform SIGTERM as an error#5536
abcxff wants to merge 1 commit into
container-runner-fixesfrom
stack/feat-container-runner-log-unexpected-platform-sigterm-as-an-error-zsrurkrz

Conversation

@abcxff

@abcxff abcxff commented Jul 31, 2026

Copy link
Copy Markdown
Contributor

No description provided.

@abcxff

abcxff commented Jul 31, 2026

Copy link
Copy Markdown
Contributor Author

@claude

claude Bot commented Jul 31, 2026

Copy link
Copy Markdown
Contributor

Review

Small, focused change (bumps the platform-SIGTERM log line from info to error and adds a diagnostic message). A couple of things worth double-checking before merge:

  1. Terminology inconsistency ("loadbalancer-sourced"). Every other comment in this file attributes this signal to "the platform" (e.g. SIGNAL_SHUTDOWN doc: "Set when the process is shutting down because the PLATFORM sent a signal", and the shutdown-sequencing comment: "Signal (platform is reclaiming the instance)"). The new message introduces "loadbalancer-sourced," a term not used anywhere else in the file/crate. If the signal is actually delivered by Cloud Run's control plane rather than a load balancer component, this could mislead whoever reads the error later. Suggest keeping consistent with "platform" unless there's a specific reason to call out the load balancer here.

  2. Hardcoded causal guess in the log message. "likely hitting OOM or running longer than 60 minutes" asserts two specific root causes without any supporting signal in the log line itself (e.g. process uptime, memory stats). The "60 minutes" figure also isn't tied to any constant in this file — SIGTERM_BUDGET defaults to 10s and is about the SIGTERM→SIGKILL grace, not max runtime — so there's nothing here that keeps the message in sync if the platform's actual timeout changes. Since this is now error! (likely feeding alerting/dashboards), consider either:

    • logging the actual elapsed actor/process runtime alongside the message so on-call can verify vs. guess, or
    • softening to something like "unexpected platform SIGTERM (possible OOM or runtime-limit termination)" without a specific hardcoded duration that can silently drift out of sync with reality.
  3. Alerting noise risk. This branch now fires error! for every signal-initiated shutdown, including benign platform-initiated ones (e.g. scale-in, revision rollout, manual instance restart) if those exist for this deployment model, not just true anomalies. Worth confirming that in this container-runner's model (one actor per instance, engine-driven graceful stop for the normal case) a platform SIGTERM really is always anomalous, since that's the premise the error! bump and message rely on.

No test coverage was added, but that seems fine here since this is purely a log level/message change, and the existing e2e signal tests (run-host-tests.sh) assert on child.rs's "sending SIGTERM to pid" line, not this one, so nothing here breaks existing coverage.

Overall: low-risk change, just want to make sure the specific diagnostic claims in the new message are accurate for this platform, since it's now going out at error severity.

@abcxff
abcxff force-pushed the container-runner-fixes branch from 858d26f to 84962e8 Compare July 31, 2026 06:03
@abcxff
abcxff force-pushed the stack/feat-container-runner-log-unexpected-platform-sigterm-as-an-error-zsrurkrz branch from 30aa498 to da132e5 Compare July 31, 2026 06:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant