Skip to content

ops: OpenCode Review Dispatch serialized on one runner (CWL central OpenCode = s1-04) — 163 queued, 22/30 recent runs cancelled after ~3h wait #2526

Description

@seonghobae

Symptom (2026-09-30 08:30 KST / 2026-09-29 23:30 UTC)

Root cause

  1. In opencode-review-dispatch.yml, all three jobs (validate-pr-metadata, coverage-evidence and opencode-review-target) use runs-on: group: CWL central OpenCode.
  2. Runner group CWL central OpenCode (id 5) contains only cwlab-s1-04.
  3. coverage-evidence has timeout-minutes: 300. A single dispatch can hold that one runner for up to 5 hours, so throughput is at most one PR at a time.
  4. Meanwhile, the CWL central control group (id 6: s1-01, s1-05, s2-01, s1-07) has an idle runner, cwlab-s1-07 (busy=false). That group's selected_workflows does not include opencode-review-dispatch.yml, so it can't take the work.

Evidence commands:

  • gh api orgs/ContextualWisdomLab/actions/runner-groups/5/runners
  • .../runner-groups/6 selected_workflows
  • gh api orgs/ContextualWisdomLab/actions/runners
  • .github/workflows/opencode-review-dispatch.yml L56/L399/L2514 (runs-on), L402 (timeout-minutes: 300)

Options (operator decision: this changes a security-isolation boundary, so I have not changed it myself)

  • A. Add more runners to the CWL central OpenCode group, for example moving cwlab-s1-07 from group 6. This keeps isolation per group and widens throughput.
  • B. Add opencode-review-dispatch.yml@refs/heads/main to group 6's selected_workflows. The isolation trade-off needs review.
  • C. Short term: run scripts/ci/opencode_queue_priority.py (operator runbook) to put maintainer-labelled review-priority PRs first.

Side note

The defect in #2273 (coverage build context missing requirements-noema-document-ci-hashes.txt) appears fixed in the current main opencode-review-dispatch.yml L718, which now stages that file. Please check whether #2273 can be closed.

🤖 Generated with Claude Code

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions