Skip to content

Bot: read-only poller status helper for dashboards - #61

Closed
Deicyde wants to merge 2 commits into
mainfrom
claude/bot-gate-status
Closed

Deicyde wants to merge 2 commits into
mainfrom
claude/bot-gate-status

Conversation

@Deicyde

@Deicyde Deicyde commented Oct 14, 2026

Copy link
Copy Markdown
Owner

Stacked on #48 (base claude/poller-fail-closed): it reads the bot-state branch and the step names that #48 introduces. Retarget to main after #48 merges.

Problem

The status dashboard (manage/status.py, which a SessionStart hook runs) could not have shown the three-week poller outage, and after #48 its bot line reads a frozen file:

  • It reads main's frozen copy. It takes the batch and PR from bot/state/bot_state.json on main (manage/status.py:29). After Make the wikidata poller fail closed #48 the live state is on the bot-state branch, and main's copy is a frozen snapshot.
  • --live uses the wrong credentials and drops failures. It runs bot/poll.py --decide with the operator's own gh login and a 12 s timeout, and prints nothing on a timeout or error (manage/status.py:44-49, 69-73). Even when it answers, it reports what the operator's token sees, not the CI token (MATHLIB_FORK_PAT) that had expired.
  • It has no alert for a red poller, an aging batch PR or a merge conflict.

Change

New bot/gate_status.py, a read-only helper for dashboards. It always exits 0 and prints one JSON object, with a top-level guard so even an unexpected error becomes an alert. Any part it cannot read, or did not check, becomes an alert, never a silent omission. It never runs decide().

python3 bot/gate_status.py          # local only: main's snapshot, no network
python3 bot/gate_status.py --live   # plus at most five read-only gh calls; allow 90 s

With --live, it reads:

  • bot_state.json from the bot-state branch. A 404 is resolved with a second call:
    • No branch: it falls back to main's snapshot. The alert says this is expected only before the first state-branch run; after that it means the branch was deleted.
    • Branch without the file: it reports an error, since every poller run then fails at restore.
  • recent wikidata-poll.yml runs on main, skipping runs the concurrency group cancelled, and the step outcomes of the newest one. From those it reports the run's decision (act, wait, error or unknown) and the first failed_step, without downloading logs. Act and wait are told apart by whether any act-gated step ran, so a setup failure after "act" is not mislabelled "wait".
  • the batch PR on leanprover-community/mathlib4.

Alerts cover:

  • a bot state it could not read, or the snapshot shown in place of bot-state
  • a latest run that is not success, with its decision and failed step
  • any red run in the last 24 h, even after a green one
  • no successful run in 24 h. Schedules here run about every 4 h, so the threshold allows for that.
  • a batch PR that conflicts (CONFLICTING, or the merge-conflict label while GitHub still says UNKNOWN)
  • a batch PR open more than 30 days
  • a batch PR closed without merging
  • any GitHub read that failed
  • offline mode, which says it checked nothing live

bot/README.md gains a "Checking the poller" section and a Files row.

manage/status.py is not changed here. The interface it can call is above.

How tested

  • New bot/test_gate_status.py (17 tests) stubs gh. Run step lists are generated from the real wikidata-poll.yml, so a renamed or reordered step fails the tests. It covers:
    • every step that can fail, mapped to its decision and failed step
    • offline mode
    • live state, including a forged source key, which cannot relabel it
    • no branch versus branch without the file
    • non-object or broken JSON
    • a red latest run, and an earlier red run after a green one
    • cancelled runs
    • no success in 24 h
    • unreadable runs, jobs and PR
    • conflicting, label-only, aging and closed PRs
    • a crash inside report()
    • the command line, run on a private copy
  • A read-only live run against GitHub returned the expected shape: the missing-branch alert (Make the wikidata poller fail closed #48 is not merged), the latest run read as wait, and the batch PR flagged as conflicting and open 64 days.
  • An adversarial review ran before this PR, and its findings are folded in.

Risks

  • It is read-only and nothing calls it yet. Wiring it into manage/status.py is a separate change.
  • Step-name coupling. It keys on the Decide and Commit poll state step names. If those are renamed, the decision reads unknown rather than a wrong answer, and the tests fail first.

What Jack must do

Nothing: no secrets, settings or operator steps.

bot/gate_status.py prints one JSON object and always exits 0: the live
state from the bot-state branch (main's snapshot only when the branch is
missing, with the hazard spelled out; a branch without the state file is
an error), what the latest poller run on main concluded (act/wait/error
plus the first failed step, from step outcomes), red runs in the last
24 h, and the batch PR, with alerts for anything red, stale, missing or
unreadable. Offline mode says it checked nothing live. It never runs
decide() with the operator's own login.

Tests: bot/test_gate_status.py builds run step lists from the real
wikidata-poll.yml, so a renamed step fails the tests.
…ts on main's snapshot

After the bootstrap_state change in the base branch, a missing branch
past the one-time bootstrap fails every poller run at restore. The
alert said the next run would act on the stale snapshot.
@Deicyde
Deicyde force-pushed the claude/bot-gate-status branch from 7590dd3 to fb18eee Compare October 14, 2026 04:43
@Deicyde Deicyde mentioned this pull request Oct 14, 2026
@Deicyde

Deicyde commented Oct 14, 2026 •

Copy link
Copy Markdown
Owner Author

Review status: superseded by #59; closing

Update 2026-10-14:

Original review (2026-10-14)

Independent adversarial review of 7590dd38 (since rebased, plus a 3-line alert wording fix; current head fb18eee0): a defect reviewer with skeptics, a design reviewer with a devil's advocate, and an arbiter.

Verdict: close #61 without merging. Its useful behaviours move into #59, which keeps the single implementation in manage/status.py.

Why

  • Nothing calls bot/gate_status.py, so on its own it changes no dashboard.
  • status.py health alerts #59 already makes the same three gh reads (bot-state, run list, batch PR) under different rules: 12 h staleness instead of 24 h, and no --branch main. Merging both leaves two poller checks that disagree.
  • Bot: read-only poller status helper for dashboards #61 does one thing better: every failed read becomes an alert. status.py health alerts #59 will pick this up for gh run list, gh pr view and the bot-state read. Today it falls back quietly and can end with "nothing waiting." for checks that never ran.
  • Both PRs skip cancelled runs as concurrency noise (bot/gate_status.py:122-124). 209 of the poller's 210 cancelled runs were ticks killed by the 30-minute timeout, so the 2026-08-17/18 stall would have looked healthy. status.py health alerts #59 gets this fix too.

Decisions for the maintainer

Merge order: don't merge. Close once #59's revision has the cancelled-run fix and the failed-read alerts; #59 stays based on main and doesn't wait for #48 or #61.

Posted by the WikiLean PR swarm (lead); updated 2026-10-14.

@Deicyde

Deicyde commented Oct 14, 2026

Copy link
Copy Markdown
Owner Author

Status review (2026-10-14): ✅ Ready (stacked on #48); pick one of #59 and #61

The code is small and read-only. Apart from the nits below, it behaved correctly on every path probed: a live run, stubbed 502, 401 and 404 responses on the bot-state and branch reads, startup_failure with no jobs, cancelled runs, and a label-only conflict while GitHub says UNKNOWN. The step names it keys on match the real jobs API.

Decide: this PR or #59. #59 implements the same poller, bot-state and batch-PR checks inside manage/status.py, with different rules:

#59 #61
Stale after 12 h 24 h
Runs read all branches main only
Failure signal scrapes logs step outcomes
Bot-state read fails fallback, no alert alert

This PR's README row says it is "meant for manage/status.py; not wired in yet", but #59 doesn't use it. Keep one engine. This one alerts on bot-state read failures and on any red run in the last 24 h, which #59 does not, so one option is to have status.py call it.

Nits

  • The Bors blind spot is here too. gate_status.py:199 alerts "CLOSED, not merged; the poller halts" for a closed batch PR. Bors closes merged PRs (mergedAt: null, title [Merged by Bors] - …), so this fires after every successful batch. Use settle.is_merged or the title prefix.
  • A failed jobs-API read is stored as poller.decision_error but never becomes an alert. That contradicts the docstring's "never dropped", and the test named for this case doesn't check for an alert.
  • The conflict alert fires on mathlib's merge-conflict label even when GitHub reports MERGEABLE; the PR body says the label is only consulted while GitHub says UNKNOWN. It also ignores the poller's own rebuild of settled PRs.

Order: after #48; then retarget to main. It conflicts with #57 in scripts/ci-python.sh; keep both run_check lines. Until #48's bootstrap_state dispatch runs, --live correctly shows the missing-branch alert.


🤖 Automated status pass by Claude Code over all 18 open PRs (#46–#63), posted for the repo owner. An independent verifier re-checked each finding. All 18 heads were also merged and tested together in a scratch tree; the only test failure that comes from combining them is #47 × #55 (three other pairs had mechanical text conflicts). Reviewed head: fb18eee0.

Base automatically changed from claude/poller-fail-closed to main October 14, 2026 16:46
@Deicyde Deicyde closed this Oct 14, 2026
Deicyde added a commit that referenced this pull request Oct 17, 2026
… a missing bot-state branch

Follow-up to #61 (closed). Its review found that the poller's long
cancelled runs were hangs, not concurrency noise: 209 of its 210 cancelled
runs lasted 30 minutes or more, the job's timeout. A job timeout ends the run
as cancelled, and status.py skipped every cancelled run, so five hung
ticks in a row left a silent "CI ?".

- wikidata-poll.yml: Install pinned Claude CLI and Clone mathlib4 get a
  10-minute step timeout and Poll tick (apply) 20 minutes, inside the
  job's 30. The runner marks a step that hits its own timeout as failed,
  so a hang ends the run as a failure instead of a cancellation, with
  time left for Commit poll state. bot/README.md lists the message, and
  a wiring test keeps the step timeouts below the job's.
- status.py ci_gate reads 20 runs with startedAt/updatedAt. A cancelled
  run that lasted 25 minutes or more counts as timed out and alerts;
  that covers older runs and hangs in the steps without a timeout of
  their own. When no listed run qualifies it alerts instead of printing
  a silent "CI ?".
- batch_pr: a failed read is an alert, not only a "gh unavailable" line.
- live_section: since #48 the bot-state branch is mandatory, so its 404
  is an alert that points at the one-time bootstrap_state dispatch and
  bot/README.md 'Restore failed'. A second read, only on a 404, tells a
  missing branch from a branch without bot_state.json; the latter is
  reported as unreadable state and the batch PR is not checked.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant