Skip to content

Give the drift alarm an inbox — assign a failure issue to mmcky - #29

Merged
mmcky merged 1 commit into
mainfrom
ci/audit-drift-notification
Aug 3, 2026
Merged

Give the drift alarm an inbox — assign a failure issue to mmcky#29
mmcky merged 1 commit into
mainfrom
ci/audit-drift-notification

Conversation

@mmcky

@mmcky mmcky commented Aug 3, 2026

Copy link
Copy Markdown
Contributor

Follow-up to #28. The strict build is this repo's early warning for data references landing in 8 upstream lecture repos, and it works — it went red on the 2026-07-27 schedule the day after QuantEcon/lecture-python.myst#1001 added the phillips suite. Then it sat there for a week, because in practice a red scheduled run notifies nobody. It only surfaced when it blocked an unrelated PR (#27), by which point two more repos had drifted and the Pages deploy had been skipped since Jul 17.

What it does

On any non-PR failure of build, the new notify job opens one issue assigned to @mmcky — or comments on it if it is already open. Dedup is by an audit-drift label, created in-band with gh label create --force, so there is no manual setup step before this works.

decision why
PRs excluded that failure is already in front of the author; an issue per red PR is noise
push to main included, not just schedule #27 is the case that matters — main went red on merge and waiting until the next Monday to say so is the bug being fixed
one issue, later failures comment bounded volume; close it when the build is green and the next failure opens a fresh one
warning lines in the body the fix for a drift failure is almost always one entry in audit_annotations.yml, and the warning names the reference — no reason to make someone open the run log first

Carrying the warnings is why the build step is now teed, and why it needs set -o pipefail: without it tee's exit code masks the strict failure and nothing fires at all.

Verification

Run locally against fresh clones of all 8 lecture repos, inducing drift by dropping one annotation. The teed step still exits 1 (so failure() fires), the collection step emits well-formed $GITHUB_OUTPUT, and the rendered body has the warnings fenced and the links absolute — relative links do not resolve reliably in issue bodies, so both are built from github.server_url.

I also confirmed the post-#28 push to main deployed successfully, so the dashboard is live again and this notifier is guarding a green baseline rather than papering over a red one.

Two notes

mmcky is hardcoded as the assignee. That is the smallest thing that answers the actual need; if this should route to a team or a rotation later, the assignee line is the only thing to change.

consumed-file-check.yml has the same silent-failure gap on push to main — it is the go-live guardrail, so a failure there matters more than a stale dashboard, though its pull_request trigger means it almost always catches problems pre-merge. I did not extend this to it, because copying ~40 lines of shell into a second workflow is the wrong shape — if you want it covered, the notifier should become a composite action first. Happy to do that as a follow-up.

🤖 Generated with Claude Code

The strict build is this repo's early warning for data references landing
in 8 upstream lecture repos, and it works: it went red on the 2026-07-27
schedule the day after lecture-python.myst#1001 added the phillips suite.
Then it sat there for a week, because a red scheduled run notifies nobody
in practice. It only surfaced when it blocked an unrelated PR (#27), by
which point two more repos had drifted and the Pages deploy had been
skipped since Jul 17.

So on any non-PR failure, open one issue assigned to mmcky, or comment on
it if it is already open. PRs are excluded — that failure is already in
front of the author. Dedup is by the audit-drift label, created in-band
with `gh label create --force` so there is no manual setup step.

The issue carries the actual warning lines, not just a run link: the fix
for a drift failure is almost always one entry in audit_annotations.yml,
and the warning names the reference. Capturing them means teeing the
build step, hence `set -o pipefail` — without it tee's exit code would
mask the strict failure and nothing would fire at all.

Verified locally against clones of all 8 repos, by dropping one
annotation to induce drift: the teed step still exits 1, the collection
step emits well-formed $GITHUB_OUTPUT, and the body renders with the
warnings fenced and the links absolute.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings August 3, 2026 06:08

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds an “inbox” for audit-dashboard drift failures by having the workflow automatically open (or update) a single tracking issue on non-PR failures, ensuring scheduled/push-to-main regressions don’t go unnoticed and block unrelated work.

Changes:

  • Tee the strict audit build output into audit.log while preserving failure exit codes (pipefail) so downstream failure handling still triggers.
  • Collect warning: lines from the audit output and expose them as a job output for reuse.
  • Add a notify job that, on non-PR workflow failures, creates/updates a single audit-drift-labeled issue assigned to mmcky, including actionable warning lines and run links.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@mmcky
mmcky merged commit 94259c6 into main Aug 3, 2026
5 checks passed
@mmcky
mmcky deleted the ci/audit-drift-notification branch August 3, 2026 06:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants