Repository navigation
e2e: fail fast on dead GH_TOKEN; guard quota parse - #652
Merged
Merged
Conversation
The 2026-08-05 full-test-suite run failed before any LLM call because RDB_TESTER_PAT_TOKEN (set Feb 25) had expired: every gh call returned HTTP 401, the quota check spewed a bash syntax error trying to compare a JSON error body against an integer, and the failure surfaced as three confusing per-test errors instead of one diagnosis. Both e2e scripts now verify the token up front (gh api user) and exit with rotation instructions when it's invalid, and the shared check_graphql_quota copies guard against non-numeric parses. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The account table said the token belongs to the remote-dev-bot account, but that account was flagged by GitHub as a bot and authorized-trigger duty moved to gnovak — every e2e test issue on rdb-test is authored by gnovak. Documented the actual state, recommended a fine-grained PAT scoped to the test repo with the permissions the suite really needs, and noted the expiry failure mode (the 2026-08-05 suite failure). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…oth tokens The previous commit documented the gnovak-PAT workaround as the standing setup. The bot-account arrangement is the intended design — the account is just flagged by GitHub right now (its profile 404s on the public API) pending a support conversation. Restore the intended setup as primary in CONTRIBUTING with a clearly-marked temporary-state note carrying the stopgap guidance and the restore path. Pre-flight hardening while here: print the token's expiration date (GitHub-Authentication-Token-Expiration header) alongside the login so future runs self-document how long the credential has left, and validate UNAUTHORIZED_PAT up front in e2e-security.sh — its failures otherwise surface only mid-gating-test. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Today's 1.0-gate full-test-suite run failed in under 3 seconds — not the new models, an expired credential:
RDB_TESTER_PAT_TOKENwas last rotated 2026-02-25, so everyghcall against the test repo gotHTTP 401: Bad credentials. Zero LLM spend. Two legibility problems made the diagnosis slower than it should have been:Pass: 0 Fail: 3— which for a security suite is an alarming way to spell "expired token").check_graphql_quotacompared a JSON error body against an integer, producing a bash syntax error as the first visible symptom.Both scripts (each carries its own copy of the quota function) now: authenticate up front via
gh api userand exit immediately with rotation instructions when the token is dead, and guard the quota parse against non-numeric responses.Test plan
bash -nboth scripts; test_cost.py (tests e2e.sh's parse function) passes🤖 Generated with Claude Code