Skip to content

The documented error table is held to ERROR_RULES - #600

Open
JArmandoAnaya wants to merge 1 commit into
mainfrom
chore/gate-error-code-table
Open

The documented error table is held to ERROR_RULES#600
JArmandoAnaya wants to merge 1 commit into
mainfrom
chore/gate-error-code-table

Conversation

@JArmandoAnaya

Copy link
Copy Markdown
Contributor

docs/api.md's "full table" is where a client author reads the error-code inventory, and nothing
held it to the code. It fell nineteen codes behind before anybody noticed.

The sync half was already delivered

Measured at the merge-base before writing anything, rather than taken from the issue body:

ERROR_RULES codes: 75      table codes: 80
In ERROR_RULES, NOT in table: []
In table, NOT in ERROR_RULES: INTERNAL_ERROR, METHOD_NOT_ALLOWED, NOT_FOUND,
                              UNAUTHORIZED, VALIDATION_ERROR
Status disagreement: []

The documentation audit #533 (d723c4b, 2026-08-10) synced the table and the issue was never
updated. The five remaining extras are exactly the five the document itself names in prose as
raised by the framework and the auth guard. So the drift is gone and this PR is the other half
— which was always the half that mattered, since an ungated mirror is a second spelling waiting to
drift back.

Mechanism: parse the markdown in place

The issue leaves the choice open between parsing the table and promoting it to a
generated-and-committed artifact on openapi.json's footing. Parsing, because the artifact route
needs a script, a --check mode, a CI step and a regeneration ritual, for a table whose rows carry
grouping and prose a generator would flatten — and the invariant is identical either way. It is
~55 lines in a module that already imports ERROR_RULES.

Placement is tests/server/test_errors.py, beside test_the_status_and_code_of_every_error: that
test already owns the ERROR_RULESkernel/errors.py correspondence, and the document is a
third side of the same triangle. frontend/ui-core/src/tokens.test.ts is the precedent for holding
a document this way.

What landed

Three tests, both directions and the status:

Test Fails when
test_every_error_rule_appears_in_the_documented_table a code ships and nobody edits the document
test_every_documented_code_still_exists a row names a code that was renamed or deleted, or sits under the wrong status
test_the_published_message_table_matches_expose_message the 5xx opt-out table and expose_message disagree either way

FRAMEWORK_CODES names the five that have no kernel class to map (VALIDATION_ERROR,
NOT_FOUND, METHOD_NOT_ALLOWED, UNAUTHORIZED, INTERNAL_ERROR) — the document is the only
place they are written down, so they are listed rather than derived.

Two decisions worth the diff:

  • Codes are collected as (code, status) pairs, not a mapping. A mapping lets one code listed
    under two statuses silently overwrite the other; as pairs it arrives as two members and the
    stale one is caught. M3 below is that case.
  • An empty parse raises. Without it, deleting or reshaping the table would make
    test_every_documented_code_still_exists pass on nothing — a gate that stops guarding exactly
    when the document is being edited carelessly, which is the only time it matters. M6 is that
    case.

Also corrects server/errors.py's own module docstring, which said four errors opt out of 5xx
opacity when it has been six since the inference work landed. The 5xx table in docs/api.md
was already right; only the docstring had drifted, and it is now gated from the document side.

Test plan

Every stage's exit code, staged against the harness ceiling:

Stage Exit
bash scripts/check.sh python (106s tests, ruff, mypy, 4/4 import contracts) 0
bash scripts/check.sh frontend generated 0
bash scripts/check.sh docs 0
bash scripts/check.sh browser (245s e2e + 81s real-server cycle) 0

tests/server/test_errors.py: 26 passed.

Mutation-verified — six cases, each turning a named test red, reverted by its own recorded
diff with a clean-tree assertion before and after:

Case Expected red Actually red
M1 · a code dropped from the table test 1 test 1
M2 · the table names a code that does not exist test 2 test 2
M3 · a code listed under the wrong status tests 1 + 2 tests 1 + 2
M4 · an opt-out row dropped test 3 test 3
M5 · the opt-out table gains a row test 3 test 3
M6 · the ## The full table heading renamed tests 1 + 2 tests 1 + 2

M1 reddened only test 1 and M2 only test 2, which is the evidence that the two directions are
independent rather than one test doing both jobs.

Found, not fixed

  • The docstring count is corrected but still a bare numeral, so it can drift again the way it just
    did. Removing the count instead was considered; the correction is what was scoped here, and the
    countless phrasing is a one-word follow-up if it is wanted.
  • A fresh worktree needs pnpm install and pnpm --dir docs-site install before check.sh
    reaches the frontend and docs stages. Pre-existing, unrelated to this diff.

Closes #524.

docs/api.md's "full table" is where a client author reads the code
inventory, and nothing checked it against the code. It fell nineteen
codes behind before anybody noticed; #533 synced it, and syncing alone
was never the fix, because an unchecked mirror is a second spelling
waiting to drift.

Three tests, the same exact-correspondence construction the module
already uses for ERROR_RULES against kernel/errors.py, with the markdown
as the other side:

- every rule appears in the table under its own status
- every documented code still exists at that status, or is one of the
  five the framework and the auth guard raise
- the 5xx opt-out table equals expose_message, exactly, both ways

Codes are collected as (code, status) pairs rather than a mapping, so a
code listed under two statuses arrives as two members instead of one
silently overwriting the other, and a table that parses as empty raises
rather than letting the second test pass on nothing.

Also corrects errors.py's own docstring, which said four errors opt out
of 5xx opacity when it has been six since the inference work landed.
@JArmandoAnaya JArmandoAnaya added this to the 0.1.0 milestone Aug 15, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

docs/api.md's error-code table is 19 codes behind ERROR_RULES: sync it and gate it

1 participant