Repository navigation
Check Agents API JSON bodies in one shared gate - #83
Merged
Merged
Conversation
Every Agents API JSON route now reads its body through readJSONObject
before route-specific decoding, validation or lookup. It requires a JSON
Content-Type, applies the route's body limit, then rejects invalid UTF-8,
malformed JSON, repeated keys at any depth and non-object roots with the
official messages, and treats a zero-length body or null as {} (HP-09..15).
The local 'Duplicate parameter' error and the metadata last-value path are
retired.
The Agent, Agent update, Vault, Credential, Credential rotation and Session
metadata scripts send raw bytes with and without the JSON Content-Type and
assert the shared gate's official messages without writes (HP-09..15). A
zero-length body or null is {} where the route accepts it.
Record the HP-09..15 alignment rows, register the JB evidence code and describe the gate in CONTRIBUTING and the Agent create annotation.
The duplicate-key scan keeps key positions in the body instead of copying every key: small objects compare hashed positions and objects with more than 16 keys use an open-addressing set of 4-byte positions, compared after unescaping. A 16 MiB body of short keys now allocates 16 MiB instead of 116 MiB, still in one linear pass. The same pass rejects string escapes that form a lone or mis-paired UTF-16 surrogate with the official parse error (req_1a9b7680d615454ca97c816b25e2f401). The JSON Content-Type is parsed with mime.ParseMediaType, so malformed media types get the Content-Type error. Tests extend the differential check to large objects, bound allocations on many short keys, prove the Beta-before-Content-Type order and drop the excluded-route test that did not reach its routes.
encoding/json matched a case variant such as Metadata, Input or a nested Role to the field and let the last copy win, so Session create, events and nested credential and input objects accepted and stored it. The official service treats such names as distinct keys (req_6ba2a50c71a4410f87a1baac855e82df). caseVariantMember makes a case variant an unknown member at any struct depth, in decodeInputObject and the Session create and events decoders, rejected with each route's existing unknown-member error before any write. The Credential auth type is read exactly too.
A PostgreSQL test sends DELETE, Files and Skills uploads, Skills update and executor credential requests with real storage and shows they keep their own body handling. The Vault, Credential and rotation scripts recount or reread their resources around the body gate checks.
…buffer The duplicate-key set stores 32 hash bits beside each key position, so probes compare keys only when the bits match. Duplicate paths are built with exact-size strings, and the deep-body test now bounds its allocation. The shared reader grows its buffer by doubling: io.ReadAll allocated about five times a large body.
caseVariantMember loaded every object level into a map and added about 231 MiB to a 16 MiB Session create. inexactMember walks the bytes once without copying them and reports the first key that is not exactly a member name, unknown or a case variant, before the decoder, which with DisallowUnknownFields formatted an error for every unknown key. The Session metadata check reads only the metadata member. A 16 MiB Session create of unknown keys now allocates about 96 MiB instead of 482 MiB, and a route-level test bounds Session create and events. The nested cases now fail only without the check.
appendUnescaped assumed paired surrogates and sliced past the end of an exact-capacity key such as "\ud800". HTTP cannot reach it, as the gate rejects these escapes first, but inexactMember could. Like encoding/json it now decodes an unpaired or mis-paired surrogate as U+FFFD, so the key matches no member, and keeps a truncated escape as it is. Tests call both functions on lone and mis-paired surrogates in keys, nested objects and arrays.
The 96 MiB Session create figure holds for unknown top-level keys; unknown keys under agent or environment keep the existing object decoding, linear and at or below main. The doubling reader allocates two to four times the body, against 4.4 to 6.1 times for io.ReadAll. Correct a stale test comment on case-insensitive names.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Every Agents API JSON request body now passes one shared gate that follows the official parse semantics. This fixes two data-integrity problems:
It also stops writes the official service rejects, such as a missing or wrong Content-Type whose update still applied. The pinned baseline is unchanged (SDK 3.13.0 / d7c41ef /
agents=v1).Behavior
All rejections are 400 with type and code
invalid_request_errorand param null, unless noted, with the exact official messages. The order is: authentication and Beta, then Content-Type, then the existing body limit and 413, then UTF-8, syntax, duplicates, root type.application/jsonandapplication/*+jsonare accepted with parameters, case-insensitively, parsed withmime.ParseMediaType. The check is not applied to DELETE, multipart Files/Skills uploads, the non-Beta Skills update,/core/v1, or the daemon, sandbox and node routes.unsupported_or_invalid_configurationfor these.[]as{}is an upstream anomaly and is not copied.null: treated as{}, so create reports the missing field and update becomes the documented empty update.Metadata,Inputor a nestedRoleno longer map to the real field through case-insensitive decoding. They are unknown members, rejected with each route's existing unknown-field error. The official service also treats them as distinct keys.x_agents_core.Evidence
metadataandMetadataare distinct keys (req_6ba2a50c71a4410f87a1baac855e82df).official-semantics-alignment.md,operation-evidence.md(register JB, rows 1 and 3) and CONTRIBUTING.Validation
Independent acceptance, written from the requirements only: real Core with PostgreSQL, exact bytes over
http.clientplus the pinned SDK, all 32 official probes replayed, no official or model calls.An addendum covers the follow-ups:
The original script was unchanged between runs (same sha256).
Tests:
encoding/jsontoken walk on random bodies, including objects with more than 16 members.Callers checked: the TS client, Core Web, the console proxy, install acceptance and the Python scripts all send
application/json. The product repository calls Core through openai-go.Server gate on this head: all
make checktargets, plus Web typecheck, core-doctor, unit tests and build, pass. The Playwright browser cases were not run on the server (no Google Chrome; the user approved the skip).make openapiis byte-identical.Independent reviews by fresh Claude Code subagents (the user-approved replacement for GPT-6 Astra):
encoding/jsonover 38 request types found no mismatch, and forced hash collisions pass. Its follow-ups are applied in the last two commits: a latent panic on an unpaired surrogate in a direct helper call, which HTTP can't reach, and precise memory wording. They were verified by focused tests and the gate on the final head.No full protocol compatibility is claimed.
Need help on this PR? Tag
@codesmithwith what you need. Autofix is disabled.