Skip to content

Align Agents API routing, Beta order and response headers - #82

Merged
SaladDay merged 12 commits into
mainfrom
codex/http-routing-headers
Sep 23, 2026
Merged

SaladDay merged 12 commits into
mainfrom
codex/http-routing-headers

Conversation

@SaladDay

@SaladDay SaladDay commented Sep 23, 2026 •

Copy link
Copy Markdown
Collaborator

Agents API routing and response headers now follow the observed official behavior. The most important fix: a path with // or .. used to get a 301 that the pinned SDK follows by turning a POST into a GET, so updates were silently lost. Such paths are now served as their cleaned form. The pinned baseline is unchanged (SDK 3.13.0 / d7c41ef / agents=v1).

Behavior

  • Path canonicalization (R1/R2):

    • CanonicalPaths starts from the path exactly as the client wrote it.
    • It percent-encodes bytes that are invalid in a path and decodes only unreserved escapes.
    • It resolves //, . and .. like ServeMux and keeps trailing slashes, so they still 404.
    • %2F, %5C and double encodings are never decoded into separators.
    • It then sets Path and RawPath to agree. It wraps both server configurations (daemon-enabled ServeMux and chi-only), outside every middleware, so chi, the ServeMux and every auth check route on the same string.
    • Every spelling reaches exactly the route group and authentication of its canonical form. The one remaining redirect is the daemon prefix /api/v1/agent-daemon → /api/v1/agent-daemon/.
  • HEAD (R3): runs the GET route without a body, after the same Beta and auth checks. These routes answer HEAD with 405 and Allow instead:

    • the events stream;
    • the File, Skill, Skill-version and Artifact content downloads;
    • the Environment Files directory list;
    • the three Runtime observation and history reads.
  • 405 (R4): Core's JSON body plus Allow in the official order, across the API router.

  • Response headers (R5/R10): every Agents API response, including errors, 401, 405 and SSE, carries:

    • X-Request-Id: req_<32 hex>, from crypto/rand, fresh per request and logged next to the trace IDs;
    • OpenAI-Version: 2020-10-01, OpenAI-Processing-Ms and X-Content-Type-Options: nosniff.

    The pinned SDK's _request_id is now set.

  • Beta before auth (R6/R7/R9):

    • The constant OpenAI-Beta check now runs before authentication on the /v1 Beta group.
    • Exactly one header value equal to agents=v1 is accepted.
    • The message is the official one.
    • Every request that passes is still authenticated before any handler.
  • 401 envelope (R8):

    • Agents API 401s use type invalid_request_error.
    • Beta routes always report a null code. Files, Skills and Core project routes report invalid_api_key only when a Bearer credential was sent and rejected.
    • Deployment-admin (Add administrator onboarding and Agent API key management #79) semantics are unchanged, and messages and WWW-Authenticate are kept.
    • Core Web's connection probe accepts both the old and the new envelope.
  • Unchanged (R11): unknown sub-routes, CORS (an architecture difference), Core's extra headers, and rejection of agents=v0 (an upstream anomaly that is not copied).

Evidence

  • HP-02..24 (campaign scan 6, one owned official Agent, deleted).
  • Recorded in official-semantics-alignment.md (RH1–RH11, with the corrected line on Beta ordering), operation-evidence.md (register RH), the README, the Web connection guide and CONTRIBUTING.

Validation

  • Independent acceptance, written from the requirements only: real Core with PostgreSQL, raw http.client plus the pinned SDK, 82 checks, no official or model calls.

    Row main candidate
    R1 ///dot paths served, update applies FAIL (301) PASS
    R2 percent-encoded IDs FAIL PASS
    R3 HEAD / 405 exclusions FAIL PASS
    R4 Allow FAIL PASS
    R5 X-Request-Id everywhere FAIL PASS
    R6/R7/R9 Beta order, single value, message FAIL PASS
    R8 401 envelope FAIL PASS
    R10 optional headers FAIL PASS
    R11 unchanged behaviors PASS PASS
    Pinned SDK: update through /v1// applies, _request_id set FAIL PASS

    A raw-request-line addendum (21/21) sends invalid bytes, non-ASCII, %2F, %2f, %5C and %252F. It confirms that no spelling reaches a daemon, node, sandbox, admin or project-api-keys handler unauthenticated, and that encoded separators never route.

  • Tests:

    • Raw TCP request-line tests in both server configurations.
    • A route-walk security test over every registered route, including Add administrator onboarding and Agent API key management #79's project API key routes and derived keys.
    • Differential fuzz targets for both configurations, asserted against an independent segment-splitting oracle.
    • A Go client test; Core Web probe tests.
    • The pinned-SDK official client suite, including the new routing script.
  • Server gate on this head: all make check targets, plus Web typecheck, core-doctor, unit tests and build, pass. The Playwright browser cases were not run on the server (no Google Chrome; the user approved the skip). make openapi is byte-identical.

  • Independent reviews by fresh Claude Code subagents (the user-approved replacement for GPT-6 Astra):

    • First review: found one blocker. %2F became a separator when the raw path contained a byte Go treats as invalid, so two spellings could reach different route groups. It was not an authentication bypass, since each group keeps its own authentication. It is fixed as described above.
    • Focused security re-review of the fix: no blocker; 7.7M executions of an independent-oracle fuzz across both configurations and raw TCP probes found no discrepancy. Its follow-ups are applied in the last commit: HEAD 405 on the three Runtime reads, and an independent oracle in the fuzz targets.

Follow-ups recorded, not in this change:

  • unknown HTTP methods get 405 with Allow before authentication (route structure only);
  • the Core Web Vite dev proxy's normalization, which predates this change.

No full protocol compatibility is claimed.


View with [code]smith Autofix with [code]smith
Need help on this PR? Tag @codesmith with what you need. Autofix is disabled.

Add an opt-in request ID context value that ContextHandler emits as
request_id beside trace_id and span_id. The shared HTTP middleware is
unchanged; a service attaches the ID in its own middleware.
Serve every request on its canonical path instead of the ServeMux 301,
which made the pinned SDK resend an update as a GET (HP-17). The
canonicalizing handler wraps the complete server handler in both
configurations, decodes percent-encoded unreserved characters (HP-18)
and resolves empty and dot segments before any routing or
authentication decision; other escapes stay encoded.

HEAD runs GET routes without a body (HP-19); the events stream and
content downloads answer 405. A 405 lists the route's methods in Allow
(HP-20). Every response carries a fresh X-Request-Id, OpenAI-Version,
OpenAI-Processing-Ms and nosniff (HP-23/24).

On the Beta group the constant OpenAI-Beta check, now requiring exactly
one agents=v1 value with the observed message, runs before
authentication (HP-02/03/05). Every 401 has type invalid_request_error;
Beta routes report a null code and Files, Skills and project extensions
invalid_api_key only for a rejected Bearer credential (HP-07).
The connection probe accepts the current 401 envelope (type
invalid_request_error, null code) and still recognizes the older
invalid_api_key code. The result copy and fixtures no longer name the
old code or Beta message.
Record rows RH1-RH11 with the campaign scan 6 evidence, register the RH
evidence code, correct the Beta check order and the 401 envelope in the
contract documents, and add the canonical path rule to CONTRIBUTING.
official_http_routing.py updates an Agent through base URL /v1//, reads
_request_id and the error request_id, and checks HEAD, Allow, the Beta
order and 401 envelopes. Raw checks in the other official scripts and
the Go client service test now expect the Beta check before
authentication and a null-code Beta 401.
When the raw path held a byte net/url considers invalid, such as '{',
'"' or non-ASCII, EscapedPath re-escaped the decoded Path, so %2F became
a separator and ..%2F a dot segment: /v1/x{/..%2F..%2Fcore/v1/sandbox/nodes
reached sandbox administration, and the same trick reached daemon
enrollment and node transport. When nothing was rewritten, chi kept the
original RawPath while the ServeMux used the re-escaped form.

Build the canonical path from RawPath (or the default encoding when Go
kept none), percent-encode invalid bytes, decode only unreserved
escapes and set Path and RawPath consistently, so chi, the ServeMux and
every middleware route on one string and %2F or %5C never separate
segments. Raw request-line tests over real listeners and two
differential fuzz targets cover both server configurations.
HEAD on the Environment Files list would run the live Worker directory
read; it now answers 405 like the stream and content routes. The root
router uses the JSON 405 with Allow for routes outside the Beta group
and for unknown methods, and Allow resolves a mounted router's own
root route, which chi's Find does not descend into.
Describe the canonical path source and the invalid-byte case, the HEAD
exclusion of the directory list, Content-Length only for buffered HEAD
bodies, the JSON 405 outside the Beta group, 401 types limited to the
Agents API handler, the remaining daemon-prefix redirect, and the
invalid_beta code in the Web guide.
The route walk and both differential fuzz targets now include the
project API key management routes (deployment administrator authority)
and a derived project key: dirty and encoded spellings, including
traversal into /core/v1/project-api-keys, reach exactly what their
canonical path reaches, and a derived key authenticates the canonical
Beta route like its static parent. The MCP credential selection
comparison ignores the per-request X-Request-Id and processing time.
The GET stream that ends after a hosted provisioning failure carries
X-Request-Id and OpenAI-Processing-Ms like every Agents API response.
HEAD on a Session Runtime observation, the observation list and Session
Runtime history would sample a provider, export an observation or query
the history backend. They now answer the JSON 405 with Allow, like the
stream, content and directory routes.
The targets compared a spelling only with CanonicalPaths' own output, so
reverting to EscapedPath still passed. An oracle now splits only on a
literal '/', decodes each segment once and resolves only segments that
decode to '.' or '..'. Both targets assert that the routed segments
equal the oracle's and that the oracle's own spelling reaches the same
route and response. With EscapedPath restored and no invalid-byte seeds,
each target finds the %2F blocker within seconds.
@SaladDay
SaladDay merged commit 99eb8d9 into main Sep 23, 2026
2 of 3 checks passed
@SaladDay
SaladDay deleted the codex/http-routing-headers branch October 7, 2026 06:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant